Execution quality and how it is measured

Execution quality is one of the few things about a broker that can be measured from the account holder's own records rather than taken on assertion. Every order produces a timestamp, a requested price and an executed price, and those three fields are enough to construct the whole picture. What makes the subject difficult is not the measurement but the reporting: a single number pulled out of a distribution can support almost any conclusion, and most published figures are exactly that.

The four things that are actually measured 

Speed is the interval between an order being accepted and being filled. It is the most quoted figure and the least informative on its own, because it says nothing about the price achieved. An order can be filled very quickly at a poor price, and a firm optimising for the published number alone would have an incentive to do exactly that.

Price difference is the distance between the requested and the executed price, signed. It is the measure that matters most and the one least often published in full, because publishing it in full means publishing both tails of a distribution and one of them is unflattering by definition.

Fill rate is the proportion of submitted orders that resulted in a trade. A low fill rate means orders are being rejected, which is a cost that does not appear in a price statistic at all, because a rejected order has no execution price to measure.

Completeness is the proportion of orders filled in their entirety rather than in part. A partially filled order leaves a position smaller than intended and, where the remainder is then filled at a different price, produces an effective price no single row on a statement shows.

Why a distribution rather than an average 

Price differences are close to symmetrical in ordinary conditions, so their average tends towards zero and conveys almost nothing. The shape carries the information: how many orders fill exactly at the requested price, how far the two tails reach, and whether they reach equally far. Two firms can report the same average and have entirely different distributions behind it, one filling almost everything at the requested price and the other producing wide differences in both directions that happen to cancel.

Whether the second profile is worse than the first depends on what an order is trying to achieve, which is why the honest presentation is the distribution itself rather than a verdict derived from it. The average, on its own, is the one summary that distinguishes the two not at all.

What a statistic is conditioned on 

Every execution figure describes a population, and the definition of that population does more work than the number. A statistic covering only market orders excludes the pending orders that trigger in fast conditions. One covering only the most liquid instrument excludes everything that is harder to fill. One covering only the busiest hours excludes the thin windows around session boundaries and the daily rollover, which is precisely when execution is hardest.

• Which order types are included. Market orders and triggered stops behave very differently, and combining them without saying so hides both.
• Which instruments. A figure aggregated across a whole catalog is dominated by whichever instruments carry the most orders.
• Which hours. Excluding the rollover window and the minutes around scheduled releases removes the hardest conditions from the sample.
• Which sizes. Small orders fill at the top of the book by construction, so a size weighted figure and an order weighted figure describe different things.
• Whether rejected orders are counted. A statistic computed only over filled orders cannot see the cost of the ones that were not.

None of those exclusions is dishonest in itself, and several of them are reasonable analytical choices. What makes a figure uninterpretable is not the conditioning but the absence of a statement of it, and a published statistic that does not say what it covers cannot be compared with one that does.

The measurement available from an account's own records 

Platforms record a requested price and an executed price on each deal ticket, along with timestamps for submission and fill. The difference between the two price fields, taken across a run of orders and grouped by instrument and by time of day, reproduces the same distribution a firm would report, on the one population that is directly relevant: this account's own orders.

Two adjustments make the result interpretable. The spread has to be removed first, because a position always opens on one side of a two way price and closes on the other, and that difference is present in every fill regardless of execution. And triggered orders have to be separated from immediate ones, because a stop enters the market during a move it is on the wrong side of and its distribution is asymmetrical for reasons unconnected to how it was handled.

What execution statistics cannot show 

No execution measurement reveals what would have happened at a different firm, because the counterfactual does not exist: the order was only submitted once. Comparisons between firms are therefore comparisons between populations of different orders in different conditions, and they are meaningful only in aggregate and over long periods.

Nor do they capture the orders that were never submitted because a price was unavailable, or the difference between the price displayed and the price a larger order would have met. Execution quality is a description of what happened to orders that were sent, and there is a category of cost that never enters it.