What execution quality actually measures

The record an order leaves 

When an instruction is accepted, a row is written. It states the instrument, the direction, the quantity, the price the instruction referenced when it was created, the price at which the position was opened or closed, whether it completed at all, and the times at which each of those things occurred. Nothing called execution quality exists apart from rows of that shape: every figure a broker publishes, and every figure an account holder can derive independently, is a count, an average or a difference computed over a pile of them. A count over past rows carries no information about an order that has not been sent yet.

Four counts are conventionally reported. The industry has settled on their names without ever quite settling on their definitions, and each of them is silent about the other three.

• Fill rate. The proportion of accepted instructions that completed rather than being returned unexecuted. It counts outcomes, not prices: an instruction filled far from the price referenced counts as a fill, exactly like one filled at the price shown.
• Execution speed. The elapsed time between two stated events, conventionally the arrival of an instruction at the broker's system and the confirmation that it has been dealt. It measures duration and nothing else. A quick fill at a worse price is quick.
• Slippage. The difference between the price an instruction referenced and the price at which it completed, in either direction. It is a price measurement rather than a count, and it exists only for instructions that filled.
• Rejection rate. The proportion of instructions returned unexecuted. In most definitions it is the complement of the fill rate, but firms differ over what belongs in the denominator, and that choice moves the figure more than most differences in execution do.

Why an average speed says nothing about the worst case 

A mean is one summary of a distribution, and it is the summary that survives publication because it fits in a sentence. Execution times are not distributed symmetrically around it. The overwhelming majority cluster tightly, because most instructions arrive in ordinary conditions and are dealt against a book that is not moving, and a small minority sit far out in a long tail. The mean is dominated by the quiet majority and moves very little whatever the tail does.

The two averages in that block are, for practical purposes, the same number. The distributions underneath them are not, and an instruction that landed in the tail met the tail rather than the average. This is why practitioners who work with execution data ask for a percentile rather than a mean: the ninety ninth, or the ninety fifth, states what the slowest one in a hundred or one in twenty looked like, which is the part of the distribution a mean is built to suppress.

The tail also matters more than its size suggests, because slow fills are not scattered randomly through the day. They concentrate where many instructions arrive at once and providers are revising their quotes hardest, which is to say where the price is moving. The rare slow fill and the fast market are one event seen from two directions.

Why the direction of slippage matters more than its frequency 

Slippage is a difference measured against a reference price, and it carries a sign. A fill worse than the reference is conventionally called negative slippage, and a fill better than it positive slippage or price improvement. Both are ordinary: in a market that moves continuously, an instruction sent at one instant and dealt at a slightly later one has no particular reason to complete at the price that was on the screen when it was created.

This is why the frequency of slippage carries so little information on its own. A figure stating that a large proportion of instructions completed away from the reference may describe a volatile fortnight rather than the firm reporting it, and the same measurement over a quiet fortnight would look entirely different with nothing having changed. Frequency counts how often the market moved between sending and dealing. What describes the mechanism is the balance between the two directions.

The argument for symmetry is simple. For an instruction small enough not to move the market itself, movement during that interval is as likely to run one way as the other, so a mechanism that passes it through regardless of direction produces a roughly even split either side of the reference over a large sample. A distribution skewed against the client, sustained across quiet and fast conditions and over a sample large enough for chance to wash out, describes a mechanism doing something else. The best known example is an asymmetric tolerance, where a fill worse than the reference is passed on while a better one is withheld and the order requoted.

The two cases in that block report the identical headline. Forty per cent of instructions completed away from the referenced price in each, and a firm quoting only that number would be quoting a true one in both. The counts underneath it point in opposite directions, and the second case has a systematic tilt the first does not.

What a rejection does to every other number 

A rejected instruction produces no fill, so it has no fill price, no difference against a reference price, and in most methodologies no execution time. It leaves both the slippage sample and the speed sample at the moment it fails.

That is a structural point, because the instructions most likely to be declined are not a random selection. Rejections concentrate in fast conditions, where a quote matched against has moved before the match is confirmed, and those are the conditions that produce the worst fills. Declining an instruction rather than dealing it at a worse price removes an unfavourable observation from the slippage distribution and a slow one from the speed distribution. The remaining figures move as a matter of arithmetic, without any fill having been different.

The cost of a rejection is real and lands outside every measure that reports it. An instruction returned unexecuted has to be worked again against a book that moved during the round trip, in exactly the conditions that caused the first attempt to fail. No slippage statistic records that sequence, because the fill it eventually produces is measured against the price referenced at the second attempt rather than the first.

What each figure is measured against 

Every one of these measures needs two reference points, and each pair is a choice rather than a fact of nature. Three choices in particular move the reported figures more than any difference in the underlying execution.

• Where the clock starts. Timing from the arrival of an instruction at the broker's system to the confirmation of a deal excludes the journey between the client's device and that system, often the largest part of the interval a person experiences. Timing from the moment a client acts includes it, and depends on a connection the broker does not control. Both are defensible, and they are not the same number.
• Which price counts as the reference. Slippage measured against the price displayed when the instruction was created is a different quantity from slippage measured against the market at the instant it arrived. The two differ by whatever the market did in transit, which is the interval the measurement is supposed to be about.
• Which instructions are in the sample. Market orders alone, or market orders together with stops and limits. A stop is conventionally executed at the next available price once its level is reached, so it differs from its trigger level by design rather than by failure, and a sample containing stops reports more slippage than one without for reasons unrelated to execution.

None of these choices is wrong, and a firm has to make all three before it can report anything. The consequence is that two published figures are comparable only when the definitions behind them match, and definitions are rarely published in the detail that would settle it. A number without its methodology is closer to a claim than to a measurement.

Reading it from an account statement 

An account statement is the one execution record a client holds that was not assembled by the firm being assessed in order to describe itself. It lists each instruction, its size, the prices, the timestamps the platform recorded and the costs charged, and it can be exported and counted independently.

The counts a statement supports are the same four, computed over one account rather than a whole book. The sequence below is the ordinary one, and each step is a count rather than a judgement.

1. The instructions that completed are separated from those returned unexecuted. The proportion is that account's own fill rate for the period, on its own definition, comparable with the same account in an earlier period rather than with any published figure.
2. Market orders are isolated from stop and limit instructions, because the latter carry a difference against their trigger level by design and mixing the two makes the count uninterpretable.
3. For each market order, the difference between the referenced price and the fill is recorded with its sign, and the two signs are counted separately. The balance between those counts, not the number of orders that differed at all, is the observation.
4. The differences are then split by condition, ordinarily by whether they fall around a scheduled release or in the ordinary run of a session, since one distribution measured across both averages two populations that behave differently.

What a published figure is, and is not 

Firms publish aggregates of their own measurement, and YAL states two of them: an average execution speed of 19 ms and a fill quality of 99.6%.

The honest reading of any published execution figure, at any firm, is that it is one number computed one way over one period, constraining the range of what happened without describing any instance of it. Where the systems producing those numbers physically sit is the subject of the next lesson.

Where practitioners disagree 

Whether speed matters at retail size is the oldest of these arguments and it has not resolved. One position holds that the interval is decisive, because the quotes at the top of an aggregated book are replaced many times a second and an instruction arriving later is worked against a different book. The other holds that for an instruction small enough to sit inside the size available at the best price, in conditions where that price is not moving, one interval of tens of milliseconds is indistinguishable from another in the fill, so the argument is about the tail rather than the mean. Both are accurate about different sizes at different moments, which is why it recurs rather than concluding.

The second disagreement is about what symmetry evidences. One tradition treats an even split either side of the reference as the signature of a mechanism indifferent to direction, reasoning that an asymmetry has to be produced by something and the something is usually a rule. Critics answer that symmetry across a mixed sample can conceal an asymmetry in fast conditions offset by a symmetric quiet majority, so an even overall split is necessary without being sufficient. With no agreed convention for splitting a sample by condition, there is no agreed test.

The third is about publication itself. One argument holds that standardised disclosure on common definitions is the only route to figures that can be compared at all, and regulators in several jurisdictions have moved in that direction. The counter argument is that a metric published for comparison becomes a target: an instruction declined is an instruction that never slipped, and a clock started late reports a short duration. Nothing about that objection is specific to trading, which is why the question keeps being reopened rather than settled.