A real order in the book, or a fake one

The bench tests claims about how an exchange order book behaves against live public data: it assembles a local book from a stream of updates, looks for large clusters of limit orders, tells the genuine ones from those posted for show, and runs hypotheses on a simulated account. There are no keys and no request signing in the project at all — it could not place an order even by accident.

The significance threshold is not a sum of money but a power function of turnover

The first version worked the textbook way: anything above a fixed sum counted as “large”. Now the threshold is the instrument’s turnover raised to a power above one, with coefficients fitted to two anchor points.

A fixed threshold is wrong in both directions at once. The same sum on a thin instrument is a slab that genuinely holds the price, while on the most liquid one it is an ordinary level eaten through in seconds. Hence a stream of false signals at the top and blindness at the bottom. The exponent above one is not invented: the capacity that turns a price grows faster than turnover, not in proportion to it.

Significance is checked by two different questions

A cluster counts as significant only if both hold: its size clears the threshold and exceeds the book’s median level many times over.

The threshold alone is not enough. On a high-turnover instrument the entire book is thick, and the largest level clears the threshold simply because everything there is large — that is not a wall but an even book the price walks through unnoticed. The two conditions answer two different questions: is there enough capacity to turn the price and is there anything standing out at all against the book’s own profile.

The cluster is measured in a price pocket, not at a single price

The volume of every level inside a narrow pocket around the candidate is summed — and the same pocket is used later when checking how much of the cluster is left.

A large participant almost never places the whole size as one order: they spread it across neighbouring prices or add in portions. Measuring at exactly one price simply will not find such a layered slab. A fixed-width pocket is the compromise: it collects the spread-out volume without stretching far enough to glue a “wall” out of smeared liquidity.

Later it turned out that a mismatch between pockets is also the source of an expensive mistake. More on that below.

Reality is proven over time, not in a snapshot

A separate check takes eight snapshots of the book a few seconds apart and looks at two things: how many different prices the cluster occupied and how much its size jumped.

An order in the book is an intention, not a fact: cancelling it costs nothing. A single snapshot fundamentally cannot tell a slab meant to be filled from one drawn to be pulled — at any instant they look identical. The only sign of seriousness visible from outside is the cost of cancelling over time: an order that survived tens of seconds without changing price or size stands out in a market where most limit orders live about a second.

High churn is not yet evidence

The monitor distinguishes not “present or absent” but six states: holding, being nibbled, eaten right at the price, pulled away from the price, re-quoted, broken through. The fork after “more than half is gone” turns on the distance to the price and on whether a new large order appeared nearby.

A naive “it changed, so it is fake” detector mostly catches an honest market maker: as the price drifts they must re-quote every few seconds. The bench’s own run showed it: the upper cluster held through all eight snapshots, while the lower one right at the touch changed price in five cases out of eight — and that was a normal refresh, not manipulation. Without separating “re-quoted” from “pulled”, the bench would exit live positions on every move the maker makes.

The book is assembled by protocol, not by “snapshot then subscribe”

The order is strict: connect to the update stream, only buffer incoming events at first, then take a snapshot, then replay the buffer discarding everything older than it, and from there apply updates tick by tick.

The obvious order — snapshot, then subscribe — leaves a gap between the two actions: events that passed through it are lost for good. The divergence does not fail loudly, it accumulates quietly: long-cancelled levels linger, a cluster holds where it no longer exists. That is the difference between a book you can act on and a book that lies more the longer the bench runs.

The mistake that cost the account was not fixed by tuning a threshold

An early version searched for the cluster in one pocket and measured the remainder of an open position in another, half as wide. The ratio “left over original” was therefore always below one, a false exit fired seconds after entry, and the simulated account fell 83%.

The obvious reaction to “it exits too often” is to raise the threshold. That would have masked the bug rather than removed it: a systematic bias fires eventually at any threshold. The fix was four independent conditions: the same pocket as in the search; a minimum hold; the remainder must stay below the threshold for several seconds; and the price must actually press against the cluster rather than drift nearby.

The safeguards stop the trading, not the trade

After that episode the engine gained cut-offs of a different order: all new entries blocked on drawdown, a pause after a streak of losses, a cooldown per instrument, a cap on trades per instrument per session. Resuming is manual only.

Risk per trade protects against one bad trade but not against a systemic failure, where the detector itself is broken and reproduces one mistake over and over. That is exactly what happened: dozens of trades, each formally within risk. And the key detail: a safeguard that lifts itself is not a safeguard.

The success criterion is written into the code before the data exists

In the factor test the protocol lives in its own dictionary: what is being tested, over what horizon, which cost model, what counts as success and when to stop. The log is append-only.

Without a criterion fixed in advance any result becomes “almost worked”: significance came out short — widen the window; wrong sign — flip the side. It was this protocol that let the bench disprove its own hypotheses: one relationship turned out to be overfitting, another was real but twenty-five times weaker than costs, a third fell apart under a stricter check.

Python · WebSocket · asyncio · standard library