MicroPrice-Rust

Queue-imbalance micro-price estimation for limit order books, in Rust. Given a stream of best bid / best ask quotes (price and size), it calibrates a Markov-chain model over discretized (imbalance, spread) states and outputs mid + G*(state), an estimate of the expected future mid-price, following the construction in Stoikov (2018) with the departures listed below.

It is aimed at people who want an auditable, reproducible implementation to study or extend, not a trading signal.

Status: research code. One real-data result, and it is negative. On one day of Binance BTCUSDT perpetual best bid/ask data (2024-01-15, first 5 million quotes, pre-registered protocol) the calibrated micro-price is worse than the naive mid-price at the pre-registered horizon of 10 events (MSE +9.67 ticks^2, 95% interval [+9.40, +9.93]; details). On the project's own synthetic data it does not beat the naive mid either (MAE 0.1094 vs 0.0988 ticks, horizon 1, 90,000 held-out events; details). One instrument-day is one sample of one regime, so this does not establish how the model behaves elsewhere. The Binance column mapping is now checked; the LOBSTER mapping is still unverified.

Quickstart

Requires a recent stable Rust toolchain. On Linux the CLI needs the system fontconfig development files (used by plotters' text rendering): sudo apt-get install libfontconfig1-dev.

git clone https://github.com/heykav/microprice-rust.git
cd microprice-rust

# Train on the deterministic synthetic generator, inspect, predict one book.
cargo run --release -p microprice-cli -- train --output /tmp/model.bin --num-events 500000
cargo run --release -p microprice-cli -- inspect --model /tmp/model.bin
cargo run --release -p microprice-cli -- predict --model /tmp/model.bin \
    --bid-price-ticks 10000 --bid-qty 500 --ask-price-ticks 10002 --ask-qty 500

# Out-of-sample evaluation on a chronological split (synthetic data).
cargo run --release -p microprice-cli -- evaluate --num-events 300000 \
    --num-imbalance-buckets 10 --spread-bucket-bounds "1,2,4"

# Library-level example: synthetic -> CSV -> ingest -> calibrate -> compare.
cargo run --release -p microprice-eval --example synthetic_then_csv

To run on your own Level-1 quotes (any CSV with timestamp, bid price/size, ask price/size), see Real-data result and docs/real-data-evaluation.md.

Live demo: Micro-Price Terminal is an interactive console over an embedded, pre-trained model (static; see site/README.md for exactly what that means).

_{The demo page (headless Chromium, 1280x800). The embedded model was trained on the project's synthetic generator, not on market data.}

Real-data result

Result: on this data the micro-price does not beat the naive mid, and at the pre-registered primary comparison it is worse. This is the whole result of the pre-registered run; nothing was tuned after seeing it.

Source	Binance USD-M futures bookTicker daily file, public archive data.binance.vision
File	BTCUSDT-bookTicker-2024-01-15.zip, 188,017,444 bytes (1,871,514,159 bytes unzipped, 19,881,490 data rows)
SHA-256 (zip)	50de14aafe593160e613bcdb46e0f172cbc9ef222ae64ec84b6e3e5ef711d37f, equal to the archive's published .CHECKSUM
Used	first 5,000,000 accepted quotes (00:00:00 to about 06:26 UTC, 23,165 s), the script's pre-registered default; 34 rows off the 0.1 tick grid were skipped
Split	first 70% (3,500,000 quotes) calibrate, last 30% (1,500,000) test; no shuffling
Config	10 imbalance buckets, spread bounds 1/2/4 ticks, alpha 0.5, resolution 2, tick 0.1, bootstrap 1000 x block 1000, seed 42
Code	commit 6787864 (this PR changes no model or evaluation logic)
Run	2026-10-30. The file was downloaded with curl and its checksum compared by hand; the evaluation command is the one the fetch script prints (below). It is deterministic: a second run reproduced the same numbers.

Test split only, error of predicting the mid h events ahead (ticks; MSE in ticks^2). Differences are micro-price minus baseline, so positive means the micro-price is worse; intervals are 95% block-bootstrap.

horizon	predictor	MAE	MSE	MSE diff vs naive mid
1	naive mid	0.0776	1.5917
1	weighted mid	0.4380	2.6774
1	micro-price	3.4998	15.1339	+13.54 [+13.32, +13.75]
10 (primary)	naive mid	0.6978	17.3655
10 (primary)	weighted mid	0.9663	17.7099
10 (primary)	micro-price	3.7527	27.0323	+9.67 [+9.40, +9.93]
100	naive mid	6.4266	258.2925
100	weighted mid	6.4672	256.2379
100	micro-price	7.5814	238.5443	-19.75 [-21.27, -18.10]

Pre-registered decision, applied mechanically: primary comparison (MSE vs the naive mid at horizon 10) is worse; secondary (vs the weighted mid) is worse (+9.32 [+8.32, +10.00]). The horizon-100 MSE row favours the micro-price, but its MAE is worse (+1.15), horizon 100 is descriptive only in the protocol, and it does not change the decision. Direction accuracy of all model predictors on price-changing events is about 0.88 to 0.92 against a majority-direction reference of about 0.51 to 0.54; the micro-price and the weighted mid are within 0.001 of each other, so the direction signal is the imbalance sign, not anything the calibration adds.

One observation made after seeing the result, not part of the protocol: the calibrated adjustment G* reaches 4.98 ticks in the most one-sided imbalance states (the report's diagnostics line), because mid changes on this feed are heavy-tailed (only 1.8% of quotes change the mid, and when they do the size varies widely). A correction that large is the wrong size for a one-event or ten-event horizon, which fits the sign flip at horizon 100. This is an interpretation, not a tested claim.

Full report: results/binance-bookticker-BTCUSDT-2024-01-15.md.

Every other run done (all exploratory, same file and code)

Per commitment 2 of the protocol, these are labelled exploratory and none of them replaces the result above.

run	report	primary-horizon MSE diff vs naive mid	decision rule output
--symmetrize (imbalance-mirror pooling)	report	+9.82 [+9.54, +10.07]	worse (h1 +13.69, h100 -19.60)
wall-clock horizons 100 ms, 1 s, 10 s (additional, not pre-registered)	report	event horizons identical to the pre-registered run	wall-clock MSE diffs: 100 ms -9.43 [-10.93, -7.94], 1 s -66.31 [-73.24, -59.89], 10 s -125.90 [-155.83, -94.60]

The wall-clock rows favour the micro-price on MSE. They are additional analyses (the model is calibrated event-to-event and asked about a time scale it was not fitted for), they are not the primary comparison, and they are reported because they were run, not because they change the verdict.

Live recording check

scripts/record_binance_live.py records N minutes of the public <symbol>@bookTicker WebSocket stream and runs the same evaluation. One 2-minute run was done as a test of the tool, not as a result: BTCUSDT, 2026-10-31 13:06:51 to 13:08:51 UTC, 14,558 quotes, 4,367 test events, 15 of 40 states unvisited in training. Primary comparison: micro-price worse (+3.17 [+0.65, +6.83] at horizon 10); horizon 1 +4.40 [+0.99, +9.12]; horizon 100 -9.37 [-20.10, +1.45]. Too short to read anything into beyond "same sign". Report: results/live-BTCUSDT-20261031T130651Z.md.

pip install websocket-client
TICK_SIZE=0.1 scripts/record_binance_live.py BTCUSDT 10   # 10 minutes

Reproduce

TICK_SIZE=0.1 scripts/fetch_binance_bookticker.sh BTCUSDT 2024-01-15

It downloads the file, checks it against the published checksum, runs the protocol and writes the report. Downloaded data is not committed: the Binance archive is under CC BY-NC-SA 4.0 with a non-commercial, no-hosting clause (see docs/real-data-evaluation.md), so only the generated reports are in the repository. Data courtesy of Binance Vision. Limitations (one instrument-day, level-1 only, no costs, event-time calibration) are in that document and apply to everything above.

What was measured on synthetic data

microprice evaluate --num-events 300000 --num-imbalance-buckets 10 --spread-bucket-bounds "1,2,4" (seed 42, horizon 1 event, 90,000 held-out events) gave micro-price MAE 0.1094 ticks versus naive-mid MAE 0.0988 ticks: the micro-price did not beat the naive mid. Direction accuracy was 0.5147, barely above a coin flip. The generator's imbalance_persistence drives price-move direction as its own Markov chain, independent of queue imbalance, so there is no imbalance signal for the model to find. This is a negative result about that synthetic dataset, not evidence about order books. The Brier score against a climatological baseline shows no skill either (about -0.0006). The same command with --symmetrize gives micro-price MAE 0.1031 vs 0.0988 for the naive mid: still no win (consistent with the generator carrying no imbalance-to-direction signal). The generator is deterministic (same seed, identical output) and exists for tests and examples, not realism.

Figures below are regenerated from the CLI output by scripts/make_figures.py (all synthetic data, seed 42):

How it works

_{Illustrative schematic of the idea, not data and not model output. The marker is drawn at the size-weighted mid for intuition; the library's adjustment comes from calibration.}

1. State. Best-bid/ask sizes give imbalance I = Qb / (Qb + Qa), split into uniform buckets; the spread in ticks is bucketed by explicit bounds. Both pack into one StateId. Prices are integer ticks, never floats.
2. Counting. Consecutive quote events form transitions state_i -> state_j. A transition is price-changing if the mid-price differs (signed tick delta retained); only non-price-changing ones enter Q.
3. Estimation. Laplace-smoothed Q, one-step expected mid change G1, and per-state P(up) (used only for the Brier score, never in G*).
4. Solve. G* = G1 + Q G* by fixed-point iteration (no matrix inverse); non-convergence is a returned error.
5. Predict. micro-price = mid + G*[state], allocation-free.

Exact definitions and degenerate cases: docs/model-spec.md.

The calibrated G* for a synthetic-data model, by imbalance bucket and spread bucket (cells never visited in training are hatched, not coloured):

Departures from the paper

Reference: Stoikov, S. (2018), "The Micro-Price: A High-Frequency Estimator of Future Prices", Quantitative Finance 18(12). (Volume/issue as recalled; page range and DOI not checked because the publisher was unreachable during this work - UNVERIFIED. Nor was the paper's text available, so the list below reflects the author's understanding of its structure and should be checked against it.)

This implementation is in the tradition of that construction, not a replication. Known differences:

• Imbalance symmetrization is optional and off by default. The paper's setting is recalled (UNVERIFIED) to have a natural mirror symmetry (I -> 1 - I, moves negated). --symmetrize imposes it by pooling each transition with its mirror image, which makes G* exactly antisymmetric (proved and tested; derivation in docs/model-spec.md). Without the flag every state is estimated independently and the surface is only approximately antisymmetric.
• The martingale property does not hold by construction here. The construction is motivated by the micro-price being a martingale (recalled, UNVERIFIED). This code's recursion G* = G1 + Q G* only involves non-price-changing transitions; the one-step drift of mid + G* is sum_j R[i][j] G*[j], generally nonzero. A diagnostic measures it and the CLI reports it. solve_full_chain (library only) solves the martingale-consistent recursion; whether the paper's G* is that one is UNVERIFIED.
• Event-to-event sampling. Every consecutive quote event is a transition, including size-only updates and (with some feeds) no-op rows. Whether that matches the paper's time scale is unverified. Fixed-time and N-event sampling are not implemented for calibration (evaluation can use wall-clock horizons, an additional analysis).
• Own discretization and smoothing. Uniform imbalance buckets, explicit spread bounds, and additive Laplace smoothing (with a zero-mean prior on G1) are this project's choices, not the paper's. With alpha = 0, calibration fails if any state is unvisited.
• Truncating integer mid in microprice-core. TopOfBook::mid_price_ticks truncates (bid + ask) / 2, so with an odd bid+ask sum the model sees a mid off by half a tick. The CSV path avoids this by using half-tick model units (--resolution 2); direct library users with odd spreads should do the same or be aware of it.
• Multi-tick moves are kept (signed delta), rather than assuming one-tick moves.

Workspace

crate	contents
microprice-core	integer-tick prices, quantities, validated TopOfBook (crossed/locked policy is explicit), imbalance, state discretization
microprice-data	MarketDataSource, deterministic synthetic generator, Level-1 CSV ingestion (generic, Binance bookTicker, LOBSTER), optional Parquet (parquet-ingestion feature, off by default)
microprice-calibration	transition counting, estimation, solver, serializable MicroPriceModel (bincode + JSON sidecar, validated on load)
microprice-eval	chronological split, metrics, compare_predictors (paired baselines with block-bootstrap intervals)
microprice-cli	microprice binary: train, predict, inspect, benchmark, evaluate, evaluate-csv, evaluate-parquet (feature parquet), visualize
microprice-python	PyO3 bindings (load/save/predict/predict_batch/metadata, train_synthetic); its own Cargo workspace, built with maturin, see docs/python-bindings.md. No Parquet/CSV ingestion from Python yet.

microprice-cli train and evaluate use the synthetic generator only; evaluate-csv is the real-data entry point. evaluate-parquet reads this project's Parquet Level-1 schema (docs/parquet-input.md) and exists only in builds with --features parquet; the default build does not compile arrow/parquet.

Correctness evidence

• The solver matches a hand-derived 2-state example (G* = [0.2, 0.225]).
• A known-truth test samples from that chain, runs the real TransitionCounter -> estimate -> solve pipeline, and converges toward the truth as samples grow (|error| about 0.028, 0.015, 0.004 at n = 1e3, 1e4, 1e5).
• Two bugs found while building it are pinned by tests: same-bucket price-changing transitions must not count toward Q[i][i], and merging non-overlapping chunk counters drops exactly the boundary transitions (documented on merge, with the overlap fix tested).
• #![forbid(unsafe_code)] in every crate; typed errors, no unwrap() in library code.
• Performance numbers, measured with Criterion on stated hardware, are in docs/benchmarking.md. Profiling concluded that SIMD, parallelism or sparse matrices are not justified at the intended state-space size; nothing else about speed is claimed here.

Build and test

cargo build --workspace
cargo test --workspace --all-features      # the test count changes; run it
cargo clippy --workspace --all-targets --all-features -- -D warnings
cargo fmt --all --check
RUSTDOCFLAGS="-D warnings" cargo doc --workspace --no-deps
cargo bench -p microprice-core             # optional; see docs/benchmarking.md
python3 scripts/make_hero.py                  # optional; regenerates the animated hero and explainer SVGs (no dependencies)
python3 scripts/make_figures.py               # optional; regenerates docs/img figures (needs matplotlib, numpy)

CI runs these on Linux and macOS plus a maturin build and smoke test of the Python bindings. The minimum supported Rust version has not been measured; CI uses current stable. See CONTRIBUTING.md and CHANGELOG.md.

Component status

All of the following exist and are tested: core types and state encoding; synthetic generator; transition counting; estimation and smoothing; solver; model serialization; CLI (train, predict, inspect, benchmark, evaluate, visualize); chronological evaluation; Parquet ingestion (library, and CLI evaluate-parquet behind --features parquet); Python bindings; static PNG visualization (three plots via plotters); CSV ingestion and evaluate-csv; profiling.

Added since: optional imbalance symmetrization, a martingale diagnostic, wall-clock evaluation horizons (additional, not pre-registered), and predict_batch in the Python bindings.

Not done: real-data results beyond one Binance instrument-day (no LOBSTER, no other symbols or dates); ingestion in the Python bindings; a measured MSRV.

No benchmark numbers, accuracy claims or example predictions are added to this README unless they come from a reproducible run on real or explicitly synthetic-and-labelled data.

License

MIT, see LICENSE.