Repository navigation
Conversation
ron has ~450 correctness tests but none that guard the asymptotic cost of parsing. That gap let ron-rs#534 regress escaped_byte_buf from O(n) to O(n^2) across v0.9.0..=v0.12.2 — four releases — with every test green, and a second O(n^2) entered next_bytes_is_float in v0.12.2 (ron-rs#602). tests/complexity_scaling.rs asserts that parse time grows at most linearly: it doubles the input and checks the time ratio (~2 linear vs ~4 quadratic; threshold 3.0), using min-of-N + median to stay robust to timing noise. The tests are #[ignore]d and run in a dedicated release CI job, so they never slow down or flake the normal test job. Two forms guard the regressions fixed in ron-rs#608 (next_bytes_is_float) and ron-rs#610 (escaped_byte_buf); two linear controls keep the harness honest. On the pre-fix tree (d0e99bc) those two forms report ratio ~4.0 and fail the 3.0 threshold; on current master they are ~2.0 and pass. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
enomado
force-pushed
the
test/parser-complexity-scaling
branch
from
July 16, 2026 10:28
4f49edc to
54a4292
Compare
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
ron has ~450 correctness tests and, until now, zero tests that guard the
asymptotic cost of parsing. That blind spot is exactly how two independent O(n²)
regressions shipped unnoticed:
escaped_byte_buffrom O(n)into O(n²). It shipped in v0.9.0 and survived through v0.12.2 — four
releases, ~two years — with every one of the ~450 correctness tests green. Fixed in
Fix quadratic escaped string parsing in escaped_byte_buf #610.
next_bytes_is_floatin v0.12.2 via the number-rangesfeature Add support for number ranges #602. Fixed in Fix quadratic float parsing in next_bytes_is_float #608.
Both fixes are now on
master. This PR adds the missing layer: a test that would havecaught either regression the day it landed — and will catch the next one. Both bugs are
the same shape (an unbounded
find(..)scanning to end-of-input, called once perelement in a loop over that same input), and no correctness test can see it: they all
check what is parsed, never how the cost grows.
What
tests/complexity_scaling.rsadds doubling-ratio tests that assert parse time growsat most linearly in input size:
assertion is on the ratio, so it does not depend on the speed of the machine
running it.
time) and the verdict is the median of the doubling ratios (one slow sample
cannot flip it). The threshold 3.0 sits in the wide gap between linear (~2) and
quadratic (~4).
#[ignore]d, so they never slow down or flake the normalcargo testjob. A dedicated
ScalingCI job runs them in--release, single-threaded.It's green — and it demonstrably bites
Now that #608 and #610 are merged, the
Scalingjob is green on this PR: itconfirms the parser is linear on the current code. Ratios from this PR's own CI run:
d0e99bc0)escaped_stringfloats_via_valueint_list(control)plain_strings(control)The right-hand column (verified locally against
d0e99bc0, the commit right before thetwo fixes) is the important one: run against the buggy tree, the two bug-catching forms
report 4.02 and 3.85 — well past the 3.0 threshold. The test fails exactly when
the regression is present and passes once it's fixed. The two linear controls stay ~2 on
both trees, so a failure is specific, not just "parsing got slower".
Note
floats_via_valueis not only about floats: on the self-describingValuepathnext_bytes_is_floatruns for every number to disambiguate int from float, so thesame blow-up hit integer sequences and map keys/values too (a 200k-entry integer map
took 103 s on the pre-fix tree).
Notes for maintainers
Scalingjob is isolated from the main test matrix, so a timing hiccup can neverblock a normal PR. If you'd prefer it non-blocking anyway,
continue-on-error: trueor moving it under the
schedule:trigger both work — happy to adjust.~1 s on CI); a form only gets slow when it is actually quadratic — which is the case
you want to be loud.
fn(usize) -> Stringand aparseclosure.🤖 Generated with Claude Code