Repository navigation
test(fuzz): fuzz the hook envelope and the config parsers, and gate the corpus on every landing (CLOUD-112) - #354
Merged
wenzowski merged 1 commit intoAug 12, 2026
Conversation
CLOUD-112 Fuzz the hook-envelope parser and the config-trust loader
Fuzzing pays off against parsers of partly-untrusted input. Batten has none today (TOML-via-serde and git output don't warrant it), so this is scheduled behind the two tickets that introduce such parsers — it becomes valuable exactly when they land, not before. Trigger (blockers): the neutral hook-envelope parser (CLOUD-43) and the two-layer Scope / acceptance
|
wenzowski
marked this pull request as ready for review
August 12, 2026 05:18
wenzowski
force-pushed
the
wenzowski/cloud-112-fuzz-the-hook-envelope-parser-and-the-config-trust-loader
branch
5 times, most recently
from
August 12, 2026 06:15
7f2d7d1 to
764db76
Compare
…he corpus on every landing
The two surfaces that read partly-untrusted input now have fuzz targets:
`hook::decode`, which reads whatever an agent harness writes to stdin, and
`config::parse`/`parse_override` plus the `trust::weakenings` clamp, which
under `--config-from <ref>` read TOML that need not come from the working tree.
The change splits fuzzing into the two things it actually is, along the line
`lock-check` already drew — a property of the commit belongs in the gate, a
property of the world belongs on a clock:
* The SEARCH (`mise run fuzz`, libFuzzer under nightly) is a property of the
world: nondeterministic, and green proves only that this seed found nothing
this time. It runs weekly, never on the landing path, where it would also
force a nightly toolchain into every local verify to satisfy
ci-local-parity.
* The REPLAY (`crates/batten/tests/fuzz_corpus.rs`) is a property of the
commit: every curated seed and every saved reproducer, on stable, in
milliseconds, inside verify. It is what makes "a reproducer becomes a
regression test" structural — committing the file is the whole follow-up.
Both drivers assert the SAME properties, because `fuzz/properties.rs` is
included by each; a reproducer that meant one thing when found and another when
replayed is the failure that split would cause. The properties go past "does
not panic": decode is a function of its input, an undecodable payload stays an
allow, a decoded event is either the host's own spelling or the documented
default, and the clamp is total, self-consistent and sorted.
The search earned its keep before it landed. Its first find, in about a thousand
execs, refuted one of these properties rather than the decoder: a payload whose
key mutated to `hookaevent_name` names no event and therefore takes the assumed
default, which the original assertion forbade. That input is kept as a named
seed, and it is what makes the replay gate demonstrably able to fail.
cargo-fuzz is deliberately not a `[tools]` entry: upstream ships x86_64 assets
only, so no attested-binary backend satisfies lock-complete's required
platforms, and the `cargo:` backend both trips no-source-built-tool and cannot
build at all under the pinned 1.85 (its deps require rustc 1.91). The task
provisions it under the nightly libFuzzer needs anyway, only when missing.
`mise-tasks/sbom` excludes the fuzz tree: it is a development harness that
ships to nobody, and its second lockfile would break sbom-package-drift by
construction (measured 175 -> 281).
Refs: CLOUD-112
wenzowski
force-pushed
the
wenzowski/cloud-112-fuzz-the-hook-envelope-parser-and-the-config-trust-loader
branch
from
August 12, 2026 06:17
764db76 to
a69729a
Compare
|
Contributor
Author
|
/fast-forward |
wenzowski
deleted the
wenzowski/cloud-112-fuzz-the-hook-envelope-parser-and-the-config-trust-loader
branch
August 12, 2026 06:23
This was referenced Aug 12, 2026
wenzowski
pushed a commit
that referenced
this pull request
Aug 12, 2026
…ing it Measured twice on 2026-08-12, on two branches that touch no part of the landing loop: PR #354 (a fuzz crate) and PR #370 (a board gate) each lost a full verify and a lap to `not ok THE DEFECT: a lease sighted before it expired is taken on the first check after`. Re-run alone, the case passes. It was never the branch and never the code under test — it was the test's own clock. The race is in the SETUP, not the measurement, which is what the original comment missed while correctly calling the row above it "the flake-proof half". The case pins a 4s TTL and then depends on a wall-clock ordering across separate process launches. `test:bats` runs under `rush --jobs` (CLOUD-386), and the process can be descheduled for longer than the whole TTL — at which point the "sight the live lease" acquire is RIGHT to succeed, and it steals instead of sighting. The output said so plainly: `took the lease 0s after ... stopped holding it`. The assertion graded the runner's scheduler. That is worse than an ordinary flake because it sits in `verify`, on the landing path, in the suite that gates every branch in the fleet. `land` says "reproduce and fix locally", nothing reproduces, and the reliable way through is to run `land` again — the reflexive drive-to-green AGENTS.md forbids, arrived at honestly. So the precondition is established rather than assumed, and never asserted through (CLOUD-249): a sighting acquire that succeeds means the lease had already expired, so no sighting happened and there is nothing to measure. SETUP is retried — never the measurement, which would be drive-to-green in the test — because a plain skip would fire often enough to erase the coverage, this having raced twice in one day. Three attempts, then a skip naming the reason. The TTL stays short deliberately. Raising it widens the window without removing the race, and every second added is paid on every run of the suite. Both halves of the obligation are checked, since either alone would let the repair convert a flaky check into one that cannot fail: A. Against the pre-fix land-lock (sighting recorded only once expired, steal at ~9-12s) the repaired case still goes RED. B. With a deschedule past the whole TTL injected before the sighting, it SKIPS naming the reason — never fails, never silently passes. `LAND_LOCK_UNDER_TEST` is added so that harness mutates a COPY: an in-place mutation makes a corrupted commit reachable from any concurrent `git add -A`, which staged a mutant into a pushed commit earlier the same day (CLOUD-418). The harness hashes both tracked files before and after and requires them byte-identical. Refs: CLOUD-448
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
The two surfaces that read partly-untrusted input now have fuzz targets:
hook::decode(crates/batten/src/hook.rs:465) — reads whatever an agent harness writes to stdin, across all ofHarness::ALL, several payload dialects each.config::parse/parse_overrideplus thetrust::weakeningsraise-only clamp — under--config-from <ref>(CLOUD-31) these read TOML that need not come from the working tree.The split, which is the design
Fuzzing is two things that are easy to confuse, and this repo already drew the line in
lock-check: a property of the commit belongs in the gate, a property of the world belongs on a clock.mise run fuzz, libFuzzer/nightly).github/workflows/fuzz.yml), never on the landing pathcrates/batten/tests/fuzz_corpus.rs)verify, like any other testPutting the search on the landing path would also force a nightly toolchain and a wall-clock search into every local
verify, sinceci-local-parityrequires every task a PR workflow runs to be oneverifyruns.The replay is what makes "a reproducer becomes a regression test" structural rather than a habit: committing the file is the entire follow-up, and the fix cannot land while it still fails.
The properties go past "does not panic"
fuzz/properties.rsisinclude!d by both fuzz targets and by the replay gate, so a reproducer means the same thing where it was found and where it is replayed. It asserts: decode is a function of its input; an undecodable payload stays an allow; a decoded event is either the host's own spelling or the documented assumed default; a write target that exists is a real path; and the clamp is total, never weakens a config against itself, and returns a sorted (byte-stable, §6) report.The search earned its keep before it landed
Its first find, in about a thousand execs, refuted one of these properties rather than the decoder: a payload whose key mutated to
hookaevent_namenames no event, soraw_eventtakes the assumed default — which the original assertion forbade. The property was wrong. Its input is kept as a named seed,fuzz/corpus/hook_decode/no-event-key-takes-the-default, and it is what makes the replay gate demonstrably able to fail: restoring the over-strong assertion turns the gate red naming exactly that file. The empty-corpus refusal was probed the same way.Toolchain
cargo-fuzzis deliberately not a[tools]entry, and that is upstream's constraint twice over: 0.13.2 publishes x86_64 assets only, so no attested-binary backend satisfieslock-complete's required platforms (linux-arm64,macos-arm64) — the same gap that keepscargo-msrvout — and thecargo:backend that would sidestep it both tripsno-source-built-tooland cannot build at all under the pinned 1.85, since its dependencies require rustc 1.91 (measured). The task provisions it under the nightly libFuzzer needs anyway, pinned and only when missing, off the landing path — the same shape as[tasks.coverage]'s component and[tasks.msrv]'s toolchain.mise-tasks/sbomexcludes./fuzz: it is a development harness that ships to nobody, and its second lockfile would break thesbom-package-driftinvariant by construction (measured 175 → 281 packages).The machine's working corpus goes to git-ignored
fuzz/work/with the curated seeds passed read-only, so a search never appends a thousand opaque blobs to the tree; the weekly job carries it in a cache instead.mise run verifygreen on the rebased HEAD.Refs: CLOUD-112
Generated by Claude Code