Skip to content

feat(perf): record the invocation cost as a series, and gate the regression (CLOUD-172) - #333

Merged
wenzowski merged 9 commits into
mainfrom
wenzowski/cloud-172-record-battens-invocation-cost-as-a-series-so-hook-latency
Aug 14, 2026
Merged

wenzowski merged 9 commits into
mainfrom
wenzowski/cloud-172-record-battens-invocation-cost-as-a-series-so-hook-latency

Conversation

@wenzowski

@wenzowski wenzowski commented Aug 11, 2026 •

Copy link
Copy Markdown
Contributor

CLOUD-207 published one number. A number measured once decays in silence: nobody notices 3 ms becoming 40 ms, they notice months later that the hook got uninstalled. This turns the snapshot into a series attributable to a SHA, and puts a comparison gate on the pull request that caused a regression — the half that reaches someone who can still cheaply act.

The tasks

  • perf-record — appends each trunk measurement to git notes on refs/notes/perf, keyed to the commit. No external service, and a regression stays attributable without re-running history. Refuses to run off main: a branch's numbers are not the trunk's, and a series mixing them cannot be read.
  • perf-pair — builds the merge-base and HEAD binaries and measures them back to back on one machine.
  • perf-compare — decides that ratio. A pure function of stdin, so tests/perf-compare.bats exercises every verdict on every commit with no build.
  • perf-gate — composes the two, and is the one name verify and CI call, so a caller cannot wire in half of it.

Three decisions, each measured rather than chosen

Wall clock, paired — instruction counts are not available. The issue specified callgrind with wall clock as a fallback "where valgrind is unavailable". mise registry valgrind reports tool not found in registry, so it cannot be pinned, and batten.toml's no-source-built-tool rule forbids compiling one. The fallback is the only branch.

What makes wall clock usable is not a better clock but a better experiment. A contended runner can double a 3 ms measurement for reasons no commit caused. What it cannot do is double one arm of a pair measured back to back on that same runner in that same second. Machine variance is common-mode across the pair and divides out. So the verdict is a ratio, never an absolute, and perf-record stamps metric=wall-clock into every entry so a later instruction-count series can never be diffed against this one and read as a step change.

Threshold 1.30×, derived from a measured null. Identical binary as both arms, 100 runs after 10 warmups, ten repeats across three paths — 30 ratios of a comparison that by construction measures nothing. They spread 0.966 to 1.102. A 10% swing is what this experiment produces when nothing changed, so a threshold near 1.10 would be a coin flip. 1.30 clears that measured maximum with room for a busier runner. mise run perf-pair --null re-measures the floor, and a bats case fails if the constant is ever tightened below the recorded one.

Recording trigger: the daily schedule, not a push-to-main workflow. AGENTS.md forbids the latter and the reason holds rather than merely applying — main advances only by fast-forward to a SHA CI already judged, so a push trigger buys a runner per merge for a measurement a clock can take. The cost is resolution, and it is affordable because the series is not the attribution mechanism: perf-gate catches a regression on the PR that caused it, at the commit that caused it.

Two corrections to what this branch first shipped

Both are against CLOUD-172's own Mechanism clause rather than against taste, and both were found by re-reading the issue beside the diff instead of beside the PR body.

The namespace was never moved. The issue specifies perf, perf:record, perf:compare, and an earlier version of this body claimed the rename had happened. It had not: the tasks landed as bench-* beside CLOUD-207's bench and bench-assert, and the workflow as bench.yml. The whole latency family moves now — six tasks, four suites, the workflow, the job, the concurrency group, and refs/notes/bench → refs/notes/perf (no notes ref exists on the remote yet, so the series orphans nothing). Hyphens rather than the issue's colons, because mise-tasks/ holds file tasks where a colon is a path separator. token-bench, token-bench-check and bench/tokens/** are deliberately untouched — they are CLOUD-119's token-economics benchmark, and freeing the bare bench for it is the reason the issue gives for this move.

The gate ran nowhere that decides a pull request. An earlier version of this body claimed a draft-gated perf job in ci.yml, in final's needs, and in CI_REQUIRED_CHECKS. None of the three existed. perf-gate was a step in [tasks.verify] only, and [tasks.ci] is depends = ["hooks", "deny"] — so no runner ever invoked it, and the Done clause "a PR that regresses past the threshold fails a check that names the regressed measurement" was unmet, because verify is what an author runs locally. The job exists now, wired into all three, with fetch-depth: 0 because perf-pair builds both arms and a shallow clone has no merge base.

It stays affordable for the reason the gate's own header argues: no crate source, manifest or lockfile in the diff means no build, and most of this repository's traffic pays a git diff --name-only.

What the roster change then broke, which is the interesting part

Adding perf to CI_REQUIRED_CHECKS failed six cases in tests/checks-green.bats and hung tests/ci-wait.bats — twice, for the better part of an hour each time.

Both suites read the real roster from mise.toml [env] on purpose, so they can never disagree with the one landing uses — and then both built their all-green fixture from a hand-written list of the names. A roster that grows leaves the fixture short, and short means a required name with no run: exit 3 in checks-green, and in ci-wait a poll that cannot tell "the roster grew" from "the run is still going". The suite that tests the poller inherited the poller's blind spot.

Both fixtures derive from the roster now, minus the absent-ok set, with an anti-vacuity guard so an empty derivation cannot make every composing case pass while asserting nothing. One expectation stays literal and gained perf by hand — the partial-set case asserts the whole with no run at all: … list, whose subject is the order and the elision, and deriving that one would compare the roster against itself.

Two more things this turned up

fuzz/Cargo.lock was four releases stale, so the first cargo invocation under fuzz/ rewrote it and verify's tree-clean step failed on a file the run had just dirtied. (Resolved against trunk during the final rebase, which had refreshed it too.)

.claude/rules/toolchain.md said the gate runs "in verify, not as a CI job". True when written, false the moment the job landed — and rules-drift exists for exactly that shape: a claim that is present and wrong, carrying the rule's authority to a reader who then acts without re-deriving it. Corrected in the same branch rather than left for the gate to catch.

Timeout budget

timeout-check takes grandfathered or p95=<n>s x3, and prose is neither. A new job cannot honestly claim the first, so the budget is measured: perf-gate forced down its full path in a worktree carrying a crate-source change — both release binaries built, both arms sampled — is 109s wall on a warm cache. The declared 200s covers that plus checkout, mise install and cache restore, with headroom for a colder runner; ×3 lands it at 10 minutes. One sample is not a p95 and the line says so; re-measure once the job has a history.

Closes CLOUD-172

Refs: CLOUD-207, CLOUD-119, CLOUD-418

@linear-code

linear-code Bot commented Aug 11, 2026 •

Copy link
Copy Markdown
CLOUD-172 Record batten's invocation cost as a series, so hook latency can't degrade invisibly

Batten runs as a hook, so its startup cost is paid on every mediated call. A policy engine slow enough to be annoying gets routed around, and the rule it enforces stops binding. Cost measured once degrades invisibly; a per-commit series makes a regression attributable to a SHA.

Mechanism

  • mise run perf — build release, measure, print.
  • mise run perf:record — same measurement, appended to git notes for the commit. Runs from main only: a branch's numbers are not the trunk's, and a series mixing them cannot be read.
  • mise run perf:compare — measure this branch against its merge-base and exit non-zero if an instruction count rose past the threshold. This is the gate. A recorded series nobody opens changes no behaviour; a failing check on the pull request that caused the regression reaches the person who can still act on it. PR measurements are used for the comparison and discarded — nothing on a branch writes to the notes.
  • Instruction counts as the primary metric; wall-clock variance on shared runners both hides real regressions and invents fake ones. Where valgrind is unavailable, degrade to wall clock and label it as such.
  • History in git notes, attached to the measured commit: no external service, regression attributable without re-running history.
  • Instrument pinned in mise.toml. A change in how it samples enters the series as a step change indistinguishable from a regression.

Blocked on

  • A real invocation path with measurable startup cost (check or hook doing work).
  • Recording trigger: per-commit-on-main implies a push-to-main workflow, and this repo has none by design. Note the two jobs are separable — the PR comparison gate runs on pull_request and needs no such trigger, so only the notes-recording half is affected. Decide whether it is in scope of that rule; record the decision in AGENTS.md beside it.
  • A regression threshold, and where measurement noise on a shared runner puts its floor.

Ready

  • An invocation path worth measuring exists.
  • Recording trigger decided against the no-push-CI rule.

Done

  • mise run perf works with and without valgrind.
  • A PR that regresses past the threshold fails a check that names the regressed measurement.
  • The series exists in git notes, written only from main; a regression is visible without re-running history.
  • Instrument pinned, with the pinning reason stated at the pin.

Decisions taken during implementation

Metric: wall clock, paired — instruction counts are not available. mise registry valgrind reports "tool not found in registry", so callgrind cannot be pinned in mise.toml, and batten.toml's no-source-built-tool rule forbids compiling it. The Mechanism's fallback ("where valgrind is unavailable, degrade to wall clock and label it as such") is therefore the only branch, not a contingency.

What makes wall clock usable is pairing rather than a better clock: perf-pair builds both the merge-base and HEAD binaries and measures them back to back on one machine, so machine-level variance is common-mode and divides out. The verdict is a ratio, never an absolute. perf-record stamps metric=wall-clock into every series entry so a later instruction-count series can never be diffed against this one and read as a regression.

Threshold: 1.30x, derived from a measured null. Identical binary as both arms, 100 runs after 10 warmups, ten repeats across three paths — 30 ratios of a comparison that by construction measures nothing. They spread 0.966 to 1.102. 1.30 clears that measured maximum with room for a busier runner. mise run perf-pair --null re-measures it, and a bats case asserts the threshold still clears the recorded floor, so tightening the constant without re-measuring fails.

Recording trigger: the daily schedule, not a push-to-main workflow. AGENTS.md forbids the latter, and the reason holds rather than merely applying — main advances only by fast-forward to a SHA CI already judged, so a push trigger buys a runner per merge for a measurement a clock can take. The cost is resolution (per-sample, not per-commit), and it is affordable because the series is not the attribution mechanism: perf-gate catches a regression on the PR that caused it, at the commit that caused it.

Where the decision is recorded. The Ready block asks for it "in AGENTS.md beside it". AGENTS.md is at exactly 199 lines, the max_lines = 199 ceiling in [budget.instructions] — there is no line to spend, and the rule the decision honours ("Don't add push-to-main triggers") is already there. It is recorded at the mechanism (perf-record's guard, perf.yml's header) and in .claude/rules/toolchain.md, which is indexed rather than always-loaded.

Namespace: bench -> perf. CLOUD-207 landed mise-tasks/bench and bench-assert; open PR #311 (CLOUD-119) adds mise-tasks/bench and bench-check for token economics. This ticket's own Mechanism names the latency tasks perf, so they were renamed here, freeing bench for #311 before it rebases.

CLOUD-207 Publish and defend the latency number: bench task + CI assertion on the no-op and hook paths

A PreToolUse hook's cost is paid on every agent tool call — the latency number is what decides whether the hook stays installed, and the reference toolchain wins adoption partly by publishing theirs. Batten asserts speed nowhere and measures it nowhere (no bench target, no hyperfine, no CI assertion; CLOUD-74 captures the budget question, CLOUD-172 the cost-series idea — this issue is the mechanism).

Scope: a bench mise task (hyperfine over batten --help, batten check on a small fixture, and — once CLOUD-202 lands — batten hook on a canned envelope); a CI assertion that p95 stays under the budget (<100ms hook path, clig's floor); the measured number published in README the way the fast tools publish theirs. Absorbs CLOUD-74's "measure instead of assert" and feeds CLOUD-172's series.

Refine against the DoR before pulling; the flakiness posture applies to the CI assertion (a shared-runner p95 needs a tolerance band, not a point equality).

CLOUD-119 Build the token-economics benchmark harness

Backs the Batten adoption proof design doc (top priority). No existing ticket.

Why. Adoption turns on a token-economics proof we can defend, not borrow. Batten's deterministic output contract (pointer-not-payload, byte-stable) is a measurable token-reduction mechanism; this harness measures it honestly. The market is full of unmethodical "cheaper" claims — the differentiator is that ours is reproducible.

Scope. A reproducible in-repo benchmark. Per capability, measure cost per completed task (the headline unit) and tokens-per-answer for the raw tool + typical agent handling (tail/re-run/dump) vs. the Batten-wrapped path, priced across cache-read / fresh / output tiers.

Acceptance.

  • Named workloads with baselines: build/test output → warning line; scan → pointer; large log → handle; re-run → cache.
  • Every figure states its workload, baseline, run count, and method; report per-capability; publish an aggregate headline only if the aggregate is itself defensible.
  • Cross-session prefix-cache hit rate measured against a provider that documents caching (hit rate + time-to-first-token).
  • Honesty gate: a capability with no defensible number reports "not measured," never a borrowed figure.
  • Runnable by anyone from the repo.

Sources (public). Cost-of-Pass: An Economic Framework for Evaluating Language Models (arXiv:2504.13359); provider prompt-caching documentation.

Positioning input (CLOUD-211 note): the benchmark proves the token-economics story that supports the completion-gate headline (CLOUD-120 owns the headline); it is not a category proof. The note's evidence-base corrections — borrowed magnitudes that drifted from their sources — are the concrete precedent this issue's honesty gate exists for: every published figure stays quote-exact against its primary.

CLOUD-418 A new gate is never shown to fail, so a test that cannot discriminate ships as coverage

Why

This repository's most-repeated failure is a claim nothing exercises. land's refusal branch was dead code for months (CLOUD-235). timeout-check's budgets were placeholders that could not fire (CLOUD-352). A shape rule whose pattern was a program could never match and read as coverage (CLOUD-401). Each was caught after the fact.

It happened again, live, while building the landing lease (CLOUD-393). A concurrency test was written for a real race — observe() reading FETCH_HEAD, which is one file per clone while the heartbeat runs beside held/release in the same checkout. The test was green. Then the buggy version was restored to check the test could catch it, and it passed on the broken code too: every process fetches the same lease ref, so a crossed read yields a different generation of the same lease rather than an observably foreign one. The test asserted nothing.

That was found only because someone chose to mutate and re-run — a discipline nothing asks for and nothing checks. The green suite before that check and the green suite after it were indistinguishable.

Root cause. The obligation is stated as "a rule ships with a runnable gate" — a gate that exists. Nothing requires evidence the gate discriminates. A test that passes on both the fixed and the broken code satisfies every rule this repo currently has.

Scope, deliberately narrow. Not mutation testing over the workspace, which is a research project and a large CI bill. The claim here is about mise-tasks/*-check and the guards — the files whose entire purpose is to refuse — where the mutation is usually a one-line inversion and the suite is bats, so a run is seconds.

Refinement — Ready

  • Source of truth (§1). The gate's own suite, run against a deliberately broken copy of the gate. A pass there is the defect; the verdict is an exit code, not a judgement.
  • Mechanism (§3). Undecided between two, and choosing is what Ready needs:
    • Author-side, checked in. Each gate declares one or more mutant cases — a stated one-line corruption and the test name that must go red. A task runs them, and a mutant nothing catches fails. Costs the repo one small fixture per gate; runs locally, off the landing path.
    • Scheduled sweep. A weekly job applies mechanical mutations to mise-tasks/*-check and reports any whose suite stays green. No per-gate authoring, weaker coverage, and it belongs beside branch-age-check in the hygiene sweep, so no new CI minutes.
  • Deliberately not in scope (§2). Mutation coverage of crates/. Different tooling, different cost, different question.
  • Output (§7). Pointer-only: the gate, the mutant, and the test that failed to notice. Never a diff of the mutated source.

Test obligation

The mechanism must catch the case that motivated it: the FETCH_HEAD mutation of mise-tasks/land-lock against tests/land-lock.bats as it stood before the structural assertion replaced it. That pair is a known-good fixture — a real gate, a real mutant, and a real suite that missed it.

Commit / bump (§6): feat(gate) — patch until 0.1.0 regardless of type.

Blockers (§8): none.

Acceptance

  • Every *-check task has at least one mutation its suite is proven to catch.
  • A gate whose suite passes on a broken copy fails.
  • The land-lock/FETCH_HEAD pair is covered as a regression fixture, so the case that motivated this cannot recur silently.

Review in Linear

@wenzowski
wenzowski marked this pull request as ready for review August 11, 2026 20:45
@wenzowski
wenzowski force-pushed the wenzowski/cloud-172-record-battens-invocation-cost-as-a-series-so-hook-latency branch 2 times, most recently from 5f5c3b4 to 7877127 Compare August 11, 2026 20:54
@wenzowski

Copy link
Copy Markdown
Contributor Author

/fast-forward

@wenzowski

Copy link
Copy Markdown
Contributor Author

Triggered from #333 (comment) by @​wenzowski.

Trying to fast forward main (5532160) to wenzowski/cloud-172-record-battens-invocation-cost-as-a-series-so-hook-latency (7877127).

Target branch (main):

commit 55321601f0fb1cf8e5d9c5a97129e36b11df62fb (HEAD -> main, origin/main)
Author: Claude <noreply@anthropic.com>
Date:   Tue Aug 11 20:04:52 2026 +0000

    chore(deps): drop dev-dependencies that duplicate the normal ones
    
    `serde_json` was listed in both `[dependencies]` and `[dev-dependencies]`, and
    so were `flate2`, `tar` and `toml` — the identical redundancy, in the same
    table, filed as one bullet because one of the four was noticed. A normal
    dependency is already in scope for every test target, so all four entries
    declared nothing; the comment above them explained a provision-suite need that
    `[dependencies]` had long since covered.
    
    `jsonschema` stays: it is the one crate the tests link and the shipped binary
    does not.
    
    Predicate is the Ready block's — the crate and its test targets still build.
    
    Refs: CLOUD-217

Pull request (wenzowski/cloud-172-record-battens-invocation-cost-as-a-series-so-hook-latency):

commit 78771272504611cf0b1ae9174dd748df98582cd5 (pull_request/wenzowski/cloud-172-record-battens-invocation-cost-as-a-series-so-hook-latency)
Author: Claude <noreply@anthropic.com>
Date:   Tue Aug 11 20:36:37 2026 +0000

    feat(perf): record the invocation cost as a series, and gate the regression
    
    CLOUD-207 published one number. A number measured once decays in
    silence: nobody notices 3ms becoming 40ms, they notice months later
    that the hook got uninstalled.
    
    `perf-record` appends each trunk measurement to git notes on
    `refs/notes/perf`, keyed to the commit, and refuses to run off main —
    a branch's numbers are not the trunk's, and a series mixing them
    cannot be read. `perf-pair` builds the merge-base and HEAD binaries
    and measures them back to back on one machine; `perf-compare` decides
    that ratio; `perf-gate` composes the two and is the one name verify
    and CI call.
    
    WALL CLOCK, PAIRED. The issue specified instruction counts with wall
    clock as a fallback "where valgrind is unavailable" — `mise registry
    valgrind` reports "tool not found in registry", and no-source-built-tool
    forbids compiling one, so the fallback is the only branch. What makes
    it usable is not a better clock but a better experiment: both arms
    measured on the same runner within seconds, so machine noise is
    common-mode and divides out. The verdict is a ratio, never an absolute,
    and every series entry carries `metric=wall-clock` so a later
    instruction-count series cannot be diffed against it as a regression.
    
    THE THRESHOLD IS MEASURED. A null comparison — identical binary as both
    arms, n=30 — spread 0.966 to 1.102. 1.30 clears that measured maximum;
    `perf-pair --null` re-measures it and a bats case fails if the constant
    is ever tightened below the recorded floor.
    
    Recording trigger: the daily schedule, not a push-to-main workflow,
    which AGENTS.md forbids and whose reason holds here. The cost is
    resolution, affordable because the series is not the attribution
    mechanism — perf-gate catches a regression on the PR that caused it.
    
    Verify pays almost nothing: perf-gate exits clean without building when
    the diff touches no crate source, manifest or lockfile, because a
    binary that cannot have changed cannot have got slower.
    
    Renames bench/bench-assert into the perf namespace the issue specifies,
    freeing `bench` for #311's token-economics harness.
    
    Refs: CLOUD-172

Can't fast forward main (5532160) to wenzowski/cloud-172-record-battens-invocation-cost-as-a-series-so-hook-latency (7877127). main (5532160) is not a direct ancestor of wenzowski/cloud-172-record-battens-invocation-cost-as-a-series-so-hook-latency (7877127). Branches appear to have diverged at 6947232:

* 78771272504611cf0b1ae9174dd748df98582cd5 feat(perf): record the invocation cost as a series, and gate the regression
| * 55321601f0fb1cf8e5d9c5a97129e36b11df62fb chore(deps): drop dev-dependencies that duplicate the normal ones
|/  
* 6947232b13bc17174986a0e3bfe528d848617519 perf(hk): glob batten-check and macos-link-check, and gate the glob's honesty

commit 6947232b13bc17174986a0e3bfe528d848617519
Author: Claude <noreply@anthropic.com>
Date:   Tue Aug 11 19:57:59 2026 +0000

    perf(hk): glob batten-check and macos-link-check, and gate the glob's honesty
    
    `hk check` ran three steps unconditionally, because a step with no `glob` is
    never skipped. `batten-check` is `cargo run -p batten -- check`, so every
    commit paid a debug build whatever it touched: 9.47s cold from a fresh clone
    (CLOUD-212 run 1, D8) against 0.16s for the cheapest gate in the same run, and
    133-141ms warm, forever.
    
    The glob is the union of what `batten check` ACTUALLY reads, which is not the
    two things the obvious reading gives. Read at run_check_with in
    crates/batten/src/lib.rs, it evaluates every `[[rule]]` over that rule's own
    glob — and those reach outside crates/, to mise.toml, .github/workflows/*.yml
    and tests/*.bats — plus `budget::measure_all`, because a declared budget is a
    gate under `check` and not only under `policy budget` (CLOUD-50). That last one
    makes AGENTS.md and .serena/project.yml genuine inputs, so "a Markdown-only
    change cannot change this verdict" is false for the most-edited Markdown file
    in the repo, and a glob written from the obvious reading would have switched
    the gate off for it. Cargo.toml and Cargo.lock are in because the task uses
    `cargo run` deliberately, to judge the engine and the config as the pair that
    ships.
    
    `macos-link-check` globs the manifests alone: its predicate is `cargo metadata
    --filter-platform` over the resolved graph, and nothing else can move that
    graph. `no-docs-tree` deliberately keeps no glob — its input is the whole
    index, including a docs/ path an earlier commit left behind, and at ~95ms that
    is affordable.
    
    Deriving batten-check's list from batten.toml by hand makes hk.pkl a second
    authority over a set the config already defines, and a second authority narrows
    silently: add a rule whose glob names a new path and the step stops running for
    commits that touch only it, with nothing going red. So the glob ships with its
    mechanism (non-negotiable 2). `mise run batten-glob-check` is a pure function
    of the two committed files — every rule glob and every declared budget path
    must be covered verbatim or subsumed by a `P/**` entry, and a config it can
    parse nothing out of is exit 2 rather than a vacuous pass.
    
    Selection is asserted as data, never as a timing: `hk check --plan --json`
    prints each step's status without running it, and tests/hk-selection.bats pins
    both directions over the repository's own hk.pkl — README.md skips
    batten-check, AGENTS.md does not.
    
    Refs: CLOUD-224

Rebase locally, and then force push to wenzowski/cloud-172-record-battens-invocation-cost-as-a-series-so-hook-latency.

@wenzowski
wenzowski force-pushed the wenzowski/cloud-172-record-battens-invocation-cost-as-a-series-so-hook-latency branch from 7877127 to 5766e2d Compare August 11, 2026 21:02
@wenzowski

Copy link
Copy Markdown
Contributor Author

/fast-forward

@wenzowski

Copy link
Copy Markdown
Contributor Author

Triggered from #333 (comment) by @​wenzowski.

Trying to fast forward main (b642e2d) to wenzowski/cloud-172-record-battens-invocation-cost-as-a-series-so-hook-latency (5766e2d).

Target branch (main):

commit b642e2dd35304790923f2a63bbbf0f9f68c9b4c2 (HEAD -> main, origin/main)
Author: Claude <noreply@anthropic.com>
Date:   Tue Aug 11 20:47:33 2026 +0000

    perf(ci): run the shell suite in parallel, since it is the pole of the gate
    
    `hk` runs two chains concurrently, and the shell one was the longer by 133s:
    on run 31531728503 the cargo chain's last step finished 166s into `mise run
    ci`, while `test:bats` ran to 247.01s of the step's 298.87s. 800 tests, one at
    a time. `verify` runs the same gate, so a `land` lap paid it twice.
    
    That put it in the exponent of the lap economics. A lap dies whenever `main`
    moves under it, so with mean inter-land gap 487s the expected CI runs per
    landed PR is e^(L/G) — which is why shortening the lap is the one lever that
    cuts latency and the bill together rather than trading one for the other.
    
    Measured here: 248s serial -> 72.73s, 808 tests, same assertions, nothing
    skipped or sampled. The remaining pole is the cargo chain at 166s.
    
    `rush` rather than GNU parallel, which bats would otherwise look for: GNU
    parallel ships as a source tarball with no aqua/ubi/release coordinate, so it
    cannot be a [tools] key at all, and an `apt-get install` in ci.yml is the
    second-authority drift `ci-tools-check` exists to refuse. rush is prebuilt per
    platform, so `mise.lock` holds a checksum for every platform this repo
    installs on.
    
    Across files, never within one. The suites that model lock and race behaviour
    are precisely the ones a within-file schedule would interleave with their own
    setup, and across-files is already enough — the largest file is 49 of 808
    tests, so the schedule is not tail-bound.
    
    Two things ship with it:
    
    - `tests/attribution-check.bats` built its end-to-end fixture in the real
      worktree and the real index. Serially that was invisible; in parallel it is
      a race against the sibling case that reads the same tracked set, plus
      .git/index.lock contention. It now builds a throwaway repo, which is the
      convention issue-guard.bats already set.
    - A speed-up is the kind of fix that rots silently — drop the flag and
      everything still passes, just slower, and `timeout-check`'s 3.4x headroom
      absorbs a return to serial without a red build. So the invocation is now a
      gate over mise.toml: serial is an exit code, not a slower Tuesday.
    
    Refs: CLOUD-386

Pull request (wenzowski/cloud-172-record-battens-invocation-cost-as-a-series-so-hook-latency):

commit 5766e2df1aca1812c5e19a1a9257a58bbd3d81cd (pull_request/wenzowski/cloud-172-record-battens-invocation-cost-as-a-series-so-hook-latency)
Author: Claude <noreply@anthropic.com>
Date:   Tue Aug 11 20:36:37 2026 +0000

    feat(perf): record the invocation cost as a series, and gate the regression
    
    CLOUD-207 published one number. A number measured once decays in
    silence: nobody notices 3ms becoming 40ms, they notice months later
    that the hook got uninstalled.
    
    `perf-record` appends each trunk measurement to git notes on
    `refs/notes/perf`, keyed to the commit, and refuses to run off main —
    a branch's numbers are not the trunk's, and a series mixing them
    cannot be read. `perf-pair` builds the merge-base and HEAD binaries
    and measures them back to back on one machine; `perf-compare` decides
    that ratio; `perf-gate` composes the two and is the one name verify
    and CI call.
    
    WALL CLOCK, PAIRED. The issue specified instruction counts with wall
    clock as a fallback "where valgrind is unavailable" — `mise registry
    valgrind` reports "tool not found in registry", and no-source-built-tool
    forbids compiling one, so the fallback is the only branch. What makes
    it usable is not a better clock but a better experiment: both arms
    measured on the same runner within seconds, so machine noise is
    common-mode and divides out. The verdict is a ratio, never an absolute,
    and every series entry carries `metric=wall-clock` so a later
    instruction-count series cannot be diffed against it as a regression.
    
    THE THRESHOLD IS MEASURED. A null comparison — identical binary as both
    arms, n=30 — spread 0.966 to 1.102. 1.30 clears that measured maximum;
    `perf-pair --null` re-measures it and a bats case fails if the constant
    is ever tightened below the recorded floor.
    
    Recording trigger: the daily schedule, not a push-to-main workflow,
    which AGENTS.md forbids and whose reason holds here. The cost is
    resolution, affordable because the series is not the attribution
    mechanism — perf-gate catches a regression on the PR that caused it.
    
    Verify pays almost nothing: perf-gate exits clean without building when
    the diff touches no crate source, manifest or lockfile, because a
    binary that cannot have changed cannot have got slower.
    
    Renames bench/bench-assert into the perf namespace the issue specifies,
    freeing `bench` for #311's token-economics harness.
    
    Refs: CLOUD-172

Can't fast forward main (b642e2d) to wenzowski/cloud-172-record-battens-invocation-cost-as-a-series-so-hook-latency (5766e2d). main (b642e2d) is not a direct ancestor of wenzowski/cloud-172-record-battens-invocation-cost-as-a-series-so-hook-latency (5766e2d). Branches appear to have diverged at 5532160:

* 5766e2df1aca1812c5e19a1a9257a58bbd3d81cd feat(perf): record the invocation cost as a series, and gate the regression
| * b642e2dd35304790923f2a63bbbf0f9f68c9b4c2 perf(ci): run the shell suite in parallel, since it is the pole of the gate
|/  
* 55321601f0fb1cf8e5d9c5a97129e36b11df62fb chore(deps): drop dev-dependencies that duplicate the normal ones

commit 55321601f0fb1cf8e5d9c5a97129e36b11df62fb
Author: Claude <noreply@anthropic.com>
Date:   Tue Aug 11 20:04:52 2026 +0000

    chore(deps): drop dev-dependencies that duplicate the normal ones
    
    `serde_json` was listed in both `[dependencies]` and `[dev-dependencies]`, and
    so were `flate2`, `tar` and `toml` — the identical redundancy, in the same
    table, filed as one bullet because one of the four was noticed. A normal
    dependency is already in scope for every test target, so all four entries
    declared nothing; the comment above them explained a provision-suite need that
    `[dependencies]` had long since covered.
    
    `jsonschema` stays: it is the one crate the tests link and the shipped binary
    does not.
    
    Predicate is the Ready block's — the crate and its test targets still build.
    
    Refs: CLOUD-217

Rebase locally, and then force push to wenzowski/cloud-172-record-battens-invocation-cost-as-a-series-so-hook-latency.

@wenzowski
wenzowski force-pushed the wenzowski/cloud-172-record-battens-invocation-cost-as-a-series-so-hook-latency branch from 5766e2d to e10072e Compare August 11, 2026 21:08
@wenzowski

Copy link
Copy Markdown
Contributor Author

/fast-forward

@wenzowski

Copy link
Copy Markdown
Contributor Author

Triggered from #333 (comment) by @​wenzowski.

Trying to fast forward main (98aceb0) to wenzowski/cloud-172-record-battens-invocation-cost-as-a-series-so-hook-latency (e10072e).

Target branch (main):

commit 98aceb05393d0fdfc91d977603ae486cf34e1b7f (HEAD -> main, origin/main)
Author: Claude <noreply@anthropic.com>
Date:   Tue Aug 11 19:14:30 2026 +0000

    feat(bench): measure the token economics instead of asserting them
    
    The adoption claim is token economics, and the differentiator is that ours is
    reproducible: anyone can say they are cheaper, we ship the harness that produces
    the number from committed inputs with no credential and no network.
    
    `mise run token-bench` executes what `bench/tokens/workloads.toml` declares and
    prices it with what `bench/tokens/method.toml` declares — no price, divisor,
    re-run coefficient or workload is baked into the program, because a constant
    written into a benchmark's code is a number nobody re-derives. Every arm runs
    three times and is compared byte-for-byte; an arm that is not byte-stable is
    reported as such rather than averaged, since byte-stability is the property the
    cache claim rests on.
    
    `mise run token-bench-check` is the mechanism that makes "reproducible"
    checkable: it regenerates the table into scratch and diffs it byte-for-byte, and
    it re-reads the published bytes for the honesty predicate — a figure without
    workload, baseline, run count and method is refused, and so is a capability
    reporting "not measured" with no reason. The gate re-reads the artifact rather
    than trusting the generator, which is what a program checking its own arithmetic
    would not do.
    
    Two capabilities carry a measured figure; `log-handle` reports "not measured"
    because the capability is not shipped, and the cross-session cache hit rate
    reports "not measured" because it needs a live provider credential and is a
    property of the world rather than of this commit. Neither borrows a magnitude
    from its neighbour. No aggregate is published: averaging across capabilities
    needs a workload mix nobody here has measured.
    
    `build-warning` is the honest one — Batten wins it by 1.47x, and the saving is
    exactly the discarded `tail` window rather than the log, which `exec` tees in
    full by design.
    
    Named `token-bench` rather than `bench`: CLOUD-207's `mise run bench` measures
    batten's invocation cost and landed under that name first. Two benchmarks with
    two subjects need two names, so the later and more specific one moves, and
    `bench/tokens/` leaves room for the sibling's own inputs beside it.
    
    Refs: CLOUD-119

Pull request (wenzowski/cloud-172-record-battens-invocation-cost-as-a-series-so-hook-latency):

commit e10072e2331d69f65d22b066ad270178e56e7e48 (pull_request/wenzowski/cloud-172-record-battens-invocation-cost-as-a-series-so-hook-latency)
Author: Claude <noreply@anthropic.com>
Date:   Tue Aug 11 20:36:37 2026 +0000

    feat(perf): record the invocation cost as a series, and gate the regression
    
    CLOUD-207 published one number. A number measured once decays in
    silence: nobody notices 3ms becoming 40ms, they notice months later
    that the hook got uninstalled.
    
    `perf-record` appends each trunk measurement to git notes on
    `refs/notes/perf`, keyed to the commit, and refuses to run off main —
    a branch's numbers are not the trunk's, and a series mixing them
    cannot be read. `perf-pair` builds the merge-base and HEAD binaries
    and measures them back to back on one machine; `perf-compare` decides
    that ratio; `perf-gate` composes the two and is the one name verify
    and CI call.
    
    WALL CLOCK, PAIRED. The issue specified instruction counts with wall
    clock as a fallback "where valgrind is unavailable" — `mise registry
    valgrind` reports "tool not found in registry", and no-source-built-tool
    forbids compiling one, so the fallback is the only branch. What makes
    it usable is not a better clock but a better experiment: both arms
    measured on the same runner within seconds, so machine noise is
    common-mode and divides out. The verdict is a ratio, never an absolute,
    and every series entry carries `metric=wall-clock` so a later
    instruction-count series cannot be diffed against it as a regression.
    
    THE THRESHOLD IS MEASURED. A null comparison — identical binary as both
    arms, n=30 — spread 0.966 to 1.102. 1.30 clears that measured maximum;
    `perf-pair --null` re-measures it and a bats case fails if the constant
    is ever tightened below the recorded floor.
    
    Recording trigger: the daily schedule, not a push-to-main workflow,
    which AGENTS.md forbids and whose reason holds here. The cost is
    resolution, affordable because the series is not the attribution
    mechanism — perf-gate catches a regression on the PR that caused it.
    
    Verify pays almost nothing: perf-gate exits clean without building when
    the diff touches no crate source, manifest or lockfile, because a
    binary that cannot have changed cannot have got slower.
    
    Renames bench/bench-assert into the perf namespace the issue specifies,
    freeing `bench` for #311's token-economics harness.
    
    Refs: CLOUD-172

Can't fast forward main (98aceb0) to wenzowski/cloud-172-record-battens-invocation-cost-as-a-series-so-hook-latency (e10072e). main (98aceb0) is not a direct ancestor of wenzowski/cloud-172-record-battens-invocation-cost-as-a-series-so-hook-latency (e10072e). Branches appear to have diverged at b642e2d:

* e10072e2331d69f65d22b066ad270178e56e7e48 feat(perf): record the invocation cost as a series, and gate the regression
| * 98aceb05393d0fdfc91d977603ae486cf34e1b7f feat(bench): measure the token economics instead of asserting them
|/  
* b642e2dd35304790923f2a63bbbf0f9f68c9b4c2 perf(ci): run the shell suite in parallel, since it is the pole of the gate

commit b642e2dd35304790923f2a63bbbf0f9f68c9b4c2
Author: Claude <noreply@anthropic.com>
Date:   Tue Aug 11 20:47:33 2026 +0000

    perf(ci): run the shell suite in parallel, since it is the pole of the gate
    
    `hk` runs two chains concurrently, and the shell one was the longer by 133s:
    on run 31531728503 the cargo chain's last step finished 166s into `mise run
    ci`, while `test:bats` ran to 247.01s of the step's 298.87s. 800 tests, one at
    a time. `verify` runs the same gate, so a `land` lap paid it twice.
    
    That put it in the exponent of the lap economics. A lap dies whenever `main`
    moves under it, so with mean inter-land gap 487s the expected CI runs per
    landed PR is e^(L/G) — which is why shortening the lap is the one lever that
    cuts latency and the bill together rather than trading one for the other.
    
    Measured here: 248s serial -> 72.73s, 808 tests, same assertions, nothing
    skipped or sampled. The remaining pole is the cargo chain at 166s.
    
    `rush` rather than GNU parallel, which bats would otherwise look for: GNU
    parallel ships as a source tarball with no aqua/ubi/release coordinate, so it
    cannot be a [tools] key at all, and an `apt-get install` in ci.yml is the
    second-authority drift `ci-tools-check` exists to refuse. rush is prebuilt per
    platform, so `mise.lock` holds a checksum for every platform this repo
    installs on.
    
    Across files, never within one. The suites that model lock and race behaviour
    are precisely the ones a within-file schedule would interleave with their own
    setup, and across-files is already enough — the largest file is 49 of 808
    tests, so the schedule is not tail-bound.
    
    Two things ship with it:
    
    - `tests/attribution-check.bats` built its end-to-end fixture in the real
      worktree and the real index. Serially that was invisible; in parallel it is
      a race against the sibling case that reads the same tracked set, plus
      .git/index.lock contention. It now builds a throwaway repo, which is the
      convention issue-guard.bats already set.
    - A speed-up is the kind of fix that rots silently — drop the flag and
      everything still passes, just slower, and `timeout-check`'s 3.4x headroom
      absorbs a return to serial without a red build. So the invocation is now a
      gate over mise.toml: serial is an exit code, not a slower Tuesday.
    
    Refs: CLOUD-386

Rebase locally, and then force push to wenzowski/cloud-172-record-battens-invocation-cost-as-a-series-so-hook-latency.

@wenzowski
wenzowski force-pushed the wenzowski/cloud-172-record-battens-invocation-cost-as-a-series-so-hook-latency branch from e10072e to 22913a4 Compare August 11, 2026 21:13
@wenzowski

Copy link
Copy Markdown
Contributor Author

/fast-forward

@wenzowski

Copy link
Copy Markdown
Contributor Author

Triggered from #333 (comment) by @​wenzowski.

Trying to fast forward main (87ed397) to wenzowski/cloud-172-record-battens-invocation-cost-as-a-series-so-hook-latency (22913a4).

Target branch (main):

commit 87ed39733221612abd4f99ce64c5f93e773439d4 (HEAD -> main, origin/main)
Author: Claude <noreply@anthropic.com>
Date:   Tue Aug 11 18:56:29 2026 +0000

    fix(verify): refuse to receipt a HEAD whose bytes were not the ones validated
    
    `verify` runs every gate over the WORKING TREE and then writes a receipt keyed
    to HEAD. Nothing asserted the two agree, so a dirty tree could attest a commit
    nobody verified: the receipt is written, `verified` matches it, `ready-guard`
    accepts it, and CI runs a commit that was never the thing that passed.
    
    Measured 2026-08-09 landing CLOUD-269 — a backgrounded `land` compiled a
    mid-edit snapshot and reported `verify failed on c5bfe05`, a commit containing
    none of that code. That direction is loud and self-correcting; the mirror is
    silent, and it is the one this closes.
    
    `mise-tasks/tree-clean` is the predicate: `git --no-optional-locks status
    --porcelain --untracked-files=normal`, exit 1 with a count and the porcelain
    lines as pointers, exit 2 when it cannot look. Untracked-but-not-ignored counts
    as dirty — decided, not omitted — because `cargo test` auto-discovers
    `crates/batten/tests/*.rs` and the bats suite globs `tests/*.bats`, so a new
    untracked file is run by `verify` with zero tracked-file change. Ignored paths
    are excluded structurally.
    
    Wired into `[tasks.verify]` at both ends of the window: in `depends` so a dirty
    tree fails in seconds rather than after ~170s, and in the body — the
    load-bearing one, since `depends` completes before a tree can be dirtied
    mid-run. Not an hk step: `pre-commit` runs over a tree that is dirty by
    definition. `land` inherits it through `verify` and re-implements nothing.
    
    Refs: CLOUD-277

Pull request (wenzowski/cloud-172-record-battens-invocation-cost-as-a-series-so-hook-latency):

commit 22913a4fc97ed5d2cc9b83e22fd7fe24b4f86ed8 (pull_request/wenzowski/cloud-172-record-battens-invocation-cost-as-a-series-so-hook-latency)
Author: Claude <noreply@anthropic.com>
Date:   Tue Aug 11 20:36:37 2026 +0000

    feat(perf): record the invocation cost as a series, and gate the regression
    
    CLOUD-207 published one number. A number measured once decays in
    silence: nobody notices 3ms becoming 40ms, they notice months later
    that the hook got uninstalled.
    
    `perf-record` appends each trunk measurement to git notes on
    `refs/notes/perf`, keyed to the commit, and refuses to run off main —
    a branch's numbers are not the trunk's, and a series mixing them
    cannot be read. `perf-pair` builds the merge-base and HEAD binaries
    and measures them back to back on one machine; `perf-compare` decides
    that ratio; `perf-gate` composes the two and is the one name verify
    and CI call.
    
    WALL CLOCK, PAIRED. The issue specified instruction counts with wall
    clock as a fallback "where valgrind is unavailable" — `mise registry
    valgrind` reports "tool not found in registry", and no-source-built-tool
    forbids compiling one, so the fallback is the only branch. What makes
    it usable is not a better clock but a better experiment: both arms
    measured on the same runner within seconds, so machine noise is
    common-mode and divides out. The verdict is a ratio, never an absolute,
    and every series entry carries `metric=wall-clock` so a later
    instruction-count series cannot be diffed against it as a regression.
    
    THE THRESHOLD IS MEASURED. A null comparison — identical binary as both
    arms, n=30 — spread 0.966 to 1.102. 1.30 clears that measured maximum;
    `perf-pair --null` re-measures it and a bats case fails if the constant
    is ever tightened below the recorded floor.
    
    Recording trigger: the daily schedule, not a push-to-main workflow,
    which AGENTS.md forbids and whose reason holds here. The cost is
    resolution, affordable because the series is not the attribution
    mechanism — perf-gate catches a regression on the PR that caused it.
    
    Verify pays almost nothing: perf-gate exits clean without building when
    the diff touches no crate source, manifest or lockfile, because a
    binary that cannot have changed cannot have got slower.
    
    Renames bench/bench-assert into the perf namespace the issue specifies,
    freeing `bench` for #311's token-economics harness.
    
    Refs: CLOUD-172

Can't fast forward main (87ed397) to wenzowski/cloud-172-record-battens-invocation-cost-as-a-series-so-hook-latency (22913a4). main (87ed397) is not a direct ancestor of wenzowski/cloud-172-record-battens-invocation-cost-as-a-series-so-hook-latency (22913a4). Branches appear to have diverged at 98aceb0:

* 22913a4fc97ed5d2cc9b83e22fd7fe24b4f86ed8 feat(perf): record the invocation cost as a series, and gate the regression
| * 87ed39733221612abd4f99ce64c5f93e773439d4 fix(verify): refuse to receipt a HEAD whose bytes were not the ones validated
|/  
* 98aceb05393d0fdfc91d977603ae486cf34e1b7f feat(bench): measure the token economics instead of asserting them

commit 98aceb05393d0fdfc91d977603ae486cf34e1b7f
Author: Claude <noreply@anthropic.com>
Date:   Tue Aug 11 19:14:30 2026 +0000

    feat(bench): measure the token economics instead of asserting them
    
    The adoption claim is token economics, and the differentiator is that ours is
    reproducible: anyone can say they are cheaper, we ship the harness that produces
    the number from committed inputs with no credential and no network.
    
    `mise run token-bench` executes what `bench/tokens/workloads.toml` declares and
    prices it with what `bench/tokens/method.toml` declares — no price, divisor,
    re-run coefficient or workload is baked into the program, because a constant
    written into a benchmark's code is a number nobody re-derives. Every arm runs
    three times and is compared byte-for-byte; an arm that is not byte-stable is
    reported as such rather than averaged, since byte-stability is the property the
    cache claim rests on.
    
    `mise run token-bench-check` is the mechanism that makes "reproducible"
    checkable: it regenerates the table into scratch and diffs it byte-for-byte, and
    it re-reads the published bytes for the honesty predicate — a figure without
    workload, baseline, run count and method is refused, and so is a capability
    reporting "not measured" with no reason. The gate re-reads the artifact rather
    than trusting the generator, which is what a program checking its own arithmetic
    would not do.
    
    Two capabilities carry a measured figure; `log-handle` reports "not measured"
    because the capability is not shipped, and the cross-session cache hit rate
    reports "not measured" because it needs a live provider credential and is a
    property of the world rather than of this commit. Neither borrows a magnitude
    from its neighbour. No aggregate is published: averaging across capabilities
    needs a workload mix nobody here has measured.
    
    `build-warning` is the honest one — Batten wins it by 1.47x, and the saving is
    exactly the discarded `tail` window rather than the log, which `exec` tees in
    full by design.
    
    Named `token-bench` rather than `bench`: CLOUD-207's `mise run bench` measures
    batten's invocation cost and landed under that name first. Two benchmarks with
    two subjects need two names, so the later and more specific one moves, and
    `bench/tokens/` leaves room for the sibling's own inputs beside it.
    
    Refs: CLOUD-119

Rebase locally, and then force push to wenzowski/cloud-172-record-battens-invocation-cost-as-a-series-so-hook-latency.

@wenzowski
wenzowski force-pushed the wenzowski/cloud-172-record-battens-invocation-cost-as-a-series-so-hook-latency branch from 22913a4 to 5df9003 Compare August 11, 2026 21:19
@wenzowski

Copy link
Copy Markdown
Contributor Author

/fast-forward

@wenzowski

Copy link
Copy Markdown
Contributor Author

Triggered from #333 (comment) by @​wenzowski.

Trying to fast forward main (42319f4) to wenzowski/cloud-172-record-battens-invocation-cost-as-a-series-so-hook-latency (5df9003).

Target branch (main):

commit 42319f43bfdff5cdb654394d2504d1ac173cb024 (HEAD -> main, origin/main)
Author: Claude <noreply@anthropic.com>
Date:   Tue Aug 11 20:44:36 2026 +0000

    ci(stop-posture): one flagging-verb set across the worth/bears openers
    
    `HEDGES` gave its two openers different verb sets — `worth
    (noting|flagging)` beside `bears (noting|mentioning)` — so `mentioning`
    was a flagging verb to this file under one opener and unknown under the
    other. Nothing distinguished them; the split was an accident of how the
    alternation was written.
    
    Measured, one sentence per row: `worth naming` and `worth mentioning`
    were silent while `bears mentioning` fired. The first of those is a
    witnessed miss. The CLOUD-347..356 audit closed its report with "One
    open thread worth naming: the census … never interrogated host
    settings" — a real finding that reached chat and nothing else, and
    became CLOUD-380 only because a human asked. Nothing else covered it
    either: `stop-guard` consults `finding-sink-check` precisely when this
    rule is silent, and that gate needs a `path:line` citation the sentence
    does not carry, so the turn's one advisory slot was never claimed.
    
    One shared `FLAG_VERB` set now, every entry either witnessed (`naming`)
    or already in the file (`noting`, `flagging`, `mentioning`). `calling
    out` stays out and is asserted to stay out: unwitnessed and not already
    present, which is the line between completing an inflection and
    inventing a phrase list. The `I would flag` / `I should note` family
    keeps `(flag|note)` — "I would name that" is not natural flagging.
    
    Not the unmeasured-literal mistake CLOUD-323 and CLOUD-326 forbid:
    those govern a new SHAPE, while this is inflection completion inside a
    construction already measured at 3/3, which this gate's own header
    sanctions and its own history already did once.
    
    Refs: CLOUD-387

Pull request (wenzowski/cloud-172-record-battens-invocation-cost-as-a-series-so-hook-latency):

commit 5df900392c9d9336ed15bf7de869f71bb3566db3 (pull_request/wenzowski/cloud-172-record-battens-invocation-cost-as-a-series-so-hook-latency)
Author: Claude <noreply@anthropic.com>
Date:   Tue Aug 11 20:36:37 2026 +0000

    feat(perf): record the invocation cost as a series, and gate the regression
    
    CLOUD-207 published one number. A number measured once decays in
    silence: nobody notices 3ms becoming 40ms, they notice months later
    that the hook got uninstalled.
    
    `perf-record` appends each trunk measurement to git notes on
    `refs/notes/perf`, keyed to the commit, and refuses to run off main —
    a branch's numbers are not the trunk's, and a series mixing them
    cannot be read. `perf-pair` builds the merge-base and HEAD binaries
    and measures them back to back on one machine; `perf-compare` decides
    that ratio; `perf-gate` composes the two and is the one name verify
    and CI call.
    
    WALL CLOCK, PAIRED. The issue specified instruction counts with wall
    clock as a fallback "where valgrind is unavailable" — `mise registry
    valgrind` reports "tool not found in registry", and no-source-built-tool
    forbids compiling one, so the fallback is the only branch. What makes
    it usable is not a better clock but a better experiment: both arms
    measured on the same runner within seconds, so machine noise is
    common-mode and divides out. The verdict is a ratio, never an absolute,
    and every series entry carries `metric=wall-clock` so a later
    instruction-count series cannot be diffed against it as a regression.
    
    THE THRESHOLD IS MEASURED. A null comparison — identical binary as both
    arms, n=30 — spread 0.966 to 1.102. 1.30 clears that measured maximum;
    `perf-pair --null` re-measures it and a bats case fails if the constant
    is ever tightened below the recorded floor.
    
    Recording trigger: the daily schedule, not a push-to-main workflow,
    which AGENTS.md forbids and whose reason holds here. The cost is
    resolution, affordable because the series is not the attribution
    mechanism — perf-gate catches a regression on the PR that caused it.
    
    Verify pays almost nothing: perf-gate exits clean without building when
    the diff touches no crate source, manifest or lockfile, because a
    binary that cannot have changed cannot have got slower.
    
    Renames bench/bench-assert into the perf namespace the issue specifies,
    freeing `bench` for #311's token-economics harness.
    
    Refs: CLOUD-172

Can't fast forward main (42319f4) to wenzowski/cloud-172-record-battens-invocation-cost-as-a-series-so-hook-latency (5df9003). main (42319f4) is not a direct ancestor of wenzowski/cloud-172-record-battens-invocation-cost-as-a-series-so-hook-latency (5df9003). Branches appear to have diverged at 87ed397:

* 5df900392c9d9336ed15bf7de869f71bb3566db3 feat(perf): record the invocation cost as a series, and gate the regression
| * 42319f43bfdff5cdb654394d2504d1ac173cb024 ci(stop-posture): one flagging-verb set across the worth/bears openers
|/  
* 87ed39733221612abd4f99ce64c5f93e773439d4 fix(verify): refuse to receipt a HEAD whose bytes were not the ones validated

commit 87ed39733221612abd4f99ce64c5f93e773439d4
Author: Claude <noreply@anthropic.com>
Date:   Tue Aug 11 18:56:29 2026 +0000

    fix(verify): refuse to receipt a HEAD whose bytes were not the ones validated
    
    `verify` runs every gate over the WORKING TREE and then writes a receipt keyed
    to HEAD. Nothing asserted the two agree, so a dirty tree could attest a commit
    nobody verified: the receipt is written, `verified` matches it, `ready-guard`
    accepts it, and CI runs a commit that was never the thing that passed.
    
    Measured 2026-08-09 landing CLOUD-269 — a backgrounded `land` compiled a
    mid-edit snapshot and reported `verify failed on c5bfe05`, a commit containing
    none of that code. That direction is loud and self-correcting; the mirror is
    silent, and it is the one this closes.
    
    `mise-tasks/tree-clean` is the predicate: `git --no-optional-locks status
    --porcelain --untracked-files=normal`, exit 1 with a count and the porcelain
    lines as pointers, exit 2 when it cannot look. Untracked-but-not-ignored counts
    as dirty — decided, not omitted — because `cargo test` auto-discovers
    `crates/batten/tests/*.rs` and the bats suite globs `tests/*.bats`, so a new
    untracked file is run by `verify` with zero tracked-file change. Ignored paths
    are excluded structurally.
    
    Wired into `[tasks.verify]` at both ends of the window: in `depends` so a dirty
    tree fails in seconds rather than after ~170s, and in the body — the
    load-bearing one, since `depends` completes before a tree can be dirtied
    mid-run. Not an hk step: `pre-commit` runs over a tree that is dirty by
    definition. `land` inherits it through `verify` and re-implements nothing.
    
    Refs: CLOUD-277

Rebase locally, and then force push to wenzowski/cloud-172-record-battens-invocation-cost-as-a-series-so-hook-latency.

@wenzowski
wenzowski force-pushed the wenzowski/cloud-172-record-battens-invocation-cost-as-a-series-so-hook-latency branch 2 times, most recently from 42953b6 to cd775a3 Compare August 11, 2026 21:30
@wenzowski

Copy link
Copy Markdown
Contributor Author

/fast-forward

@wenzowski

Copy link
Copy Markdown
Contributor Author

Triggered from #333 (comment) by @​wenzowski.

Trying to fast forward main (ac7da09) to wenzowski/cloud-172-record-battens-invocation-cost-as-a-series-so-hook-latency (cd775a3).

Target branch (main):

commit ac7da09517068209d6d27197e908d2ef617762ef (HEAD -> main, origin/main)
Author: Claude <noreply@anthropic.com>
Date:   Tue Aug 11 19:02:37 2026 +0000

    docs(agents): price the metered model call, and index every memory the repo ships
    
    The workflow contract opened with a two-row price list — "Every CI run costs
    real minutes; your own execution costs nothing" — and a token-consuming model
    call is in neither row. It is not local and it is not free, so an agent reading
    only the always-loaded surface correctly concluded its own fan-out was free.
    
    Three categories now: local execution, free; a CI run, metered; a token-consuming
    model call, metered in the same category, a subagent spawn above all. The
    derivation stays out of the always-loaded surface and the clause points at
    `mem:workflow/agent-fanout` for it — which is the memory that had no trigger
    anywhere, along with `core` and `memory_maintenance`. All three get an index row,
    so the routing table now covers every memory the repo ships.
    
    Both were paid for out of reclaimed lines rather than appended: the surface sits
    at its 199-line target with zero headroom, so `land`'s step, the governing
    paragraph and two background paragraphs were repacked and their *why* clauses
    dropped where `.claude/rules/toolchain.md` already carries them. 199 lines before
    and after, 28 insertions against 28 deletions.
    
    Refs: CLOUD-288

Pull request (wenzowski/cloud-172-record-battens-invocation-cost-as-a-series-so-hook-latency):

commit cd775a319b56dc8d152ab6760d323abf2d0e6fab (pull_request/wenzowski/cloud-172-record-battens-invocation-cost-as-a-series-so-hook-latency)
Author: Claude <noreply@anthropic.com>
Date:   Tue Aug 11 20:36:37 2026 +0000

    feat(perf): record the invocation cost as a series, and gate the regression
    
    CLOUD-207 published one number. A number measured once decays in
    silence: nobody notices 3ms becoming 40ms, they notice months later
    that the hook got uninstalled.
    
    `perf-record` appends each trunk measurement to git notes on
    `refs/notes/perf`, keyed to the commit, and refuses to run off main —
    a branch's numbers are not the trunk's, and a series mixing them
    cannot be read. `perf-pair` builds the merge-base and HEAD binaries
    and measures them back to back on one machine; `perf-compare` decides
    that ratio; `perf-gate` composes the two and is the one name verify
    and CI call.
    
    WALL CLOCK, PAIRED. The issue specified instruction counts with wall
    clock as a fallback "where valgrind is unavailable" — `mise registry
    valgrind` reports "tool not found in registry", and no-source-built-tool
    forbids compiling one, so the fallback is the only branch. What makes
    it usable is not a better clock but a better experiment: both arms
    measured on the same runner within seconds, so machine noise is
    common-mode and divides out. The verdict is a ratio, never an absolute,
    and every series entry carries `metric=wall-clock` so a later
    instruction-count series cannot be diffed against it as a regression.
    
    THE THRESHOLD IS MEASURED. A null comparison — identical binary as both
    arms, n=30 — spread 0.966 to 1.102. 1.30 clears that measured maximum;
    `perf-pair --null` re-measures it and a bats case fails if the constant
    is ever tightened below the recorded floor.
    
    Recording trigger: the daily schedule, not a push-to-main workflow,
    which AGENTS.md forbids and whose reason holds here. The cost is
    resolution, affordable because the series is not the attribution
    mechanism — perf-gate catches a regression on the PR that caused it.
    
    Verify pays almost nothing: perf-gate exits clean without building when
    the diff touches no crate source, manifest or lockfile, because a
    binary that cannot have changed cannot have got slower.
    
    Renames bench/bench-assert into the perf namespace the issue specifies,
    freeing `bench` for #311's token-economics harness.
    
    Refs: CLOUD-172

Can't fast forward main (ac7da09) to wenzowski/cloud-172-record-battens-invocation-cost-as-a-series-so-hook-latency (cd775a3). main (ac7da09) is not a direct ancestor of wenzowski/cloud-172-record-battens-invocation-cost-as-a-series-so-hook-latency (cd775a3). Branches appear to have diverged at b1669d6:

* cd775a319b56dc8d152ab6760d323abf2d0e6fab feat(perf): record the invocation cost as a series, and gate the regression
| * ac7da09517068209d6d27197e908d2ef617762ef docs(agents): price the metered model call, and index every memory the repo ships
|/  
* b1669d6f503804d7102754c992931a9c31359d6d docs(readme): lead with the completion gate, not the policy-engine category

commit b1669d6f503804d7102754c992931a9c31359d6d
Author: Claude <noreply@anthropic.com>
Date:   Tue Aug 11 19:17:41 2026 +0000

    docs(readme): lead with the completion gate, not the policy-engine category
    
    "Policy engine" as the lead noun claims membership in a category whose
    repo-level population measures near-zero, and it invites an expressiveness
    benchmark against authorization engines Batten does not implement and must not
    grow. The category is conformance checking: the subject is always the
    repository's own state, and the question is always whether observed state
    conforms to the committed model of how work lands here. A principal/action/
    resource request shape has no field for "is this ref on `main`" or "did the
    required checks conclude green for this exact SHA".
    
    So the lead becomes the gate itself, hook-first, with CI and pre-commit demoted
    to what they actually are — bypass defense over the same predicate, not two
    more adoption surfaces. "Permission hooks" becomes "conformance gates" for the
    same reason: the hook's allow/deny channel is the only authorization-shaped
    part, and the verdict inside it is a conformance verdict.
    
    Every "done" claim now carries its bound. Landed-plus-green is the minimum
    falsifiable completion predicate: it kills false success, it does not certify
    correctness, and review still gates release. Stated because the evidence
    behind it demonstrates that merge alone is not a verification event — it does
    not establish that the compound predicate certifies anything more.
    
    The token-economics story moves in under the headline as the supporting
    narrative it is, naming the three pains it addresses (tail-calling,
    lost-in-the-middle, context rot) against the two contract properties that
    answer them — pointer-not-payload and byte-stable output — plus the refusal
    that points at the fix. Magnitude is deliberately absent: the benchmark
    carries it, per workload with a stated baseline and run count, and a
    capability with no defensible number reports "not measured" rather than
    borrowing one.
    
    The crate keywords go with the description: "policy" as a discovery term
    recruits exactly the category readers this change is spending effort to stop
    attracting.
    
    Untouched on purpose: the exit-code table and the whole extension-surfaces
    section, both asserted against the compiled binary by tests that would fail if
    a rewrite clipped them.
    
    Refs: CLOUD-120

Rebase locally, and then force push to wenzowski/cloud-172-record-battens-invocation-cost-as-a-series-so-hook-latency.

@wenzowski
wenzowski force-pushed the wenzowski/cloud-172-record-battens-invocation-cost-as-a-series-so-hook-latency branch from cd775a3 to 2f16b27 Compare August 11, 2026 21:36
@wenzowski

Copy link
Copy Markdown
Contributor Author

/fast-forward

@wenzowski

Copy link
Copy Markdown
Contributor Author

Triggered from #333 (comment) by @​wenzowski.

Trying to fast forward main (2d3acc3) to wenzowski/cloud-172-record-battens-invocation-cost-as-a-series-so-hook-latency (2f16b27).

Target branch (main):

commit 2d3acc3416b5d9ed3ca9ec5c3280194b11ef757f (HEAD -> main, origin/main)
Author: Claude <noreply@anthropic.com>
Date:   Tue Aug 11 18:52:11 2026 +0000

    ci(graph-check): make a frontier omission attributable, and say "could not look"
    
    `graph-check` is the scheduler as well as the gate, and its header claims every
    agent computing the frontier independently gets the same answer. It did not:
    `:92` ran ready-lint as `| "$lint" >/dev/null 2>&1 || continue`, so exit 1 ("this
    Ready block is wrong") and exit 2 ("stdin is not a payload with a .description
    field") arrived as the same silent absence — one missing `frontier` line and
    nothing else, exit 0 both times over a 25-payload closure.
    
    The same shape reached the verdict: `:40` required only id and status, so a set
    whose relations the caller projected away was reported an acyclic, non-dangling
    board with no edges in it at all.
    
    Three channels now, not one. `report()` keeps exit 1 for a board signalling
    falsely; `unjudged()` takes exit 2 for anything the gate could not read; `note()`
    attributes an honest exclusion without moving the exit code, because a board full
    of unrefined issues is not an incoherent board. ready-lint's rule ids are
    forwarded rather than re-derived, and its ::error:: summary dropped — annotating
    a failure at an exit code that denies one is the false signal this repo catches.
    
    The missing-blockedBy report is keyed to the `graph` pseudo-id, not to each
    issue: `released`'s `refusal_for` greps this stderr for `^<id> <rule>`, and a
    per-id line would turn every In Review issue in a relations-free sweep into a
    REFUSED. An explicit `blockedBy: []` is data and is judged; only an absent key
    is unjudgeable.
    
    Exit 2 outranks exit 1 when both hold, with both report sets printed: a verdict
    over a set the gate could only partly read is not a verdict, and the caller's
    next action is a re-fetch.
    
    Refs: CLOUD-251

Pull request (wenzowski/cloud-172-record-battens-invocation-cost-as-a-series-so-hook-latency):

commit 2f16b275d2100232e515d0fdfbb81f15f1f6c010 (pull_request/wenzowski/cloud-172-record-battens-invocation-cost-as-a-series-so-hook-latency)
Author: Claude <noreply@anthropic.com>
Date:   Tue Aug 11 21:35:17 2026 +0000

    docs(fanout): record that a bundle serializes on land, inside one session
    
    The fan-out memory reasoned about land contention BETWEEN sessions and
    said nothing about within one. A session handed a bundle meets the same
    constraint as strict serialization for a mechanical reason: `land`
    drives the working tree, so ticket N+1 cannot start while N's land is in
    flight.
    
    Measured on this bundle — each land ran 5+ laps at roughly 5-7 minutes,
    so wall clock was dominated by laps the session could only wait out. The
    unblock is a worktree per ticket, and `batten worktree` already models
    the question.
    
    Refs: CLOUD-172

Can't fast forward main (2d3acc3) to wenzowski/cloud-172-record-battens-invocation-cost-as-a-series-so-hook-latency (2f16b27). main (2d3acc3) is not a direct ancestor of wenzowski/cloud-172-record-battens-invocation-cost-as-a-series-so-hook-latency (2f16b27). Branches appear to have diverged at ac7da09:

* 2f16b275d2100232e515d0fdfbb81f15f1f6c010 docs(fanout): record that a bundle serializes on land, inside one session
* eeb04963d014cf21d05b250879aa7bccc8fe7280 feat(perf): record the invocation cost as a series, and gate the regression
| * 2d3acc3416b5d9ed3ca9ec5c3280194b11ef757f ci(graph-check): make a frontier omission attributable, and say "could not look"
|/  
* ac7da09517068209d6d27197e908d2ef617762ef docs(agents): price the metered model call, and index every memory the repo ships

commit ac7da09517068209d6d27197e908d2ef617762ef
Author: Claude <noreply@anthropic.com>
Date:   Tue Aug 11 19:02:37 2026 +0000

    docs(agents): price the metered model call, and index every memory the repo ships
    
    The workflow contract opened with a two-row price list — "Every CI run costs
    real minutes; your own execution costs nothing" — and a token-consuming model
    call is in neither row. It is not local and it is not free, so an agent reading
    only the always-loaded surface correctly concluded its own fan-out was free.
    
    Three categories now: local execution, free; a CI run, metered; a token-consuming
    model call, metered in the same category, a subagent spawn above all. The
    derivation stays out of the always-loaded surface and the clause points at
    `mem:workflow/agent-fanout` for it — which is the memory that had no trigger
    anywhere, along with `core` and `memory_maintenance`. All three get an index row,
    so the routing table now covers every memory the repo ships.
    
    Both were paid for out of reclaimed lines rather than appended: the surface sits
    at its 199-line target with zero headroom, so `land`'s step, the governing
    paragraph and two background paragraphs were repacked and their *why* clauses
    dropped where `.claude/rules/toolchain.md` already carries them. 199 lines before
    and after, 28 insertions against 28 deletions.
    
    Refs: CLOUD-288

Rebase locally, and then force push to wenzowski/cloud-172-record-battens-invocation-cost-as-a-series-so-hook-latency.

@wenzowski
wenzowski force-pushed the wenzowski/cloud-172-record-battens-invocation-cost-as-a-series-so-hook-latency branch from 2f16b27 to 4ec18ac Compare August 11, 2026 21:42
@wenzowski

Copy link
Copy Markdown
Contributor Author

/fast-forward

@wenzowski

Copy link
Copy Markdown
Contributor Author

Triggered from #333 (comment) by @​wenzowski.

Trying to fast forward main (eee9bb4) to wenzowski/cloud-172-record-battens-invocation-cost-as-a-series-so-hook-latency (4ec18ac).

Target branch (main):

commit eee9bb4bade0799923aff7d983911a169743d839 (HEAD -> main, origin/main)
Author: Claude <noreply@anthropic.com>
Date:   Tue Aug 11 21:10:41 2026 +0000

    feat(config): gate the predecessor consumer's repo name
    
    CLOUD-73 asks what stays in the consumer's repo and what moves into Batten. Its
    structural half is settled by history — this repository was scaffolded fresh, so
    none of the domain modules it names (entity grep, pdfium, CA privacy detectors,
    evidence and provenance semantics, the ICM design protocol) has ever been here,
    and there is no `tools/batten/` to keep them out of. Its other half is
    non-negotiable rule 1, and that one had a hole.
    
    CLOUD-26's acceptance named four literals. Two are rows; the comment beside them
    dropped the other two on the ground that `forbid` was a case-sensitive literal
    substring check, so a literal with an English reading was wrong in both
    directions at once. CLOUD-283 gave `forbid` a `regex` alternative, which retires
    that premise — so the two exclusions are re-decided on the merits rather than
    inherited from a limit that no longer exists.
    
    The predecessor consumer's repo name goes in, as `(?i)complian`: it is squarely
    what rule 1 forbids, `(?i)` catches the capitalised form the old literal would
    have missed, and it costs nothing — zero lines under crates/ carry it, and this
    engine's vocabulary is policy / gate / predicate / verdict.
    
    The vendor name stays out, and the comment now says that is a decision rather
    than a gap: it names the vendor, not a consumer, `github.com/button-inc/batten`
    is Batten's own home, and the consumer-facing half is already gated by
    `no-origin-literal-in-fixtures`.
    
    The fixture test spells the new shape with a capital, which is precisely the
    half a literal could not have caught.
    
    Refs: CLOUD-73

Pull request (wenzowski/cloud-172-record-battens-invocation-cost-as-a-series-so-hook-latency):

commit 4ec18ac76112329a4b9408ef858c01f6ab5ac073 (pull_request/wenzowski/cloud-172-record-battens-invocation-cost-as-a-series-so-hook-latency)
Author: Claude <noreply@anthropic.com>
Date:   Tue Aug 11 21:35:17 2026 +0000

    docs(fanout): record that a bundle serializes on land, inside one session
    
    The fan-out memory reasoned about land contention BETWEEN sessions and
    said nothing about within one. A session handed a bundle meets the same
    constraint as strict serialization for a mechanical reason: `land`
    drives the working tree, so ticket N+1 cannot start while N's land is in
    flight.
    
    Measured on this bundle — each land ran 5+ laps at roughly 5-7 minutes,
    so wall clock was dominated by laps the session could only wait out. The
    unblock is a worktree per ticket, and `batten worktree` already models
    the question.
    
    Refs: CLOUD-172

Can't fast forward main (eee9bb4) to wenzowski/cloud-172-record-battens-invocation-cost-as-a-series-so-hook-latency (4ec18ac). main (eee9bb4) is not a direct ancestor of wenzowski/cloud-172-record-battens-invocation-cost-as-a-series-so-hook-latency (4ec18ac). Branches appear to have diverged at 2d3acc3:

* 4ec18ac76112329a4b9408ef858c01f6ab5ac073 docs(fanout): record that a bundle serializes on land, inside one session
* d1b933c3c59db10d7c278e4e72cf0cb443b80bcf feat(perf): record the invocation cost as a series, and gate the regression
| * eee9bb4bade0799923aff7d983911a169743d839 feat(config): gate the predecessor consumer's repo name
|/  
* 2d3acc3416b5d9ed3ca9ec5c3280194b11ef757f ci(graph-check): make a frontier omission attributable, and say "could not look"

commit 2d3acc3416b5d9ed3ca9ec5c3280194b11ef757f
Author: Claude <noreply@anthropic.com>
Date:   Tue Aug 11 18:52:11 2026 +0000

    ci(graph-check): make a frontier omission attributable, and say "could not look"
    
    `graph-check` is the scheduler as well as the gate, and its header claims every
    agent computing the frontier independently gets the same answer. It did not:
    `:92` ran ready-lint as `| "$lint" >/dev/null 2>&1 || continue`, so exit 1 ("this
    Ready block is wrong") and exit 2 ("stdin is not a payload with a .description
    field") arrived as the same silent absence — one missing `frontier` line and
    nothing else, exit 0 both times over a 25-payload closure.
    
    The same shape reached the verdict: `:40` required only id and status, so a set
    whose relations the caller projected away was reported an acyclic, non-dangling
    board with no edges in it at all.
    
    Three channels now, not one. `report()` keeps exit 1 for a board signalling
    falsely; `unjudged()` takes exit 2 for anything the gate could not read; `note()`
    attributes an honest exclusion without moving the exit code, because a board full
    of unrefined issues is not an incoherent board. ready-lint's rule ids are
    forwarded rather than re-derived, and its ::error:: summary dropped — annotating
    a failure at an exit code that denies one is the false signal this repo catches.
    
    The missing-blockedBy report is keyed to the `graph` pseudo-id, not to each
    issue: `released`'s `refusal_for` greps this stderr for `^<id> <rule>`, and a
    per-id line would turn every In Review issue in a relations-free sweep into a
    REFUSED. An explicit `blockedBy: []` is data and is judged; only an absent key
    is unjudgeable.
    
    Exit 2 outranks exit 1 when both hold, with both report sets printed: a verdict
    over a set the gate could only partly read is not a verdict, and the caller's
    next action is a re-fetch.
    
    Refs: CLOUD-251

Rebase locally, and then force push to wenzowski/cloud-172-record-battens-invocation-cost-as-a-series-so-hook-latency.

@wenzowski
wenzowski force-pushed the wenzowski/cloud-172-record-battens-invocation-cost-as-a-series-so-hook-latency branch from 4ec18ac to 553c89c Compare August 11, 2026 21:47
@wenzowski

Copy link
Copy Markdown
Contributor Author

/fast-forward

@wenzowski
wenzowski force-pushed the wenzowski/cloud-172-record-battens-invocation-cost-as-a-series-so-hook-latency branch 2 times, most recently from dd2dd08 to 7b75370 Compare August 13, 2026 07:26
@wenzowski
wenzowski marked this pull request as draft August 13, 2026 07:26
@wenzowski
wenzowski force-pushed the wenzowski/cloud-172-record-battens-invocation-cost-as-a-series-so-hook-latency branch from 7b75370 to edfa3bd Compare August 13, 2026 07:26
@sonarqubecloud

Copy link
Copy Markdown

❌ The last analysis has failed.

See analysis details on SonarQube Cloud

@wenzowski
wenzowski marked this pull request as ready for review August 13, 2026 07:28
@wenzowski
wenzowski force-pushed the wenzowski/cloud-172-record-battens-invocation-cost-as-a-series-so-hook-latency branch from edfa3bd to 351ec6f Compare August 13, 2026 07:28
@wenzowski
wenzowski marked this pull request as draft August 13, 2026 07:34
@wenzowski
wenzowski force-pushed the wenzowski/cloud-172-record-battens-invocation-cost-as-a-series-so-hook-latency branch 5 times, most recently from 83a8115 to e8fad29 Compare August 14, 2026 01:47
The `forbid` kind bans literals named in advance, and a credential is the banned
shape nobody can enumerate -- so a committed secret passed every gate this engine
had. Detection is adopted prior art (a pinned scanner, run as a child process);
what this adds is CONTAINMENT, because the scanner prints the byte it matched and
Batten copies what it sees into channels that retain it.

Not a `command` rule pointed at the same binary, and the difference is the whole
kind: a command rule's contract is the exit code alone with both child streams
nulled, so it yields one batch verdict per glob -- no per-secret path:line, no
per-secret identity, nothing to key. Reading the scanner's output is exactly what
makes this kind expressible and exactly what makes it dangerous.

Two controls, neither of which anyone has to remember. Every match is wrapped
into `identity::SecretSpan` at the parse boundary, and that type has no route
back to `&str`, so the unkeyed `code_fingerprint` cannot be handed one -- a
compile error rather than a recall obligation. And `Finding` has no field a span
could occupy, so pointer-only output is structural in text and `-J` alike rather
than a property of the renderer. `settling_check` returns `Reevaluate` and
deliberately not an `Argv`: an argv here would put the scanner's invocation on
the record, and re-running the scan is the honest answer anyway.

THE PARSER ANCHORS ON THE PATHS IT PASSED IN, never on separator position.
`path:line:span` reads like a `split(':')` problem and is not, twice over: a path
may contain a colon and so may a secret (`user:password@host` is a credential
shape the scanner matches). Splitting there does not fail -- it silently keys a
TRUNCATED span, a wrong answer wearing a right answer's clothes that no
colon-free fixture would catch. Longest-match, because `a.rs` is a prefix of
`a.rs.bak` and a first-match parser would credit the wrong file. A line naming a
path the batch never passed is a parse failure, not a guess.

`--only-matching` is pinned beside the parser rather than offered as an option:
it is what makes the third field the matched literal alone. The span has to be
the literal because `secret_code_fingerprint` forces `Verbatim` normalization,
and a whole-line span would fold every unrelated edit on that line into the
secret's identity.

Fail-closed, all exit 3, all pointer-only -- an unparseable line names its
ordinal and never its bytes, which matters most here because the line that failed
to parse is a line the scanner emitted BECAUSE it found a secret in it. The
exit-status/parse-count cross-check refuses both directions: a clean exit with
matches parsed would report secrets the tool says it did not find, and a found
exit with nothing parsed would report a clean tree the tool says is not clean --
the second being the silent false green the clause exists to prevent. Any other
code, and any signal, is neither verdict.

A missing scanner is exit 1 naming `batten provision apply`, never a silent pass:
a scanner that did not run is not evidence of a clean tree. The provision entry
name is a constant rather than a rule column -- the kind permits no new column,
and a `tool = "..."` key would let a config point the secret kind at an arbitrary
binary and read its output as secret spans, which is a wider capability than
"scan for credentials" and a worse one to hand a config file.

The provision manifest is threaded through the runner because the adapter has to
resolve a pinned binary and `run_rule` could not see one. `run_static` takes the
slice and ignores it: it refuses every spawning kind before any work, so the one
kind that would read it never reaches the dispatch.

THE CACHE ROOT IS CANONICALIZED, NOT RESOLVED THROUGH GIT, and the difference is
a hermeticity bug caught by the corpus fixture. The engine's anchor is a RELATIVE
path (`.`) whenever the config sits in the cwd, and `state::derive_repo_name`
cannot name `.` -- so the first version failed with "cannot derive a repository
name" instead of the missing-scanner refusal. Reaching for `git::repo_root`
instead would have been worse than wrong: fixtures materialize under `target/`,
inside THIS repository, so a fixture asserting "no scanner installed" would have
resolved to the real cache and started passing or failing on whatever the
developer happened to have warmed.

The corpus fixture is the uninstalled case on purpose. The happy path needs a
stub binary in a cache the corpus harness cannot populate -- it materializes a
tree and runs argv, nothing more -- and the cold-cache posture is the one that
matters most on a fresh machine anyway: a scanner that did not run must never
read as a clean tree. The happy path lands beside it in a suite that stands up
its own XDG_DATA_HOME.

BREAKING CHANGE: `rules::run_static` and `rules::run_all` take the provision
manifest as a second parameter, surfaced by `mise run semver` as
`function_parameter_count_changed`. The adapter has to resolve a PINNED binary and
`run_rule` could not see one, so the manifest had to reach it; the alternative was
a `tool = "..."` rule column, which would let any config point the secrets kind at
an arbitrary binary and have its stdout parsed as secret spans -- a materially
wider capability than the kind is for. `run_static` takes the slice and ignores
it, since it refuses every spawning kind before any work and the one kind that
reads it therefore never reaches the dispatch.

Refs: CLOUD-59
…ed binary

The claim this kind rests on is a NEGATIVE -- that no surface Batten writes can
carry a matched byte -- and a negative needs a witness that would be found if it
were false. Every case plants a synthetic token and then searches for it in each
place the run could have put it: stdout, stderr, the `-J` document, and every
file under the state root, which is where the key and the provision cache live.

THE SUITE'S FIRST RUN FAILED ON ITS OWN FIXTURE, and the fix is the interesting
part. Five cases reported "the token reached a file under the state root" --
correctly: `provision` caches the exact artifact bytes by contract, and the stub
scanner carried the token as a shell literal, so installing it wrote the token
under the state root.

The tempting repair was to exclude the provision cache from the search. That
would have disabled the assertion in the one place it matters most: the state
root is where Batten writes files nobody reads, so a leak there is invisible
indefinitely, and narrowing the sweep to make a test pass removes exactly the
coverage the test exists for. The stub now reconstructs the token from shell
fragments instead, so no artifact holds the contiguous string and the sweep stays
whole.

The planted tokens are assembled from fragments at runtime for the same family of
reason: consumer #1's `no-secrets` rule globs the tree, so a committed literal
credential in this file would be a standing violation of the rule this suite
exists to prove -- the repository failing its own gate, forever, on its own test
fixture.

The scanner is a stub provisioned through `file://`, because what is under test
is the ADAPTER rather than ripsecrets' detection. It emits the output shape
measured against 0.1.11, which is what makes the disagreement branches drivable
at all: a real scanner cannot be asked to contradict its own exit status.
Detection is adopted prior art and consumer #1's own gate exercises it once the
pin lands.

Proved rather than asserted: exit 2 with a pointer-only stdout and the token
absent from all four surfaces; the emitted identity secret-class (the version tag
is what shows the KEYED path was taken) and unequal to `code_fingerprint` of the
same span; two repositories minting two keys and two identities for one secret,
which is the machine-scoped trade observed rather than described; byte-identical
output across two runs, ordered by path rather than by emission order; a cold
cache as exit 1 naming `batten provision apply`; and all four fail-closed
branches as exit 3.

Two cases exist to keep the rest honest. A clean scan still exits 0 with no
output -- without it every assertion above would pass on an engine that always
errored. And a glob matching nothing is driven with a stub that would FAIL the
run if executed, so "no match means no spawn" is proved by the run succeeding
rather than by trusting the code path.

Refs: CLOUD-59
`clippy::format_collect` over `shell_token`. A plain loop pushing the quotes and
the fragment says the same thing without allocating per part, and the helper's
whole job is to keep the contiguous token out of the artifact -- which a loop
does as well as a chain.

Also records the semver lesson this bundle measured twice, in
`mem:toolchain-and-hooks`: `semver` is verify-time, so a break declared late has
to be walked back onto the commit that MAKES it, and declaring it on whichever
commit happens to be HEAD satisfies the gate while leaving a `test(...)!` in
history claiming an API break it did not make. Both instances were authored by
someone who knew the change was breaking -- once with the reason written in the
commit body -- so knowing and marking are independent acts, and the second is the
one that gets skipped.

Recorded rather than mechanised, with the reason: a commit-time semver check
needs a baseline rustdoc build per commit, which is the cost `verify` exists to
amortise.

Refs: CLOUD-59
@wenzowski
wenzowski force-pushed the wenzowski/cloud-172-record-battens-invocation-cost-as-a-series-so-hook-latency branch from e8fad29 to 146ddc1 Compare August 14, 2026 01:59
…ression

CLOUD-207 published one number and holds it to an absolute ceiling. A
number measured once still decays invisibly, and a ceiling cannot see a
branch that got slower while staying under it.

`bench-record` appends each trunk measurement to git notes on
`refs/notes/bench`, keyed to the commit, and refuses to run off main —
a branch's numbers are not the trunk's, and a series mixing them cannot
be read. `bench-pair` builds this branch's binary AND its merge base's
and measures them back to back on one machine; `bench-compare` decides
that ratio; `bench-gate` composes the two.

WALL CLOCK, PAIRED. The issue specified instruction counts with wall
clock as a fallback "where valgrind is unavailable" — `mise registry
valgrind` reports "tool not found in registry", and no-source-built-tool
forbids compiling one, so the fallback is the only branch. What makes it
usable is not a better clock but a better experiment: both arms on the
same runner seconds apart, so machine noise is common-mode and divides
out. The verdict is a ratio, never an absolute, and every series entry
carries `metric=wall-clock` so a later instruction-count series cannot
be diffed against it as a regression.

THE THRESHOLD IS MEASURED. A null comparison — identical binary as both
arms, n=30 — spread 0.966 to 1.102. 1.30 clears that measured maximum;
`bench-pair --null` re-measures it and a bats case fails if the constant
is ever tightened below the recorded floor.

IN VERIFY, NOT AS A CI JOB. `ready-guard` refuses `gh pr ready` without
a verify receipt for this exact HEAD, so a regression stops the PR
before it can spend a matrix — earlier and cheaper than a check that
reds after the runner is paid for. It costs the common change nothing:
`bench-gate` exits clean without building when the diff against the
merge base touches no crate source, manifest or lockfile.

Recording runs on the existing daily schedule rather than a push-to-main
trigger, which AGENTS.md forbids and whose reason holds here.

Refs: CLOUD-172
…heckout

The base arm is a binary built from the merge base, and both arms ran in
the repo root — so the older binary was handed HEAD's committed
batten.toml. A head that adds a config key the base binary does not know
makes that binary exit 1 at load, hyperfine abort on its first warmup,
and the whole gate answer 2.

Measured: a head carrying a `[worktree]` table produced "unknown field
`worktree`" from the base arm and took bench-gate down — a
could-not-look manufactured by the gate's own setup, on exactly the
class of change the gate exists to judge.

Every arm now runs in the materialised fixture, whose config is pinned
and loadable by both binaries. That also makes the number reproducible:
measuring against this repository's live config moved the baseline
whenever the config changed.

tests/bench-pair.bats pins the fix as a static property, because
reproducing it needs two real binaries built an hour apart.

Refs: CLOUD-172
…gate it in CI

Two corrections to what this branch shipped, both against CLOUD-172's own
Mechanism clause rather than against taste.

THE NAMESPACE. The issue specifies `perf`, `perf:record` and `perf:compare`, and
this branch's own PR body claimed the rename had happened. It had not: the tasks
landed as `bench-compare`, `bench-gate`, `bench-pair` and `bench-record` beside
CLOUD-207's `bench` and `bench-assert`, and the workflow as `bench.yml`. The
whole latency family moves here — six tasks, four suites, the workflow, the job,
the concurrency group and `refs/notes/bench` -> `refs/notes/perf`. No notes ref
exists on the remote yet, so the series orphans nothing.

Hyphens, not the colons the issue writes: `mise-tasks/` holds file tasks, where a
colon is a path separator. `perf-record` is the same name in the convention this
repository actually uses.

`token-bench`, `token-bench-check` and `bench/tokens/**` are deliberately NOT
renamed. They are CLOUD-119's token-economics benchmark, a different subject with
a name of its own, and freeing the bare `bench` for it is the reason the issue
gives for this move.

THE GATE RAN NOWHERE THAT DECIDES A PULL REQUEST. `perf-gate` was a step in
`[tasks.verify]` only. `[tasks.ci]` is `depends = ["hooks", "deny"]`, so no
runner ever invoked it: the Done clause "a PR that regresses past the threshold
fails a check that names the regressed measurement" was unmet, because `verify`
is what an author runs locally and a regression would have reached `main` past a
green required check. A draft-gated `perf` job now runs it, seated in `final`'s
`needs:` and named in `CI_REQUIRED_CHECKS`, with `fetch-depth: 0` because
`perf-pair` builds both arms and a shallow clone has no merge base.

It stays affordable for the reason the gate's own header argues: no crate source,
manifest or lockfile in the diff means no build, and most of this repository's
traffic pays a `git diff --name-only`. The recording half stays on the schedule —
`perf-record` refuses to run off `main`, and a push-to-`main` trigger is
forbidden.

AND THE ROSTER HAS A SECOND READER THAT HAND-COPIED IT. `tests/ci-wait.bats`
reads the real $CI_REQUIRED_CHECKS deliberately, but built its green fixture from
a hand-written list of the names -- so adding `perf` left the fixture answering
green for every name it knew while `ci-wait` waited for the one it did not. The
suite HUNG rather than failed, twice, for the better part of an hour each time:
a poll with no answer is precisely what that task is built to do, and it cannot
tell "the roster grew" from "the run is still going". `green_body` now derives
its rows from the roster minus the absent-ok set, with an anti-vacuity guard, so
the copy that drifted no longer exists.

BREAKING CHANGE: `mise run bench`, `bench-assert`, `bench-compare`, `bench-gate`,
`bench-pair` and `bench-record` no longer exist; every one is `perf`-prefixed.
The series ref moves from `refs/notes/bench` to `refs/notes/perf`. Task names are
not a Rust API, so release-plz reads this as a patch either way, but a caller
with the old name in a script gets "task not found" and the honest type is what
puts that in the changelog.

Refs: CLOUD-172
`timeout-check` takes two forms and prose is neither: `grandfathered` for a value
that predates measurement, or `p95=<n>s x3` where the declared minutes equal
ceil(p95 x 3 / 60). A new job cannot honestly claim the first — nothing here
predates anything — so it gets the second, from a measurement rather than a
guess.

`mise run perf-gate` was forced down its full path, in a worktree carrying a
crate-source change so the skip could not fire: both release binaries built, both
arms sampled, 109s wall on a warm cache. The declared 200s covers that plus the
job's own preamble — checkout, mise install, cache restore — and headroom for a
colder runner, which lands the budget at 10 minutes.

Stated at the line rather than implied, because one sample is not a p95 and the
next reader deserves to know which it is. Re-measure once the job has a run
history.

Refs: CLOUD-172
…econd copy

`mandatory_green` wrote out the roster by name, under a header already calling it
"$CI_REQUIRED_CHECKS minus $CI_ABSENT_OK_CHECKS" — a second copy of a list the
suite deliberately reads from mise.toml so it could never disagree with the one
landing uses. Adding `perf` made it disagree: six cases failed at once, each on a
branch they were not written to exercise, because a reading missing a required
name answers "no run at all" and exit 3 before it reaches the bucket under test.

The same hand-listing in tests/ci-wait.bats did worse and is fixed in the commit
that introduced `perf`: there the missing name HUNG the suite rather than failing
it, twice, because polling until every required check is green cannot tell a
roster that grew from a run still going.

Derived, both fixtures are correct for whatever the roster says today, with an
anti-vacuity guard so an empty derivation cannot make every composing case pass
while asserting nothing.

One expectation stays literal and gains `perf` by hand: the partial-set case
asserts the whole "with no run at all: …" list, whose subject is the ORDER and
the elision of tolerated names. Deriving that one would compare the roster with
itself and assert nothing about the message. It fails with a readable diff when
the roster moves, which is the sensor working — the failure mode a silently
short fixture does not have.

Refs: CLOUD-172
… did not

The rule claimed "in `verify`, not as a CI job" and named the reason — a regression
caught before the PR spends a matrix. That half is still true and still the point.
What was wrong is the exclusion: `perf-gate` is now also the `perf` job in ci.yml,
because `verify` is what an author runs and CLOUD-172's Done clause asks for a
CHECK. `[tasks.ci]` is `depends = ["hooks", "deny"]`, so the gate ran on no runner
at all for its whole life.

`rules-drift` exists for exactly this shape: a claim that is present and WRONG,
carrying the rule's authority to a reader who then acts without re-deriving it.
Landing the job without this edit would have left one.

Refs: CLOUD-172
@wenzowski
wenzowski marked this pull request as ready for review August 14, 2026 02:10
@wenzowski
wenzowski force-pushed the wenzowski/cloud-172-record-battens-invocation-cost-as-a-series-so-hook-latency branch from 146ddc1 to ad982c3 Compare August 14, 2026 02:10
@sonarqubecloud

Copy link
Copy Markdown

@wenzowski

Copy link
Copy Markdown
Contributor Author

/fast-forward

@wenzowski
wenzowski merged commit ad982c3 into main Aug 14, 2026
19 checks passed
@wenzowski
wenzowski deleted the wenzowski/cloud-172-record-battens-invocation-cost-as-a-series-so-hook-latency branch August 14, 2026 02:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant