Repository navigation
Conversation
… that lets it CLOUD-398 was closed Done on PR #606, which landed slice 2 only — the four pure-Rust jobs made absent on a diff they cannot judge. Slice 1, moving the shell suite off the shared runner, was fully refined and never implemented, and three of that issue's acceptance clauses are unmet verbatim. The arithmetic that made it look marginal has moved by 6x. CLOUD-398 costed the split on an 857s gate with a 697s suite and concluded ~-167s. Measured on run 33244045030: the `ci` job is 1983s of a 1992s run — the entire critical path, with `perf` finishing at 9.2 min and idling for 24 more — `mise run ci` is 1832s of that, and `test:bats` ~83% of that again. CLOUD-386 swept the worker count at the runner's real width and found every direction worse, because each point redistributes workers inside one saturated 2-core box; its own point D measured the suite alone with the box to itself at 623s against 829s contended. A second runner is the experiment that sweep could not contain, and it turns the gate's wall clock from a sum of two chains into a max. `HK_SKIP_STEPS` rather than a profile or a second task, and each alternative fails for its own reason. A profile cannot express it: hk resolves multiple profiles on a step with AND and the CLI value supplants the config list, so carving one step out needs the whole lane partition CLOUD-398 costed at ~160s of duplication and explicitly sequenced after this. A second mise task would put a name in the workflow that `verify` does not run, which is `ci-local-parity` property 3 — so `test:bats` is named in `verify:gated`'s depends instead, beside the `ci`/`ci:quick` pair that has overlapped there for as long as the quick tier has existed. THE FALSE GREEN THIS SHIPS ITS OWN GATE FOR. `hook-profile-check` reads `hk check --all --plan` and asserts every slow-tagged step is included there; it cannot see what a workflow job passes, which is why it reads the git hook's invocation line for its other half. Measured: `HK_SKIP_STEPS=test:bats` takes that plan from 58 steps to 57 with reason `env_exclude`, while the gate's own reading stays green — so the suite could be carved out of the `ci` job and run by no job at all, with every gate in this repository green over a suite that never executed. `ci-suite-lane` refuses exactly that direction. The opposite mistake, a name matching no step, runs the step in both jobs and needs no gate because it has a symptom. A module rather than a shell gate because `shell-retirement` governs `mise-tasks/**` and `tests/**/*.bats` on the added side as well as the edited one, so the engine is the only route a new predicate has. `bats-invocation` had to move with it. Its `install_args` clause asserted `rush` was installed in the `ci` job, which was correct exactly while that job ran the suite; after the move it would have been green over a `bats` job with no rush — the same false green one layer over. The job is derived from which one invokes the suite now, and the fixture carries the invocation as well as the list, because one with only the list could no longer reach the assertion it exists for. Refs: CLOUD-1140
CLOUD-1140 CLOUD-398's slice 1 never landed: the shell suite is still 83% of a 30-minute `ci` job it shares a runner with, and the split is now worth 6x what it was costed at
Why CLOUD-398 was closed Done on PR #606, which landed slice 2 only — the four pure-Rust jobs made absent on a diff they cannot judge. Slice 1, moving the shell suite off the shared runner, was fully refined and never implemented. Three of that issue's acceptance clauses are unmet verbatim:
This row is that slice. It is filed rather than reopening CLOUD-398, which is released. **The arithmetic that made it marginal has moved by 6x. **CLOUD-398 costed the split on an 857s gate with a 697s suite and concluded ~-167s, -20% — real but thin against a metered runner. Measured 2026-08-29 on run
**Why the worker knob cannot deliver this instead. **CLOUD-386 swept it at the runner's real width and every direction is worse ( What CLOUD-398 got right and this must not re-derive. Its correction of 2026-08-21 stands: the lane partition is an optimisation of the split, not its enabling mechanism. The suite does not need a lane — it needs to not be in the other job. **The false green this must close, and it is the reason this is not a two-line change. ** Refinement — Ready (move the suite to its own job, and gate that the two jobs still cover the whole gate)
Acceptance
CLOUD-398 The hk gate's two chains contend for four cores, so parallelising inside it returns a third of what the same work returns alone
Why
CLOUD-386 measured what that costs. The same bats suite, alone on a 4-core box, ran 72.73s; inside the gate on CI (run 31535645426) it ran 192.59s. That reading is wrong, and CLOUD-439 retired it before this row was refined — corrected in the body 2026-08-21 rather than left in a comment, because a headline number nobody reads past is the whole case. The two figures came from different machines: 72.73s local, 192.59s on This is the issue CLOUD-386 explicitly listed as out of scope, and I had separately argued against it. That argument was: a split converts Both preconditions have cleared, and the cheap option was tried and rejected — 2026-08-20This issue said it should not be pulled before two things. Both are now Done, so the hold no longer applies. CLOUD-439 was recorded here as a blocker and as "a cheaper thing to try first": cap CLOUD-393's laps-per-landed-PR arithmetic also exists now, which was the second hold. The arithmetic has moved, by a lot — measured 2026-08-20The case above was built on a 242s gate. On run 32395938706:
The suite is 2,300 cases. The tuning that chose 4 workers was measured at 1,039. What a split is actually worth, on the corrected 1.11× rather than the 2.6× above: moving the suite to its own runner leaves
So slice 1 is a wall-clock change and not a cost one, and it needs deciding on that basis. The 697s is a slow 4-core runner executing 2,300 cases; the levers on it are the suite's size (CLOUD-312's retirements) and sharding, neither of which is a lane partition. What the split breaks, concretelyAll satisfiable, all load-bearing:
Mechanism, decided — a lane partition with a totality gateCorrection, 2026-08-21: the lane partition is not required for the cheap job, and the section below over-specifies it. The reasoning that led here was that hk profiles are additive, so a subset cannot be selected. That holds for selecting the slow half alone. It does not hold for the cheap half, which is already one flag: The partition already exists in What the lane work still buys, stated so it is a trade rather than a discard. Without it there is no way to say "slow only", so the path-gated full job re-runs the cheap steps: about 160s of the 857s gate, paid twice on a diff that touches code. The lane partition removes that duplication and nothing else. It is an optimisation of the split, not the enabling mechanism, and it should be sequenced accordingly. A constraint on the shape, from a gate rather than from taste. The concrete case that surfaced this, measured 2026-08-21. That 697s is provably unable to change its verdict on such a diff: every bats suite that touches memories builds its subject under a temp fixture rather than reading the real tree — Two questions the implementation must answer rather than assume. Whether The profiles question this issue was waiting on. hk 1.54's semantics are that an untagged step always runs and a tagged step runs only when its profile is enabled, so Two options were on the table and both were rejected: tagging every step makes the pre-commit hook's behaviour depend on a flag, and The answer is the first option plus the mechanism that makes its objection safe, which is this repo's standard move rather than a new one:
Slice 2 — the pure-Rust jobs are absent, never skipped — LANDED 2026-08-21, PR #606Folded in here rather than filed separately, because splitting it would reproduce the two-authorities problem this issue is about.
The shape is fixed by the The filter ships with a gate over its own honesty, the shape CLOUD-224 set: cases asserting the jobs are selected for a Refinement — Ready (a lane partition proven total by a gate, plus an absent-not-skipped path filter)
Acceptance
|
ℹ️ Recent review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Free Run ID: 📒 Files selected for processing (7)
Included review availability: Your plan provides up to 1 included review per hour; 0 remain after this review. 📝 WalkthroughWalkthroughThe CI workflow now runs Merge Risk: ⚪ Minimal · up to This PR moves the shell suite to its own runner and adds safeguards for lane coverage and invocation tracking; no actionable merge-blocking risk remains after normal checks and review. Warning Git: CodeRabbit could not clone the repository, so clone-backed analysis was skipped and this review may be incomplete. Verify repository clone access, such as SSH credentials, before requesting another full review. If clone access is intentionally unavailable, use Note 🎁 Summarized by CodeRabbit FreeYour organization is on the Free plan. CodeRabbit will generate a high-level summary and a walkthrough for each pull request. For a comprehensive line-by-line review, please upgrade your subscription to CodeRabbit Pro by visiting https://app.coderabbit.ai/settings/billing. Comment |
…re's job
Three of the four failures a full `verify` found, all of them mine and none in
the change's substance.
`verify:gated`'s `depends` array was the wrong home for `test:bats`. Adding an
eighth element pushed the line past taplo's width, `mise run fmt` exploded it to
a multi-line array, and three suites read that key with
`awk '/^depends = /{print; exit}'` — one line, so every substring assertion over
it failed at once. The array is restored and the call moved into the body, which
is a better home on its own merits: `depends` entries run in PARALLEL, so an
entry there would have started a second full suite run concurrently with the `ci`
leg that already runs it through hk, where the body call lands after the gate set
has finished and answers from the step receipt in ~0.3s. It is still named where
`ci-local-parity` property 3 reads, since that gate's block covers both the
`verify` and `verify:gated` tables.
`bats-invocation`'s Rust fixture carried an install list and no invocation, which
was fine while the install clause read `jobs.ci.steps` by name and is not now
that it derives the job from `mise run test:bats`. Left alone the derivation was
empty, the clause vacuous, and the case that exists to catch a missing parallel
backend passed over a workflow with none. The fixture carries both halves now.
Refs: CLOUD-1140
|
❌ The last analysis has failed. |
|
Closing unlanded. The branch Why
Neither mechanism is wrong. Together they mean the required-check roster cannot grow until Three ways around it, and why none was taken
What happens insteadCLOUD-1143 is filed to retire What is green and stays on the branch
Two findings from this branch are worth keeping regardless of when it lands:
Refs: CLOUD-1140, CLOUD-1143 Generated by Claude Code |
Closes CLOUD-1140.
What this is
CLOUD-398 was closed Done on #606, which landed slice 2 only — the four pure-Rust jobs made absent on a diff they cannot judge. Slice 1, moving the shell suite off the shared runner, was fully refined and never implemented. Three of that issue's acceptance clauses are unmet verbatim:
ciwall clock is measurably lower, and the runner-seconds delta is reported rather than assumed."This is that slice.
The arithmetic moved by 6x
CLOUD-398 costed the split on an 857s gate with a 697s suite and concluded ~−167s, −20% — real but thin against a metered runner. Measured 2026-08-29 on run
33244045030(green):cijobperffinished at 9.2 min and idled for 24 moremise run ciinside ittest:batsinsidemise run ciciis the entire critical path. CLOUD-386 swept the worker count at the runner's real width and found every direction worse, because each point redistributes workers inside one saturated 2-core box. Its own point D measured the number this needs: the suite alone with the box to itself is 623s, against 829s contended. A second runner is the experiment that sweep could not contain, and it turns the gate's wall clock from a sum of two chains into a max — paid twice perlandlap, sinceverifyruns the same gate locally.Why
HK_SKIP_STEPS, and not the two obvious alternativesverifydoes not run —ci-local-parityproperty 3. Sotest:batsis named inverify:gated'sdependsinstead, beside theci/ci:quickpair that has overlapped there for as long as the quick tier has existed. Naming it in a comment would satisfy the grep and nothing else, which is the shape that gate's own header condemns.The false green this ships its own gate for
hook-profile-checkreadshk check --all --planand asserts everyslow-tagged step isincludedthere. It cannot see what a workflow job passes — which is why it reads.claude/hooks/git-hook.sh's invocation line for its other half.Measured:
HK_SKIP_STEPS=test:batstakes that plan from 58 steps to 57 with reasonenv_exclude, while the gate's own reading stays green. So the suite could be carved out of thecijob and run by no job at all, with every gate in this repository green over a suite that never executed.ci-suite-lanerefuses exactly that direction. The opposite mistake — a name matching no step — runs the step in both jobs, which shows up in the bill and needs no gate because it has a symptom.A Rego module rather than a shell gate because
shell-retirementgovernsmise-tasks/**andtests/**/*.batson the added side as well as the edited one (delta.added, mutant rowshell-added-unchecked), so the engine is the only route a new predicate has. Hence theMUTANT-EXEMPTrow and the compiled-binary second tier.bats-invocationhad to move with itIts
install_argsclause assertedrushwas installed in thecijob. That was correct exactly while thecijob ran the suite; after the move it would have been green over abatsjob with norush— the same false green one layer over. The job is now derived from which one invokesmise run test:bats, so it follows the suite wherever it goes next. Its fixture changed too: the old one carried an install list and norun, so with the job derived it could no longer reach the assertion it exists for.Files
.github/workflows/ci.ymlcijob setsHK_SKIP_STEPS: test:bats; newbatsjob;finalneeds[ci, bats, perf]mise.tomlbatsinCI_REQUIRED_CHECKS;test:batsinverify:gated'sdependspolicy/ci-suite-lane.regopolicy/bats-invocation.regobatten.toml[[verdict]]rows with routescrates/batten/tests/ci_suite_lane.rsVerification
Green locally before pushing:
policy-test(22 bundles, 254 cases),batten-check(exit 0 over this tree),actionlint,ci-tools-check,timeout-check(39 jobs),ci-local-parity(all 17 properties), and the full hk fast tier at commit time.The measurement CI itself has to return is the one no local run can give: the
ciandbatsjobs' own durations. The wall-clock delta and the runner-seconds delta both get reported on this PR, whichever way they fall — a negative result closes CLOUD-398's deferred question rather than being dropped.Not in this change
cijob still runs the cheap steps thebatsjob's preamble also pays for. Judgement call recorded on CLOUD-1140 §2.runs-on:in this repository areubuntu-latest= 2 cores, and the 2026-08-11 sweep measured the same suite at 210.7s (2 workers) vs 100.3s (4) — a 2.1x from cores alone. That is a billing decision, not a code one. Judgement call recorded on CLOUD-1140 §2.cijob'stimeout-minutes: 87. Itsp95=1730sis about a job that no longer runs the suite, so it is now over-declared. It needs 25 runs of the new shape beforetimeout-driftcan answer; CLOUD-1140's acceptance carries it.Generated by Claude Code