Repository navigation
perf(ci): give the shell suite its own runner, and gate the carve-out that lets it - #766
Conversation
CLOUD-1140 CLOUD-398's slice 1 never landed: the shell suite is still 83% of a 30-minute `ci` job it shares a runner with, and the split is now worth 6x what it was costed at
Why CLOUD-398 was closed Done on PR #606, which landed slice 2 only — the four pure-Rust jobs made absent on a diff they cannot judge. Slice 1, moving the shell suite off the shared runner, was fully refined and never implemented. Three of that issue's acceptance clauses are unmet verbatim:
This row is that slice. It is filed rather than reopening CLOUD-398, which is released. The arithmetic that made it marginal has moved by 6x. CLOUD-398 costed the split on an 857s gate with a 697s suite and concluded ~-167s, -20% — real but thin against a metered runner. Measured 2026-08-29 on run
Why the worker knob cannot deliver this instead. CLOUD-386 swept it at the runner's real width and every direction is worse ( What CLOUD-398 got right and this must not re-derive. Its correction of 2026-08-21 stands: the lane partition is an optimisation of the split, not its enabling mechanism. The suite does not need a lane — it needs to not be in the other job. The false green this must close, and it is the reason this is not a two-line change. BLOCKED on CLOUD-1143: the roster change obliges an edit
|
| rank | program | suite cost | governed siblings | verdict |
|---|---|---|---|---|
| 1 | ci-wait |
102.5s | 0 | SELF-CONTAINED |
| 2 | derived-check |
210.8s | 1 (man-pages.sh) |
CLOSES |
| 3 | land-lock |
145.1s | 3 | BLOCKED |
| 4 | land |
91.0s | 6 | BLOCKED, worst |
ci-wait is the leaf and the cheapest thing here. Nothing resolves it by path except its own suite. Its one live caller is mise-tasks/land.sh:1838 — mise run ci-wait, a task name, which is exactly the shape 42cbee1 records as why the claim gate's port landed cleanly: "its only callers named it by task name, so mise.toml could answer for it with no frozen file touched at all." It is not in $MUTANT_GATES and carries no #MUTANT row, so there is no gate to drop. Two deleted paths, two ledger arms, zero governed sibling edits.
derived-check is the largest prize and it closes. Nothing resolves it by path; hk.pkl:289 calls it by task name. Its one drag-in is not a caller but its own header — tests/derived-check.bats:2 declares # subject: mise-tasks/derived-check.sh mise-tasks/man-pages.sh, and SubjectFacts::died requires every declared subject to die ("a suite declaring two subjects of which one still stands has work left"). man-pages.sh is cheap: no suite, not in $MUTANT_GATES, and its only other reference is mise.toml:1356, ungoverned. Three deleted paths, three ledger arms, zero governed sibling edits. Independent of schema-check — no path reference, no task-name call, no shared subject header; the coupling is rhetorical only.
Its 210.8s is process-spawn tax, not compilation: ~77 cargo run -p batten invocations per gate run × 10 @test cases ≈ 770 dispatches.
land-lock and land **are blocked, and not negotiably. **mise-tasks/ci-lease-precondition.sh:176 fetches land-lock.sh's blob from main and execs it, branching on its exit code. Removing that block deletes lines naming no deleted path, so only_drops_a_retired_reference fails; repoints_at_the_declared_successor cannot help either, since it demands replace(was, gone, succ) == line byte-for-byte and no single-token substitution turns a fetch-write-chmod-exec block into a compiled-verb call. ci-lease-precondition.sh is itself non-retirable: $MUTANT_GATES-listed, own suite, fetched by path from main by five workflows. land adds ci-slow-needed.sh and ci-local-parity.sh:1086-1088, the latter literally opening and grepping land.sh's bytes as its own predicate. 42cbee1's ready-lint → graph-check shape, two levels deeper.
The ordering that matters to THIS row
CLOUD-1143 — this row's blocker — must repoint six checks-green readers, and ci-wait.sh:46 is one of them (verdict_of="$(dirname "$0")/checks-green.sh", by path). Retiring ci-wait first removes one of those six, so it shrinks this row's blocker rather than competing with it. land.sh and land-lock.sh are two more of the six and are blocked — which is why CLOUD-1143 ends up entrenching them as live bash with rewritten exit-branching.
A hazard CLOUD-1143 does not carry: changing checks-green's exit table (1/3 → stdout) breaks the exit-semantics assertions in tests/ci-wait.bats, tests/land.bats and tests/land-lock.bats. Each such edit is V-SHELL-RULE-EDITED with only R-PORT-AND-RETIRE as a route. That row names none of the three suites. Retiring ci-wait first removes a third of that hazard too.
Recommended sequence: retire ci-wait (102.5s, free), then derived-check + man-pages (210.8s, closes) — 313.3s, 25.2% of serial suite cost, from five deleted paths and no governed sibling edited. Both need new rows; neither exists. Leave land and land-lock until ci-lease-precondition has a route, and note CLOUD-880 — Done — already concluded "land stays bash", so that pair is a decision to reopen rather than work to schedule.
|
Warning Review limit reachedNext included review available in 29 minutes. View limit detailsLimit details: You’ve used the included review currently available. Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. Review configuration: ⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Free Run ID: 📒 Files selected for processing (7)
Note 🎁 Summarized by CodeRabbit FreeYour organization is on the Free plan. CodeRabbit will generate a high-level summary and a walkthrough for each pull request. For a comprehensive line-by-line review, please upgrade your subscription to CodeRabbit Pro by visiting https://app.coderabbit.ai/settings/billing. Comment |
… that lets it CLOUD-398 was closed Done on PR #606, which landed slice 2 only — the four pure-Rust jobs made absent on a diff they cannot judge. Slice 1, moving the shell suite off the shared runner, was fully refined and never implemented, and three of that issue's acceptance clauses are unmet verbatim. The arithmetic that made it look marginal has moved by 6x. CLOUD-398 costed the split on an 857s gate with a 697s suite and concluded ~-167s. Measured on run 33244045030: the `ci` job is 1983s of a 1992s run — the entire critical path, with `perf` finishing at 9.2 min and idling for 24 more — `mise run ci` is 1832s of that, and `test:bats` ~83% of that again. CLOUD-386 swept the worker count at the runner's real width and found every direction worse, because each point redistributes workers inside one saturated 2-core box; its own point D measured the suite alone with the box to itself at 623s against 829s contended. A second runner is the experiment that sweep could not contain, and it turns the gate's wall clock from a sum of two chains into a max. `HK_SKIP_STEPS` rather than a profile or a second task, and each alternative fails for its own reason. A profile cannot express it: hk resolves multiple profiles on a step with AND and the CLI value supplants the config list, so carving one step out needs the whole lane partition CLOUD-398 costed at ~160s of duplication and explicitly sequenced after this. A second mise task would put a name in the workflow that `verify` does not run, which is `ci-local-parity` property 3 — so `test:bats` is named in `verify:gated`'s depends instead, beside the `ci`/`ci:quick` pair that has overlapped there for as long as the quick tier has existed. THE FALSE GREEN THIS SHIPS ITS OWN GATE FOR. `hook-profile-check` reads `hk check --all --plan` and asserts every slow-tagged step is included there; it cannot see what a workflow job passes, which is why it reads the git hook's invocation line for its other half. Measured: `HK_SKIP_STEPS=test:bats` takes that plan from 58 steps to 57 with reason `env_exclude`, while the gate's own reading stays green — so the suite could be carved out of the `ci` job and run by no job at all, with every gate in this repository green over a suite that never executed. `ci-suite-lane` refuses exactly that direction. The opposite mistake, a name matching no step, runs the step in both jobs and needs no gate because it has a symptom. A module rather than a shell gate because `shell-retirement` governs `mise-tasks/**` and `tests/**/*.bats` on the added side as well as the edited one, so the engine is the only route a new predicate has. `bats-invocation` had to move with it. Its `install_args` clause asserted `rush` was installed in the `ci` job, which was correct exactly while that job ran the suite; after the move it would have been green over a `bats` job with no rush — the same false green one layer over. The job is derived from which one invokes the suite now, and the fixture carries the invocation as well as the list, because one with only the list could no longer reach the assertion it exists for. Refs: CLOUD-1140
…re's job
Three of the four failures a full `verify` found, all of them mine and none in
the change's substance.
`verify:gated`'s `depends` array was the wrong home for `test:bats`. Adding an
eighth element pushed the line past taplo's width, `mise run fmt` exploded it to
a multi-line array, and three suites read that key with
`awk '/^depends = /{print; exit}'` — one line, so every substring assertion over
it failed at once. The array is restored and the call moved into the body, which
is a better home on its own merits: `depends` entries run in PARALLEL, so an
entry there would have started a second full suite run concurrently with the `ci`
leg that already runs it through hk, where the body call lands after the gate set
has finished and answers from the step receipt in ~0.3s. It is still named where
`ci-local-parity` property 3 reads, since that gate's block covers both the
`verify` and `verify:gated` tables.
`bats-invocation`'s Rust fixture carried an install list and no invocation, which
was fine while the install clause read `jobs.ci.steps` by name and is not now
that it derives the job from `mise run test:bats`. Left alone the derivation was
empty, the clause vacuous, and the case that exists to catch a missing parallel
backend passed over a workflow with none. The fixture carries both halves now.
Refs: CLOUD-1140
48f6388 to
e340f79
Compare
|
❌ The last analysis has failed. |
|
/fast-forward |
Closes CLOUD-1140
Why
ciis the entire critical path of a CI run. Measured on run33244045030(green):cijob, end to endmise run ciinside ittest:batsinside thatperffinished at 9.2 minutes and idled for 24 more. Moving the suite off the shared runner is worth roughly −1100s of wall clock, about −60%, and it is paid twice perlandlap becauseverifyruns the same gate locally before every push.This is CLOUD-398's slice 1. That issue closed Done having shipped slice 2 only, leaving three of its acceptance clauses unmet verbatim.
Why the worker knob cannot deliver it instead: CLOUD-386 swept it at the runner's real width and every direction is worse — 4 workers costs the gate 173s, giving bats the box costs 50%, capping nextest costs 20%. Every point in that sweep redistributes workers inside one saturated 2-core box, so each robs one chain to pay another. A second runner is the experiment the sweep could not contain, and point D already measured what it needs: the suite alone with the box to itself is 623s against 829s contended.
Why it lands now, when it didn't yesterday
This is a replay of PR #742 (
bafefee), which was closed unlanded against exactly one blocker.ci-local-parityproperty 5 requires a newpull_requestjob to be named in$CI_REQUIRED_CHECKS, both directions — a job added and not listed is silently unrequired. Sobatsjoins the roster. That madetests/checks-green.bats:102fail:a deliberate literal in a file
policy/shell-retirement.regogoverns, whoseV-SHELL-RULE-EDITEDdeclares one route (R-PORT-AND-RETIRE) with no override and nobypass_env.#758 retired that suite and its program, so the roster now reaches the decision as a flag rather than being pinned by a bats literal. The two commits cherry-picked onto the new
mainwith no conflicts, and:No governed file is edited. That empty output is the shared acceptance clause of this row and CLOUD-1143 — "adding a name to
$CI_REQUIRED_CHECKSno longer requires editing any.batsfile, demonstrated by CLOUD-1140 landing on top of this without one."The false green this closes
hook-profile-checkjudgeshk check --all --planand asserts everyslow-tagged step isincludedthere. It never reads what a workflow job passes. So acijob that skips the suite would be green on that gate while CI silently stopped running it — theprofiled-step-not-in-checkclass, one layer up where the gate cannot see.policy/ci-suite-lane.regois that gate: it refuses a step carved out byHK_SKIP_STEPSthat no job runs. This is why the change is 753 lines rather than two.What's here
.github/workflows/ci.ymlcigainsHK_SKIP_STEPS: test:bats; newbatsjob with a measuredtimeout-minutes;finalgainsbatsinneeds:policy/ci-suite-lane.regocrates/batten/tests/ci_suite_lane.rsbatten.toml[[rule]]row registering the module and its[[verdict]]rowspolicy/bats-invocation.regomise run test:batsinstead of hard-codingjobs.cicrates/batten/tests/bats_invocation.rsrunhalf it needs once the job is derivedmise.tomlbatsinCI_REQUIRED_CHECKS;test:batsinvoked fromverify:gated's bodyThe
verify:gateddetail is load-bearing.test:batsgoes in the task body, notdepends: taplo reflows a longdependsarray to multi-line and three suites read it withawk '/^depends = /{print; exit}'. Putting it independsalso double-runs the suite, sincedependsentries run in parallel with the body.Verification
mise run ci-local-parity— green, all 17 properties, including property 5 both directions with the new job.mise run verify— running; the bats tier is the one that matters here, since it must now pass with the grown roster.Generated by Claude Code