Weekly test health: enforced check needs triage #3154

test:health garrytan/gstack, last 7 day(s), 2026-10-12T10:09:52.162Z
- PR full-fallback rate: 45.0% (58/129 runs)
    rule: share of evals.yml pull_request runs whose paid-plan manifest has prCoverage.mode = full-fallback
    dependents: 28
    pr: 43
    full-fallback: 58
- Reused PR trial records: 0 of 6772 trial records
    rule: trial-outcomes records with execution = reused across non-cancelled PR runs
- Recorded PR-lane spend: $3454.32 recorded
    rule: sum of executed trial cost_usd in PR runs; per merged PR = / PRs merged in the window; per validated revision = / distinct head SHAs with a success or failure verdict
    per merged PR: $75.09 (46 merged)
    per validated revision: $28.31 (122 revisions)
- Cancelled slice-minutes per week: 1939 min/week (44 cancelled run(s))
    rule: wall minutes of eval-slices jobs in cancelled evals.yml PR runs, scaled to 7 days
- PR lane push-to-verdict: p50 12.0 min, p95 1160.7 min (122 runs)
    rule: created_at to updated_at of evals.yml pull_request runs that reached success or failure (includes queue and debounce)
- Periodic census reliability (CEO-06/22): 2.00 expected failed trials/run (6/538 over 3 run(s)); P(red) 90.5% in force, 91.2% single-trial
    rule: trailing 4 main censuses with trial outcomes (scheduled runs and main dispatches): expected failed trials per run, raw trial failure rate per kind and class, and P(lane red) from pooled per-case rates under the recorded panels vs single-trial aggregation (every failure treated as nondeterminism: upper bounds for known-good inputs)
    kind behavior: 1/207 failed (0.5%)
    kind rule: 4/256 failed (1.6%)
    kind judge: 1/75 failed (1.3%)
    class assertion: 6
- Gate census reliability (CEO-06/22): 0.33 expected failed trials/run (1/308 over 3 run(s)); P(red) 33.3% in force, 33.3% single-trial
    rule: trailing 4 main censuses with trial outcomes (scheduled runs and main dispatches): expected failed trials per run, raw trial failure rate per kind and class, and P(lane red) from pooled per-case rates under the recorded panels vs single-trial aggregation (every failure treated as nondeterminism: upper bounds for known-good inputs)
    kind rule: 1/308 failed (0.3%)
    class assertion: 1
- INFRA re-dispatches (ENG-10): 0 of 4 census(es) re-dispatched; longest paid slice job 11.2 min
    rule: trailing 4 main census runs whose redispatch job succeeded; longest paid slice job wall in the same runs
- Quarantine evidence (DX-12): 0 of 236 blocking case(s) have enough evidence; 236 insufficient evidence (not the same as healthy)
    rule: blocking cases with >= 10 qualifying v3 trials on their current series, over the last 10 main census runs
    896 older-policy trial(s) display only
- Free-suite flaky-pass rate: 13.0% (6/46 main runs since 2026-10-05 needed the retry pass; 5 file(s))
    rule: share of main free-tests runs (success or failure) with at least one flaky-pass ledger entry; per-file rate = runs with that file / runs
    browse/test/terminal-agent-publication-lock.test.ts: 2/46 runs (4.3%)
    test/gstack-docs-candidate.test.ts: 1/46 runs (2.2%)
    test/gstack-next-version.test.ts: 1/46 runs (2.2%)
    test/plan-design-sdk-fixture.test.ts: 1/46 runs (2.2%)
    test/qa-functional-observer.test.ts: 1/46 runs (2.2%)
- Free-suite and Windows wall: free-tests 4.3 min (45 runs); windows-free-tests n/a (0 runs)
    rule: p50 of run_started_at to updated_at for successful main runs of free-tests.yml and windows-free-tests.yml
- Free-suite serial time: 2440 s over 1055 files
    rule: sum of scripts/free-test-durations.json (the committed seed)
- Net complexity (CEO-32, reported, no target): HEAD: 96637 lines, 57 package scripts (pass --complexity-base <ref> for the delta)
    rule: lines under scripts, test/helpers, .github/workflows and package.json script count, HEAD vs --complexity-base
Enforced checks:
- [PASS] Free-suite flaky files (W7a): no file above 5% over 46 main runs
- [FAIL] Free test files missing from the duration seed (CEO-10): stale seed: 147 unseeded file(s) > 5: browse/test/bun-child-config.test.ts, browse/test/cli-chain-stdin.test.ts, browse/test/cli-config-mismatch-message.test.ts, browse/test/path-security-realpath.test.ts, browse/test/proxy-env-daemon.test.ts, browse/test/proxy-env.test.ts, browse/test/runtime-root-server-bundle.test.ts, browse/test/sidepanel-font-size.test.ts, test/auq-substance-panel.test.ts, test/autoplan-answer-invalidation.test.ts, +137 more
    fix: Refresh the seed: `bun run test:ubicloud --record-durations` (or a CI recording; a 4-core laptop recording is not acceptable), then commit scripts/free-test-durations.json (docs/TESTING_INTERNALS.md#free-suite-duration-seed).