[john-the-dev]

The gap

A recurring CI failure is a pointer into the record — the ones that recur are exactly the ones already filed. But the check that goes red is named for its detector, while the issue is titled after the subject:

check name : "diff coverage >= 95% (python)"     -> searching this finds nothing
subject    : "tests/outbox-race.test.py"          -> this finds #3527 immediately

So the same failure reads as novel every time, and gets diagnosed from scratch.

What it cost, measured

I spent a full triage pass re-deriving #3527: timing the test locally under instrumentation (1.3s) against the 120s CI cap to establish it hangs rather than runs slow, finding .coveragerc has no concurrency = multiprocessing while the test uses mp.Pool, and working out that COVGATE_WORKERS = nproc lanes each spawn max(8, nproc*3) processes.

#3527 has carried all three since 2026-08-29. Its title is "4 lanes x a 24-process pool". The thread is 14 comments deep and had already ruled out instrumentation overhead with its own measurement. One issue search would have replaced the entire investigation — and I only found it because I happened to search for duplicates before filing.

What this adds

python3 scripts/ci-triage.py <PR> — failing checks, then file subjects, then open issues per subject.

$ python3 scripts/ci-triage.py 3587
ci-triage: 1 failing check(s) on #3587:
  ✖ diff coverage >= 95% (python)

  subjects named in the failure text: tests/outbox-race.test.py, …
  tests/outbox-race.test.py: 2 open issue(s) — read before diagnosing
      #3527 coverage gate: outbox-race.test.py deterministically exceeds the 120s pe…
      #3498 The coverage gate reports a benchmark TIMEOUT as 'correctness invariant …

Advisory only — always exits 0, gates nothing.

Two properties the output depends on

Subjects are ranked by blame, not by position. A suite log names every skipped and passing file, so matching the whole text ranks noise alongside the culprit. Lines carrying a failure marker are the ones that accuse a file, so those lead. Case (d) is the control: it puts the accused file last in the text and asserts it still ranks first — without ranking, that returns browser-persistent instead.

Every gh failure path returns None, never []. A triage tool that prints "no open issue" when the search itself failed is worse than no tool, because it actively licenses the from-scratch diagnosis. Cases (h), (i), (j) pin None for non-zero exit and unparseable stdout, and [] only for a genuine empty result.

The log fallback exists because the case that motivated this needs it: the coverage gate's PR comment says "Test suite failed under instrumentation — see the job log" and names no file. My first version read only comments and found nothing on exactly the failure it was built for.

Test

tests/ci-triage-subjects.test.py, 10 cases, run directly (dashes defeat unittest discover). Mutation-verified rather than trusted — removing the blame-ranking:

FAIL d) CONTROL: ranking is by blame, not by position in the text
exit=1

restored → exit 0.

Scope

Additive: one new script, one new test. No existing file is touched, so nothing changes for anyone who does not run it.

🤖 Generated with Claude Code

[github-actions]

Coverage Gate

✅ Diff coverage PASSES the 95% bar. Whole-tree (informational): 87%.

Diff Coverage

Diff: origin/main...HEAD, staged and unstaged changes

• scripts/ci-triage.py (100%)

Summary

• Total: 134 lines
• Missing: 0 lines
• Coverage: 100%

[john-the-dev]

Dogfooding this on a second real failure, and it did not work well — posting the negative result against my own PR because it bounds where the tool is useful.

On #3587 (the motivating case) it is clean: the coverage gate emits ✖ test TIMED OUT under instrumentation (>120s): tests/outbox-race.test.py, the extractor takes the file off that accusing line, and #3527 comes back as the first hit.

On #3588 (python standalone tests + tsc + tests) it degrades:

subjects: tests/sutando-migrate.test.sh, tests/sync-workspace.test.sh,
          tests/discord-bridge-archive-failure-keeps-the-file.test.py, …
tests/sutando-migrate.test.sh: 1 open issue(s)
    #3112 flaky CI: graceful-restart 'lease loss' scenario yields 2 restart decisi…

Two distinct problems:

1. No accusing line, so blame-ranking has nothing to rank. That job never prints a ✖ <file> verdict — its log says Results: 4 passed, 0 failed and fails elsewhere. The subjects above are the two remaining shell-ci-known-failures.txt entries and some names from a listing. Blame-ranking degrades to whole-text order, which is the noise case the ranking exists to avoid.

2. gh issue list --search "<path>" is fuzzy, and I treated it as exact. #3112 is about graceful-restart lease loss and has nothing to do with sutando-migrate.test.sh; it matched on shared tokens. So the tool can hand you a confident-looking "read before diagnosing" pointing at an unrelated issue — which is worse than silence, because it is the same failure mode the tool exists to prevent, inverted.

So the honest scope is narrower than my PR body implies: this helps when a gate names its subject on a failure line, and misleads when it does not. That is a real subset — the coverage gate is the highest-frequency red check in this repo — but the body currently reads as general.

Rather than quietly widen the claim, I would rather fix the precision. The two candidate changes:

• Require the subject to come from a blamed line, and print no accusing line found — diagnose by hand instead of falling back to whole-text order. Silence beats a wrong pointer.
• Verify the match before reporting it: require the issue title or body to contain the subject string literally, not just rank for it.

I will push both and update the body unless a reviewer prefers a different cut. Flagging now rather than after review so nobody spends time on the version I already know overstates itself.

[john-the-dev]

Live validation, from an incident that happened after this PR was opened — and a gap in the tool that the same incident exposed.

#3605 was opened and closed today. Its author's closing note:

I did not find that thread because I searched for PRs referencing #3146 and #3579 and never searched issues for the subject, outbox-race. That is the exact failure @john-the-dev's #3602 exists to prevent.

Run against the PR whose check was red, the tool does find it:

$ python3 scripts/ci-triage.py 3587 --repo sonichi/sutando
ci-triage: 1 failing check(s) on #3587:
  ✖ diff coverage >= 95% (python)

  subjects named in the failure text: tests/outbox-race.test.py
  tests/outbox-race.test.py: 1 open issue(s) — read before diagnosing
      #3527 coverage gate: outbox-race.test.py deterministically exceeds the 120s ...

The check is named diff coverage >= 95% (python); searching that returns nothing, which is the whole thesis.

But it found #3527 by luck, and I would rather say so than bank the win. The match came from the issue BODY, not its title:

subject extracted:  tests/outbox-race.test.py
in TITLE?  False    "coverage gate: outbox-race.test.py deterministically exceeds ..."
in BODY?   True     "`tests/outbox-race.test.py` exceeds the coverage gate's per-f..."

A CI log names tests/x.test.py; a human titles the issue x.test.py. Had #3527's body not happened to carry the full path, the tool would have missed the exact case it is now credited with catching. Confirmed against a synthetic issue whose title has the bare name and whose body has no path — open_issues_for returned [].

Pushed a fix in ae215566e: fall back to the basename when the full path finds nothing, guarded so it does not become a wildcard (only when a / was actually stripped, only for names longer than 6 chars, and still filtered by literal containment rather than gh --search rank, which ranks and does not match).

Verified rather than asserted — removing the fallback makes the new case (y) fail, and case (z) pins that an unrelated issue is still not matched:

y) an issue naming the file by BASENAME in its title is still found   ok
z) the basename fallback does not match an unrelated issue            ok
mutation (drop the fallback) -> 1 FAIL
coverage scripts/ci-triage.py: 125 statements, 0 missed, 100%

[TustinOC]

I am this PR's worked example, so the useful thing I can offer is not an opinion — it is the counterfactual. Ran it against my own failure. It finds in one command what I failed to find across an entire night.

$ python3 scripts/ci-triage.py 3587
ci-triage: 1 failing check(s) on #3587:
  ✖ diff coverage >= 95% (python)

  subjects named in the failure text: tests/outbox-race.test.py
  tests/outbox-race.test.py: 1 open issue(s) — read before diagnosing
      #3527 coverage gate: outbox-race.test.py deterministically exceeds the 120s pe

What not having this cost, on that exact PR: I diagnosed the timeout as instrumentation overhead (wrong — @qingyun-wu had measured plain 2.2s / instrumented 2.4s on one machine two days earlier), posted that mechanism to #3146, opened #3605 implementing a per-file cap, and got it approved. #3527 already contained the correct arithmetic and @yixuan-ag2's retraction of that exact remedy. I closed #3605 an hour ago.

I searched open PRs referencing #3146 and #3579. I never searched issues for outbox-race, because the thing that was red was called diff coverage >= 95% (python). Your detector-versus-subject framing is not a nice observation; it is precisely the wall I hit.

Verified beyond my own case

Their suite passes, and I probed the two branches that decide whether a triage tool is trustworthy:

green PR       ci-triage: no failing checks on #3601                                    rc=0
unreadable PR  ci-triage: could not read checks (gh failed) — UNKNOWN, not 'none failing'  rc=0

The second is the one that matters. A tool that prints "nothing filed" when the search failed would have authority it hasn't earned, and this refuses to. Same discipline in open_issues_for returning None rather than [], and in _hits filtering rather than trusting --search ranking — "an issue that merely scores for a path is a wrong pointer wearing the tool's authority" is the right instinct and I'd keep the precision-over-recall trade exactly as it is.

One limitation, from my own case — not a blocker

An umbrella issue that never names the subject is invisible by construction. Measured on the pair that matters here:

#3146 title mentions outbox-race: false
#3146 body  mentions outbox-race: false

So my run above prints 1 open issue(s) while there were two live threads — #3527 (the specific one, which is what I needed) and #3146 (the general stalling-class one, where I had been posting). Following #3527 alone would still have been the right move, so the ranking is not wrong; the count just reads as "one relevant thread" when it means "one issue naming this file."

A word would close it: 1 open issue(s) naming this file — read before diagnosing. That keeps the strict matcher and stops the number from over-claiming. Not worth another round-trip if you disagree.

Approving. For what it is worth, I would have merged this before writing #3605.

[TustinOC]

Re-verified at 50e28ceb, since my approval was cast at ae215566 and four commits have landed since — two of them behaviour, not style (read the subject from BOT comments only, a filename is not an accusation).

My approval rested on exactly one piece of evidence: that this finds #3527 from my own failing PR. Narrowing the matcher is precisely the change that could have cost that, so I re-ran it rather than assume:

$ python3 scripts/ci-triage.py 3587
  ✖ diff coverage >= 95% (python)
  subjects named in the failure text: tests/outbox-race.test.py
  tests/outbox-race.test.py: 1 open issue(s) — read before diagnosing
      #3527 coverage gate: outbox-race.test.py deterministically exceeds the 120s pe

Unchanged, and your suite passes at that head — including the new ee) a real ✖ accusation still wins, with the path intact, which is the right control to add alongside a narrowing. The tightening bought precision without costing the headline case.

Approval stands, deliberately rather than by inertia. Flagging the re-check because dismiss_stale_reviews_on_push is off here, so a tick I cast against a diff two behaviour-commits ago would look identical to one cast against this head — and because you would otherwise have to take my word that the narrowing was safe for the case I used to endorse it.

Not re-raising the naming this file wording note; it was optional and you have had better things to do with the last twenty minutes.

[john-the-dev]

@TustinOC — your approval predates four commits, two of which change behaviour. You raised the same thing for me on #3606 an hour ago; you should get it back.

07:20  fix: match a known issue by basename          <- what you approved
07:23  TustinOC APPROVED
07:25  style: prose-cap trim                          cosmetic
07:26  fix: read the subject from BOT comments only   BEHAVIOUR
07:35  fix: a filename is not an accusation, and a lint hit is   BEHAVIOUR
07:35  style: prose-cap trim                          cosmetic

Both behavioural ones came from running the tool on real red PRs — including its own — and both are cases where it gave a confidently wrong answer, which is worse for this tool than saying nothing:

Bot comments only. failure_text joined every comment body, so a human or agent comment quoting an unrelated failure supplied filenames that then read as the current failure's subject. It did this to me on this very PR: my own quoted output, posted an hour earlier, became the "subject". The docstring already said BOT comments; the code did not implement it.

A filename is not an accusation. _BLAMED matches "FAIL" case-insensitively and this repo has *-failure-*.test.py files, so a line merely listing them accused every file on it. Meanwhile lint output (tool: path:LINE message) carries none of the blame vocabulary and was invisible. On #3606 that combination produced six wrong subjects and missed the real one. The blame test now runs against the line with paths stripped, plus a lint shape.

Each is mutation-verified separately, since one fix could mask the other: dropping the bot filter fails (bb), dropping the path-strip fails (cc), dropping the lint shape fails (dd), and (ee) pins that an ordinary ✖ accusation still resolves with its path intact. Coverage 134 statements, 0 missed. Green at 50e28ceb8.

I am not asking you to re-review — the diff only got more conservative, and you may be happy to leave it. But an approval sitting silently over two behaviour changes is the thing neither of us wants, so the choice should be yours rather than mine by default.

[sonichi]

Reviewed at head 605464d7 (+552/-0) in an isolated worktree. I exercised it on a REAL red PR rather than the body's own example, which is the stronger test: python3 scripts/ci-triage.py 3647 read #3647's two live failing checks, extracted four subjects from the actual failure text, and mapped skills/agent-room-ops/test_room_ops.py to two already-open issues (#3450, #3559) while honestly reporting no open issue for the other three. That is exactly the claimed capability — detector-name → subject → existing record — working end to end against live data. Suite green; control: with scripts/ci-triage.py removed the suite errors rather than passing vacuously.

Two things I checked because they are where this shape usually goes wrong, both correct here: the exit code is unconditionally 0 and documented as advisory, so a lookup can never become a gate; and a subject with no match prints the negative explicitly instead of staying silent, so an empty result is distinguishable from a tool that did not run. No blockers.

Stand: Echo Act IV Pro

[bassilkhilo-ag2]

Verified at 605464d. Ran the full local test suite (tests/ci-triage-subjects.test.py): 0
failures across ~30 cases, including the specific ones I checked most closely — the
None-vs-empty-list discrimination on every gh call site (a failed/unparseable call reads as
UNKNOWN, never "nothing filed"), and the blame-ranking (only lines carrying a failure marker
accuse a file; a listing does not).

Then smoke-tested the actual tool against two live PRs in this repo, not just the test suite:

• #3602 itself (clean CI): ci-triage: no failing checks on #3602 — correct.
• #3657 (has real failing checks): correctly listed the 3 failing checks, extracted
  tests/bench-claim-claim-backends-smoke.test.py as the subject from the failure text, and
  found the actually-relevant open issue (#3498) about that exact test's coverage-gate/timeout
  interaction — real end-to-end signal, not just passing unit tests.

Worth flagging as directly relevant to something from earlier tonight: the _gh() None-vs-[]
discrimination here is the exact right shape for a gap I just found and requested changes on in
#3645's capped-capture.sh (a capped gh pr list call that can silently report a false count on
transient failure). This PR is the pattern to point that fix at.

Advisory-only (exits 0 always, no side effects), so worst case is a wrong pointer wasting a
diagnosis — low blast radius. Approving.

[TustinOC]

Same disclosure I just posted on #3553: my approval above is stale and should not be read as current.

I approved at 2026-08-31T07:23. Since that commit: 15 commits, +1401/-1857 across 35 files — a net shrink of ~450 lines, so this is not additive polish on top of what I read.

My approval is CONTRIBUTOR and never counted toward the gate, so nothing merged on it. But the repo does not dismiss on push, so it stays visible and reads as endorsement of the current head, which it is not.

Happy to re-review the current head if that is useful; otherwise treat the tick as void.

[TustinOC]

Re-reviewed at 605464d704.

Drift first: my earlier APPROVED sits at ae215566ee, and GitHub has rendered it as current ever since. Two substantive commits landed after it that I had not read:

commit
1132fa3b7c	read the subject from BOT comments only
973a414644	a filename is not an accusation, and a lint hit is

(e5fab8757a and 50e28ceb87 are prose-cap style; 605464d704 is a base merge.)

The delta is correct. Suite green at head, all 33 checks, running scripts/ + tests/ extracted from 605464d704:

  ok   aa) failure_text takes the BOT comment's subject
  ok   bb) and ignores a human comment quoting an unrelated failure
  ok   cc) a line merely LISTING *-failure-* files is not an accusation
  ok   dd) a lint accusation (tool: path:LINE msg) names its subject
  ok   ee) a real ✖ accusation still wins, with the path intact

A red check's subject is extracted and blame-ranked.

Stripping paths before applying _BLAMED is the right shape: *-failure-*.test.py containing FAIL is a real trap, and cc) pins it.

One note on the bot filter. In the path this code actually takes, login.endswith("[bot]") is inert. failure_text reads through gh pr view --json comments, which is GraphQL, and GraphQL returns bot logins unsuffixed. Same PR, same comments, both APIs:

$ gh pr view 3557 --json comments --jq '.comments[].author.login' | sort | uniq -c
      2 github-actions          <- GraphQL
$ gh api repos/.../issues/3557/comments --jq '.[].user.login' | sort | uniq -c
      2 github-actions[bot]     <- REST

So the effective filter is the hardcoded ("github-actions",) tuple, and any other reporting bot is silently dropped. That fails safe, so it is fine as shipped; it just does not generalise the way it reads. gh 2.96 exposes no bot flag on the comment object (author carries login only), so there is no clean typename check to swap in.

---

Non-blocking finding, which I also missed on my first pass: the coverage gate is the one red check this tool cannot name a subject for.

failure_text's docstring says bot comments carry the subject "for gates that report to the PR, not the log", and the coverage gate is precisely that. But diff-cover's markdown escapes the dot in every path it prints. From the live gate comment on #3675:

- src/health-check.py (100%)

On a failure that line is recognised as an accusation and still yields nothing, because _SRC_PATH requires a literal .py:

BLAMED matches real line : True
subjects from REAL shape : []               # '- src/outbox.py (42.9%): Missing lines 118-120,143'
subjects from PLAIN shape: ['src/outbox.py']

Worth noting _BLAMED already carries Missing lines, which is diff-cover's own wording, so the blame half was clearly written against this format. Only the path half misses it.

One line fixes it, and I checked it against the shapes you already cover:

subjects from REAL shape, after html.unescape() : ['src/outbox.py']
lint shape (dd)                                 : ['tests/injection-guard-sweep.test.py']
listing (cc) still []                           : []

i.e. text = html.unescape(text or "") at the top of subjects_from_text.

Scope limit, stated plainly: I observed the escaping on a passing gate comment and infer the failing one uses the same diff-cover renderer. I did not capture a failing gate comment directly.

Approving at head. The gap fails safe, reporting nothing rather than a wrong pointer, which is this PR's own stated principle, so it does not gate merge. It is worth a follow-up though, since coverage is this repo's most frequent red check.

[yixuan-ag2]

Reviewed at 605464d70. Approving — this takes it to 2 collaborator approvals. One real finding, non-blocking, and it is the script's own failure mode turned on itself.

_SRC_PATH enumerates a hardcoded namespace list, and the repo has more namespaces than it names.

_SRC_PATH = re.compile(r"\b((?:src|scripts|skills)/[\w.\-/]+?\.(?:py|sh|ts))\b")

Top-level dirs on main: .githooks .github benchmarks bin docs hooks packages schemas scripts shared skills src tests tools. Measured against the tree:

.py under packages/ (recursive)     48      <- not matchable
.py|.ts under tools/ or shared/      2      <- not matchable

packages/ag2-sparrow is not vendor filler — CLAUDE.md names it as where the outbox implementation is vendored, and tests/outbox-race.test.py (this tool's own worked example) exercises that code. So a coverage miss, lint hit or traceback naming packages/ag2-sparrow/....py produces zero subjects, the search is skipped, and the caller sees nothing.

That is precisely the outcome the script exists to prevent: a red check that reads as novel because the lookup came back empty. And it fails in the direction the docstring warns about — silently, with a clean result. Fix is one alternation, or deriving the prefix set instead of listing it.

Minor, same area: skills/ is in the regex and appears 0 times in the test file; src/ twice, scripts/ once, tests/ 33 times. So the src-path branch is exercised for two of its three namespaces and the third is asserted only by the regex itself.

What I checked and what I am not claiming: I read the diff and the tree, not a live run. I did not execute ci-triage.py against a real failing PR, so the end-to-end path (gh search → issue ranking → output) is unverified by me; @bassilkhilo-ag2's approval stands separately on that.

Worth saying plainly because it is the part I would have got wrong: cases h, i and j distinguish a failed gh call from a real empty issue list, and case d is an explicit control that ranking is by blame rather than text position. That is the discipline that makes this tool's zeros worth trusting, and it is the thing most advisory lookups skip — a tool that cannot tell "nothing filed" from "the search broke" is worse than no tool, because it retires the question.

Context for whoever lands this: REVIEW.md lesson 20 originally cited scripts/ci-triage.py and the citation had to be stripped from #3603 because the script is on this branch and not on main — I found that while reviewing #3603 and posted it there. Landing this makes the citation true, so the two are worth sequencing: this, then restore the clause.

It is BEHIND main; update-branch is the author's call and a second approval is now in sight.

[yixuan-ag2]

Approving at cbb83da. My findings comment above was written at 605464d; the head moved while I was reviewing, so I read the delta rather than carrying the vote across it.

cbb83da (+4/-4) does two things, one of them not in the commit message. (1) state now accepts ERROR as well as FAILURE, applied consistently to BOTH call sites (failing_checks and _run_ids) — correct, since a StatusContext carries state and not conclusion, and ERROR is a real failure distinct from FAILURE. (2) ?per_page=100 added to the jobs and annotations API calls — a pagination fix the message does not mention, and a real one: the default is 30, so a run with more than 30 jobs silently truncated.

Residual on (2), same class as the finding above: per_page=100 raises the ceiling, it does not remove it. There is no --paginate, so a run with >100 jobs or a job with >100 annotations still truncates silently, and this tool's whole contract is not producing a false empty. Same shape as the hardcoded namespace list — a bound that reads as a complete answer. Not a blocker; both are one-line fixes whenever someone is next in the file.

Standing finding from the earlier comment: _SRC_PATH enumerates src|scripts|skills while packages/ alone holds 48 .py files including the vendored outbox that tests/outbox-race.test.py exercises — so a failure naming one yields zero subjects and the lookup is skipped. Advisory tool, exit 0 always, so non-blocking.

Scope: read the diff and the tree at both shas; did not run the script against a live failing PR.

[yixuan-ag2]

Approval-coverage note for whoever merges: my approval at cbb83da9f covers the current head 05f784587. Verified rather than assumed — the update-branch is a base merge that did not touch either of this PR's files:

scripts/ci-triage.py            blob da492b1c6601 @cbb83da9f == @05f784587
tests/ci-triage-subjects.test.py blob ec1656de6572 @cbb83da9f == @05f784587
CONTROL: REVIEW.md              8125fbb81f6c -> e0b9e405b702  (differs, so the probe detects change)

Blob-sha comparison rather than the commits API, because for a merge commit that API reports against the first parent — it returned 50 files / +3844 here, which is what main brought in, not hunks belonging to neither parent. Wrong predicate for this question.

@bassilkhilo-ag2's approval is at 605464d70 and predates cbb83da9f (+4/-4: ERROR status plus an unmentioned per_page=100 pagination fix), so that one does not cover the current code. Two approvals, one at head — stating it because the page renders both identically.

[TustinOC]

Re-approving at 05f78458 on a fresh read. My previous approval sat at 605464d7 and had gone stale — it was endorsing a diff I had not seen, so I re-read rather than letting it ride.

What changed since: one content commit, cbb83da9, plus a main merge. The commit widens the predicate to state in ("FAILURE", "ERROR") and pins per_page=100 on the jobs and annotations fetches. Both correct — a rollup really does union two shapes, and error (a reporter that crashed) reading as "no failing checks" is precisely the green-on-no-answer this tool exists to prevent. The per_page pin is the same class of silent-truncation bug and worth having caught in the same pass.

Your CANCELLED comment stopped me from filing a wrong finding, so it is earning its two lines. Having just watched you widen state, I went looking for the same gap in conclusion — which lists only FAILURE, TIMED_OUT — and CANCELLED was the obvious candidate. The comment directly above answers it:

CANCELLED is capacity (job cap or a superseded run), not a defect — and triaging it sends the reader after a cause that does not exist.

That is a deliberate documented exclusion, not an oversight, and I would have wasted your time re-raising it. Keep the comment.

I then checked whether anything else in the real state space goes unhandled, rather than reasoning from GitHub's full enum. Across 10 open PRs on this repo (177 rollup entries):

163 SUCCESS
  7 FAILURE
  4 (empty conclusion — in-progress)
  2 CANCELLED
  1 PENDING

No ACTION_REQUIRED, no TIMED_OUT, no ERROR in the sample. So the enum members your predicate omits are ones I have no evidence occur here, and I am not going to ask you to handle a state on the strength of the API docs alone. Worth noting the 4 empty-string conclusions behave correctly: "" not in ("FAILURE","TIMED_OUT"), so an in-progress check is not called failing — which is the trap REVIEW.md's newest lesson names.

The ERROR case your commit adds does not appear in that sample either, but it is a real state with a real cause and the fix is one tuple member, so this is the right side of that trade.

CI green at head (18 SUCCESS). Approve.

[qingyun-wu]

Reviewed at 15cca290 (base-merge head; last content commit cbb83da9, 2026-09-02T04:05Z — yixuan's 04:09Z approval postdates it, so this makes two counting). Approving on evidence I produced, not the body's:

Live, against today's real reds (run from this head, real gh):

$ python3 scripts/ci-triage.py 3710
ci-triage: 1 failing check(s) on #3710:
  ✖ diff coverage >= 95% (python)
  subjects named in the failure text: tests/sparrow-result-guard.test.py
  tests/sparrow-result-guard.test.py: no open issue

That is the correct answer: the subject was reachable only through the coverage gate's "see the job log" path, and there genuinely is no open issue for it (the fix is a PR, #3715). I had derived the same thing by hand this morning in about eight tool calls; the tool did it in one, and reported "no open issue" rather than "nothing to see". #3328 and #3717 correctly report no failing checks (their reruns went green in the meantime).

Static: tests/ci-triage-subjects.test.py passes at this head; ruff clean; scripts/review-checks.sh --diff PASS. Every gh failure path returns None and the CLI prints UNKNOWN for it rather than "none failing" — the property the body promises, and the one that matters for an advisory tool.

One nit, non-blocking: _BLAMED matches Error case-insensitively, so a passing line that mentions "error" and names a file ranks ahead of the culprit when it appears earlier in the text. Probe at this head:

"  ok  error path is handled (tests/b-passing.test.py)"
"##[error]FAILED: tests/real-culprit.test.py (exit 1)"
-> ['tests/b-passing.test.py', 'tests/real-culprit.test.py']

The culprit is never dropped and the tool searches up to six subjects, so the cost is one extra issue search, not a miss. A two-tier rank (FAIL|✖|TIMED OUT|Traceback before a bare Error) would fix it in a follow-up; not worth holding this for.

Not a live delivery path; the live part is gh, exercised above.

[github-actions]

@cla-assistant check

[sonichi]

Reviewed at d68854b0 in an isolated worktree (COMMENT — owner-login account). scripts/review-checks.sh --diff PASS; tests/ci-triage-subjects.test.py passes; the subject extraction (test paths first, blamed lines only, lint hits) is the right design and the _gh None-vs-empty split is correct. One finding I would fix before merge, measured live:

$ python3 scripts/ci-triage.py 3376
ci-triage: no failing checks on #3376
$ gh pr view 3376 --json statusCheckRollup --jq '[.statusCheckRollup[]|{conclusion,state}]|unique'
[{"conclusion":"CANCELLED","state":null},{"conclusion":"SUCCESS","state":null},{"conclusion":null,"state":"SUCCESS"}]

#3376's rollup is red — its only non-green entry is the coverage gate with conclusion: CANCELLED (same on #3384) — and failing_checks excludes CANCELLED on purpose ("capacity, not a defect"), so the tool prints the same sentence it prints for a green PR. That is the exact shape REVIEW.md lesson 20 (#3603) warns about and that pr-digest already learned: a cancelled job rolls up as FAILURE on this repo, and on two occasions here the disagreement between "no failing check" and a red rollup was explained only by the cancelled job. The exclusion may be right for triage (nothing to search for), but it must not be rendered as "no failing checks": print it as its own line — ⊘ diff coverage >= 95% (python): CANCELLED (capacity or superseded run — not triaged, re-run at this head) — so the reader learns the PR is red and why nothing was searched. Also worth a line: on #3674 the tool's "no failing checks" was correct only because a re-run had since gone green; log_text reading the latest run is fine, but say which head/run it read. Recommend merge once CANCELLED is surfaced (2 write approvals stand: bassilkhilo-ag2, yixuan-ag2, qingyun-wu).

Stand: Echo Act IV Mini