Repository navigation
Description
Story
As a maintainer relying on pr-review to actually review each PR,
I want the ~49% of pr-review runs that are currently cancelled while pending to either not be started or not be surfaced as failures,
so that review coverage is not silently dependent on which run survives the concurrency slot, and a cancelled run stops rendering as a red check.
Problem — measured, not inferred
Over a 3-hour window on 2026-08-02 (epic #1402's own PR traffic), pr-review-trigger.yml runs resolved as:
| conclusion | count |
|---|---|
| cancelled | 49 |
| success | 34 |
| skipped | 17 |
That is a ~49% cancellation rate — roughly the same magnitude as the "long-standing ~34% cancel rate" that #534 was filed to fix.
Mechanism. pr-review.yml sets cancel-in-progress: false with a per-PR concurrency group, and its inline comment (.github/workflows/pr-review.yml:97-107) reasons:
"with cancellation on, each new event killed the in-flight review … Same-head-SHA reviews are idempotent … so racing runs are safe to QUEUE rather than cancel: the first reviews, the rest no-op. No lost work, no cancelled-check noise."
That is correct about in-flight runs but not about pending ones. GitHub's concurrency model admits one running + one pending per group; when a third run arrives for the same group, the older pending run is cancelled. cancel-in-progress: false protects the in-flight run only. So under high event volume — precisely what this epic documents as the trigger fan-out — most runs are cancelled before they start, and the comment's "no cancelled-check noise" conclusion no longer holds.
Cancellations by triggering event in the same window:
| event | cancelled |
|---|---|
pull_request_review |
21 |
repository_dispatch |
17 |
pull_request |
5 |
check_suite |
5 |
workflow_dispatch |
1 |
Observed impact. On PR #1421, gh pr checks reported review / review fail, which reads as a failing required check; the underlying run conclusion was actually cancelled. A cancelled run rendering as a failure is misleading to both humans and to any automation keying on check state.
Why it belongs to this epic. This is the epic's three problems interacting: the trigger fan-out (redundancy) collapses four trigger types into one concurrency slot, which drops review attempts (convergence) and emits red checks that mean nothing (noise). It is also directly adjacent to #1408 — narrowing pr-review-sweep reduces one of the re-trigger paths that currently backstops a dropped review, so this should be understood before that lands.
Note the reusable already records the intended direction as a deferred refinement: "Future refinement: key the group by head SHA so a new push doesn't queue behind a now-stale review; deferred because the SHA isn't in every trigger payload."
Acceptance Criteria
- The pr-review run-outcome mix (success / cancelled / skipped, split by triggering event) is measured and reported on an existing report — no new cron — so the cancellation rate is a tracked number rather than an anecdote. The 3-hour figures above are recorded as the starting observation.
- A cancelled run no longer surfaces as a failing check. Whichever mechanism is chosen,
gh pr checksmust not reportfailfor a run whose conclusion iscancelled(the feat: implement issue #1405 — [Phase 2] Add the "How to add an agentic role" runbook and a PR-template interaction-contract section #1421 observation is the regression fixture). - The cancellation rate is materially reduced. Any of these is acceptable if justified and measured: coalescing/debouncing the four trigger types before they reach the concurrency group; keying the group by head SHA (the deferred refinement — note it must handle triggers whose payload lacks a SHA, e.g.
workflow_dispatchwithpr_url); or dropping trigger types that are provably redundant given the others. - Review coverage is proven not to depend on which run survives. A test or documented argument shows that for any push, at least one review runs to completion at the final head SHA — this is the property the current design assumes but does not demonstrate.
- The
cancel-in-progress: falsecomment block inpr-review.ymlis corrected: it currently asserts "No lost work, no cancelled-check noise", which the measurements contradict. It should state whatcancel-in-progress: falsedoes and does not protect (in-flight yes, pending no). - Interaction with [Phase 4] Narrow pr-review-sweep's cron scope to genuinely un-eventable cases and surface a sweep-hit-rate metric in daily health #1408 is stated explicitly: whether narrowing the sweep removes a backstop that currently compensates for dropped reviews, and if so, what replaces it.
Tasks / Subtasks
- Add the run-outcome-by-event breakdown to an existing report (
pr_review_health.shalready collects run history and computes p50/p95 — extend it rather than adding a collector). (AC: test issue from agent #1) - Fix the cancelled-renders-as-failed surface; add the feat: implement issue #1405 — [Phase 2] Add the "How to add an agentic role" runbook and a PR-template interaction-contract section #1421 case as a regression fixture. (AC: Go-live improvements for PR review agent #2)
- Choose and implement a fan-in reduction, with before/after cancellation numbers in the PR. (AC: Add @claude delegation, auto-merge, and rebase handling #3)
- Demonstrate the coverage property — at least one completed review at final head per push. (AC: Optimize review: small-PR and incremental fast paths #4)
- Correct the
pr-review.ymlconcurrency comment. (AC: feat: add Copilot engine support via REVIEW_ENGINE toggle #5) - Record the [Phase 4] Narrow pr-review-sweep's cron scope to genuinely un-eventable cases and surface a sweep-hit-rate metric in daily health #1408 interaction. (AC: feat: feature ideation framework — adversarial proposer/challenger/synthesizer #6)
Dev Notes
- Do not simply re-enable
cancel-in-progress: true— that is what pr-review: per-PR cancel-in-progress cancels racing runs (concurrency thrash) #534 fixed, and it kills in-flight reviews (a strictly worse failure than dropping a pending duplicate). The fix is upstream of the concurrency group: fewer, better-targeted triggers. scripts/pr_review_health.shalready enumerates pr-review run history and computes duration percentiles that are discarded rather than surfaced — [Phase 4] Capture the pre-rollout baseline for the three success metrics, including a new no-action-comment noise metric #1411 flagged this too. Surfacing the outcome mix and the percentiles together is a small, single change.- The idempotency marker (
<!-- pr-review-agent v1 sha=<HEAD> -->) is what makes a surviving-run-only outcome usually correct; AC Optimize review: small-PR and incremental fast paths #4 exists because "usually" is the current evidence standard and this story should raise it. - Relevant to the classification work:
pr-review-trigger.ymlis Class 1 with four webhook subscriptions; this is a concrete case where "subscribe to the narrowest event(s) that carry the signal" (§2 Class 1 rule) has practical teeth — the four subscriptions are the fan-in source.
Project Structure Notes
scripts/pr_review_health.sh (+ its report doc) for the measurement; .github/workflows/pr-review.yml for the concurrency comment and any keying change; .github/workflows/pr-review-trigger.yml if trigger reduction is the chosen path. Tests under tests/, registered in lint.yml's bats list.
References
.github/workflows/pr-review.yml:97-107(thecancel-in-progress: falserationale this corrects).github/workflows/pr-review-trigger.yml(the four trigger subscriptions)scripts/pr_review_health.sh(existing run-history collector; unsurfaced percentiles)- pr-review: per-PR cancel-in-progress cancels racing runs (concurrency thrash) #534 (the original ~34% cancel-rate fix, which addressed in-flight cancellation only)
- Observed on PR feat: implement issue #1405 — [Phase 2] Add the "How to add an agentic role" runbook and a PR-template interaction-contract section #1421 (
review / reviewreportedfail, run conclusioncancelled) docs/agentic-interaction-model.md§2 (Class 1: subscribe to the narrowest events that carry the signal)
Likely target surface
scripts/pr_review_health.sh,.github/workflows/pr-review.yml
Filed from overnight monitoring of epic #1402. Measured on the epic's own PR traffic: 49 cancelled / 34 success / 17 skipped over three hours.
Activity
github-actions commentedon Aug 2, 2026
Thanks for the thorough write-up and the measured data — having concrete cancellation counts split by triggering event makes this much easier to act on than a vague "things seem flaky" report. This is clearly a bug: the workflow comment asserts 'no cancelled-check noise' but the measurements show the opposite. Before we move to implementation, the key design decision is AC #3: of the three acceptable paths (debouncing/coalescing the four trigger types upstream, keying the concurrency group by head SHA with a fallback for payloads that lack one, or dropping provably-redundant trigger subscriptions), does the team have a preferred direction, or is the implementer free to choose based on what the before/after numbers support?
don-petry commentedon Aug 2, 2026
Severity escalation — cancelled checks are blocking merges, not just misleading
I filed this as misleading-noise. Live evidence shows it is the root cause of a merge deadlock.
PR #1421 has zero genuinely-failed checks. Queried
statusCheckRollupforconclusion == FAILURE: none — all non-pass are CANCELLED/SKIPPED. Yet the PR sits atBLOCKED, becausemainrequires checks via thecode-quality/pr-qualityrulesets and a cancelled required check never satisfies them.Confirmed on two separate runs:
gh pr checksrenders all three asfail.The causal chain, end to end:
BLOCKED— nothing re-runs them without a new eventpush: main→auto-rebasenever fires →DIRTYPRs stay dirty ([Phase 4] Conflicted-PR / queue-wide deadlock has no detection or resolution path — auto-rebase fires only on push:main, and #1410's stall detector keys on REVIEW_REQUIRED #1424)So #1422 and #1424 are one chain, not two coincidences. At the time of writing, the queue has been fully wedged for ~50 minutes with the last merge at 02:50Z.
This raises two ACs in priority:
Worth adding as a new AC: a cancelled required check must be automatically re-run or otherwise resolved, since today the only remedies are a human re-run or a new push.
Generated with Claude Code
don-petry commentedon Aug 2, 2026
Correction to my previous comment — the blocking mechanism is different
My last comment claimed cancelled required checks were leaving PRs
BLOCKED. That is wrong, and the correction changes where the fix should go.What the data actually shows on #1421. The required checks — per
repos/.../rules/branches/mainthese areSonarCloud,CodeQL,agent-shield / AgentShield,dependency-audit / Detect ecosystems— are all SUCCESS. The cancelled checks (dev-lead / dispatch,dev-lead / ci-relay,review / review,guard) are not in the required set, so they are not what blocks the merge.The real blocker is
REVIEW_REQUIRED:donpetry-botis the code-owner approver, and it has no live approval at the current head — its prior approvals showDISMISSED(stale-on-push after each dev-lead fix commit), and its latest markers show it withholding (status=rate-limited, advisory bots recovering; plus aci-pending-ack). CodeRabbit'sAPPROVEDdoes not satisfy the code-owner requirement.What still stands, and what does not:
failingh pr checksis genuinely misleading, and it misled me into the wrong diagnosis, which is a fair demonstration of the cost.I also attempted the unwedge from #1424 AC #5 — re-ran the cancelled
dev-leadandreviewruns — and it did not unblock the PR, consistent with this correction: re-running non-required checks cannot produce the missing code-owner approval. That is useful evidence for #1424's unwedge procedure: re-running cancelled checks is not a sufficient remedy.Generated with Claude Code
don-petry commentedon Aug 2, 2026
Live instance of the approve → push → dismiss cycle, observed end to end
The redirect in my correction above ("the merge-blocking mechanism to investigate is the approve → push → dismiss-stale → re-review cycle") just played out completely on PR #1421 within 25 minutes:
@donpetry-bot(the documented re-engage path)donpetry-botAPPROVED — code-owner approval finally obtained after ~2.5ha7b02171+d1063f6e, addressing a relayed review findingREVIEW_REQUIREDState right now: 0 unresolved threads, no genuine check failures,
mergeable. The only blocker is the absent code-owner approval — dismissed by the very commits that resolved the findings which had blocked it.Why this matters beyond one PR. Every review round costs a full approval re-acquisition, and the re-acquisition is not automatic: pr-review's re-review is subject to the advisory gate, the ci-pending skip, and (per this issue) ~49% run cancellation. So each round has a real probability of not producing a fresh approval, and the PR sits until something external nudges it. That is the convergence tax the epic set out to measure — and it compounds with review count rather than with change size.
Note the mechanism is GitHub's
dismiss_stale_reviews_on_push, which is desirable for human approvals: a human approving code should not have that approval survive arbitrary new commits. The question this issue should answer is whether the code-owner bot approval should be re-earned per push in the same way, or whether an approval that only ever gates on "the agent's own fix commits" is being dismissed for a risk that does not apply. I do not think that should be decided here without a security review — but it is the right question, and it belongs on the record.Generated with Claude Code
don-petry commentedon Aug 2, 2026
Maintainer decision — keep
dismiss_stale_reviews_on_push; narrow this issue's scopeThe approval-policy question raised earlier on this issue is settled: leave the behaviour as-is. A code-owner approval continues to be dismissed by every push, including dev-lead's own review-fix commits. No exemption will be scoped.
Rationale to record: an approval that survives arbitrary later commits is exactly the property the setting exists to prevent, and carving out "commits authored solely by dev-lead" would make the exemption's safety depend on correctly identifying the author of every commit — the same shared-identity problem that produced #1415 and #1417. The convergence cost is real but it is the safe side of the trade.
Therefore this issue is narrowed to the two mechanical defects, and the approval cycle becomes documented context rather than a work item:
cancel-in-progress: true— that is what pr-review: per-PR cancel-in-progress cancels racing runs (concurrency thrash) #534 fixed, and killing in-flight reviews is strictly worse than dropping a pending duplicate.cancel-in-progress: falsecomment block, which currently asserts "No lost work, no cancelled-check noise". The measurements contradict it: it protects the in-flight run, not the pending one.Dropped from scope: AC #6's approval-cycle question, and the earlier suggestion of auto-re-running cancelled required checks (already retracted — required checks were never the blocker).
The approve → push → dismiss cycle remains documented here as the accepted cost of the safe setting: convergence expense scales with review rounds rather than change size, which is an argument for fewer review rounds, not for a weaker gate.
Unblocking for implementation.
don-petry commentedon Aug 3, 2026
Dev-Lead Implementation Plan
Issue: #1422 — [Phase 4] pr-review runs are ~49% cancelled-while-pending
Scope
Add a run-outcome-by-event metric to the existing daily health report, drop the provably-redundant
pull_request_reviewtrigger (the largest measured cancellation source) to reduce fan-in, correct the misleadingcancel-in-progress:falsecomment, and pin the #1421 "cancelled-renders-as-fail" case as a regression fixture.Implementation Checklist
scripts/lib/pr-review-outcomes.sh(outcome-mix-by-event table + cancellation-rate + the 3-hour starting observation), sourced byscripts/pr_review_health.sh; add.eventto the run fetch and surface the table (with the already-computed p50/p95) in the deterministic report section.review / reviewconclusion=cancelled → not failing) as a regression fixture; document that first-party check-state classification (compute_ci_status, pr-review CI gate should gate on REQUIRED checks only — non-required CANCELLED *and* FAILED checks falsely block reviews #608/[Phase 1] Pin all callers off @main to the stable tag (consumer repos + self-review/dev duty) #497) already treats it non-failing and that the residual GitHub-native red is eliminated at the source by AC#3.pull_request_reviewsubscription from.github/workflows/pr-review-trigger.yml(provably a same-head-SHA no-op = pure churn; 21/49 = 43% of measured cancellations). Regeneratetests/fixtures/caller-stub-freeze/pr-review-trigger.block; updateinteraction-contracts/pr-review.ymlandpersonas/pr-review/interaction.ymlevent lists.cancel-in-progress:falsecomment in.github/workflows/pr-review.ymlto state it protects in-flight (yes) not pending (no); remove the false "no cancelled-check noise" claim.Tests to Write
tests/pr_review_outcomes.bats— outcome-mix-by-event counts, cancellation-rate, empty/malformed input, starting-observation note (registered inlint.yml).tests/test_ci_status.bats— explicit feat: implement issue #1405 — [Phase 2] Add the "How to add an agentic role" runbook and a PR-template interaction-contract section #1421 cancelledreview / reviewregression fixtures.Test Command
bash scripts/run-bats.sh tests/pr_review_outcomes.bats tests/test_ci_status.batsandbash .dev-lead/scripts/dev-lead-lint.shdon-petry commentedon Aug 3, 2026
Dev-Lead: Implementation Complete
Plan Execution
review / reviewconclusion=cancelled) pinned as a regression fixture; first-party check-state classification already treats it non-failing.pull_request_reviewsubscription from the ring-0 trigger stub.cancel-in-progress:falsecomment corrected (protects in-flight, not pending).How each mechanism works
AC#1 (measurement).
scripts/lib/pr-review-outcomes.sh(pure, unit-tested) renders a per-event table (success/cancelled/skipped/failure/total) + overall cancellation rate.scripts/pr_review_health.shnow fetches.eventper run and writes the table into the deterministic report section (alongside the already-computed p50/p95 that were previously discarded, #1411), before the model output so truncation can't drop it. The cancellation rate is now a tracked number.AC#3 (fan-in reduction) — justification. A
pull_request_reviewevent never changes the head SHA, so any review it triggers is a guaranteed idempotent no-op at the current head (the<!-- pr-review-agent v1 sha=<HEAD> -->marker short-circuits it) — pure churn that still consumes the per-PR concurrency slot. It was the single largest cancellation source: 21 of 49 (~43%) in the measured window. Removing the subscription eliminates those runs by construction with zero coverage loss — the signals that warrant a (re-)review, new code (pull_request: synchronize) and CI going green (check_suite: completed), remain subscribed. This is the §2 Class-1 "narrowest events that carry the signal" rule with teeth. Live before/after is now tracked by the AC#1 metric going forward. Per the Dev Notes,cancel-in-progress: truewas not re-enabled (that reverts #534 and kills in-flight reviews — strictly worse).AC#2 (cancelled ≠ fail).
compute_ci_status(scripts/lib/ci-status.sh) — the classifier every first-party automation keys on — already treats a cancelledreview / reviewas non-failing (own-check filter #497 + CANCELLED-non-blocking #608). The #1421 case is now pinned by explicit fixtures so it cannot silently regress. The residual GitHub-native red on rawgh pr checks(which we cannot re-render) is eliminated at the source by the AC#3 fan-in reduction.AC#4 (coverage, documented argument). For any push, ≥1 review reaches the final head SHA independent of which racing run survives, because: (1) the head-SHA marker means a survivor at a stale SHA doesn't count as reviewed; (2) a push emits
synchronizethencheck_suite:completed— either re-enters the slot at the final SHA; (3)pr-review-sweep.yml(≤15-min) backstops any un-eventable residue. A cancelled pending run self-heals; it never strands the PR.AC#6 (#1408 interaction). #1408 narrows the sweep cron but keeps the
workflow_runfast path. Droppingpull_request_reviewremoves a re-trigger path, but a pull_request_review-triggered review is a same-head no-op — nothing for the sweep to backstop there, so no compensating backstop is removed. The backstop that matters (a review cancelled while pending on a real push) is still covered by synchronize/check_suite + the sweep's retained fast path + un-eventable cron residue.Test Results
Files Changed
scripts/lib/pr-review-outcomes.sh(new): pure outcome-mix-by-event helpers + baseline note.tests/pr_review_outcomes.bats(new): unit tests for the above.scripts/pr_review_health.sh: source the lib, fetch.event, surface the table in the deterministic section..github/workflows/pr-review-trigger.yml: droppull_request_reviewsubscription (documented).tests/fixtures/caller-stub-freeze/pr-review-trigger.block: regenerated baseline for the on: change.interaction-contracts/pr-review.yml,personas/pr-review/interaction.yml: removepull_request_reviewfrom events..github/workflows/pr-review.yml: correct thecancel-in-progress:falsecomment; add coverage argument + [Phase 4] Narrow pr-review-sweep's cron scope to genuinely un-eventable cases and surface a sweep-hit-rate metric in daily health #1408 note.tests/test_ci_status.bats: feat: implement issue #1405 — [Phase 2] Add the "How to add an agentic role" runbook and a PR-template interaction-contract section #1421 regression fixtures..github/workflows/lint.yml: registertests/pr_review_outcomes.bats.Notes
pr-review.ymlkeeps itspull_request_reviewif:handling for consumer fleet callers that subscribe to it; only the ring-0 self-host trigger drops the subscription.don-petry commentedon Aug 3, 2026
Dev-Lead: cannot implement issue #1422 — needs human attention
A stage timed out (exit 124) while implementing this issue. A timeout is treated as a first-class, non-retryable condition — retrying with the same budget would just time out again — so this escalates for human review on the first occurrence instead of wasting a same-budget retry.
timeoutThis hit the max action budget. Split this issue into smaller, independently-implementable pieces, or raise
vars.ACTION_TIMEOUT_SEC, then re-apply thedev-leadlabel.don-petry commentedon Aug 12, 2026
Parked (maintainer decision 2026-08-11): labels removed to free this issue's in-flight slot on epic #1402. The 2700s timeout on attempt 1 (2026-08-03) suggests this story needs splitting before a retry can succeed — a same-budget retry would likely repeat it. Re-release via the initiative driver (or by re-applying
dev-lead) once the scope has been revisited.don-petry commentedon Aug 12, 2026
Dev-Lead Implementation Plan
Issue: #1422 — [Phase 4] pr-review runs are ~49% cancelled-while-pending
Scope
Surface the pr-review run-outcome mix (split by triggering event) on the existing health report; correct the misleading
cancel-in-progress: falserationale; reduce cross-SHA pending-eviction by keying the concurrency group on head SHA (the deferred refinement); and document the final-head-SHA coverage guarantee plus the #1408 sweep interaction. A cancelled run is proven non-failing in the surface our automation keys on, with the #1421 case pinned as a regression fixture.Implementation Checklist
scripts/lib/pr-review-outcomes.sh(new) — pure, sourceable helpers: outcome counts by triggering event + markdown renderer (mirrors thescripts/lib/*.shpure-lib pattern). (AC test issue from agent #1)scripts/pr_review_health.sh— fetch the runeventfield, source the lib, write the by-event outcome-mix section into the deterministic report block (alongside the already-surfaced p50/p95). (AC test issue from agent #1).github/workflows/pr-review.yml— correct thecancel-in-progress: falsecomment (in-flight protected, pending NOT); key the concurrency group by head SHA when the trigger payload carries one, falling back to the per-PR group for SHA-less triggers (workflow_dispatch/repository_dispatch). (AC Add @claude delegation, auto-merge, and rebase handling #3, feat: add Copilot engine support via REVIEW_ENGINE toggle #5)docs/pr-review-agent/concurrency-and-coverage.md(new) — documented argument that at least one review completes at the final head SHA per push; explicit statement of the [Phase 4] Narrow pr-review-sweep's cron scope to genuinely un-eventable cases and surface a sweep-hit-rate metric in daily health #1408 sweep-narrowing interaction. Linked from the corrected comment + documentation-index. (AC Optimize review: small-PR and incremental fast paths #4, feat: feature ideation framework — adversarial proposer/challenger/synthesizer #6)docs/metrics-baseline.md— record the dated 3-hour starting observation (49 cancelled / 34 success / 17 skipped; cancellations by event) as the tracked baseline. (AC test issue from agent #1).github/workflows/lint.yml— register the new bats file in the test list.Tests to Write
tests/pr_review_outcomes.bats(new) — by-event aggregation + renderer, including the feat: implement issue #1405 — [Phase 2] Add the "How to add an agentic role" runbook and a PR-template interaction-contract section #1421 cancelledpull_request_reviewrun counted ascancelled(notfailure), and empty/malformed-input guards.tests/test_ci_status.bats— add the feat: implement issue #1405 — [Phase 2] Add the "How to add an agentic role" runbook and a PR-template interaction-contract section #1421 regression case: a rollup whose ownreview / reviewcheck concludedcancelledclassifies as passing, never failing (the surface automation keys on).Test Command
scripts/run-bats.sh tests/pr_review_outcomes.bats tests/test_ci_status.bats(full suite vialint.yml), plusbash .dev-lead/scripts/dev-lead-lint.shNote on AC #2/#3 measurement
Live before/after cancellation numbers require post-merge run telemetry, which this environment cannot produce. The instrumentation added here (AC #1) is what makes that number tracked; the SHA-keying removes the dominant cross-SHA eviction path. Narrowing the four trigger subscriptions (the larger fan-in cut) is intentionally deferred because it is coupled to the #1408 sweep-backstop decision — documented under AC #6 rather than changing the byte-frozen ring-0 trigger stub blind.
don-petry commentedon Aug 12, 2026
✅ Issue #1422 — Phase 4 implementation complete
pr-review cancelled-while-pending runs addressed on three fronts: measure it, stop it surfacing as a failing check on our automation surface, and reduce the dominant cancellation path via per-head-SHA concurrency keying — with the coverage guarantee and the #1408 coupling written down.
Acceptance criteria
pr_review_health.shreport — no new cron. Pure aggregation extracted to a unit-tested lib. 2026-08-02 starting figures (49 cancelled / 34 success / 17 skipped) recorded.scripts/lib/pr-review-outcomes.sh,scripts/pr_review_health.sh,docs/metrics-baseline.mdcompute_ci_status(the automation surface our tooling keys on) already bucketsCANCELLEDas non-blocking; pinned with two #1421 regression fixtures (ownreview / reviewCANCELLED → passing; CANCELLED alongside external SUCCESS → passing).gh pr checks' own bucketing ofcancelledasfailis a platform behaviour code can't change — the remedy is AC#3 (fewer cancelled runs) + honest docs.tests/test_ci_status.bats<PR>-<head-sha>instead of<PR>, removing the dominant cross-SHA eviction path (a run for a new push evicted by churn on an older, stale SHA). SHA-less triggers (workflow_dispatchw/ onlypr_url, mention w/o sha) fall back to the exact prior per-PR group → strictly additive..github/workflows/pr-review.ymlS, every event forSmaps toG(S)and no event for a different SHA does;cancel-in-progress:falseprotects the running member, so at least one run reaches execution atSand the idempotency marker makes it a completion.docs/pr-review-agent/concurrency-and-coverage.mdcancel-in-progress:falsecomment block: in-flight PROTECTED, pending NOT protected; the old "no cancelled-check noise" claim was wrong (2026-08-02 measured ~49% cancelled). References #1421 / #534 / the new doc..github/workflows/pr-review.ymlci-pending/ci-failing → greenre-trigger (SHA keying does not backstop it — no run was ever started for the green state). Land+verify SHA-keyed coverage before #1408 removes any path. Dependency is one-directional.docs/pr-review-agent/concurrency-and-coverage.mdConstraints honoured
cancel-in-progress: falseretained — did not re-enable thetruethat pr-review: per-PR cancel-in-progress cancels racing runs (concurrency thrash) #534 fixed.pr-review-trigger.yml(byte-frozen ring-0 caller stub) untouched; only the non-frozen reusablepr-review.ymledited.Files changed (9)
New
scripts/lib/pr-review-outcomes.sh— pure by-event outcome helperstests/pr_review_outcomes.bats— 10 tests incl. feat: implement issue #1405 — [Phase 2] Add the "How to add an agentic role" runbook and a PR-template interaction-contract section #1421 regression casedocs/pr-review-agent/concurrency-and-coverage.md— coverage argument + [Phase 4] Narrow pr-review-sweep's cron scope to genuinely un-eventable cases and surface a sweep-hit-rate metric in daily health #1408 couplingModified
scripts/pr_review_health.sh— source lib, fetchevent, render outcome-mix section.github/workflows/pr-review.yml— head-SHA concurrency key + corrected comment (AC#3/feat: add Copilot engine support via REVIEW_ENGINE toggle #5)tests/test_ci_status.bats— two feat: implement issue #1405 — [Phase 2] Add the "How to add an agentic role" runbook and a PR-template interaction-contract section #1421 regression fixtures (AC#2)docs/metrics-baseline.md— 2026-08-02 starting observation + cancellations-by-event table (AC#1)docs/pr-review-agent/documentation-index.md— index link for the new doc.github/workflows/lint.yml— registerpr_review_outcomes.batsVerification
Measurement caveat (AC#3)
Live before/after cancellation numbers require post-merge run telemetry not available in this environment. The change removes the dominant cross-SHA eviction path structurally; the residual same-SHA cancellation (three+ events for one SHA in quick succession) is documented honestly and requires the trigger fan-in cut coupled to #1408 to eliminate fully. The new health-report table makes the rate a tracked number so the effect is observable post-merge.
feat: implement issue #1422 — [Phase 4] pr-review runs are ~49% cance…
chore: empty commit to refresh checks (no functional change)
feat: implement issue #1644 — [qa-lead S1] Provision <id>:hands-off l…