Skip to content

PR Review Agent — failures detected 2026-11-13 #2523

Closed

Description

@github-actions

Convergence latency (deterministic)

Workflow duration across 1000 completed run(s) in the window — computed by pr_review_health.sh, not the model.

Workflow Runs min p50 p95 max
pr-review-trigger.yml 1000 1s 1m18s 5m24s 15m7s

Run-outcome mix by triggering event

pr-review run conclusions over the window, split by the event that triggered each run — computed deterministically by pr-review-outcomes.sh, not the model. A high cancelled share means runs were evicted while PENDING (concurrency admits one running + one pending per group); a cancelled run is non-blocking but historically rendered as a failing check (#1421).

Event success cancelled skipped failure total cancel %
check_suite 135 283 15 0 433 65%
pull_request 68 1 0 0 69 1%
pull_request_review 8 22 56 0 86 25%
repository_dispatch 82 26 0 0 108 24%
workflow_dispatch 293 9 0 2 304 2%
TOTAL 586 341 71 2 1000 34%

Sweep hit-rate (deterministic)

How often the pr-review-sweep scheduled backstop actually had to re-review a PR — computed deterministically by pr-review-sweep-metrics.sh, not the model. The scheduled (cron) path re-dispatches only for genuinely un-eventable cases (GitHub-App / cross-repo checks like SonarCloud that emit no workflow_run); the workflow_run fast path handles every eventable case, so a scheduled re-dispatch is always a case the fast path did not cover.

Metric Value
Scheduled sweep ticks 4
Ticks that re-dispatched a review (hits) 4
Sweep hit-rate 100%

Interpretation: a low rate confirms the timer stays exception-only (the event fast path is carrying the load); a rising rate is a signal that an eventable case is leaking onto the timer — a hole in the narrowing worth investigating.

1. Executive Summary

Status: WARNING
Period: 2026-11-12T23:48Z – 2026-11-13T13:30Z (last 1 day)
Result: 2 of 1000 runs failed (0.2%). 341 were cancelled (34.1%).

Key findings:

  • Real failures are negligible: only #111828 (12:13Z) and #111686 (09:49Z) failed. No failed-run logs were supplied, so the cause is unknown.
  • 34% of runs were cancelled. This is almost certainly concurrency cancellation of superseded runs, in bursts of 10–50 within seconds (e.g. 07:09–07:16Z, 04:05–04:20Z). It is probably not a fault.
  • About 7% of runs were skipped (the rest of the 1000). They are likely events where the reusable's gating found nothing to review.
  • Duration is healthy: p50 1m18s, p95 5m24s, max 15m7s.

Action required: Retrieve the logs for runs #111828 and #111686 to identify the failure cause, and confirm the cancellations come from intended concurrency groups.

2. Failure Breakdown

Failure Category Affected Runs Example Error Message
Other / unknown 2 (#111828, #111686) Not available: the "Failed Run Logs" section is empty
CLI breaking change 0 evidenced –
Permission / auth error 0 evidenced –
GitHub API rate limit 0 evidenced –
Missing token scope 0 evidenced –
Engine rate limit 0 evidenced –
Timeout / infrastructure 0 evidenced (max 15m7s, no timeout confirmed) –
Cancelled (not a failure) 341 No log. Likely superseded by a newer run (concurrency).

3. Error Patterns

  • No error messages are available. The failed-run log section is empty, so I can't quote any error, step or script. I won't guess a root cause.
  • Failure timing: #111828 failed at 12:13:13Z, one second before #111829 succeeded, in a burst that included five cancellations (#111823–#111827). #111686 failed at 09:49:47Z, between successful runs, and was followed by a recovery. Both look isolated and transient. Neither is part of a sustained outage.
  • Cancellation bursts: Many runs were cancelled within seconds of each other. Examples are ~50 runs at 07:09–07:16Z and ~70 runs at 04:05–04:20Z. A synchronize/check_suite storm, such as a mass rebase or a bulk push, would produce this pattern. Each new event cancels the earlier in-flight run. The trigger fires on check_suite, pull_request_review and pull_request, so one PR update can start several runs.
  • Skipped runs: These are scattered, and some cluster (e.g. 02:38–02:40Z, 01:10–01:14Z). They are consistent with job-level if: gating inside the reusable.

4. Token Scope Analysis

  • Scopes present (from the workflow source): contents: read, pull-requests: write, checks: read. secrets: inherit is also passed.
  • Missing or insufficient scopes: None are evidenced. There were no 401/403 errors, and no gh auth status output was supplied.
  • Recommendation: Make no change. If the two failures turn out to be 403s, check whether the reusable needs issues: write or statuses. The caller can only grant what the reusable asks for, so this must be confirmed from the logs first.

5. Recommendations

  1. [MEDIUM] Capture logs for the failed runs. Run gh run view 111828 --log-failed and gh run view 111686 --log-failed. Then fix the report generator so the "Failed Run Logs" section is populated. Impact: lets future reports diagnose failures instead of guessing.
  2. [LOW] Verify the cancellation behaviour. Check that pr-review.yml@pr-review/v1-next uses a concurrency group keyed per PR with cancel-in-progress: true. Confirm that the cancelled runs were superseded and not lost reviews, by checking that each PR's latest head SHA has a successful run. Impact: confirms that 34% cancellation is intended and not wasted compute.
  3. [LOW] Reduce trigger fan-out. If one PR update starts multiple runs through check_suite, pull_request and pull_request_review, consider narrowing the events or filtering earlier. Do this in the reusable's gating, not in this stub, which is a thin caller. Impact: fewer cancelled and skipped runs and lower Actions minutes.
  4. [LOW] Watch the 15m7s maximum. Check whether the slowest runs were deep or security review passes or were hung. Add or confirm a timeout-minutes limit in the reusable if none exists. Impact: bounds the worst case.

6. Health Score

Health: 9/10 — Real failures are 0.2% and look transient, but the failed-run logs are missing and the 34% cancellation rate should be confirmed as intentional.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    automated-reportCreated by automated workflowhealth-checkAutomated health check report

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions