Skip to content

PR Review Agent — failures detected 2026-11-15 #2548

Closed

Description

@github-actions

Convergence latency (deterministic)

Workflow duration across 495 completed run(s) in the window — computed by pr_review_health.sh, not the model.

Workflow Runs min p50 p95 max
pr-review-trigger.yml 495 1s 55s 4m7s 8m13s

Run-outcome mix by triggering event

pr-review run conclusions over the window, split by the event that triggered each run — computed deterministically by pr-review-outcomes.sh, not the model. A high cancelled share means runs were evicted while PENDING (concurrency admits one running + one pending per group); a cancelled run is non-blocking but historically rendered as a failing check (#1421).

Event success cancelled skipped failure total cancel %
check_suite 86 189 15 2 292 64%
pull_request 49 0 0 0 49 0%
pull_request_review 5 15 39 0 59 25%
repository_dispatch 67 8 0 0 75 10%
workflow_dispatch 14 0 0 6 20 0%
TOTAL 221 212 54 8 495 42%

Sweep hit-rate (deterministic)

How often the pr-review-sweep scheduled backstop actually had to re-review a PR — computed deterministically by pr-review-sweep-metrics.sh, not the model. The scheduled (cron) path re-dispatches only for genuinely un-eventable cases (GitHub-App / cross-repo checks like SonarCloud that emit no workflow_run); the workflow_run fast path handles every eventable case, so a scheduled re-dispatch is always a case the fast path did not cover.

Metric Value
Scheduled sweep ticks 4
Ticks that re-dispatched a review (hits) 4
Sweep hit-rate 100%

Interpretation: a low rate confirms the timer stays exception-only (the event fast path is carrying the load); a rising rate is a signal that an eventable case is leaking onto the timer — a hole in the narrowing worth investigating.

1. Executive Summary

Status: WARNING
Period: 2026-11-14 13:38Z to 2026-11-15 11:17Z (last 1 day)
Result: 8 of 495 runs failed (1.6%). Excluding 212 cancelled and 54 skipped runs, that is 8 of 229 executed runs (3.5%).

Key findings:

  • The failed-run log section was empty, so no error text is available and the root cause cannot be determined from this input.
  • 5 of the 8 failures cluster in a 25-minute window on 11-14 (20:27–20:52Z: #113324, #113326, #113327, #113328, #113330). Runs #113329 and #113331 succeeded immediately before and after, so this was likely transient or tied to specific PRs.
  • The other 3 failures (#113160, #113443, #113471) are isolated. Each sat between two successes within about a minute (for example #113442 success, #113443 failure, #113444 success).
  • 212 cancellations (43%) are probably concurrency-group supersession during bursts of synchronize and check_suite events (for example 40+ runs between 20:58 and 21:02Z). They are likely benign, but they are not confirmed.
  • Run duration is healthy: p50 is 55s and p95 is 4m7s. The 8m13s maximum is a one-off.

Action required: Re-run the log fetch so failed-job logs are captured, then inspect #113324–#113330 first.

2. Failure Breakdown

Failure Category Affected Runs Example Error Message
Unknown (no logs supplied) 8: #113471, #113443, #113330, #113328, #113327, #113326, #113324, #113160 Not available. The "Failed Run Logs" section was empty.
Cancelled (likely concurrency supersession, not a failure) 212 No log. Cancellation was inferred from event bursts.
Skipped (likely gating conditions in the reusable) 54 No log.

No category can be confirmed. I did not identify any CLI breaking change, auth error, rate limit, scope problem or timeout, because there is no evidence for or against them.

3. Error Patterns

  • No exact error messages are available. I can't name the step or script behind any failure, and I won't guess at one.
  • Observed timing pattern:
    • The #113324–#113330 cluster interleaves failures with successes (#113323, #113329 success), so it is not a total outage. It suggests a per-PR or per-event cause, or a transient dependency. Examples are an API error, an engine quota limit or a bad head SHA.
    • The three isolated failures each follow a success by about 10 seconds. That points to a second event on the same PR, such as check_suite right after pull_request, hitting a race or a state the first run did not.
  • Workflow source observations:
    • The trigger is a thin stub pinned to pr-review.yml@pr-review/v1-next, the ring-0 canary channel. A regression in the next channel would show up here first. That would fit a cluster of failures, but it is only a hypothesis.
    • All substantive logic and failures live in the reusable and its scripts (engine.sh, review-one-pr.sh), not in this stub.

4. Token Scope Analysis

  • Scopes present (from the workflow permissions block): contents: read, pull-requests: write, checks: read. secrets: inherit passes the caller's secrets through.
  • No gh auth status output was supplied, so actual token scopes could not be checked.
  • Scopes that appear missing: none can be shown from the evidence. No 401 or 403 errors were provided.
  • Recommendation: when logs are re-collected, grep for 403, Resource not accessible by integration and read:org. If found, add only the specific permission needed, such as issues: write for comments, rather than broadening permissions.

5. Recommendations

  1. [HIGH] Fix log capture for the health report.
    • What to change: the log-fetch step that feeds this analysis. It should use gh run view <id> --log-failed for run IDs 113324, 113326, 113327, 113328, 113330, 113160, 113443 and 113471.
    • Why: the report is blind without failure text.
    • Impact: it enables root-cause classification on the next report.
  2. [MEDIUM] Investigate the 11-14 20:27–20:52Z cluster.
    • What to change: compare the PRs and events behind #113324–#113330 with the channel tag pr-review/v1-next at that time. Check whether the tag moved.
    • Why: five failures in 25 minutes is the strongest signal of a real defect or a transient dependency problem.
    • Impact: it confirms or rules out a regression in the next channel. If confirmed, roll next back before promotion to stable.
  3. [MEDIUM] Check the three isolated failures for a same-PR race.
    • What to change: look at the concurrency settings in pr-review.yml, and check whether check_suite and pull_request events collide.
    • Why: all three failures were adjacent to successful runs on probable sibling events.
    • Impact: it could remove the roughly 1% baseline failure rate.
  4. [LOW] Reduce cancellation noise.
    • What to change: filter check_suite events earlier, for example by checking at job level that the event is relevant, and keep concurrency cancel-in-progress for the same PR.
    • Why: 43% of runs are cancelled, which wastes runner minutes and clutters the run history.
    • Impact: fewer runs and a clearer health signal.
  5. [LOW] Report cancelled and skipped runs separately from failures in the health computation.
    • Why: the 1.6% figure understates the executed-run failure rate of 3.5%.
    • Impact: more accurate status thresholds.

None are CRITICAL: the workflow is not failing 100% of the time.

6. Health Score

Health: 8/10 — Most executed runs succeed (221 of 229) and failures are intermittent, but the 5-failure cluster is unexplained and the missing logs prevent root-causing it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    automated-reportCreated by automated workflowhealth-checkAutomated health check report

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions