Skip to content

PR Review Agent — failures detected 2026-10-15 #2335

Closed

Description

@github-actions

Convergence latency (deterministic)

Workflow duration across 446 completed run(s) in the window — computed by pr_review_health.sh, not the model.

Workflow Runs min p50 p95 max
pr-review-trigger.yml 446 1s 59s 8m40s 21m18s

Run-outcome mix by triggering event

pr-review run conclusions over the window, split by the event that triggered each run — computed deterministically by pr-review-outcomes.sh, not the model. A high cancelled share means runs were evicted while PENDING (concurrency admits one running + one pending per group); a cancelled run is non-blocking but historically rendered as a failing check (#1421).

Event success cancelled skipped failure total cancel %
check_suite 57 28 13 2 100 28%
pull_request 17 2 0 0 19 10%
pull_request_review 7 1 28 0 36 2%
repository_dispatch 143 16 0 0 159 10%
workflow_dispatch 129 2 0 1 132 1%
TOTAL 353 49 41 3 446 10%

Sweep hit-rate (deterministic)

How often the pr-review-sweep scheduled backstop actually had to re-review a PR — computed deterministically by pr-review-sweep-metrics.sh, not the model. The scheduled (cron) path re-dispatches only for genuinely un-eventable cases (GitHub-App / cross-repo checks like SonarCloud that emit no workflow_run); the workflow_run fast path handles every eventable case, so a scheduled re-dispatch is always a case the fast path did not cover.

Metric Value
Scheduled sweep ticks 5
Ticks that re-dispatched a review (hits) 5
Sweep hit-rate 100%

Interpretation: a low rate confirms the timer stays exception-only (the event fast path is carrying the load); a rising rate is a signal that an eventable case is leaking onto the timer — a hole in the narrowing worth investigating.

1. Executive Summary

Status: WARNING
Period: 2026-10-14 14:57Z → 2026-10-15 09:56Z (last 1 day)
Result: 3 of 446 runs failed (0.7%); a further 49 were cancelled (11.0%)

Key findings:

  • All 3 failures (#103286, #103287, #103288) happened within 23 seconds at 2026-10-14T15:41:59–15:42:21Z. No later run failed, and 353 runs succeeded.
  • The "Failed Run Logs" section is empty, so the root cause of the 3 failures can't be determined from this data.
  • 49 cancellations and about 44 skips come in bursts (for example 15:00–15:21Z on 10-14). That looks like concurrency-group cancellation when several events hit the same PR, not an error.

Action required: Re-fetch the logs for runs #103286–#103288 and confirm the cause. Until then, treat it as a one-off transient failure.

2. Failure Breakdown

Failure Category Affected Runs Example Error Message
Other / unknown (no logs captured) 3 (#103286, #103287, #103288) Not available. The failed-run log section was empty.
CLI breaking change 0 n/a
Permission / auth error 0 observed n/a
GitHub API rate limit 0 observed n/a
Missing token scope 0 observed n/a
Engine rate limit 0 observed n/a
Timeout / infrastructure 0 confirmed n/a. Max run duration was 21m18s, which may be near a job timeout.
Cancelled (concurrency or superseded) 49 n/a. This is not a failure.

3. Error Patterns

  • Unknown failures (#103286–#103288): No log text was provided, so I can't quote an error message, step or script.
    • The three runs were 9–23 seconds apart and followed cancelled run #103285 (15:41:58Z). Run #103284 succeeded just before them, and #103289 succeeded just after.
    • The tight timing points to one shared trigger, such as a burst of events on one PR or a brief GitHub or runner hiccup. A deterministic code or config bug is less likely, because every run since has passed.
    • This is an inference, not something the logs confirm.
  • Cancellations: These cluster in bursts of rapid events, such as the 15:00–15:21Z window on 10-14. They are consistent with concurrency cancel-in-progress on the reusable workflow. The caller stub defines no concurrency block, so the cancelling is probably set inside pr-review.yml. I haven't verified that.
  • Skips: The skipped runs are probably check_suite or pull_request_review events that the reusable's gating if conditions filtered out. This is expected.

4. Token Scope Analysis

  • Scopes present (from the workflow permissions block): contents: read, pull-requests: write, checks: read. No gh auth status output was in the logs.
  • Missing or insufficient: None indicated. No 401 or 403 errors appear, and the success rate was 79% overall and 100% after the failure cluster.
  • Recommendation: Make no scope changes. If the re-fetched logs show a 403, check whether secrets: inherit supplies the token that engine or gh calls need. Don't widen scopes without evidence.

5. Recommendations

  1. [MEDIUM] Capture logs for the three failed runs.
    • What to change: run gh run view 103286 --log-failed -R combsmith/.github-private (and the same for 103287 and 103288). Also check why the report generator received no failed-run logs, in case the log fetch is silently failing.
    • Why: the cause is currently unknown.
    • Expected impact: confirms whether the failures were transient or a real defect.
  2. [LOW] Review the cancellation rate.
    • What to change: check the concurrency group and cancel-in-progress settings in pr-review.yml at pr-review/v1-next.
    • Why: 11% of runs were cancelled, so some review work is wasted. If the group key is too coarse (for example per-workflow instead of per-PR), unrelated PRs may be cancelling each other.
    • Expected impact: fewer wasted runs and fewer missed reviews.
  3. [LOW] Watch the long tail of run duration.
    • What to change: compare the p95 of 8m40s and the max of 21m18s against the job timeout-minutes.
    • Why: runs close to the timeout would show up later as unexplained failures.
    • Expected impact: an early warning of timeout-related failures.
  4. [LOW] Only if the failures recur: hold off on promoting pr-review/v1-next to v1-stable until the cause is understood. This repo is the ring-0 canary, so a regression here should block promotion.

6. Health Score

Health: 9/10 — 99.3% of runs succeeded or were skipped or cancelled by design, the 3 failures came in one 23-second burst with no recurrence, and their cause is unknown because no logs were captured.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    automated-reportCreated by automated workflowhealth-checkAutomated health check report

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions