Skip to content

[Fleet Monitor] petry-projects/.github β€” .github/workflows/pr-auto-review-sweep.ymlΒ #947

Closed

Description

@don-petry

🟑 WARNING: petry-projects/.github / .github/workflows/pr-auto-review-sweep.yml

Metric Value
Failure Rate 11.1% (1 / 9 runs)
Status WARNING
p50 Duration 22s
p95 Duration 17456s
Successful Runs 8
Cancelled Runs 0

Workflow: .github/workflows/pr-auto-review-sweep.yml

Last updated by Fleet Monitor run on 2026-08-07.
Threshold: failure rate > 10% over the monitored window.

Activity

  1. don-petry commented on Aug 7, 2026

    @don-petry
    ContributorAuthor

    Dev-Lead Implementation Plan

    Issue: #947 β€” [Fleet Monitor] petry-projects/.github β€” .github/workflows/pr-auto-review-sweep.yml

    Scope

    Fault-isolate the catch-up sweep so a single transient GitHub API error on one PR no longer aborts the entire scheduled run (the source of the 11.1% failure rate).

    Root Cause

    sweep-dispatch.sh runs under set -euo pipefail. The repository_dispatch POST (gh api --method POST .../dispatches) inside the candidate loop is unguarded and executes with errexit active β€” a single transient failure (secondary rate limit / 5xx / network blip) aborts the whole run, fails the workflow, and strands the remaining candidates. The gh pr view / gh api graphql fact-gathering calls in evaluate_pr can also transiently fail and yield silently-wrong decisions. The sweep is idempotent and runs every 15 min, so a transient per-PR failure should be logged and retried next cycle, not fail the run.

    Implementation Checklist

    • .github/scripts/pr-auto-review/sweep-dispatch.sh β€” add a bounded-retry dispatch_review helper; on persistent failure log ::warning:: and continue (do not abort). Track/report failed dispatches.
    • .github/scripts/pr-auto-review/sweep-dispatch.sh β€” harden evaluate_pr fact-gathering: guard gh pr view / gh api graphql; on hard fetch failure emit a skip-error class and skip the PR this cycle instead of dispatching on garbage.
    • .github/scripts/pr-auto-review/README.md β€” document the resilience behaviour.

    Tests to Write

    • test/workflows/pr-auto-review/sweep-dispatch.bats β€” a transient dispatch failure on one PR does not abort the sweep; the other PR still dispatches; run exits 0; warning logged.
    • test/workflows/pr-auto-review/sweep-dispatch.bats β€” a transient gh pr view failure on one PR is skipped (skip-error) and the loop continues.
    • test/workflows/pr-auto-review/sweep-dispatch.bats β€” a dispatch that fails once then succeeds is retried (no lost dispatch, no failure).

    Test Command

    bats --print-output-on-failure test/workflows/pr-auto-review/ plus shellcheck -x .github/scripts/pr-auto-review/**/*.sh and bash .dev-lead/scripts/dev-lead-lint.sh

  2. don-petry commented on Aug 7, 2026

    @don-petry
    ContributorAuthor

    Dev-Lead: Implementation Complete

    Plan Execution

    • Root-caused the 11.1% failure rate: sweep-dispatch.sh runs under set -euo pipefail; the repository_dispatch POST inside the candidate loop was unguarded and ran with errexit active, so a single transient API error (secondary rate limit / 5xx / network blip) on one PR aborted the whole scheduled run and stranded the remaining candidates.
    • Added a bounded-retry dispatch_review helper; on persistent failure it logs a ::warning:: and the loop continues (the run still exits 0, since the sweep is idempotent and re-runs every 15 min).
    • Hardened evaluate_pr fact-gathering (gh pr view, review-threads GraphQL): a transient fetch failure now yields a skip-error class and skips that PR this cycle instead of dispatching on incomplete facts.
    • Documented the fault-isolation behaviour in the pr-auto-review README.
    • Wrote failing end-to-end tests first (TDD), then implemented until green.

    Test Results

    === bats (test/workflows/pr-auto-review/) ===
    ok 74 candidates: empty stdin emits nothing
    ok 75 candidates: non-array API error body emits nothing
    ok 76 candidates: an object missing isDraft is treated as non-draft
    ok 77 candidates: all drafts β†’ nothing
    77 tests, 0 failures   (73 pre-existing + 4 new sweep-dispatch resilience tests)
    
    === new sweep-dispatch.bats ===
    ok 1 sweep: both ready PRs dispatch and the run succeeds
    ok 2 sweep: a transient dispatch failure on one PR does not abort the run
    ok 3 sweep: a transient 'gh pr view' failure skips that PR and continues
    ok 4 sweep: a dispatch that fails once is retried and then succeeds
    
    === shellcheck -x (ready-check.sh, sweep.sh, sweep-dispatch.sh) ===
    shellcheck OK
    
    === .dev-lead/scripts/dev-lead-lint.sh ===
    [lint] all checks passed
    

    Files Changed

    • .github/scripts/pr-auto-review/sweep-dispatch.sh: bounded-retry dispatch_review + per-PR fault isolation so one transient error no longer aborts the sweep; new DISPATCH_RETRIES / DISPATCH_RETRY_SLEEP knobs; failed-dispatch count in the summary line.
    • test/workflows/pr-auto-review/sweep-dispatch.bats: new end-to-end resilience suite driving the orchestrator with a scripted gh stub.
    • .github/scripts/pr-auto-review/README.md: documents the per-PR fault-isolation behaviour ([Fleet Monitor] petry-projects/.github β€” .github/workflows/pr-auto-review-sweep.ymlΒ #947).

    Notes

  3. don-petry commented on Aug 10, 2026

    @don-petry
    ContributorAuthor

    βœ… Auto-resolved by Fleet Monitor. This workflow's failure rate has been below the 10% threshold (or the workflow was removed) since 2026-08-07 (3 days) β€” the monitor stopped flagging it. Closing. It will reopen automatically if the workflow crosses the threshold again.

  4. added a commit that references this issue on Aug 18, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions