Skip to content

feat: implement issue #947 — [Fleet Monitor] petry-projects/.github — .github/workflows/pr-auto-review-sweep.yml - #950

Merged
don-petry merged 13 commits into
mainfrom
dev-lead/issue-947-20260807-0756
Aug 18, 2026
Merged

don-petry merged 13 commits into
mainfrom
dev-lead/issue-947-20260807-0756

Conversation

@don-petry

@don-petry don-petry commented Aug 7, 2026 •

Copy link
Copy Markdown
Contributor

User description

Closes #947

Implemented by dev-lead agent. Please review.

Summary by CodeRabbit

  • Reliability

    • Catch-up sweeps now isolate failures per pull request instead of stopping entirely.
    • Failed metadata lookups and empty results are skipped and retried in later scheduled cycles.
    • Dispatch attempts retry within configurable limits and delays, with warnings after exhausted retries.
    • Completion reporting now identifies dispatch failures while allowing successful work to continue.
  • Documentation

    • Added guidance on per-pull-request fault isolation and retry behavior.
  • Tests

    • Added coverage for successful dispatches, retries, fetch failures, and missing pull-request data.

CodeAnt-AI Description

Keep scheduled pull request review sweeps running when individual checks or dispatches fail

What Changed

  • A PR with unavailable metadata, deleted data, incomplete review-thread results, or a temporary API failure is skipped for the current cycle instead of stopping the sweep or being dispatched on unreliable information.
  • Review dispatches retry a configurable number of times; persistent failures are logged and deferred to the next scheduled run while other PRs continue.
  • Added end-to-end coverage and usage guidance for retries, fault isolation, and recovery.

Impact

✅ Fewer failed scheduled review sweeps
✅ Fewer pull requests stranded by transient GitHub API errors
✅ Safer handling of deleted or incomplete PR data

💡 Usage Guide

Checking Your Pull Request

Every time you make a pull request, our system automatically looks through it. We check for security issues, mistakes in how you're setting up your infrastructure, and common code problems. We do this to make sure your changes are solid and won't cause any trouble later.

Talking to CodeAnt AI

Got a question or need a hand with something in your pull request? You can easily get in touch with CodeAnt AI right here. Just type the following in a comment on your pull request, and replace "Your question here" with whatever you want to ask:

@codeant-ai ask: Your question here

This lets you have a chat with CodeAnt AI about your pull request, making it easier to understand and improve your code.

Example

@codeant-ai ask: Can you suggest a safer alternative to storing this secret?

Preserve Org Learnings with CodeAnt

You can record team preferences so CodeAnt AI applies them in future reviews. Reply directly to the specific CodeAnt AI suggestion (in the same thread) and replace "Your feedback here" with your input:

@codeant-ai: Your feedback here

This helps CodeAnt AI learn and adapt to your team's coding style and standards.

Example

@codeant-ai: Do not flag unused imports.

Retrigger review

Ask CodeAnt AI to review the PR again, by typing:

@codeant-ai: review

Check Your Repository Health

To analyze the health of your code repository, visit our dashboard at https://app.codeant.ai. This tool helps you identify potential issues and areas for improvement in your codebase, ensuring your repository maintains high standards of code health.

… .github/workflows/pr-auto-review-sweep.yml
@don-petry
don-petry requested a review from a team as a code owner August 7, 2026 08:06
@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

@codeant-ai

codeant-ai Bot commented Aug 7, 2026 •

Copy link
Copy Markdown

🤖 CodeAnt AI — Review Status

Status Commit Started (UTC) Finished (UTC)
✅ Incremental review completed 60ea743 Aug 18, 2026 · 16:51 16:52
✅ Incremental review completed 65328c4 Aug 11, 2026 · 19:09 19:10
✅ Incremental review completed 1821c49 Aug 09, 2026 · 08:39 08:39
✅ Reviewed your PR 8002033 Aug 07, 2026 · 08:06 08:08

@codeant-ai

codeant-ai Bot commented Aug 7, 2026

Copy link
Copy Markdown

Thanks for using CodeAnt! 🎉

We're free for open-source projects. if you're enjoying it, help us grow by sharing.

Share on X ·
Reddit ·
LinkedIn

@coderabbitai

coderabbitai Bot commented Aug 7, 2026 •

Copy link
Copy Markdown

Review Change Stack

📝 Walkthrough

Walkthrough

The sweep now isolates PR data and dispatch failures. It skips invalid candidates, retries dispatches with configurable limits, tracks exhausted failures, and continues processing. New Bats coverage validates successful dispatch, retry recovery, skipped candidates, and persistent failures.

Changes

Per-PR sweep fault isolation

Layer / File(s) Summary
Candidate data validation
.github/scripts/pr-auto-review/sweep-dispatch.sh, test/workflows/pr-auto-review/sweep-dispatch.bats
PR metadata and review-thread requests now skip candidates when commands fail, responses are empty, or GraphQL data is null.
Dispatch retry and completion handling
.github/scripts/pr-auto-review/sweep-dispatch.sh, test/workflows/pr-auto-review/sweep-dispatch.bats
Dispatches retry with configurable attempts and delays. Exhausted failures are counted and reported without stopping other candidates.
Behavior documentation and test harness
.github/scripts/pr-auto-review/README.md, test/workflows/pr-auto-review/sweep-dispatch.bats
The README documents skip and retry behavior. Tests provide configurable gh responses and validate the sweep outcomes.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Sequence Diagram(s)

sequenceDiagram
  participant sweep-dispatch.sh
  participant gh
  participant GitHub API
  sweep-dispatch.sh->>gh: Fetch PR metadata
  gh-->>sweep-dispatch.sh: Return metadata or failure
  sweep-dispatch.sh->>GitHub API: Fetch review threads
  GitHub API-->>sweep-dispatch.sh: Return thread data or null data
  sweep-dispatch.sh->>gh: Dispatch review with bounded retries
  gh-->>sweep-dispatch.sh: Return dispatch result
  sweep-dispatch.sh-->>sweep-dispatch.sh: Continue sweep and report failures
Loading

Possibly related PRs

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Linked Issues check ✅ Passed The changes directly address issue #947 by improving sweep reliability through failure isolation, retries, logging, and continued execution.
Out of Scope Changes check ✅ Passed The script changes, tests, and documentation all support the linked issue’s objective to improve workflow reliability.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title identifies issue #947 and the affected PR auto-review sweep workflow, which matches the main changes.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch dev-lead/issue-947-20260807-0756

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@qodo-code-review

Copy link
Copy Markdown

ⓘ Qodo reviews are paused because your trial has ended. Ask your workspace admin to add credits to resume reviews. Manage billing

@codeant-ai codeant-ai Bot added the size:L This PR changes 100-499 lines, ignoring generated files label Aug 7, 2026

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request implements per-PR fault isolation and retry mechanisms in the sweep-dispatch.sh script to prevent transient GitHub API errors from failing the entire scheduled workflow. It introduces configurable dispatch retries, handles API fetch failures gracefully by skipping the affected PR, and adds comprehensive end-to-end BATS tests. The reviewer feedback recommends improving the BATS tests by replacing logical negation on grep with explicit exit status checks ([ "$status" -eq 1 ]) to avoid false positives from unrelated errors.

@don-petry
don-petry enabled auto-merge (squash) August 7, 2026 08:08
@don-petry
don-petry disabled auto-merge August 7, 2026 08:08
donpetry-bot
donpetry-bot previously approved these changes Aug 7, 2026

@donpetry-bot donpetry-bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review — APPROVED ✓

Risk: MEDIUM
Reviewed commit: 8002033f76a43ff5ff464c536695c74dfd45a10e
Review mode: triage-approved (single reviewer)

Summary

Adds per-PR fault isolation and bounded dispatch retries to sweep-dispatch.sh so one flaky GitHub API call no longer aborts the whole scheduled sweep under set -euo pipefail. Fact-gathering failures (gh pr view, review-threads GraphQL) now emit a skip-error class and skip only that PR; the repository_dispatch POST retries up to DISPATCH_RETRIES (default 3) and logs a ::warning:: on exhaustion while the run still exits 0. New bats suite covers the happy path, isolated fetch/dispatch failures, and retry recovery; README documents the behavior. Triage's low-risk assessment is confirmed — the change is defensive error handling in org-private orchestration, no new permissions, secrets, or injection surfaces.

Linked issue analysis

Closes #947 (Fleet Monitor WARNING: pr-auto-review-sweep.yml failure rate 11.1%). Root cause — an unguarded dispatch POST inside the candidate loop with errexit active — is directly fixed. The "deferral, not lost dispatch" rationale is sound because the sweep is idempotent and re-runs every 15 minutes. Substantively addressed.

Findings

No blocking findings. Verified the outstanding bot-review threads (all non-blocking by org convention — human-authored threads only gate readiness):

  1. codeant "Major" — duplicate dispatch on ambiguous network failure: mitigated. The consumer (pr-review-trigger.yml → repository_dispatch: pr-review-mention) enforces head-SHA idempotency (force_review exists precisely to bypass it), so a double-fired dispatch at the same SHA no-ops. Worst case is a redundant workflow run, strictly better than the prior whole-sweep failure.
  2. codeant — GraphQL errors array with non-empty body treated as zero blocking threads: largely moot; gh api graphql exits non-zero when the response carries GraphQL errors, which the new guard catches. Residual partial-data edge is pre-existing behavior.
  3. gemini (low, x2) — ! echo "$output" | grep -q absence assertions in bats: valid nit. Negated non-final commands are errexit-exempt in bats, so the line-130 absence assertion is effectively a no-op; the adjacent dispatched 1 of 2 count assertion covers the same behavior, so test intent still holds. Suggested follow-up (non-blocking): switch to run ! grep … (bats ≥1.5 is already required) or explicit [ "$status" -eq 1 ].

Minor doc nit (non-blocking): README says "retried up to 3 times" — DISPATCH_RETRIES=3 is 3 total attempts (2 retries).

Secret scan: the run_secret_scanning MCP tool was unavailable in this run; the gitleaks CI check passed and the diff contains no credentials (test stub uses a dummy token).

CI status

All validation checks green: ShellCheck ✓, Lint ✓, PR Auto-Review Tests (lint + bats, includes the new suite) ✓, CodeQL ✓, Secret scan (gitleaks) ✓, Agent Security Scan ✓, AgentShield ✓, SonarCloud quality gate ✓, dependency audits ✓/skipped. The three CANCELLED Dev-Lead Agent runs are the authoring agent's own orchestration jobs superseded by newer events — not code-validation checks. mergeStateStatus: BLOCKED reflects only the pending review requirement this review satisfies.


Reviewed automatically by the PR-review agent (single-reviewer mode: fable 5). Reply if you need a human review.

@don-petry

Copy link
Copy Markdown
Contributor Author

Dev-Lead — fix-reviews (applied)

Changes committed and pushed.

@don-petry
don-petry enabled auto-merge (squash) August 7, 2026 08:13
@don-petry
don-petry disabled auto-merge August 7, 2026 08:15
@donpetry-bot

Copy link
Copy Markdown
Contributor

CI checks on this PR are still running. Once they complete, re-mention @donpetry-bot to trigger a fresh review.

Posted by the donpetry-bot PR-review cascade.

@don-petry

Copy link
Copy Markdown
Contributor Author

Dev-Lead — review-changes (applied)

Changes committed and pushed.

@don-petry
don-petry enabled auto-merge (squash) August 7, 2026 08:24
@don-petry
don-petry disabled auto-merge August 7, 2026 08:25
@donpetry-bot

Copy link
Copy Markdown
Contributor

CI checks on this PR are still running. Once they complete, re-mention @donpetry-bot to trigger a fresh review.

Posted by the donpetry-bot PR-review cascade.

@don-petry

Copy link
Copy Markdown
Contributor Author

Dev-Lead — waiting on PR blockers (intent: review-changes)

PR: #950
No changes were committed, but the PR still has blocking checks or reviews (failing or cancelled checks, or changes-requested reviews). The retry cron will re-attempt automatically. Next attempt after: 2026-08-07T08:57:38Z

@don-petry

Copy link
Copy Markdown
Contributor Author

Note

@don-petry I reviewed this PR and no code changes were needed, but it still has blocking checks or reviews (failing or cancelled checks, or changes-requested reviews), so I cannot mark it done yet. I'll re-check automatically.
Next attempt after: 2026-08-07T08:57:38Z

@don-petry
don-petry enabled auto-merge (squash) August 7, 2026 08:27
@don-petry
don-petry disabled auto-merge August 7, 2026 08:30
@don-petry
don-petry enabled auto-merge (squash) August 7, 2026 08:33
@codeant-ai

codeant-ai Bot commented Aug 11, 2026

Copy link
Copy Markdown

Thanks for using CodeAnt! 🎉

We're free for open-source projects. if you're enjoying it, help us grow by sharing.

Share on X ·
Reddit ·
LinkedIn

@codeant-ai codeant-ai Bot added size:L This PR changes 100-499 lines, ignoring generated files and removed size:L This PR changes 100-499 lines, ignoring generated files labels Aug 11, 2026
@don-petry
don-petry disabled auto-merge August 11, 2026 19:10
@don-petry

Copy link
Copy Markdown
Contributor Author

Dev-Lead — review-changes (applied)

Changes committed and pushed.

@don-petry
don-petry enabled auto-merge (squash) August 11, 2026 19:23
@don-petry
don-petry disabled auto-merge August 11, 2026 19:23
@don-petry

Copy link
Copy Markdown
Contributor Author

Dev-Lead — waiting on PR blockers (intent: review-changes)

PR: #950
No changes were committed, but the PR still has blocking checks or reviews (failing or cancelled checks, or changes-requested reviews). The retry cron will re-attempt automatically. Next attempt after: 2026-08-11T19:56:06Z

@don-petry

Copy link
Copy Markdown
Contributor Author

Note

@don-petry I reviewed this PR and no code changes were needed, but it still has blocking checks or reviews (failing or cancelled checks, or changes-requested reviews), so I cannot mark it done yet. I'll re-check automatically.
Next attempt after: 2026-08-11T19:56:06Z

@don-petry
don-petry enabled auto-merge (squash) August 11, 2026 19:26

@donpetry-bot donpetry-bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review — APPROVED ✓

Risk: MEDIUM
Reviewed commit: 194ee9eabfd8f1c58b195e167d52c04b34a7c364
Review mode: triage-approved (single reviewer)

Summary

Confirms the triage assessment: adds per-PR fault isolation and bounded dispatch retries to the pr-auto-review sweep so one transient GitHub API failure no longer aborts the whole scheduled run (issue #947). Guards cover failed/empty 'gh pr view', GraphQL null pullRequest, GraphQL errors array, and null reviewThreads; dispatches retry up to DISPATCH_RETRIES with a stable dispatch_id for dedup. 7 new bats tests pin the behavior. All required CI checks green; latest CodeRabbit review is APPROVED.

Linked issue analysis

Issue #947 is a Fleet Monitor WARNING for pr-auto-review-sweep.yml (11.1% failure rate). Root cause: unguarded gh calls inside the candidate loop under set -euo pipefail meant a single transient API error failed the workflow and stranded remaining PRs. The PR isolates faults per PR (skip-error class, skip this cycle, retried next scheduled run) and retries the repository_dispatch POST with bounded, configurable retries. This directly and substantively addresses the failure-rate warning. The sweep's idempotent re-run-every-cycle design makes skip-and-defer safe.

Findings

  • Risk classified MEDIUM: non-trivial error-handling logic in an org automation shell script; no auth/secret/workflow-security surface changed.
  • One review thread (codeant-ai, Major: validate GraphQL errors/null data before counting threads) is technically unresolved on GitHub, but the fix is present in this diff (three guards at sweep-dispatch.sh plus matching bats tests) and was verified in-thread three times; latest CodeRabbit review on this SHA is APPROVED. Treated as resolved in substance.
  • set -e safety verified: all newly-failing commands are tested inside if conditions; arithmetic assignments are errexit-safe; retry loop sleeps only between attempts.
  • DISPATCH_RETRIES / DISPATCH_RETRY_SLEEP are validated as positive integer / non-negative number before use.
  • Minor (non-blocking): 2>/dev/null on gh calls hides root-cause API error text (warnings still emitted); RANDOM-based dispatch_id is weakly unique but adequate as a dedup hint; node_modules/.package-lock.json is committed (pre-existing repo pattern — only the name field changed pr-940→pr-950).
  • Secret scan: run_secret_scanning MCP tool unavailable in this environment; gitleaks CI check on this SHA is green. No secrets or credentials in the diff.

CI status

All required checks green on 194ee9e: SonarCloud ✓, CodeQL ✓, agent-shield / AgentShield ✓, dependency-audit / Detect ecosystems ✓. Also green: Lint, ShellCheck, Lint and bats, Secret scan (gitleaks), Agent Security Scan, npm audit, CodeRabbit, Graphite AI Reviews. Two 'dev-lead' orchestration jobs show CANCELLED — these are the dev-lead agent's own dispatch/relay runs cancelled by concurrency (a later run at 19:23:13Z succeeded) and are not required checks.


Reviewed automatically by the PR-review agent (single-reviewer mode: fable 5). Reply if you need a human review.

@don-petry
don-petry disabled auto-merge August 11, 2026 20:08
@don-petry
don-petry enabled auto-merge (squash) August 11, 2026 20:13
@don-petry
don-petry disabled auto-merge August 11, 2026 20:13
@don-petry

Copy link
Copy Markdown
Contributor Author

Dev-Lead — fix-bot-comment (no-changes)

Agent reasoning
- Review states: CodeRabbit **APPROVED**, donpetry-bot **APPROVED**
- No `CHANGES_REQUESTED` reviews
- No failed/timed-out checks
---
## Summary
**Bot:** SonarCloud  
**Issues addressed:** 0  
**Skipped (informational):** 0  
The SonarCloud Quality Gate has **passed** with **zero new issues** detected. All CI checks are green and the PR has approvals from both CodeRabbit and the dev-lead review agent. There are no actionable issues from the SonarCloud bot to address.
**No changes needed.**

@don-petry
don-petry enabled auto-merge (squash) August 11, 2026 20:14
@codeant-ai

codeant-ai Bot commented Aug 18, 2026

Copy link
Copy Markdown

Thanks for using CodeAnt! 🎉

We're free for open-source projects. if you're enjoying it, help us grow by sharing.

Share on X ·
Reddit ·
LinkedIn

@codeant-ai codeant-ai Bot added size:L This PR changes 100-499 lines, ignoring generated files and removed size:L This PR changes 100-499 lines, ignoring generated files labels Aug 18, 2026
@sonarqubecloud

Copy link
Copy Markdown

@don-petry
don-petry disabled auto-merge August 18, 2026 16:53
@don-petry
don-petry enabled auto-merge (squash) August 18, 2026 16:57
@don-petry
don-petry disabled auto-merge August 18, 2026 16:58
@don-petry

Copy link
Copy Markdown
Contributor Author

Dev-Lead — fix-bot-comment (no-changes)

Agent reasoning
**Bot:** SonarCloud (Quality Gate Passed)
**Issues addressed:** 1
- **GraphQL validation guard (codeant-ai thread)**: Already fixed in code at lines 147-165 of sweep-dispatch.sh. Added reply and resolved thread. [replied + thread resolved]
**Files in PR:**
- `.github/scripts/pr-auto-review/sweep-dispatch.sh` 
- `.github/scripts/pr-auto-review/README.md`
- `test/workflows/pr-auto-review/sweep-dispatch.bats`
**SonarCloud findings:** 0 new issues, 0 security hotspots, 0.0% coverage/duplication — Quality Gate passes cleanly. No action needed from SonarCloud.
**Tier 1 blockers:** None. All CI checks passing, all reviews approved.
**Conclusion:** The PR is clean. All code review findings from CodeRabbit and codeant-ai have been addressed and verified in the code. The SonarCloud quality gate shows no issues. The unresolved codeant-ai thread has been replied to and resolved.

@don-petry
don-petry merged commit 5397785 into main Aug 18, 2026
26 of 30 checks passed
@don-petry
don-petry deleted the dev-lead/issue-947-20260807-0756 branch August 18, 2026 16:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size:L This PR changes 100-499 lines, ignoring generated files

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Fleet Monitor] petry-projects/.github — .github/workflows/pr-auto-review-sweep.yml

2 participants