Skip to content

fix(pr-review): set FORCE_REVIEW only from an explicit dispatch flag (#2034) - #2078

Merged
don-petry merged 2 commits into
mainfrom
claude/gallant-johnson-4a7sox
Oct 9, 2026
Merged

don-petry merged 2 commits into
mainfrom
claude/gallant-johnson-4a7sox

Conversation

@don-petry

@don-petry don-petry commented Oct 9, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

FORCE_REVIEW (the break-glass that skips every review gate) is now set only when the dispatch carries an explicit client_payload.force_review: true. Before, any repository_dispatch set it. A dispatch without the flag now runs with every gate armed and gets only the narrow FORCE_RE_REVIEW same-SHA bypass, which applies only to an orphaned head marker.

Problem

Fixes #2034. The central mention listener (petry-projects/.github pr-review-mention-reusable.yml) sends the same {pr_url} payload for two very different events:

  • a human @donpetry-bot comment, and
  • a machine-added review_requested: donpetry-bot, which our own automation adds together with needs-human-review.

Because pr-review.yml set FORCE_REVIEW from event_name alone, every automated review request became a human-grade gate bypass. Each one approved past the maintainer-comment gate, and the next event run dismissed the approval (#1813). On #2022 this repeated until the per-PR automation budget was exhausted. The 10-07 health report shows 25 of 26 pr-review failures came from the repository_dispatch path.

Risk

Medium. This changes which runs may bypass the review gates. The change is stricter: no run gains a bypass, and dispatched runs lose the gate bypass until the listener sends the flag. Trade-off: a genuine human @donpetry-bot mention no longer skips the gates or re-runs a review that already has a verdict at the same SHA. It re-reviews with gates armed, which matches what the next event run enforced anyway. Restoring the human break-glass needs a listener change in petry-projects/.github (send client_payload[force_review]=true only for trusted human comment mentions, never for review_requested). I can't push to that repo from this session, so that is a follow-up.

Test plan

  • New tests/test_pr_review_force_review_wiring.bats (registered in lint.yml). It asserts that FORCE_REVIEW requires the explicit payload flag and is never derived from the event name alone, and that a dispatch gets FORCE_RE_REVIEW. The second test fails against the previous line.
  • Ran bats tests/test_pr_review_force_review_wiring.bats tests/test_review_one_pr_force_ci_failing.bats: 5/5 pass.
  • Ran bash scripts/dev-lead-lint.sh: passes. Parsed both changed workflows with PyYAML. actionlint is not installed here.

Rollback

Revert this PR. No non-revertible side effects.

Monitoring

In PR Review Agent — Trigger runs with event repository_dispatch, the log should show FORCE_REVIEW: false and FORCE_RE_REVIEW: true. The approve→dismiss pairs on a single head SHA (Dismissing approval due to a PR issue comment lacking a verified disposition (#1813)) should stop. The daily PR Review Agent health report's repository_dispatch failure share should drop.

Interaction contract

  • N/A — this PR does not add or change an agentic role.

Checklist

  • shellcheck scripts/*.sh passes for any changed shell scripts (no shell scripts changed; repo lint passes).
  • Docs updated where behavior or standards changed (the workflow's own FORCE flag comment block).

🤖 Generated with Claude Code

https://claude.ai/code/session_019K6UsD7r25pAcRjJL3G6Kx


Generated by Claude Code

View guided diff Turn on auto-fix

…2034)

FORCE_REVIEW bypasses every review gate, but it was derived from
event_name == repository_dispatch alone. The mention listener also
dispatches for machine-added 'review_requested: donpetry-bot' events, so
automated re-review requests approved past the maintainer-comment gate
and the next event run dismissed the approval, looping until the per-PR
automation budget ran out.

FORCE_REVIEW now requires client_payload.force_review == true. A dispatch
without it runs with every gate armed plus the narrow FORCE_RE_REVIEW
idempotency bypass, so it can neither approve over an open finding nor
re-run a cascade whose verdict already stands at the head.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019K6UsD7r25pAcRjJL3G6Kx
@don-petry
don-petry requested a review from a team as a code owner October 9, 2026 01:25
@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, you can upgrade your account or add credits to your account and enable them for code reviews in your settings.

@qodo-code-review

Copy link
Copy Markdown

ⓘ Qodo reviews are paused because your trial has ended. Ask your workspace admin to add credits to resume reviews. Manage billing

@coderabbitai

coderabbitai Bot commented Oct 9, 2026 •

Copy link
Copy Markdown

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 32 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

Learn how review limits work.

Review configuration:

⚙️ Run configuration
  • Configuration used: Organization UI
  • Review profile: ASSERTIVE
  • Plan: Advanced
  • Run ID: 07128262-81f9-42b4-814c-fe9993a05511
📥 Commits

Reviewing files that changed from the base of the PR and between 403d91f and bc35b81.

📒 Files selected for processing (3)
  • .github/workflows/lint.yml
  • .github/workflows/pr-review.yml
  • tests/test_pr_review_force_review_wiring.bats
  • Autopilot · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request adds a new BATS test file, tests/test_pr_review_force_review_wiring.bats, to verify that the FORCE_REVIEW workflow bypass requires an explicit client_payload.force_review flag and is not triggered by the event name alone. The feedback suggests using a structural YAML parser like yq instead of raw text regexes with grep to make the assertions on the workflow file more robust against formatting changes.

@codeant-ai

codeant-ai Bot commented Oct 9, 2026

Copy link
Copy Markdown

CodeAnt PR Risk: Low Risk

  • The PR appears safe to merge; the full review-gate bypass now requires an explicit force_review payload flag.
  • Ordinary repository dispatches retain only the narrow re-review bypass, and Bats tests check both workflow expressions.

Assessed commit: 367d2e656dcf

@cubic-dev-ai cubic-dev-ai Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 3 files

Reply with feedback, questions, or to request a fix.

View guided diff | Turn on auto-fix | Re-trigger cubic

…sions (#2034)

A substring check would still pass if the dispatch predicate were broadened
(e.g. an added || 'true'). Compare each key's value against the full expected
expression, and require exactly one definition of each key.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019K6UsD7r25pAcRjJL3G6Kx
@don-petry don-petry added the dev-lead:hands-off Exclude from dev-lead agent automation label Oct 9, 2026 — with Claude
@sonarqubecloud

sonarqubecloud Bot commented Oct 9, 2026

Copy link
Copy Markdown

@donpetry-bot

Copy link
Copy Markdown
Contributor

No description provided.

@donpetry-bot donpetry-bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review — APPROVED ✓

Risk: MEDIUM
Reviewed commit: bc35b81b7e01e0dd34df68a0cf57b2fd37b72d08
Review mode: triage-approved (single reviewer)

Summary

Narrows the FORCE_REVIEW break-glass so it is set only by an explicit client_payload.force_review: true on repository_dispatch, not by the event name alone. Dispatches without the flag get only the narrow FORCE_RE_REVIEW idempotency bypass. A new bats wiring test is registered in lint.yml. The change is stricter: no run gains a bypass. Triage's low-risk call holds. I rate it MEDIUM only because it changes review-gate control flow in a workflow.

Linked issue analysis

Fixes #2034. The root cause is addressed: pr-review.yml no longer derives FORCE_REVIEW from github.event_name, so machine-added review_requested dispatches can no longer approve past the maintainer-comment gate and trigger the approve/dismiss loop. Suggested-fix item 1 and the first half of item 4 (a dispatch without the flag runs with gates armed) are done. Item 2 (listener-side filtering in petry-projects/.github) is out of this repo's reach, and the PR body says so. Item 3 (defence in depth under FORCE_REVIEW) is not done. While the listener sends no flag, FORCE_REVIEW cannot be reached at all, so that gap cannot be triggered today.

Findings

No blocking findings.

Verified correct:

  • FORCE_REVIEW expression: checking both == true and == 'true' is needed, because GitHub expression loose equality does not treat the string 'true' as boolean true. A repository_dispatch without the field evaluates to 'false'.
  • FORCE_RE_REVIEW: for workflow_dispatch/workflow_call the input behaves as before (a boolean false falls through to 'false'). For repository_dispatch it evaluates to 'true', which keeps the old same-SHA re-run for orphaned head markers only (scripts/review-one-pr.sh:661-662). It also still skips carry-forward (:733-734), matching the previous dispatch behaviour.
  • The tests compare the exact expressions, so a broadened predicate fails. All gemini and cubic threads are resolved.

Non-blocking notes (follow-ups):

  1. Until the listener sends force_review, a genuine human @mention no longer re-engages a PR held by needs-human-review/escalation (review-one-pr.sh:813-821). It also no longer bypasses the cycle cap (:835) or the undispositioned-comment and maintainer-thread gates. Several operator-facing emit_verdict hints still name "@mention (FORCE_REVIEW)" as the remedy (:510, :559, :699, :706, :821), and those hints are now misleading. Consider rewording them, or ship the listener change soon.
  2. Fixes #2034 will close the issue while items 2 and 3 (listener filtering, defence in depth under a forced run) are still open. Open a tracking issue for the petry-projects/.github listener change so it is not lost.

CI status

All checks at head bc35b81 that completed are green: Lint, bats, unit-tests, shellcheck, actionlint, CodeQL, SonarCloud (quality gate passed), gitleaks, AgentShield, and the stub/permission validators. The CANCELLED entries (dev-lead dispatch/resume/ci-relay, review, Dismiss, guard, sync-scope-guard) are runs superseded by concurrency groups. Every required job with the same name also has a SUCCESS run. Third-party reviewers (CodeRabbit, Codex, Qodo) were rate-limited or billing-blocked. CodeAnt rated the PR Low Risk.


Reviewed automatically by the PR-review agent (single-reviewer mode: opus 5.5 [opus 4.8, opus 4.7]). Reply if you need a human review.

@don-petry
don-petry merged commit 1df133f into main Oct 9, 2026
78 of 110 checks passed
@don-petry
don-petry deleted the claude/gallant-johnson-4a7sox branch October 9, 2026 01:58
@don-petry

Copy link
Copy Markdown
Collaborator Author

Dev-Lead — review-changes (applied)

Changes committed and pushed. Requested items addressed:

  • tests/test_pr_review_force_review_wiring.bats:36 — applied
  • tests/test_pr_review_force_review_wiring.bats:27 — applied
  • tests/test_pr_review_force_review_wiring.bats:14 — applied

@don-petry

Copy link
Copy Markdown
Collaborator Author

dev-lead is withholding action on this item.

It is labeled dev-lead:hands-off (deliberately kept off-limits to dev-lead (e.g. changes to the dev-lead automation itself)), so dev-lead will not pick it up while that label is present. This notice is posted once so the withhold is visible rather than looking like a stalled run.

To re-enable automated pickup: remove the dev-lead:hands-off label.

don-petry added a commit that referenced this pull request Oct 9, 2026
…via #2078; superseded) — keep main + follow-up tree
don-petry added a commit that referenced this pull request Oct 9, 2026
…iled deferral (#2045 follow-up) (#2087)

* fix(pr-review): set FORCE_REVIEW only from an explicit dispatch flag (#2034)

FORCE_REVIEW bypasses every review gate, but it was derived from
event_name == repository_dispatch alone. The mention listener also
dispatches for machine-added 'review_requested: donpetry-bot' events, so
automated re-review requests approved past the maintainer-comment gate
and the next event run dismissed the approval, looping until the per-PR
automation budget ran out.

FORCE_REVIEW now requires client_payload.force_review == true. A dispatch
without it runs with every gate armed plus the narrow FORCE_RE_REVIEW
idempotency bypass, so it can neither approve over an open finding nor
re-run a cascade whose verdict already stands at the head.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019K6UsD7r25pAcRjJL3G6Kx

* fix(reviews): assert FORCE_REVIEW wiring structurally with yq and exact expressions

Test-Change-Justification: file is new in this PR (#2078); review threads asked for yq parsing and full-expression assertions

* fix(dev-lead): trusted-author tracker links; no auto-merge after a failed deferral (#2045 follow-up)

Follow-up to #2049 for the two major findings in its final pr-review cycle.

- deferred-thread-verify: a tracking issue can back a deferral only through
  text a trusted author wrote. The issue body counts only when the issue's
  author_association is OWNER/MEMBER/COLLABORATOR, and only comments by such
  authors count. Previously anyone able to open an issue with the tracker
  title, or comment on it, could authorize resolving a merge-blocking bot
  thread. A link from an untrusted author now reports `untrusted-mention`
  and the thread stays open.
- fix-reviews / review-changes / fix-bot-comment: a failed deferral now sets
  _REVIEW_INCOMPLETE, so try_enable_auto_merge never runs on a pass that
  did not finish resolving its review items (the #1567 guard).
  fix-bot-comment's guard now honors the flag too.
- Tests: untrusted issue body, untrusted comment, trusted comment on an
  untrusted issue, an untrusted tracker leaving the thread open end to end,
  and no auto-merge after a failed deferral on a commit pass. Existing
  tracker fixtures now carry author_association.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019K6UsD7r25pAcRjJL3G6Kx

* fix(dev-lead): keep auto-merge off after a failed deferral, including the EXIT-trap restore

_REVIEW_INCOMPLETE stopped try_enable_auto_merge, but the EXIT trap's
restore_auto_merge still re-enabled auto-merge that hold_auto_merge had turned
off at the start of the pass. The three deferral-failure branches now also set
_AM_NEEDS_RESTORE=0, matching the existing #2037 path.

Tests: a failed deferral leaves held-off auto-merge off; a successful one still
turns it back on (the stub can now start a PR with auto-merge on).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019K6UsD7r25pAcRjJL3G6Kx

---------

Co-authored-by: donpetry-bot <{}+donpetry-bot@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
don-petry pushed a commit that referenced this pull request Oct 9, 2026
…n-holds an escalated PR (#2090)

On 2026-10-09 pr-review re-ran its full cascade on #1902 three times in
ten minutes after escalating it to a human, and the fix-request path
removed the needs-human-review hold. The run logs show every one of those
runs came in as a pr-auto-review repository_dispatch with FORCE_REVIEW=true:
the pr-review/v1-next channel still served pr-review.yml from f34a765,
before #2078, where any repository_dispatch set the human break-glass
flag. #2078 (now on v1-next) derives FORCE_REVIEW only from an explicit
client_payload.force_review.

These tests replay the #1902 shapes end to end against review-one-pr.sh:
an automated dispatch (FORCE_RE_REVIEW only) after auto-rebase moved the
head, the same with a verdict already at the new head, and an unforced
event run all stay paused on the hold and never call --remove-label. The
controls show a human removing the label, or an @mention, still
re-engages. A structural test pins the #2078 FORCE_REVIEW expression; it
fails against the f34a765 pr-review.yml that ran.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019K6UsD7r25pAcRjJL3G6Kx
don-petry added a commit that referenced this pull request Oct 9, 2026
…d; re-evaluate after a self-dismissed approval; pin dev-lead to ring0 (#2091)

Fixes #2089, #2090, #1933.

- dev-lead's retry/resume paths honour every hold label (needs-human-review, dev-lead:needs-human, dev-lead:hands-off) and fail closed on an unreadable label set; a superseded hold notice no longer claims the hold was lifted.
- Regression tests pin that an automated dispatch never re-reviews or un-holds an escalated PR (#2078 behaviour).
- pr-review re-runs the full gated cascade at the same head after its own gate dismissed its newest approval.
- This repo's dev-lead pins dev-lead/v139-ring0; the SC2 guard now requires a promoted channel (ring<N> or stable, never next).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019K6UsD7r25pAcRjJL3G6Kx
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

dev-lead:hands-off Exclude from dev-lead agent automation

Projects

None yet

2 participants