Skip to content

pr-review claims 'Review posted' but no review lands — silent approval-submission failure #1297

Closed

Description

@don-petry

Summary

The PR Review Agent's approve path completed and logged success, but no review was actually submitted to the GitHub API. It believes it approved; nothing landed. This silently blocks merges that the agent has genuinely cleared.

Evidence (#1293)

The run reached the approve path and logged, verbatim:

[tier1] escalate=false risk=LOW signals=[all CI checks pass ...; marker
enforcement is mechanically guaranteed, read/write token split isolates agent
from cross-repo posting capability; no hardcoded secrets ...]
[approve] triage cleared — running single confirmation (claude-fable-5)
...
[done]  https://github.com/petry-projects/.github-private/pull/1293
Review posted (1/10)
Summary: 1 reviews posted, 0 no-ops skipped, 0 failures (processed 1/1 candidates)

Yet the API disagrees:

$ gh api repos/petry-projects/.github-private/pulls/1293/reviews \
    --jq '[.[]|select(.user.login=="donpetry-bot")]|length'
0

Zero donpetry-bot reviews on the head. reviewDecision stayed REVIEW_REQUIRED across ~10 minutes and two separate dispatches. The run reported 1 reviews posted, 0 failures — so the submit failure was not surfaced; it counted a non-existent review as posted.

Not the usual suspects (ruled out)

Why this is dangerous

It is the fail-open pattern this org keeps hitting (see combsmith/.github#755), now in the approval mechanism itself: an action that reports success while silently doing nothing. Summary: 1 reviews posted, 0 failures reads as "approved" — but the merge gate never moves. A maintainer watching the summary would believe the PR is approved. And because the count is fabricated, no alert fires.

If it happens on a PR nobody is watching closely, that PR silently never merges; worse, if the reverse ever occurs (claims a decline that didn't happen), the failure mode inverts.

What #1293 needed in the meantime

Admin-merged, disclosed, because the review substantively passed and only the submission glitched. That is a workaround, not a fix.

Suspected area

The gap is between the confirmation step (running single confirmation (claude-fable-5)) and the actual POST /pulls/{n}/reviews. Candidates:

  • the submit call 4xx'd (e.g. 422 "Can not approve your own pull request" edge, or a transient) and the result was swallowed;
  • the confirmation returned a shape the submit step mis-parsed as "posted" without calling the API;
  • require_last_push_approval / a same-actor edge caused GitHub to accept-then-drop the review, and the agent didn't re-read state to confirm.

Acceptance criteria

  • The agent verifies the review actually exists via the API after submitting, and treats a missing review as a failure, not a success.
  • N reviews posted counts only reviews confirmed present via GET /pulls/{n}/reviews — never the intent to post.
  • A submit 4xx/5xx is surfaced as a failure in the run summary, with the API error body.
  • Regression coverage for "submit reported success but review absent" → run fails loudly.
  • Fails closed: if the agent cannot confirm its own approval landed, it must not report the PR as approved.

Activity

  1. github-actions commented on Jul 18, 2026

    @github-actions
    Contributor

    Thanks for the thorough write-up — the evidence trail here (verbatim log output, API cross-check, and ruling out the common suspects) makes this much easier to act on. The single thing that would most help narrow down which of your three candidates is the actual culprit: do you have any way to capture or re-run with verbose HTTP logging so we can see the raw response body from the POST /pulls/{n}/reviews call — specifically whether it returned a non-2xx status that was swallowed, or returned 200 and GitHub silently dropped the review? That distinction will determine whether the fix lives in error-handling around the submit call or in the post-submit verification step.

  2. don-petry commented on Jul 18, 2026

    @don-petry
    CollaboratorAuthor

    Root-cause detail from a fresh occurrence (#1310): the run logs Review posted (1/10) / Summary: 1 reviews posted, 0 failures while GET /pulls/1310/reviews shows zero donpetry-bot reviews. So the failure is in the gap between the confirmation step (running single confirmation (claude-fable-5)) and the actual POST /pulls/{n}/reviews — and it is swallowed: the run counts the review as posted without ever re-reading the API to confirm it landed. That's the core defect. The fix (already in acceptance criteria) is: after submitting, GET the reviews and treat a missing review as a FAILURE, and only count reviews confirmed present. Also worth checking the submit's HTTP status is surfaced (a 4xx/5xx there is currently invisible). Note this is INTERMITTENT — approvals landed fine on #779/#1302/#1303/#1305/#1298/#1299/#1301.

  3. don-petry commented on Aug 7, 2026

    @don-petry
    CollaboratorAuthor

    Fresh repro: combsmith/broodly#432 (2026-08-07)

    Same symptom, different repo/PR — the approve path logs success but nothing lands.

    Two consecutive occurrences on the same PR, both via manual workflow_dispatch with force_review=true (not just the cron sweep):

    • Run 31131326886 — completed, success, no review landed
    • Run 31133544800 — completed, success, no review landed

    Log from the second run, verbatim:

    review decision:
    [approve] triage cleared — running single confirmation (claude-fable-5)
    ...
    Review posted (1/10)
    Summary: 1 reviews posted, 0 no-ops skipped, 0 failures (processed 1/1 candidates)
    

    API disagrees, same as the original report:

    $ gh api repos/petry-projects/broodly/pulls/432/reviews --jq '[.[]|select(.user.login=="donpetry-bot")]|length'
    0
    $ gh pr view 432 --repo petry-projects/broodly --json reviewDecision -q .reviewDecision
    (empty)
    

    Also consistent with "not the usual suspects" in the original report:

    • Not a review-required gap: dev-lead had already applied a real fix earlier in the same session (review-changes (applied) at 14:54:47Z, committing and pushing) and confirmed clean at 14:57:30Z — the PR was genuinely ready.
    • Not self-approval: PR author is don-petry, approver identity is donpetry-bot.
    • Intermittent, not systemic: in the same batch of 7 PRs processed the same way (manual workflow_dispatch + force_review=true), 5 others (repo-template#77-adjacent batch: .github#775, markets#354, markets#351, plus earlier incubator#84, #67, etc.) got real donpetry-bot/coderabbitai approvals and merged cleanly. Only this PR's approve path silently no-opped, twice in a row.

    Given #1297 has been open since 2026-07-18 with no fix yet, flagging this as a live, reproducible, current-day data point.

  4. don-petry commented on Sep 1, 2026

    @don-petry
    CollaboratorAuthor

    Closing — not reproducing

    This reported that pr-review's approve path logged success while no review was submitted to the GitHub API — a silent approval failure blocking merges the agent had genuinely cleared.

    pr-review is demonstrably landing review decisions now. All of these are from 2026-09-01:

    PR Review decision
    #1639 APPROVED
    #1638 CHANGES_REQUESTED
    #1636 APPROVED
    #1626 APPROVED
    #1625 APPROVED

    #1638 is the strongest signal: it went CHANGES_REQUESTED, the findings were addressed, and it merged — the full review path exercised end to end.

    Closing as not reproducing rather than as definitively fixed, since the original was an intermittent silent failure and no specific fix is attributable. Reopen on recurrence — the diagnostic to capture is whether the approve call's API response is being checked, not just logged.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugBug reportsneeds-human-reviewFlagged by automated PR review agent

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions