Skip to content

advisory-bot gate strands PRs forever: pushedDate=null disables both absent-bot timeout fallbacks #577

Closed

Description

@don-petry

Summary

The advisory-bot gate (scripts/lib/advisory-review-gate.sh) strands PRs in waiting-for-advisory-bots indefinitely whenever one of the four monitored bots doesn't review a given PR (common — Codex/SonarCloud only review a subset). The two timeout fallbacks that were specifically built to handle absent bots are silently disabled because they depend on GitHub's pushedDate, which now returns null.

Root cause

The gate requires all four monitored advisory bots to submit before approving:

gemini-code-assist, copilot-pull-request-reviewer, sonarqubecloud, chatgpt-codex-connector

Two fallbacks (lines 216-224) are supposed to proceed anyway when bots are absent:

  1. head-age > 20 min — head_age_sec from pushedDate
  2. quiescence > 10 min — time_since_last_sub, but guarded on head_time_raw being non-empty (line 200)

Both derive from this query (line 180-183):

query($url:URI!){resource(url:$url){...on PullRequest{commits(last:1){nodes{commit{pushedDate}}}}}}

pushedDate is deprecated and now returns null:

$ gh api graphql -f query='...pushedDate committedDate...' -f url=.../pull/518
{"committedDate":"2026-06-11T21:39:01Z","pushedDate":null}

With pushedDate=null:

The code intentionally refuses to fall back to committedDate (lines 173-177, to stop cherry-picks bypassing the gate) — so there is no backup head-age source. Result: the gate return 1 (wait) forever for any PR missing a full 4/4 bot set.

Evidence (run 27376131558, 2026-06-11)

Every advisory-gate warning in the batch run shows the tell-tale zeros:

[advisory-gate] WARNING: Only 3/4 advisory bots submitted so far (head age: 0s, last submission: 0s ago)

head age: 0s on PRs that are hours/days old = pushedDate resolving to null across the board.

PRs stranded as waiting-for-advisory-bots this run: #479, #518, #550 (.github-private) plus ~21 more org-wide (24 total).

On #518, only 3 of 4 monitored bots submitted (copilot-pull-request-reviewer, gemini-code-assist, sonarqubecloud) — chatgpt-codex-connector (Codex) never reviewed it, so the gate waits indefinitely. The only escape today is FORCE_REVIEW=true.

Impact

Any PR that isn't reviewed by all four advisory bots is parked in waiting-for-advisory-bots and never approved by pr-review — a meaningful slice of the "PRs pending review" backlog. Because Codex (and often SonarCloud-as-a-review) are selective, this is the common case, not the edge case.

Proposed fix

Restore a working head-age source so the absent-bot fallbacks fire:

  • Replace pushedDate with a reliable timestamp — e.g. the head commit's committer date via REST (GET /repos/{o}/{r}/commits/{sha} → .commit.committer.date), or the PR's headRefOid commit time, or the latest pull_request timeline pushedAt. Keep the cherry-pick guard by anchoring quiescence to the later of head-time and latest-submission (already done at lines 206-211).
  • And/or only require bots that have ever participated on the PR (treat Codex/SonarCloud as optional unless they've actually posted), instead of a hard 4/4.
  • Decouple the quiescence fallback from head_time_raw so it can still fire from latest_sub_at alone when head-time is unavailable.

A scheduled re-trigger for stranded PRs (see #573) would also let the (repaired) timeout fire without waiting for a pull_request_review event that never comes.

Related


Filed from an investigation into "pr-review agent not working — many PRs pending review."

Activity

  1. don-petry commented on Jun 11, 2026

    @don-petry
    CollaboratorAuthor

    Dev-Lead Implementation Plan

    Issue: #577 — advisory-bot gate strands PRs forever: pushedDate=null disables both absent-bot timeout fallbacks

    Scope

    Replace the deprecated pushedDate GraphQL field (now returns null) with the head commit's committer date from the REST API (GET /repos/{owner}/{repo}/commits/{sha} → .commit.committer.date), and decouple the quiescence fallback from head_time_raw so it can fire from latest_sub_at alone when head time is unavailable.

    Implementation Checklist

    • scripts/lib/advisory-review-gate.sh — Add _get_head_committer_date() helper that fetches the PR's head commit SHA via gh pr view --json headRefOid then retrieves .commit.committer.date via REST API
    • scripts/lib/advisory-review-gate.sh — Replace the pushedDate GraphQL query block with a call to _get_head_committer_date()
    • scripts/lib/advisory-review-gate.sh — Decouple quiescence fallback from head_time_raw guard so time_since_last_sub is computed from latest_sub_at even when head_time_raw is empty
    • tests/dev-lead/unit/test_advisory_review_gate.bats — Add _make_mock_gh_dir_null_pushed_date mock (graphql=empty, REST=old timestamp)
    • tests/dev-lead/unit/test_advisory_review_gate.bats — Add _make_mock_gh_dir_no_head_time mock (head time unavailable, old submissions)
    • tests/dev-lead/unit/test_advisory_review_gate.bats — Add failing test: recovers when pushedDate=null via committer date (REST)
    • tests/dev-lead/unit/test_advisory_review_gate.bats — Add failing test: quiescence fires when head time unavailable and submissions are old
    • tests/dev-lead/unit/test_advisory_review_gate.bats — Update existing mock helpers to use REST pattern (headRefOid + /commits/)

    Tests to Write

    • Gate runtime: recovers when pushedDate is null — uses committer date (REST) for head age — expects return 0 when committer date is old (>20 min)
    • Gate runtime: quiescence fallback fires when head time unavailable and submissions are old — expects return 0 when submissions are >10 min old and head time is unknown

    Test Command

    bats tests/dev-lead/unit/test_advisory_review_gate.bats --formatter tap

  2. don-petry commented on Jun 11, 2026

    @don-petry
    CollaboratorAuthor

    Dev-Lead: Implementation Complete

    Plan Execution

    • scripts/lib/advisory-review-gate.sh — Added _get_head_committer_date() helper that fetches headRefOid via gh pr view --json headRefOid then retrieves .commit.committer.date from GET /repos/{owner}/{repo}/commits/{sha}
    • scripts/lib/advisory-review-gate.sh — Replaced the pushedDate GraphQL query block with a call to _get_head_committer_date()
    • scripts/lib/advisory-review-gate.sh — Decoupled quiescence fallback from head_time_raw guard; time_since_last_sub is now computed from latest_sub_at even when head_time_raw is empty
    • tests/dev-lead/unit/test_advisory_review_gate.bats — Added _make_mock_gh_dir_null_pushed_date mock (graphql=empty, REST=old committer date)
    • tests/dev-lead/unit/test_advisory_review_gate.bats — Added _make_mock_gh_dir_no_head_time mock (both graphql and REST unavailable, old submissions present)
    • tests/dev-lead/unit/test_advisory_review_gate.bats — Added failing-then-passing test: "Gate runtime: recovers when pushedDate is null — uses committer date (REST) for head age"
    • tests/dev-lead/unit/test_advisory_review_gate.bats — Added failing-then-passing test: "Gate runtime: quiescence fallback fires when head time unavailable and submissions are old"
    • tests/dev-lead/unit/test_advisory_review_gate.bats — Updated _make_mock_gh_dir, _make_mock_gh_dir_recent, and _make_mock_gh_dir_old_commit_recent_push to use REST pattern (headRefOid + /commits/) instead of GraphQL

    Test Results

    1..39
    ok 1 Advisory gate: script is executable
    ok 2 Advisory gate: script has correct shebang
    ok 3 Advisory gate: script uses set -euo pipefail
    ok 4 Advisory gate: no polling loops (non-blocking design)
    ok 5 Advisory gate: uses check_advisory_reviews (not wait_for_advisory_reviews)
    ok 6 Advisory gate: returns 0 when bots submitted, 1 when waiting
    ok 7 Advisory gate: no sleep/polling delays in main logic
    ok 8–22 [structural/config/integration tests all pass]
    ok 28 Advisory gate: shellcheck passes
    ok 29 Advisory gate: script is minimal (non-blocking means fewer lines)
    ok 30 Gate runtime: returns 1 when no bots have submitted
    ok 31 Gate runtime: returns 0 when all 4 bots have submitted
    ok 32 Gate runtime: returns 1 when only 1 bot submitted and PR is recent
    ok 33 Gate runtime: returns 0 after timeout even with partial bot submissions
    ok 34 Gate runtime: stale submission from previous HEAD does not trigger quiescence for fresh commit
    ok 35 Gate runtime: returns 1 for recently-pushed old cherry-picked commit
    ok 36 Gate runtime: recovers when pushedDate is null — uses committer date (REST) for head age  [NEW]
    ok 37 Gate runtime: quiescence fallback fires when head time unavailable and submissions are old  [NEW]
    ok 38 Gate runtime: returns 2 when gh command fails (API error, not normal wait)
    ok 39 Gate runtime: format_bot_status renders known states
    
    39/39 tests pass. shellcheck passes. dev-lead-lint.sh passes.
    

    Files Changed

    • scripts/lib/advisory-review-gate.sh: Replaced deprecated pushedDate GraphQL query with _get_head_committer_date() REST helper; decoupled quiescence fallback from head_time_raw guard
    • tests/dev-lead/unit/test_advisory_review_gate.bats: Added 2 new regression tests, 2 new mock helpers, updated 3 existing mock helpers to REST pattern

    Notes

    • Cherry-pick guard is preserved: committer.date reflects when git cherry-pick ran (≈push time), not the original author date — so recently cherry-picked old commits still have small head_age_sec and don't bypass the gate
    • Quiescence decoupling trade-off: When the REST commit API is unreachable, quiescence now anchors to latest_sub_at alone (no cherry-pick protection for that fallback). The issue explicitly accepted this trade-off to avoid indefinite stranding
    • TDD order maintained: tests 36 and 37 were confirmed red against the original implementation before the fix was applied
    • No change to the 4-bot requirement logic — this fix only restores the two timeout fallbacks that were silently disabled
  3. added a commit that references this issue on Jun 11, 2026
  4. added 5 commits that reference this issue on Jun 12, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugBug reportsdev-leadFor dev-lead agent pickup

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions