Skip to content

pr-review: deadline-armed retries for consumer-repo advisory-bot and rate-limit waits (generalise #1994) #2533

Open

Description

@don-petry

Problem

#2530 extends the central stuck-review sweep to consumer repos. That sweep runs on a cron, and GitHub drops scheduled runs under load (3-7 hours observed, #1952), so it is only the eventual backstop.

Two consumer-repo waits have a deadline known when pr-review skips the PR:

  • waiting-for-advisory-bots: the advisory gate proceeds once the head commit is older than ADVISORY_HEAD_AGE_TIMEOUT_SEC (1200 s in scripts/lib/advisory-review-gate.sh). The deadline is the head commit time plus 1200 s.
  • Rate-limited advisory bot: the marker carries reset=<ISO>.

In this repository, pr-review-delayed-retry.yml (#1994) already turns a near reset into a deterministic retry, armed from the sweep's defer branch. It only handles this repository's PRs and only the rate-limit wait.

Decided on #2530 (Solution Architect advisory, adopted by the Maintainer): the time limit comes from deadline-armed retries, and the cron is the backstop.

Scope for dev-lead

Generalise the existing delayed retry rather than adding a new workflow:

Acceptance criteria

Activity

  1. github-actions commented on Nov 13, 2026

    @github-actions
    Contributor

    Thanks for the detailed write-up and for tying this back to #1994, #1952 and #2530; the scope and acceptance criteria are very clear. One question to help dev-lead size this: when a retry fires and the PR's head is unchanged but the advisory bot is still pending or still rate-limited (for example, the reset time slipped), should it re-dispatch once and leave further retries to the cron backstop, or re-arm for the new deadline?

  2. don-petry commented on Nov 14, 2026

    @don-petry
    CollaboratorAuthor

    Dev-Lead Implementation Plan

    Issue: #2533 — pr-review: deadline-armed retries for consumer-repo advisory-bot and rate-limit waits (generalise #1994)

    Scope

    Generalise pr-review-delayed-retry.yml (#1994) so the sweep arms it for both deadline-known waits (rate-limit reset= and the advisory head-age timeout), for any pr-review consumer repo. The retry re-enters the sweep, and the sweep routes the PR to that repo's own entrypoint (#2530 path).

    Implementation Checklist

    • scripts/lib/delayed-retry-arming.sh (new): pure helpers
      • delayed_retry_arm_decision <wait> <deadline_epoch> <now_epoch> <horizon> <armed_head> <current_head> <in_flight> <consumer> → arm or skip:<reason> (unknown-wait, no-deadline, elapsed, beyond-horizon, head-moved, in-flight, not-consumer)
      • pr_review_consumer_repo <repo> <agent_repo> [manifest]: true for the agent repo or a repo whose consumer-manifest.json refs include .github-private/.github/workflows/pr-review.yml
      • advisory_head_age_deadline <committer_iso> <timeout_sec> → epoch
    • scripts/sweep-stuck-reviews.sh
      • arm_delayed_retry takes a wait kind and passes -f wait=<kind>. The go/no-go comes from the pure decision. The existing comment-gate, MAX_DISPATCH and DRY_RUN handling stays.
      • New advisory-wait defer on the stuck-green path. When the head commit is younger than ADVISORY_HEAD_AGE_TIMEOUT_SEC and the advisory gate (same check_advisory_reviews, in a subshell) returns "waiting", defer instead of dispatching a review that would only skip, and arm a retry for head time + timeout. If the head time is unknown or the gate is clear or errors, it dispatches as before. DEFER_ADVISORY_WAIT=false restores the old behaviour.
    • scripts/delayed-rate-limit-retry.sh: new WAIT_KIND (rate-limit default, or advisory-bots). The rate-limit marker re-check applies only to rate-limit. Non-consumer repos no-op. Still delegates to the sweep with ARM_DELAYED_RETRY=false and never forces.
    • .github/workflows/pr-review-delayed-retry.yml: new wait input passed as WAIT_KIND. Run-name dedup (pr_url @ head_sha) unchanged. Still workflow_dispatch only (Class 1).
    • Docs: the docs/agentic-interaction-model.md row and the AGENTS.md exception line mention both waits.

    Tests to Write

    • tests/test_delayed_retry_arming.bats (new): decision arms each wait when the deadline is near. It skips for head moved, in flight, not a consumer, elapsed, beyond the horizon, and no deadline. Also consumer-manifest lookup and deadline maths.
    • tests/test_sweep_stuck_reviews.bats: a consumer PR with a young head and the advisory gate waiting is deferred, and one retry is armed with wait=advisory-bots. An old head dispatches normally. A consumer rate-limit PR arms with wait=rate-limit. A non-consumer PR arms nothing.
    • tests/test_delayed_rate_limit_retry.bats: an advisory-bots retry at an unchanged head re-dispatches the consumer repo's own entrypoint once, without force_review. A non-consumer is a no-op.

    Test Command

    bats tests/test_delayed_retry_arming.bats tests/test_sweep_stuck_reviews.bats tests/test_delayed_rate_limit_retry.bats (full: bash scripts/run-bats.sh), plus bash .dev-lead/scripts/dev-lead-lint.sh

  3. don-petry commented on Nov 14, 2026

    @don-petry
    CollaboratorAuthor

    Dev-Lead: Progress (in progress — not yet durable)

    Provisional. Nothing is pushed yet. The automation posts the authoritative
    completion record (with PR number + head SHA) after the push. Treat the notes
    below as intent, not delivered results.

    Plan Execution (intended)

    • Pure arming decision delayed_retry_arm_decision plus pr_review_consumer_repo and advisory_head_age_deadline in scripts/lib/delayed-retry-arming.sh, with bats tests in tests/test_delayed_retry_arming.bats.
    • Sweep: arm_delayed_retry takes a wait kind (rate-limit | advisory-bots), arms only through the decision, and passes -f wait=. On the stuck-green path, a PR whose head is younger than ADVISORY_HEAD_AGE_TIMEOUT_SEC, and which the advisory gate (same check_advisory_reviews) says is still waiting, is deferred instead of dispatched. A retry is armed for head commit time + timeout. An undeterminable head time or a gate that clears or errors dispatches as before. DEFER_ADVISORY_WAIT=false restores the old behaviour.
    • Retry script and workflow: new wait input (WAIT_KIND). The marker re-check applies only to rate-limit. A PR in a non-consumer repo is a no-op. The retry still delegates to the sweep with ARM_DELAYED_RETRY=false, so a consumer PR goes to its own entrypoint and is never forced. Run name and concurrency stay keyed on pr_url @ head_sha. Still workflow_dispatch only.

    Notes

    • "pr-review consumer" means the agent repo, or a repo whose scripts/lib/consumer-manifest.json refs include .github-private/.github/workflows/pr-review.yml.
    • In this local workspace, test_retry_payload_fields, test_batch_fallback, test_engine_chain_config and test_gemini_quota fail the same way on an untouched tree (shallow clone / environment). They are unrelated to this change.
  4. don-petry commented on Nov 14, 2026

    @don-petry
    CollaboratorAuthor

    Dev-Lead: Implementation Complete — PR #2536

    The implementation for issue #2533 is durable: commits are pushed and a pull request is open. This record is backed by verifiable artifacts you can confirm in one click.

    Review happens on the PR. Acceptance criteria and test results are evidenced by the PR's diff and CI checks — not asserted here. If a later stage fails, this claim is retracted in place.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    dev-leadFor dev-lead agent pickupenhancementFeature requests

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions