Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
24 commits
Select commit Hold shift + click to select a range
b4ead0f
feat: implement issue #2533 — pr-review: deadline-armed retries for c…
don-petry Nov 14, 2026
758ab69
fix(reviews): exit 2 on unparseable PR_URL in delayed-rate-limit-retry
don-petry Nov 14, 2026
138d5ba
fix(bot): add shell options to delayed-retry-arming.sh and reorder ad…
don-petry Nov 14, 2026
e5a043b
fix(bot): match both pr-review and pr-review-reusable.yml paths in co…
don-petry Nov 14, 2026
6489319
fix(reviews): key delayed-retry dedup on wait kind; persist advisory …
don-petry Nov 14, 2026
0bce4ee
fix(reviews): exempt advisory-held PRs from scheduled sweep eventable…
don-petry Nov 14, 2026
c7360a8
fix(bot): update concurrency_lane contract to include wait kind, add …
don-petry Nov 14, 2026
2b65177
fix(bot): use [[ ... ]] conditional syntax on lines 805 and 812
don-petry Nov 14, 2026
a4835ed
fix(bot): update entrypoint detection to accept pr-review-reusable.ym…
don-petry Nov 14, 2026
ebebd13
fix(reviews): accept pr-review-reusable.yml in pr_review_job_block fl…
don-petry Nov 14, 2026
d6b8492
fix(bot): add pr-review-reusable.yml to entrypoint file search
don-petry Nov 14, 2026
66af868
fix(reviews): list pr-review-reusable.yml in resolve_dispatch_target …
don-petry Nov 14, 2026
aa1a0df
Merge branch 'main' into dev-lead/issue-2533-20261114-0101
don-petry Nov 14, 2026
6eeb5e2
fix(bot): use [[ ]] instead of [ ] for conditional test on WAIT_KIND
don-petry Nov 14, 2026
b84109f
Merge branch 'main' into dev-lead/issue-2533-20261114-0101
don-petry Nov 14, 2026
fa4cc47
Merge branch 'main' into dev-lead/issue-2533-20261114-0101
don-petry Nov 14, 2026
8d9b9c6
Merge branch 'main' into dev-lead/issue-2533-20261114-0101
don-petry Nov 15, 2026
fc54a6e
Merge branch 'main' into dev-lead/issue-2533-20261114-0101
don-petry Nov 19, 2026
97ed771
Merge branch 'main' into dev-lead/issue-2533-20261114-0101
don-petry Nov 19, 2026
f320398
Merge branch 'main' into dev-lead/issue-2533-20261114-0101
don-petry Nov 19, 2026
e07ca1a
Merge branch 'main' into dev-lead/issue-2533-20261114-0101
don-petry Nov 19, 2026
4ed0e3e
Merge branch 'main' into dev-lead/issue-2533-20261114-0101
don-petry Nov 19, 2026
46eff7d
Merge branch 'main' into dev-lead/issue-2533-20261114-0101
don-petry Nov 19, 2026
572c53d
Merge branch 'main' into dev-lead/issue-2533-20261114-0101
don-petry Nov 19, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .github/workflows/lint.yml
Original file line number Diff line number Diff line change
Expand Up @@ -193,6 +193,7 @@
tests/test_seed_repo_template.bats \
tests/test_sweep_stuck_reviews.bats \
tests/test_delayed_rate_limit_retry.bats \
tests/test_delayed_retry_arming.bats \
tests/test_standing_approval.bats \
tests/test_validate_cases.bats \
tests/test_review_registry.bats \
Expand Down Expand Up @@ -374,7 +375,7 @@
timeout-minutes: 5
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
# NOTE: persist-credentials is intentionally left at the default (true) —

Check warning on line 378 in .github/workflows/lint.yml

View workflow job for this annotation

GitHub Actions / Lint

378:9 [comments-indentation] comment not indented like content
# vci_resolve_reusable needs `git fetch origin` to authenticate against this
# private repo when resolving same-repo channel tags. Setting
# persist-credentials: false would silently push every such caller into the
Expand Down
27 changes: 22 additions & 5 deletions .github/workflows/pr-review-delayed-retry.yml
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ name: PR Review Agent — Delayed Rate-Limit Retry
# The run name carries pr_url + head_sha so the sweep's arm_delayed_retry can
# dedup against an already in-flight retry (gh run list --json displayTitle)
# instead of re-dispatching on every defer tick (#1994).
run-name: "pr-review-delayed-retry ${{ inputs.pr_url }} @ ${{ inputs.head_sha }}"
run-name: "pr-review-delayed-retry ${{ inputs.pr_url }} @ ${{ inputs.head_sha }}${{ inputs.wait == 'advisory-bots' && ' (advisory-bots)' || '' }}"

# Deterministic near-reset retry for a rate-limited PR review (#1994).
#
Expand All @@ -17,6 +17,17 @@ run-name: "pr-review-delayed-retry ${{ inputs.pr_url }} @ ${{ inputs.head_sha }}
# then delegates to the sweep scoped to that one PR — re-dispatching the review
# through exactly the path the sweep uses, never force-reviewing.
#
# GENERALISED (#2533). The sweep also arms this retry for the advisory gate's
# head-age wait (`wait: advisory-bots` — the review skipped as
# waiting-for-advisory-bots, and the gate proceeds once the head commit is
# ADVISORY_HEAD_AGE_TIMEOUT_SEC old, so `not_before` is that deadline). `pr_url`
# may be in any pr-review consumer repo (scripts/lib/consumer-manifest.json): the
# delegated sweep re-dispatches it through that repo's own entrypoint (#2530).
# The run name and concurrency lane are keyed on pr_url + head_sha + wait kind, so
# there is at most one armed retry per (PR, head, wait): an advisory-bots retry never
# suppresses or cancels a later rate-limit retry at the same head, whose reset may be
# later than the advisory deadline. The rate-limit run-name has no suffix.
#
# CLASSIFICATION (docs/agentic-interaction-model.md §4). Class 1 by declaration:
# `workflow_dispatch` only, no `schedule` and no `timer_role`. The in-run sleep is
# a deterministic delay anchored to an armed `not_before`, NOT a scheduled trigger
Expand All @@ -36,18 +47,23 @@ on:
workflow_dispatch:
inputs:
pr_url:
description: "PR html_url whose rate-limited review to retry"
description: "PR html_url (this repo or any pr-review consumer) whose deferred review to retry"
required: true
type: string
head_sha:
description: "Head SHA the rate-limit marker was armed on (supersession guard)"
description: "Head SHA the wait was armed on (supersession guard)"
required: true
type: string
not_before:
description: "ISO-8601 reset time to wait for before retrying"
description: "ISO-8601 deadline to wait for before retrying (rate-limit reset, or advisory head-age deadline)"
required: false
default: ""
type: string
wait:
description: "Which wait was armed: rate-limit (marker reset) or advisory-bots (head-age deadline)"
required: false
default: "rate-limit"
type: string
dry_run:
description: "If true, delegate in dry-run (log, do not dispatch)"
required: false
Expand All @@ -65,7 +81,7 @@ concurrency:
# (PR, head SHA)" idempotency without duplicate sleeping runners (#1994 AC3). A
# newer push changes head_sha → a new lane → the stale retry no-ops on its own
# supersession guard.
group: pr-review-delayed-retry-${{ inputs.pr_url }}-${{ inputs.head_sha }}
group: pr-review-delayed-retry-${{ inputs.pr_url }}-${{ inputs.head_sha }}-${{ inputs.wait || 'rate-limit' }}
Comment thread
cubic-dev-ai[bot] marked this conversation as resolved.
Resolved
cancel-in-progress: true

jobs:
Expand Down Expand Up @@ -110,5 +126,6 @@ jobs:
PR_URL: ${{ inputs.pr_url }}
HEAD_SHA: ${{ inputs.head_sha }}
NOT_BEFORE: ${{ inputs.not_before }}
WAIT_KIND: ${{ inputs.wait || 'rate-limit' }}
DRY_RUN: ${{ inputs.dry_run || 'false' }}
run: bash scripts/delayed-rate-limit-retry.sh
7 changes: 4 additions & 3 deletions .github/workflows/pr-review-sweep.yml
Original file line number Diff line number Diff line change
Expand Up @@ -20,9 +20,10 @@ name: PR Review Agent — Stuck-Review Sweep
# core CI workflow finishes, the sweep runs SCOPED to just that run's PR(s)
# (scripts/sweep-stuck-reviews.sh derives them from the event payload), so a PR
# that just went green is re-reviewed in seconds;
# • for a PR DEFERRED on an un-elapsed rate-limit marker whose reset is near, the
# sweep ARMS pr-review-delayed-retry.yml (#1994), a deterministic retry that
# fires within minutes of the reset without depending on the cron.
# • for a PR DEFERRED on an un-elapsed rate-limit marker whose reset is near, or
# on the advisory gate's head-age wait (#2533), the sweep ARMS
# pr-review-delayed-retry.yml (#1994), a deterministic retry that fires within
# minutes of the deadline without depending on the cron.
# Neither path can strand a PR: whatever they miss, a later cron tick still catches
# — the cron is the eventual, not the time-bounded, backstop.
#
Expand Down
5 changes: 4 additions & 1 deletion AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -270,7 +270,10 @@ fails, diff it against `main` before calling it a regression, and treat CI as au
(`workflow_dispatch`-only) companion to `pr-review-sweep.yml`: when the sweep defers a PR on an un-elapsed
rate-limit marker whose `reset` is within a bounded horizon, it arms this workflow, which sleeps until the
reset, re-checks the marker still applies at the same head, then re-dispatches through the sweep — so a
rate-limited review retries within minutes of its reset instead of waiting for the best-effort cron. Its
rate-limited review retries within minutes of its reset instead of waiting for the best-effort cron. Since
#2533 the sweep also arms it for the advisory gate's head-age wait (`waiting-for-advisory-bots`, deadline =
head commit time + `ADVISORY_HEAD_AGE_TIMEOUT_SEC`), and for PRs in any pr-review consumer repo; the retry
re-enters the sweep, which dispatches the consumer's own entrypoint. Its
in-run sleep is an armed delay, not an `on: schedule` timer, so it is Class 1 by declaration. It must not be
removed by template syncs. If the org template gains an equivalent, remove this exception and defer to it.
- **Exception:** `readme-refresh.yml` (weekly org README refresh) is a documented repo-specific workflow
Expand Down
6 changes: 3 additions & 3 deletions docs/agentic-interaction-model.md
Original file line number Diff line number Diff line change
Expand Up @@ -200,7 +200,7 @@ keeps its own file) carries **no** job qualifier and remains exactly one row.
| `.github/workflows/initiative-planner.yml` | 1 | — | `workflow_dispatch` only — human-explicit bridge for the `discussion:[labeled]` signal that cannot be subscribed to inline; §2 Class 1 by declaration (no schedule, no `timer_role`). |
| `.github/workflows/decision-brief.yml` | 1 | — | `workflow_dispatch` only — human-explicit bridge for the `discussion:[labeled]` (`needs-brief`) signal that cannot be subscribed to inline, plus the `from_queue` sweep a caller's own timer dispatches; §2 Class 1 by declaration (no schedule, no `timer_role`). |
| `.github/workflows/pr-auto-review.yml` | 1 | — | `workflow_run:[completed]`, `check_suite:[completed]`, `pull_request_review:[submitted, dismissed]`, `pull_request:[opened, reopened, synchronize, ready_for_review]` — multi-event readiness gate; no schedule. |
| `.github/workflows/pr-review-delayed-retry.yml` | 1 | — | `workflow_dispatch` only — armed on demand by `pr-review-sweep.yml` to retry a rate-limited PR review near its `reset` without depending on the best-effort cron (#1994). §2 Class 1 by declaration (no schedule, no `timer_role`); the in-run sleep is a deterministic armed delay anchored to `not_before`, not a scheduled trigger — the fix for the convergence-clock pattern, not an instance of it. |
| `.github/workflows/pr-review-delayed-retry.yml` | 1 | — | `workflow_dispatch` only — armed on demand by `pr-review-sweep.yml` to retry a deferred PR review near a known deadline — a rate-limit marker's `reset` (#1994) or the advisory gate's head-age deadline (#2533) — in this repo or any pr-review consumer, without depending on the best-effort cron. §2 Class 1 by declaration (no schedule, no `timer_role`); the in-run sleep is a deterministic armed delay anchored to `not_before`, not a scheduled trigger — the fix for the convergence-clock pattern, not an instance of it. |
| `.github/workflows/pr-review-sweep.yml` | 2 | backstop | `schedule: '2,17,32,47 * * * *'` backstop **plus** `workflow_run:[completed]` fast path (#898) scoped to the completing CI run's PR(s); idempotent, per-branch `cancel-in-progress`. Dispatches a consumer PR to that repo's own pr-review entrypoint and skips PRs held at the maintainer-comment gate (#2530, ADR-0017). |
| `.github/workflows/initiative-driver.yml` | 2 | safety-net | `issues:[closed, labeled]` fast path **plus** `schedule: '23 */6 * * *'` explicitly "safety net for missed close events"; sweeps every open `initiative:auto` epic idempotently. |
| `.github/workflows/dev-lead-retry.yml` | 2 | self-heal | `schedule: '15 */2 * * *'` + `workflow_dispatch`; re-dispatches `status=rate-limited` PRs once the limit clears. **Leak flagged in §6** — no event fast-path, so it behaves as a de-facto convergence clock rather than a true backstop. |
Expand Down Expand Up @@ -301,8 +301,8 @@ re-discovers the actionable state by scanning — so it **must** satisfy the ful
contract in §6 (stop condition, idempotency, human-gated markers). `pr-review-sweep.yml`'s
scheduled backstop is exactly this: the net under its `workflow_run` fast path, nominally
≤15 min but best-effort — GitHub delays or drops scheduled runs under load (observed 3–7 h
in #1952), so a time-bound retry must not rely on it (the rate-limit case is armed explicitly by
`pr-review-delayed-retry.yml`, #1994).
in #1952), so a time-bound retry must not rely on it (the rate-limit and advisory head-age waits are
armed explicitly by `pr-review-delayed-retry.yml`, #1994/#2533).

**Never** attempt to cross the boundary by having `GITHUB_TOKEN` emit the triggering event
and hoping it fires — it will not, and the failure is silent.
Expand Down
7 changes: 4 additions & 3 deletions interaction-contracts/pr-review.yml
Original file line number Diff line number Diff line change
Expand Up @@ -16,8 +16,9 @@
#
# pr-review-delayed-retry.yml (#1994) is a Class-1 (workflow_dispatch-only) member
# of this runtime: the sweep ARMS it when it defers a PR on an un-elapsed rate-limit
# marker whose reset is near, and it fires a deterministic retry within minutes of
# that reset instead of waiting for the best-effort cron. Its in-run sleep is an
# marker whose reset is near, or on the advisory gate's head-age wait (#2533), in
# this repo or any pr-review consumer, and it fires a deterministic retry within
# minutes of that deadline instead of waiting for the best-effort cron. Its in-run sleep is an
# armed delay, not a scheduled timer — it has no `on: schedule`, so it adds no timer
# to this contract; it only re-dispatches through the same sweep path.
#
Expand Down Expand Up @@ -59,7 +60,7 @@ interaction:
- "workflow_dispatch:pr-review-trigger"
- "workflow_dispatch:pr-review-delayed-retry"
idempotency_key: "pr_url + head_sha (the review marker makes a same-head re-review a fast no-op)"
concurrency_lane: "pr-review-pr-<pr_url> / pr-review-batch (pr-review.yml); pr-review-sweep-<branch|event_name> (pr-review-sweep.yml); pr-review-delayed-retry-<pr_url>-<head_sha> (pr-review-delayed-retry.yml)"
concurrency_lane: "pr-review-pr-<pr_url> / pr-review-batch (pr-review.yml); pr-review-sweep-<branch|event_name> (pr-review-sweep.yml); pr-review-delayed-retry-<pr_url>-<head_sha>-<wait> (pr-review-delayed-retry.yml)"
stop_markers:
- needs-human-review
- pr-review:hands-off
Expand Down
57 changes: 49 additions & 8 deletions scripts/delayed-rate-limit-retry.sh
Original file line number Diff line number Diff line change
Expand Up @@ -26,10 +26,22 @@
# the next cron sweep still catches the PR. This path only makes the common
# near-reset case fast.
#
# Generalised for consumer repos (#2533). The sweep also arms this retry for the
# advisory gate's head-age wait (WAIT_KIND=advisory-bots: review-one-pr.sh skipped
# the PR as waiting-for-advisory-bots, and the gate proceeds once the head commit
# is ADVISORY_HEAD_AGE_TIMEOUT_SEC old). That wait leaves no marker, so only the
# rate-limit wait re-checks one; both re-check the head. The PR may be in any
# pr-review consumer repo: the delegated sweep sends it to that repo's own
# entrypoint (#2530). A PR in a repo that is not a consumer is a no-op.
#
# Env / inputs:
# PR_URL (required) the deferred PR's html_url
# HEAD_SHA (required) the head the rate-limit marker was armed on
# NOT_BEFORE ISO-8601 reset time to wait for (the marker's reset=)
# HEAD_SHA (required) the head the wait was armed on
# NOT_BEFORE ISO-8601 deadline to wait for (the marker's reset=, or
# the head-age deadline for WAIT_KIND=advisory-bots)
# WAIT_KIND rate-limit (default) | advisory-bots — which wait was armed
# CONSUMER_MANIFEST consumer manifest naming the pr-review consumers
# (default: lib/consumer-manifest.json)
# DELAYED_RETRY_BUFFER_SEC seconds added after NOT_BEFORE before retrying (default 120)
# DELAYED_RETRY_HORIZON_SEC the sweep's arming horizon (default 3600; same variable
# and default as sweep-stuck-reviews.sh)
Expand All @@ -43,10 +55,14 @@
set -euo pipefail

SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
# shellcheck source=lib/delayed-retry-arming.sh
source "$SCRIPT_DIR/lib/delayed-retry-arming.sh"

PR_URL="${PR_URL:-}"
HEAD_SHA="${HEAD_SHA:-}"
NOT_BEFORE="${NOT_BEFORE:-}"
WAIT_KIND="${WAIT_KIND:-rate-limit}"
CONSUMER_MANIFEST="${CONSUMER_MANIFEST:-$SCRIPT_DIR/lib/consumer-manifest.json}"
AGENT_REPO="${AGENT_REPO:-combsmith/.github-private}"
SWEEP_SCRIPT="${SWEEP_SCRIPT:-$SCRIPT_DIR/sweep-stuck-reviews.sh}"
DELAYED_RETRY_BUFFER_SEC="${DELAYED_RETRY_BUFFER_SEC:-120}"
Expand Down Expand Up @@ -82,15 +98,37 @@ if [ -z "$PR_URL" ] || [ -z "$HEAD_SHA" ]; then
echo "::error::delayed-rate-limit-retry: PR_URL and HEAD_SHA are required"
exit 2
fi
case "$WAIT_KIND" in
rate-limit|advisory-bots) ;;
*)
echo "::error::delayed-rate-limit-retry: WAIT_KIND='$WAIT_KIND' is not rate-limit or advisory-bots"
exit 2 ;;
esac

echo "=== Delayed rate-limit retry ==="
echo "=== Delayed pr-review retry ==="
echo " PR: $PR_URL"
echo " Wait: $WAIT_KIND"
echo " Armed head: ${HEAD_SHA:0:8}"
echo " Not before: ${NOT_BEFORE:-<none>}"

# Only pr-review consumers are retried (#2533): the delegated sweep would otherwise
# dispatch a review for a repo pr-review does not serve.
if [[ "$PR_URL" =~ ^https?://[^/]+/([^/]+/[^/]+)/pull/[0-9]+ ]]; then
pr_repo="${BASH_REMATCH[1]}"
else
echo "::error::delayed-rate-limit-retry: PR_URL '${PR_URL:-<empty>}' is unparseable"
exit 2
fi
if ! pr_review_consumer_repo "$pr_repo" "$AGENT_REPO" "$CONSUMER_MANIFEST"; then
Comment thread
cubic-dev-ai[bot] marked this conversation as resolved.
Resolved
Comment thread
cubic-dev-ai[bot] marked this conversation as resolved.
Resolved
echo " no-op: $pr_repo is not a pr-review consumer — nothing to retry"
exit 0
fi
Comment thread
don-petry marked this conversation as resolved.
Resolved

# still_applicable <phase>
# The armed retry must still apply: the PR open, un-merged, at the SAME head the
# marker was armed on, and that head's rate-limit marker still present. Returns
# wait was armed on, and — for WAIT_KIND=rate-limit — that head's rate-limit marker
# still present. The advisory-bots wait leaves no marker; the delegated sweep
# re-checks the gate and the PR's state. Returns
# non-zero (caller no-ops) when it does not. Without the marker check, a retry
# whose marker was removed would fall through the delegated sweep's ordinary
# REVIEW_REQUIRED + green-CI path and launch a plain review instead of no-oping.
Expand Down Expand Up @@ -118,6 +156,7 @@ still_applicable() {
echo " no-op ($_phase): head advanced ${HEAD_SHA:0:8} -> ${_current_head:0:8} — superseded by a newer push; its event drives a fresh review"
return 1
fi
[[ "$WAIT_KIND" = "rate-limit" ]] || return 0
_has_marker=$(jq -r --arg m "<!-- pr-review-agent rate-limited v1 sha=${HEAD_SHA} " \
'[((.reviews // []) + (.comments // []))[]? | (.body // "" | tostring) | select(contains($m))] | length > 0' <<< "$_snapshot" 2>/dev/null || echo "false")
if [ "$_has_marker" != "true" ]; then
Expand Down Expand Up @@ -170,19 +209,21 @@ still_applicable "post-sleep" || exit 0

# ---------------------------------------------------------------------------
# 3. Delegate to the sweep scoped to this one PR. The sweep re-validates the
# rate-limit marker, standing verdict, CI and exclusions and re-dispatches
# through the normal trigger (never force). ARM_DELAYED_RETRY=false so it
# cannot re-arm another retry, and the event name is unset so the #1408
# rate-limit marker or advisory gate, standing verdict, CI and exclusions and
# re-dispatches through the PR's own repo entrypoint (never force).
# ARM_DELAYED_RETRY=false so it cannot re-arm another retry, and the event
# name is unset so the #1408
# scheduled-narrowing does not apply to this targeted retry.
# ---------------------------------------------------------------------------
one_pr_file="$(mktemp)" || { echo "::error::Failed to create temporary file"; exit 1; }
trap 'rm -f "$one_pr_file"' EXIT
printf '%s\n' "$PR_URL" > "$one_pr_file"

echo " retry: delegating to the sweep for $PR_URL (head ${HEAD_SHA:0:8})"
echo " retry: delegating to the sweep for $PR_URL (wait $WAIT_KIND, head ${HEAD_SHA:0:8})"
SWEEP_PRS_FILE="$one_pr_file" \
ARM_DELAYED_RETRY=false \
AGENT_REPO="$AGENT_REPO" \
CONSUMER_MANIFEST="$CONSUMER_MANIFEST" \
DRY_RUN="${DRY_RUN:-false}" \
GITHUB_EVENT_NAME="" \
bash "$SWEEP_SCRIPT"
Loading
Loading