Repository navigation
Closed
Labels
Description
Summary
Bot review threads that dev-lead never processes are never retried. They sit with no reply and no resolution, and under required_review_thread_resolution the PR cannot merge. This is the review-thread counterpart of #2017, whose fix (#2022) covers only undispositioned PR issue comments.
Evidence — PR #1953 (as of 2026-10-03 13:45Z)
Eleven unresolved bot threads have no reply at all, though both bots are trusted (dev_lead_trusted=yes in scripts/lib/reviewer-sources.tsv):
| Reviewer | Posted | Threads (file:line) |
|---|---|---|
| cubic-dev-ai | 2026-10-01 11:35Z | canary_report.sh 121, 659, 220; tests/canary_report.bats 199 |
| chatgpt-codex-connector | 2026-10-02 22:49–23:19Z | canary_report.sh 195, 192, 314, 619 |
| chatgpt-codex-connector | 2026-10-03 13:37Z | canary_report.sh 475, 226; token-report.yml 50 |
dev-lead's pass history on the PR shows why they were missed:
- Every pass after the cubic review (2026-10-01 11:35Z through 23:14Z) ran with
intent=fix-bot-comment, the PR issue-comment path. None was a review-thread pass. - The only
intent=fix-reviewspass (2026-10-02 23:10Z, sha 571cc02) revisited just the five older, already-deferred Codex threads and reportedstatus=not-applied. It never picked up the cubic threads or the newer Codex threads.
The job logs were not available from the investigating session, so the exact reason is unconfirmed. Possible causes:
- The
pull_request_review-triggered run was superseded or cancelled in thedev-lead-pr-<n>lane (cf. Harness cannot resolve a bot review thread whose correct response is 'no change needed' — false positives block merge permanently #1743 AC9, where bot chatter cancels real work). - Intent classification routed the event to
fix-bot-comment. - The pass enumerated a stale thread set.
Proposed fix
Extend the #2017 retry scan (scripts/dev-lead-retry.sh / scripts/lib/bot-comment-retry.sh) to review threads:
- For each open PR, find unresolved, non-outdated, bot-originated review threads from a trusted reviewer source whose originating comment has no our-account reply, after the same grace period the issue-comment scan uses.
- Dispatch one deduplicated
fix-reviewspass per PR covering those thread ids. Reuse the retry-marker claim, attempt caps (per rate-limit window and in total) and rate-limit awareness from feat: implement issue #2017 — A lost fix-bot-comment run is never retried — a bot comment stays undispositioned and the PR stalls at the maintainer-comment gate forever (#2009) #2022. - When attempts are exhausted, apply
needs-human-reviewor post a notice so the stall is visible. This was also raised as a non-blocking note on feat: implement issue #2017 — A lost fix-bot-comment run is never retried — a bot comment stays undispositioned and the PR stalls at the maintainer-comment gate forever (#2009) #2022. - Also check whether the
fix-reviewsthread query enumerates all unresolved bot threads, not only those from the triggering reviewer or review.
Acceptance criteria
- An unreplied, unresolved bot thread older than the grace period triggers exactly one retry
fix-reviewspass. The pass is deduplicated across the cron scan and pr-review's gate hook. - Threads with an our-account reply, and outdated or maintainer threads, are not retried.
- The attempt caps are honoured, and exhaustion is surfaced (label or notice), not silent.
- A
fix-reviewspass sees every unresolved trusted-bot thread on the PR. A regression test covers threads from multiple reviewers and from reviews older than the triggering one. - bats coverage for the decision rules.
Generated by Claude Code
Activity
don-petry commentedon Oct 3, 2026
Dev-Lead Implementation Plan
Issue: #2046 — dev-lead: unprocessed bot review threads are never retried — review-thread counterpart of #2017
Scope
Extend the #2017 retry sweep to unreplied trusted-bot review threads (one deduplicated
fix-reviewsretry per PR, capped, exhaustion surfaced by a notice), and make everyfix-reviews/review-changespass enumerate all unresolved threads.Likely root cause for AC4
dev-lead-fix-reviews.shbuildsOPEN_THREADS_JSONfromreviewThreads(first:50)without pagination. GitHub returns threads oldest first, so on a PR with more than 50 threads (PR #1953) a pass only sees the oldest page. On #1953 that page held the five already-deferred Codex threads, and the cubic and newer Codex threads never reached the engine. This matches thestatus=not-appliedpass on sha 571cc02.Implementation Checklist
scripts/lib/open-review-threads.sh(new):fetch_open_review_threads <repo> <pr>. Paginated (first:100,$endCursor) and not filtered by reviewer. Returns every unresolved thread.scripts/dev-lead-fix-reviews.sh: the fix-reviews and review-changes branches both use the helper instead of the single 50-thread page.scripts/lib/bot-thread-retry.sh(new):btr_retry_decisionswith per-thread reasons:replied,grace-period,retry-pending,rate-limited,retry-attempts-exhausted,dispatch;Botfrom a trusted reviewer source. Maintainer, outdated and resolved threads are never candidates. The grace period, pending window, attempt caps and rate-limit awareness mirrorbot-comment-retry.sh, and only our own automation's markers count.scripts/dev-lead-retry.sh:scan_pr_for_unreplied_bot_threads.pr_resume_suppressed.dev-lead-reviews-retry intent_type=fix-reviewsdispatch per PR.scan_repoafter the bot-comment scan, keeping one dispatch per PR per scan.scripts/review-one-pr.sh: the gate hook runs the thread scan when the bot-comment scan dispatched nothing. Both callers share the same marker claim, so the dispatch is deduplicated.tests/dev-lead/integration/test_prompt_coverage.sh: point the__typenameselection check at the new helper, where the query now lives.Tests to Write
tests/dev-lead/unit/test_bot_thread_retry.bats, decision matrix:fix-reviewsdispatch;fetch_open_review_threadsregression: a two-page response with threads from multiple reviewers and older reviews returns every unresolved thread, and the query paginates.Test Command
bats tests/dev-lead/unit/+bash tests/dev-lead/integration/test_prompt_coverage.sh+bash .dev-lead/scripts/dev-lead-lint.shdon-petry commentedon Oct 3, 2026
Dev-Lead: Implementation Complete — PR #2048
The implementation for issue #2046 is durable: commits are pushed and a pull request is open. This record is backed by verifiable artifacts you can confirm in one click.
b0f4d6521598e6a7a3d0a55194930bad7aad403dmain: 7 file(s), +1003/-29 linesReview happens on the PR. Acceptance criteria and test results are evidenced by the PR's diff and CI checks — not asserted here. If a later stage fails, this claim is retracted in place.
fix(docs): add missing function docstrings to improve coverage
don-petry commentedon Oct 4, 2026
Root cause found for one cause of this pattern: dev-lead's
fix-reviewspass readsreviewThreads(first:50)with no pagination, so on PR #1953 (66 threads) the single open thread was invisible — filed as #2056.Generated by Claude Code
fix(dev-lead): follow-on retry scans defer to a recent dispatch guard (…
fix(dev-lead): post a bot-thread exhaustion notice only when nothing …
fix(dev-lead): harden the dispatch-guard check; never lose an exhaust…
feat(dev-lead): retry unprocessed bot review threads (#2046) (#2048)