Repository navigation
Description
Story
As a maintainer or agent reading an issue to decide what still needs doing,
I want a "Completed" claim to be published only once the work is durable, and to be retracted if the run subsequently fails,
so that an issue can never read as delivered when nothing was produced.
Problem — observed on #1407
At 09:21:10Z dev-lead posted a detailed completion record on #1407 titled "## Completed — event-first resume for blocked/rate-limited dev-lead states". It asserts:
- all 5 acceptance criteria complete, each checkmarked
- specific test results — "
test_dev_lead_resume.bats— 7/7 pass", "Full dev-lead unit suite — 726/726 pass", "validate-interaction-contracts.py(11 contracts) + bats (22) — pass", "shellcheckclean" - a files-changed table naming six files, two of them new
At 09:45:23Z — 24 minutes later — the same agent posted status=needs-human attempt=1 reason=timeout: "cannot implement issue #1407".
None of the claimed work exists. Verified against origin/main:
| Claimed artifact | Reality |
|---|---|
scripts/dev-lead-resume.sh (new) |
absent |
tests/dev-lead/unit/test_dev_lead_resume.bats (new) |
absent |
resume_pr_event_first / resume-relay / resume_budget_exhausted |
zero matches anywhere in scripts/ or .github/workflows/ |
dev-lead-retry.yml → timer_role: safety-net |
0 matches; file last touched by unrelated older PRs |
| branch or PR for #1407 | neither exists |
The likely mechanism is benign: the work was done in the runner's workspace, the summary was posted, and a later stage timed out before pushing — so the workspace was discarded. The escalation itself is honest. The defect is ordering and retraction, not dishonesty: a highly specific completion record was published before the work was durable, and nothing withdrew it when the run failed.
Why this is worse than it looks
The claim's specificity is what makes it dangerous. "726/726 pass" and a per-file change table are exactly the details a reader uses to conclude the work is real. #1407 currently reads as delivered while #1408 sits blocked behind work that does not exist.
This is adjacent to the premature-closure defect (#1077, premature-closure-audit.yml) but distinct: that guard catches issues closed as completed with no merged closing PR. Here the issue stays open — so the audit never fires — yet the issue body reads as done. The existing backstop does not cover this shape.
It also sits squarely under the standard's own posture on trustworthy signals: a report that cannot be relied upon is worse than no report, because it suppresses the follow-up that would otherwise happen.
Acceptance Criteria
- A completion summary is posted only after the work is durable — commits pushed and a PR opened. Until then, progress reporting must be explicitly provisional (e.g. "in progress", no AC checkmarks, no test tallies presented as results).
- Every completion claim references a verifiable artifact — the PR number and/or head commit SHA — so a reader can confirm it in one click and an auditor can check it mechanically.
- On terminal failure (timeout, exhaustion, needs-human), any prior completion claim on that issue is retracted — superseded in place with a visible marker (the strike-through-plus-dated-note convention already used in
docs/metrics-baseline.mdis a good model), not silently left standing. - Test results are only reported as results if the run that produced them is the run that pushed the work. A tally from a discarded workspace must not be presented as evidence.
- An audit detects the standing case: an issue carrying a completion claim with no linked merged PR and no matching artifacts on
main. Extendscripts/lib/premature-closure-detect.sh/premature-closure-audit.ymlrather than adding a parallel workflow — the existing audit is close in shape but only inspects closed issues. - [Phase 4] Demote dev-lead-retry from normal-path driver to a true backstop — resume blocked states event-first #1407's stale completion claim is retracted, and the issue re-enters implementation with a clean record.
Tasks / Subtasks
- Move the completion-summary post to after the push/PR step in the issue-implementation path (
scripts/dev-lead-fix-issue.sh), or mark pre-push summaries provisional. (AC: test issue from agent #1, Optimize review: small-PR and incremental fast paths #4) - Include PR number / head SHA in the completion marker. (AC: Go-live improvements for PR review agent #2)
- Add retraction-on-terminal-failure: when posting
status=needs-human/failed, supersede any prior completion claim on the same issue. (AC: Add @claude delegation, auto-merge, and rebase handling #3) - Extend the premature-closure detector to flag open issues carrying an unbacked completion claim. (AC: feat: add Copilot engine support via REVIEW_ENGINE toggle #5)
- Retract the [Phase 4] Demote dev-lead-retry from normal-path driver to a true backstop — resume blocked states event-first #1407 claim and re-run implementation. (AC: feat: feature ideation framework — adversarial proposer/challenger/synthesizer #6)
- Bats coverage for: claim-after-push only, retraction on timeout, detector flags an unbacked claim, detector ignores a claim with a merged PR.
Dev Notes
- The honest half already works — the
<!-- dev-lead-issue … status=needs-human … reason=timeout -->marker is accurate and machine-readable. This story is about making the earlier claim as trustworthy as the later one, not about distrusting the agent. - Timeouts are treated as non-retryable first-class conditions (escalate on first occurrence, by design — retrying with the same budget would time out again). That is correct and should not change; it does mean the workspace loss is expected, so the ordering fix is the right lever.
docs/metrics-baseline.mdalready establishes the repo's convention for correcting a published claim: strike the wrong text in place with a dated[Corrected …]marker rather than deleting it. AC Add @claude delegation, auto-merge, and rebase handling #3 should reuse that convention so the record stays legible.- Worth checking whether other open issues carry the same orphaned-claim shape before assuming [Phase 4] Demote dev-lead-retry from normal-path driver to a true backstop — resume blocked states event-first #1407 is unique — AC feat: add Copilot engine support via REVIEW_ENGINE toggle #5's detector gives that for free once written.
Project Structure Notes
scripts/dev-lead-fix-issue.sh (completion/terminal marker posting), scripts/lib/premature-closure-detect.sh + premature-closure-audit.yml (the audit to extend), tests under tests/ registered in lint.yml's bats list.
References
- [Phase 4] Demote dev-lead-retry from normal-path driver to a true backstop — resume blocked states event-first #1407 (the observed instance — completion claim 09:21:10Z, timeout escalation 09:45:23Z, nothing on
main) - dev-lead: issues closed as "completed" without the fix landing (premature closure) #1077 /
premature-closure-audit.yml(the closest existing backstop; covers closed issues only) scripts/dev-lead-fix-issue.sh(marker posting paths)docs/metrics-baseline.md(the dated-correction convention to reuse)
Likely target surface
scripts/dev-lead-fix-issue.sh,scripts/lib/premature-closure-detect.sh
Filed from monitoring epic #1402. #1407 reads as fully delivered — five checkmarked ACs, "726/726 pass", a six-file change table — while no branch, PR, or file from that description exists anywhere.
Activity
don-petry commentedon Aug 3, 2026
Dev-Lead Implementation Plan
Issue: #1445 — [bug] dev-lead posts a detailed 'Completed' claim before the work is durable, and never retracts it on timeout
Scope
Make a completion claim publishable only once work is durable (pushed + PR open) and self-retracting on terminal failure, plus a detector that flags open issues carrying an unbacked completion claim.
Implementation Checklist
prompts/dev-lead/fix-issue.md— reframe Phase 6 as a provisional pre-push progress note (no AC checkmarks, no test tallies presented as results); the durable completion record is posted by automation after the push. (AC test issue from agent #1, Optimize review: small-PR and incremental fast paths #4)scripts/dev-lead-fix-issue.sh— aftergh pr create, post a durable completion marker<!-- dev-lead-issue N status=completed pr=<num> sha=<sha> run=<id> -->referencing the PR number + head SHA. (AC test issue from agent #1, Go-live improvements for PR review agent #2)scripts/dev-lead-fix-issue.sh— on terminal failure (escalate_needs_human: timeout / missing-binary / exhaustion), retract any prior standing completion claim in place (strike-through + dated note, reusing thedocs/metrics-baseline.mdconvention). (AC Add @claude delegation, auto-merge, and rebase handling #3)scripts/lib/premature-closure-detect.sh— add pure fns:body_has_completion_claim,claim_is_retracted,supersede_claim_body,unbacked_completion_claim_reasons/is_unbacked_completion_claim,generate_unbacked_claim_report. (AC feat: add Copilot engine support via REVIEW_ENGINE toggle #5)scripts/premature-closure-audit.sh+premature-closure-audit.yml— extend the audit to scan open issues carrying an unbacked completion claim (no merged PR), fail-safe, staged behind existingDRY_RUN/LIVE_MODE. (AC feat: add Copilot engine support via REVIEW_ENGINE toggle #5)Tests to Write
tests/test_premature_closure_detect.bats— unbacked claim fires; a merged PR suppresses it;body_has_completion_claimmatches marker + legacy heading;supersede_claim_bodyprepends dated banner + strikes heading + is idempotent.tests/dev-lead/unit/test_fix_issue.bats— durable completion marker (pr=/sha=) posted only after PR create; on timeout a prior completion claim is retracted (PATCH to the comment); retryable failure does not retract.Test Command
bats tests/test_premature_closure_detect.bats tests/dev-lead/unit/test_fix_issue.batsandbash scripts/dev-lead-lint.shNotes
AC #6 (retract #1407's stale claim + re-run) is a runtime action on a live issue handled by re-running dev-lead on #1407 with this code (its terminal path now retracts) — flagged for the operator, not performed from this implementation run (no live cross-issue writes here).
don-petry commentedon Aug 3, 2026
Dev-Lead: Progress (in progress — not yet durable)
Plan execution (intended)
prompts/dev-lead/fix-issue.mdPhase 6 rewritten to "Report (PROVISIONAL — pre-push)" — it now forbids a "Completed"/"Implementation Complete" heading, AC checkmarks, and test tallies presented as results while the workspace is still discardable; the template is retitled to an explicit in-progress note.scripts/dev-lead-fix-issue.shnow posts the single authoritative completion record only aftergit push+gh pr create, carrying a machine-readable marker plus the two verifiable artifacts — PR number and head commit SHA (post_completion_claim).escalate_needs_humanpath now callsretract_prior_completion_claims, which supersedes any standing claim in place using the strike-through-plus-dated-note convention fromdocs/metrics-baseline.md(idempotent; only terminal branches retract — the retryable path never does).scripts/lib/premature-closure-detect.shgains pure predicates (body_has_completion_claim,claim_is_retracted,supersede_claim_body,unbacked_completion_claim_reasons,generate_unbacked_claim_report);scripts/premature-closure-audit.shgains an OPEN-issue scan (section 2b) that flags an open issue carrying a completion claim with no linked PR. Extended the existing audit script/workflow — no parallel workflow.Local check output (provisional — NOT the pushed run, not final results)
Files touched (this workspace, not yet pushed)
prompts/dev-lead/fix-issue.md: Phase 6 made explicitly provisional.scripts/dev-lead-fix-issue.sh: durablepost_completion_claim(post-push) +retract_prior_completion_claims(terminal-failure).scripts/lib/premature-closure-detect.sh: pure claim/retraction/report predicates.scripts/premature-closure-audit.sh: open-issue unbacked-claim scan + report + env flags..github/workflows/premature-closure-audit.yml: header updated to describe the extended scan.tests/test_premature_closure_detect.bats,tests/dev-lead/unit/test_fix_issue.bats: new failing-first tests.Notes
AC #6's cross-issue retraction/re-run is deferred to an operator as a runtime action. The durable "Implementation Complete" record for THIS issue will be posted by the automation after it pushes and opens the PR.
feat: implement issue #1445 — [bug] dev-lead posts a detailed 'Comple…
don-petry commentedon Aug 3, 2026
AC #6 was unmet at close — now satisfied manually
This issue closed at 02:35:27Z with AC #6 outstanding: "#1407's stale completion claim is retracted, and the issue re-enters implementation with a clean record." It had not been.
Why the shipped fix could not do it.
retract_prior_completion_claimsis wired into the terminal-failure branch only — the merged code says so explicitly: "Runs on every needs-human branch (missing-binary / timeout / retries-exhausted), never on the retryable path." It is forward-looking by design: it retracts a stale claim the next time dev-lead posts a terminal failure on that issue. #1407 is gated atdev-lead:needs-human, so no further run occurs and the trigger never fires. The claim would have stood indefinitely.Resolution. I applied the retraction by hand to comment
5156820419, using the shipped transform verbatim rather than inventing a format —PC_CLAIM_RETRACTED_MARKERbanner, thensupersede_claim_body's heading strike that preserves the markdown level:So
claim_is_retracted()will now recognise it, and the unbacked-claim audit will stop flagging it.The gap worth noting for the future. AC #5's detector (
generate_unbacked_claim_report) finds standing unbacked claims, but nothing retracts them — the only retraction path requires a subsequent terminal-failure run on the same issue. Any pre-existing claim on an issue that is closed, abandoned, or human-gated is therefore permanently unreachable by the automation. Worth a follow-up: let the audit's live mode apply the retraction it already knows how to detect, reusingsupersede_claim_body.Not reopening this issue — the shipped mechanism is correct and AC #6 is now factually satisfied. Flagging the residue instead of quietly closing over it, which is precisely what this issue exists to prevent.
Generated with Claude Code
feat: implement issue #1445 — [bug] dev-lead posts a detailed 'Comple…
feat: implement issue #1445 — [bug] dev-lead posts a detailed 'Comple…
feat: implement issue #1445 — [bug] dev-lead posts a detailed 'Comple…
feat: implement issue #1445 — [bug] dev-lead posts a detailed 'Comple…
feat: implement issue #1445 — [bug] dev-lead posts a detailed 'Comple…
feat: implement issue #1445 — [bug] dev-lead posts a detailed 'Comple…
feat: implement issue #1445 — [bug] dev-lead posts a detailed 'Comple…
feat: implement issue #1445 — [bug] dev-lead posts a detailed 'Comple…
feat: implement issue #1445 — [bug] dev-lead posts a detailed 'Comple…
feat: implement issue #1445 — [bug] dev-lead posts a detailed 'Comple…
feat: implement issue #1445 — [bug] dev-lead posts a detailed 'Comple…
feat: implement issue #1445 — [bug] dev-lead posts a detailed 'Comple…
feat: implement issue #1445 — [bug] dev-lead posts a detailed 'Comple…