Skip to content

feat: implement issue #1735 — Bot-acknowledged threads are permanently unresolvable after #1691 — harness must scan the whole thread, not comments(last:1) - #1752

Merged
don-petry merged 5 commits into
mainfrom
dev-lead/issue-1735-20260910-1150
Sep 12, 2026
Merged

don-petry merged 5 commits into
mainfrom
dev-lead/issue-1735-20260910-1150

Conversation

@don-petry

@don-petry don-petry commented Sep 10, 2026 •

Copy link
Copy Markdown
Collaborator

User description

Closes #1735

Implemented by dev-lead agent. Please review.


CodeAnt-AI Description

Prevent addressed review threads from remaining permanently unresolved when a bot acknowledges the response afterward

What Changed

  • Finds the latest addressed reply from the authorized account anywhere in the full thread, rather than only checking the last comment
  • Resolves the thread when later bot comments clearly acknowledge the response
  • Keeps threads open for new bot findings, human comments, ambiguous bot messages, and rejected acknowledgements
  • Adds coverage for marker placement, acknowledgement detection, blocking feedback, and fail-closed behavior

Impact

✅ Fewer permanently stuck review threads
✅ Bot acknowledgements no longer block resolution
✅ New or unclear feedback keeps review threads open

💡 Usage Guide

Checking Your Pull Request

Every time you make a pull request, our system automatically looks through it. We check for security issues, mistakes in how you're setting up your infrastructure, and common code problems. We do this to make sure your changes are solid and won't cause any trouble later.

Talking to CodeAnt AI

Got a question or need a hand with something in your pull request? You can easily get in touch with CodeAnt AI right here. Just type the following in a comment on your pull request, and replace "Your question here" with whatever you want to ask:

@codeant-ai ask: Your question here

This lets you have a chat with CodeAnt AI about your pull request, making it easier to understand and improve your code.

Example

@codeant-ai ask: Can you suggest a safer alternative to storing this secret?

Preserve Org Learnings with CodeAnt

You can record team preferences so CodeAnt AI applies them in future reviews. Reply directly to the specific CodeAnt AI suggestion (in the same thread) and replace "Your feedback here" with your input:

@codeant-ai: Your feedback here

This helps CodeAnt AI learn and adapt to your team's coding style and standards.

Example

@codeant-ai: Do not flag unused imports.

Retrigger review

Ask CodeAnt AI to review the PR again, by typing:

@codeant-ai: review

Check Your Repository Health

To analyze the health of your code repository, visit our dashboard at https://app.codeant.ai. This tool helps you identify potential issues and areas for improvement in your codebase, ensuring your repository maintains high standards of code health.

…y unresolvable after #1691 — harness must scan the whole thread, not comments(last:1)
@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

@codeant-ai

codeant-ai Bot commented Sep 10, 2026 •

Copy link
Copy Markdown

🤖 CodeAnt AI — Review Status

Status Commit Started (UTC) Finished (UTC)
✅ Incremental review completed 9355790 Sep 12, 2026 · 17:43 17:43
✅ Incremental review completed 175e160 Sep 10, 2026 · 19:07 19:08
✅ Reviewed your PR 333e71f Sep 10, 2026 · 12:08 12:11

@qodo-code-review

Copy link
Copy Markdown

ⓘ Qodo reviews are paused because your trial has ended. Ask your workspace admin to add credits to resume reviews. Manage billing

@codeant-ai

codeant-ai Bot commented Sep 10, 2026

Copy link
Copy Markdown

Thanks for using CodeAnt! 🎉

We're free for open-source projects. if you're enjoying it, help us grow by sharing.

Share on X ·
Reddit ·
LinkedIn

@coderabbitai

coderabbitai Bot commented Sep 10, 2026 •

Copy link
Copy Markdown

Warning

Review limit reached

Next included review available in 29 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Learn how review limits work.

Review configuration:

⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Advanced

Run ID: d0b5e965-43a2-4799-99dd-9268aacf1e58

📥 Commits

Reviewing files that changed from the base of the PR and between d8394f6 and 9355790.

📒 Files selected for processing (4)
  • scripts/dev-lead-fix-reviews.sh
  • scripts/lib/addressed-claim-verify.sh
  • tests/dev-lead/unit/test_addressed_claim_verify.bats
  • tests/dev-lead/unit/test_fix_reviews.bats

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@codeant-ai codeant-ai Bot added the size:L This PR changes 100-499 lines, ignoring generated files label Sep 10, 2026
@don-petry

Copy link
Copy Markdown
Collaborator Author

Dev-Lead — review-changes (no-changes)

No changes were needed for this PR.

@don-petry
don-petry enabled auto-merge (squash) September 10, 2026 12:10

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request implements a full-thread scan to identify the latest addressed-marker reply instead of only checking the last comment, preventing threads from becoming permanently unresolvable when a bot acknowledgement lands after a marker. It introduces helper functions to find the latest marker index, classify post-marker bot comments, and verify that no unaddressed comments have landed since the marker, along with comprehensive unit tests. The review feedback recommends optimizing acv_latest_marker_index and acv_post_marker_clear by replacing loop-based jq subprocess spawning with single jq invocations and using here-strings to avoid unnecessary subshells.

Comment thread scripts/lib/addressed-claim-verify.sh Resolved
Comment thread scripts/lib/addressed-claim-verify.sh Resolved
Comment thread scripts/lib/addressed-claim-verify.sh Resolved
Comment thread scripts/lib/addressed-claim-verify.sh Resolved
@don-petry
don-petry disabled auto-merge September 10, 2026 12:11
@donpetry-bot

Copy link
Copy Markdown
Contributor

CI checks on this PR are still running. Once they complete, re-mention @donpetry-bot to trigger a fresh review.

Posted by the donpetry-bot PR-review cascade.

1 similar comment
@donpetry-bot

Copy link
Copy Markdown
Contributor

CI checks on this PR are still running. Once they complete, re-mention @donpetry-bot to trigger a fresh review.

Posted by the donpetry-bot PR-review cascade.

@don-petry

Copy link
Copy Markdown
Collaborator Author

Dev-Lead — fix-reviews (partial)

A commit was pushed, but not every requested change was applied. Per requested item:

  • scripts/lib/addressed-claim-verify.sh:175 — not applied
  • scripts/lib/addressed-claim-verify.sh:250 — not applied
  • scripts/lib/addressed-claim-verify.sh:75 — applied
  • scripts/lib/addressed-claim-verify.sh:230 — not applied
  • tests/dev-lead/unit/test_fix_reviews.bats:1392 — applied

The unaddressed items above still need work.

@don-petry
don-petry enabled auto-merge (squash) September 10, 2026 12:22
@don-petry
don-petry disabled auto-merge September 10, 2026 12:24
@don-petry

Copy link
Copy Markdown
Collaborator Author

Dev-Lead — fix-bot-comment (no-changes)

Agent reasoning
Issues addressed: 0
Status: Quality Gate Passed — 0 issues, 0 hotspots
Tier 1 blockers: 0
Action: None required
```
No changes needed. The PR is ready.

@don-petry
don-petry enabled auto-merge (squash) September 10, 2026 12:25
@donpetry-bot

Copy link
Copy Markdown
Contributor

Advisory bots were rate-limited; auto-approval is withheld until they recover. pr-review-sweep will re-review this PR after 2026-09-10T13:26:39Z.

@donpetry-bot

donpetry-bot commented Sep 10, 2026 •

Copy link
Copy Markdown
Contributor
Superseded by automated re-review at 96902039b98de849cb983737adeaa042fa1cd2b5 — click to expand prior review.

Review — fix requested (cycle 1/3)

The automated review identified the following issues. Please address each one:

Findings to fix

Automated review — NEEDS HUMAN REVIEW

Risk: MEDIUM
Reviewed commit: 2c3ffe85c8334561ac47c43b32b257123d64c13e
Review mode: triage-approved (single reviewer)

Summary

Implementation of #1735 is correct and complete — full-thread marker scan, narrow bot-ack/finding classifiers with fail-closed defaults, #1415 human-block preserved via the shared discriminator, #1692 claim gate composed rather than bypassed, and all five AC6 cases covered at both unit and harness level. All CI checks green. Escalating on a single gate: one unresolved review thread (a security suggestion on scripts/lib/addressed-claim-verify.sh) whose refutation carries no addressed-marker, so under harness-only resolution (#1691) only a human can resolve it — it blocks the merge regardless of approval and warrants owner ratification since the accepted CodeAnt instruction is itself a policy decision.

Linked issue analysis

Closes #1735. All seven acceptance criteria are substantively addressed:

  • AC1 — comments(last:1) read replaced by acv_latest_marker_index, a full-thread scan for the latest marker reply from our account (single jq pass).
  • AC2 — acv_bot_comment_is_acknowledgement classifies post-marker bot comments; the finding regex is checked FIRST so mixed ack+finding comments block; codeant's "Customized review instruction saved" shape clears.
  • AC3 — any marker-less User comment after our marker blocks unconditionally, reusing review_thread_is_agent_authored — no second classifier introduced.
  • AC4 — fail-closed throughout: empty/ambiguous bodies rc2, negated acknowledgements ("cannot acknowledge") explicitly guarded, unknown author types rc2, unparseable JSON rc2.
  • AC5 — the [#1621 S2] Verify the addressed-marker against the pushed diff + honour maintainer dispositions #1692 claim gate still runs on the marker body; both gates must pass (acv_parse_claim + verification are unchanged and now fed marker_body).
  • AC6 — all five mandated cases present as pure unit tests (test_addressed_claim_verify.bats, 22 new cases) plus four end-to-end harness cases (test_fix_reviews.bats) asserting actual mutation/no-mutation.
  • AC7 — ShellCheck and bats jobs green.

Dev-notes constraints honoured: #1691 untouched (harness remains sole resolver), #1692 built upon (same function, claim gate preserved).

Findings

Blocking (requires human action):

  1. Unresolved review thread at scripts/lib/addressed-claim-verify.sh:251 — CodeAnt flagged (security, Major) that review_thread_is_agent_authored is text-only, so a copied marker bypasses the human-comment guard. The dev-lead refuted it with sound reasoning (the harness runs as the owner account, so author identity cannot discriminate; Bot-acknowledged threads are permanently unresolvable after #1691 — harness must scan the whole thread, not comments(last:1) #1735 AC3 explicitly mandates reusing this discriminator; marker-copying is self-limiting), and CodeAnt accepted, saving a custom instruction. However: (a) the refutation reply carries no addressed-marker, so the harness — before or after this fix — will never resolve this thread; under [#1621 S1] Harness-only thread resolution: remove the model's resolveReviewThread path #1691 only a human can. It blocks required_review_thread_resolution indefinitely. (b) The accepted instruction ("do not add author-based checks") is an org-wide review-policy decision that should be ratified by the owner, not auto-approved past. Action: owner reviews the refutation, resolves the thread manually, and merge can proceed on re-review.

Non-blocking notes:

  1. Branch is BEHIND main — auto-rebase should handle after thread resolution.
  2. Secret-scan MCP tool not available in this run; the gitleaks CI check is green — no secret-shaped content in the diff (shell + bats only).
  3. Residual accepted risk, documented for the record: a human comment that quotes an automation marker verbatim would be classified agent-authored and not block. This is the trade-off the refutation defends and the issue mandates; fail-closed defaults bound the exposure.

CI status

All required checks green: shellcheck, bats, unit-tests, CodeQL, SonarCloud, gitleaks, actionlint, agent-shield, and all governance validators SUCCESS. The three CANCELLED dev-lead /* jobs are the authoring agent's own orchestration runs, not verification gates. No CI security warnings.


Reviewed automatically by the PR-review agent (single-reviewer mode: fable 5). Reply if you need a human review.

Additional tasks

  1. Resolve all unresolved review thread comments from other reviewers
  2. Ensure all CI checks pass after your changes
  3. Rebase on the target branch if behind
  4. Do NOT modify files unrelated to the findings above

The review cascade will automatically re-review after new commits are pushed.

@don-petry
don-petry disabled auto-merge September 10, 2026 19:11
@don-petry

Copy link
Copy Markdown
Collaborator Author

Dev-Lead — fix-bot-comment (no-changes)

Agent reasoning
Issues addressed: 0
Files changed: none
Skipped (informational): 1
- SonarCloud Quality Gate Passed — no actionable defects found
```
No changes required. The PR passes all quality gates with zero new issues and zero security hotspots. The gemini-code-assist feedback about jq optimization is helpful context for future iterations but is not a blocking concern.

@don-petry
don-petry enabled auto-merge (squash) September 10, 2026 19:11
@donpetry-bot

Copy link
Copy Markdown
Contributor

Advisory bots were rate-limited; auto-approval is withheld until they recover. pr-review-sweep will re-review this PR after 2026-09-10T20:12:40Z.

@don-petry
don-petry disabled auto-merge September 10, 2026 19:33
@don-petry

Copy link
Copy Markdown
Collaborator Author

Dev-Lead — fix-bot-comment (no-changes)

Agent reasoning
Issues addressed: 0
- Quality Gate Passed — no actionable defects reported
Files changed: none
Skipped (informational): 1
```
**No changes required.** The PR passes all quality gates and CI checks. The in_progress checks (`bats`, `unit`) have not failed. No bot findings are actionable.

@don-petry
don-petry enabled auto-merge (squash) September 10, 2026 19:34
@donpetry-bot

donpetry-bot commented Sep 10, 2026 •

Copy link
Copy Markdown
Contributor
Superseded by automated re-review at 93557907a0c0a30e168d57fa29ffa3744c2bf9de — click to expand prior review.

Review — fix requested (cycle 2/3)

The automated review identified the following issues. Please address each one:

Findings to fix

Automated review — NEEDS HUMAN REVIEW

Risk: MEDIUM
Reviewed commit: 96902039b98de849cb983737adeaa042fa1cd2b5
Review mode: triage-approved (single reviewer)

Summary

Carry-forward escalation. The implementation of #1735 was assessed as correct and complete at the prior review (sha 2c3ffe8); the only commits since are merges of main that do not touch this PR's four files, and all CI is green at the new head. The single blocking gate is unchanged: one unresolved review thread on scripts/lib/addressed-claim-verify.sh:251 whose refutation carries no addressed-marker, so under harness-only resolution (#1691) only a human can resolve it. It blocks required_review_thread_resolution regardless of approval, and the CodeAnt custom instruction accepted in that thread is an org-wide review-policy decision that warrants owner ratification. No AI-actionable fix exists — owner action required.

Linked issue analysis

Closes #1735. All seven acceptance criteria remain substantively addressed (verified against the current diff, unchanged since the prior review):

Findings

Blocking (requires human action, carried forward from the prior review — unchanged):

  1. Unresolved review thread at scripts/lib/addressed-claim-verify.sh:251 — CodeAnt flagged (security, Major) that review_thread_is_agent_authored is text-only, so a copied marker could bypass the human-comment guard. The dev-lead refuted it with sound reasoning (the harness runs as the owner account so author identity cannot discriminate; Bot-acknowledged threads are permanently unresolvable after #1691 — harness must scan the whole thread, not comments(last:1) #1735 AC3 mandates reusing this discriminator) and CodeAnt accepted, saving a custom instruction. But (a) the refutation reply carries no addressed-marker, so the harness — before or after this fix — will never resolve the thread; under [#1621 S1] Harness-only thread resolution: remove the model's resolveReviewThread path #1691 only a human can, and it blocks the merge indefinitely. (b) The accepted instruction ("do not add author-based checks") is an org-wide review-policy decision that should be ratified by the owner. Action: owner reviews the refutation, resolves the thread manually; merge can then proceed on re-review. No new comments have landed on this thread since the prior review.

Non-blocking notes:

  1. Commits since the prior review (2c3ffe8..9690203) are merge-of-main only (docs(adr): ADR-0007 — one agent-ingress stub per repo, not one stub per role #1710 ADR, docs: refresh org READMEs (automated) #1695 README refresh, chore(deps): bump anthropics/claude-code-action from 1.0.211 to 1.0.217 in the actions group #1755 dependabot, feat: implement issue #1715 — Adopt the #1686 advisory context contract in the remaining eight persona prompts #1753); none touch this PR's files. Branch is now MERGEABLE.
  2. Secret-scan MCP tool not available in this run; the gitleaks CI check is green — diff is shell + bats only, no secret-shaped content.
  3. Residual accepted risk, documented: a human comment quoting an automation marker verbatim would be classified agent-authored and not block. This is the trade-off the refutation defends and the issue mandates; fail-closed defaults bound the exposure.

CI status

All checks green at 9690203: shellcheck, ShellCheck, bats, unit-tests, unit, CodeQL, Analyze (actions/python), SonarCloud (Quality Gate passed), gitleaks, actionlint, agent-shield, CodeRabbit, and all governance validators SUCCESS. Skipped jobs are ecosystem-audit and dev-lead orchestration lanes, not verification gates. No CI security warnings.


Reviewed automatically by the PR-review agent (single-reviewer mode: fable 5). Reply if you need a human review.

Additional tasks

  1. Resolve all unresolved review thread comments from other reviewers
  2. Ensure all CI checks pass after your changes
  3. Rebase on the target branch if behind
  4. Do NOT modify files unrelated to the findings above

The review cascade will automatically re-review after new commits are pushed.

@sonarqubecloud

Copy link
Copy Markdown

@don-petry
don-petry disabled auto-merge September 12, 2026 17:47
@don-petry

Copy link
Copy Markdown
Collaborator Author

Dev-Lead — fix-bot-comment (no-changes)

Agent reasoning
- **SonarCloud comment**: Reports "Quality Gate passed" with 0 new issues, 0 security hotspots, 0 accepted issues. This is a passing status report, not an actionable defect.
- **CI checks**: All completed checks show `conclusion: "success"`. No failures, timeouts, or action-required states.
- **Review states**: gemini-code-assist provided feedback in `COMMENTED` state (not `CHANGES_REQUESTED`), so it's not a Tier 1 blocker.
- **Per instructions**: A passing quality gate is not an actionable finding — it describes a successful scan, not a specific issue to fix.
**Conclusion:**
No changes needed. The PR has:
- ✓ Zero Tier 1 blockers (no failed checks, no CHANGES_REQUESTED reviews)
- ✓ SonarCloud Quality Gate passed with no issues identified
- ✓ All security and linting checks passed
The SonarCloud comment is confirmation that the code quality is good, not a defect to fix.

@don-petry
don-petry enabled auto-merge (squash) September 12, 2026 17:48

@donpetry-bot donpetry-bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review — APPROVED ✓

Risk: MEDIUM
Reviewed commit: 93557907a0c0a30e168d57fa29ffa3744c2bf9de
Review mode: triage-approved (single reviewer)

Summary

All blocking gates from prior cycles are now clear. The implementation of #1735 was verified correct and complete in cycles 1–2 (full-thread marker scan via acv_latest_marker_index, narrow fail-closed bot-ack/finding classifiers, #1415 human-block preserved, #1692 claim gate composed, all five AC6 cases covered at unit and harness level); the diff to this PR's four files is byte-identical since then — only merges of main landed (none touching these files). The single prior blocker — the unresolved CodeAnt thread at scripts/lib/addressed-claim-verify.sh:251 — has been resolved: the owner ratified the dev-lead's refutation and all five review threads on the PR are now resolved. CI is fully green at the reviewed head. Approving.

Linked issue analysis

Closes #1735 (open). All seven acceptance criteria remain substantively addressed, re-verified against the current diff:

  • AC1 — comments(last:1) replaced by acv_latest_marker_index, a single-jq-pass full-thread scan for the latest marker reply from our account; foreign markers do not authorize.
  • AC2 — acv_bot_comment_is_acknowledgement classifies post-marker bot comments; the finding regex is checked FIRST so mixed ack+finding blocks; the codeant 'Customized review instruction saved' shape clears.
  • AC3 — any marker-less human comment after our marker blocks unconditionally via the shared review_thread_is_agent_authored discriminator — no second classifier (#1415 preserved).
  • AC4 — fail-closed throughout: empty/ambiguous bodies rc2, negated acknowledgements ('cannot acknowledge') guarded before the ack match, unknown author types rc2, unparseable JSON rc2.
  • AC5 — the #1692 claim gate still runs on the marker body; both gates must pass.
  • AC6 — all five mandated cases present as pure unit tests (22 new cases in test_addressed_claim_verify.bats) plus four end-to-end harness cases in test_fix_reviews.bats asserting actual mutation/no-mutation, with harness exit status captured.
  • AC7 — ShellCheck and bats green.
    Dev-notes constraints honoured: #1691 untouched (harness remains sole resolver), #1692 built upon rather than bypassed.

Findings

No blocking findings.

  1. Prior blocker resolved — the CodeAnt thread at scripts/lib/addressed-claim-verify.sh:251 (text-only agent-authorship discriminator) that forced escalation in cycles 1 and 2 is now resolved, along with all four other review threads (gemini jq-loop performance ×2, codeant negated-ack, codeant harness exit-status — each fixed in code and confirmed). The owner has thereby ratified the accepted CodeAnt custom instruction, which was the human action the prior reviews required. No new comments or threads have appeared since.
  2. Commits since prior review (9690203..9355790) are a merge of main only (#1724 agent-ingress reference, lint.yml, seed-repo-template.sh); none touch this PR's four files.
  3. Secret scan: the run_secret_scanning MCP tool is not available in this run; the gitleaks CI check is green and the diff is shell + bats only — no secret-shaped content.
  4. Residual accepted risk, documented: a human comment quoting an automation marker verbatim classifies as agent-authored and does not block. This is the trade-off the ratified refutation defends and #1735 AC3 mandates; fail-closed defaults bound the exposure.

CI status

All checks green at 9355790: shellcheck, ShellCheck, bats, unit-tests, unit, CodeQL, Analyze (actions/python), SonarCloud, gitleaks (Secret scan), actionlint, agent-shield, CodeRabbit, holdout-guard, and all governance validators SUCCESS. Skipped jobs are ecosystem-specific audit lanes and dev-lead orchestration, not verification gates. No CI security warnings. PR is MERGEABLE.


Reviewed automatically by the PR-review agent (single-reviewer mode: fable 5). Reply if you need a human review.

@don-petry
don-petry merged commit fd5c58f into main Sep 12, 2026
68 checks passed
@don-petry
don-petry deleted the dev-lead/issue-1735-20260910-1150 branch September 12, 2026 17:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size:L This PR changes 100-499 lines, ignoring generated files

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Bot-acknowledged threads are permanently unresolvable after #1691 — harness must scan the whole thread, not comments(last:1)

2 participants