Skip to content

feat: implement issue #1918 — Approval gate can never clear bot status comments — SonarCloud re-posts 'Quality Gate passed' on every push and #1911's resolve script is self-scoped, so the #1894 queue is still deadlocked - #1920

Merged
don-petry merged 6 commits into
mainfrom
dev-lead/issue-1918-20260924-0304
Sep 24, 2026

Conversation

@don-petry

@don-petry don-petry commented Sep 24, 2026 •

Copy link
Copy Markdown
Collaborator

Problem

Approval gate can never clear bot status comments — SonarCloud re-posts 'Quality Gate passed' on every push and #1911's resolve script is self-scoped, so the #1894 queue is still deadlocked

From the issue: PR #1913 (implements #1908) was approved by pr-review twice, at 11:45Z on e42886f9 and 12:08Z on 103d0513, with no blocking findings. Both approvals were dismissed with Dismissing approval due to a PR issue comment lacking a verified disposition (#1813). The gate log (run 35858596981):

Risk

Low — changes automation shell logic under scripts/, covered by shellcheck (--severity=warning) and the bats suite.

Test plan

Tests added/updated: tests/dev-lead/unit/test_maintainer_comment_gate.bats,tests/dev-lead/unit/test_maintainer_resolve_comment.bats tests/test_reviewer_sources.bats. Verification: bash scripts/dev-lead-lint.sh (shellcheck --severity=warning) ran pre-commit; the bats suite runs in CI.

Rollback

Revert this PR. No non-revertible side effects (no tags, migrations, or external state).

Monitoring

This PR's Lint (shellcheck) and bats checks show pass/fail; watch subsequent dev-lead / pr-review runs for behavioral regressions.

Closes #1918

Review in cubic

Rollout: a pr-review channel cut IS required after merge

maintainer-comment-gate.sh runs inside pr-review, which in this repo executes from the channel tag pr-review/v1-next (.github/workflows/pr-review-trigger.yml). That tag is at eee2898e (2026-09-23 03:55Z) and does not contain #1911 either. Until the channel moves, this merge changes nothing at runtime, because the autocut only watches the reusable-workflow blob (#1592).

After merge:

scripts/cut-release.sh pr-review <next-version> --ref main --channel v1-next --dry-run
scripts/cut-release.sh pr-review <next-version> --ref main --channel v1-next --push
git fetch --tags --force && git merge-base --is-ancestor <merge-sha> pr-review/v1-next && echo live

Review follow-up (cd07490)

Summary by CodeRabbit

  • New Features
    • Clean SonarCloud status comments with zero new issues and zero security hotspots no longer block PR approval.
    • Maintainers can resolve comments from registered status bots by providing a reason. Replies are recorded on the original comment, and comments from other people remain protected.
  • Bug Fixes
    • PR comments without a verified resolution continue to block approval. Failed or unclear status comments, reported findings, and unverifiable authorship or registry information are handled safely.
  • Documentation
    • Updated guidance explains how PR comments are reviewed, resolved, and handled when status information cannot be verified.

…s comments — SonarCloud re-posts 'Quality Gate passed' on every push and #1911's resolve script is self-scoped, so the #1894 queue is still deadlocked
@qodo-code-review

This comment has been minimized.

@coderabbitai

This comment has been minimized.

@don-petry

Copy link
Copy Markdown
Collaborator Author

Dev-Lead — waiting on PR blockers (intent: review-changes)

PR: #1920
No changes were committed, but the PR still can't be marked done: required checks SonarCloud, agent-shield / AgentShield are still pending. The retry cron will re-attempt automatically. Next attempt after: 2026-09-24T03:49:31Z

@don-petry

Copy link
Copy Markdown
Collaborator Author

Note

@don-petry I reviewed this PR and no code changes were needed, but I can't mark it done yet: required checks SonarCloud, agent-shield / AgentShield are still pending. I'll re-check automatically.
Next attempt after: 2026-09-24T03:49:31Z

@don-petry
don-petry enabled auto-merge (squash) September 24, 2026 03:19
@donpetry-bot

Copy link
Copy Markdown
Contributor

Advisory bots were rate-limited; auto-approval is withheld until they recover. pr-review-sweep will re-review this PR after 2026-09-24T04:20:22Z.

@don-petry

Copy link
Copy Markdown
Collaborator Author

No description provided.

@don-petry
don-petry disabled auto-merge September 24, 2026 03:20

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a data-driven mechanism to prevent clean informational status comments from registered reviewer bots (such as SonarCloud's 'Quality Gate passed') from deadlocking PR approvals, and extends the maintainer resolution script to allow manual resolution of bot comments with a mandatory reason. The review feedback highlights several opportunities to improve shell script safety and efficiency, including avoiding command substitutions in conditional lists under 'set -e', adding error handling for 'mktemp', using pure Bash substring checks and parameter expansions instead of loops and subshells, and utilizing jq's 'sub' function for cleaner regex replacements.

Comment thread scripts/lib/maintainer-comment-gate.sh Resolved
Comment thread scripts/maintainer-resolve-comment.sh Resolved
Comment thread scripts/maintainer-resolve-comment.sh Resolved
Comment thread scripts/lib/maintainer-comment-gate.sh Resolved
Comment thread scripts/maintainer-resolve-comment.sh Resolved
Comment thread scripts/maintainer-resolve-comment.sh Resolved
Comment thread scripts/maintainer-resolve-comment.sh Resolved
Comment thread scripts/lib/maintainer-comment-gate.sh Resolved
…th requires a Bot actor

Review follow-up for #1918 (PR #1920).

1. The sonarqubecloud info_status_pattern was the bare words "Quality Gate
   passed". A SonarCloud gate can PASS while still reporting new issues or
   security hotspots, depending on its conditions. Those are findings, and #1918
   AC1 says a status comment clears only when it has "no new issues". The pattern
   now pins the headline AND both zero-count lines:
     \*\*Quality Gate passed\*\*[\s\S]*\[0 New issues\][\s\S]*\[0 Security Hotspots\]
   The leading "[" stops "[10 New issues]" from matching.
   Gate tests now use SonarCloud's real comment shape and add blocking cases for
   2 new issues, 1 hotspot, "10 New issues", and bare headline text. The four new
   cases fail on the previous pattern and pass on this one.

2. maintainer-resolve-comment.sh authorized the registered-bot path on login
   alone. It now also reads author.__typename and requires "Bot", so a person
   whose login matches a registered bot's is refused (exit 3) like any other
   human. A shim-backed test covers User → refused with no write and Bot →
   authorized. The User test fails on the previous code.

AGENTS.md is updated to match. 139/139 in the gate, resolve, registry and
scorecard suites; shellcheck and markdownlint clean.

Refs #1918

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@don-petry

Copy link
Copy Markdown
Collaborator Author

Dev-Lead — fix-bot-comment (no-changes)

Agent reasoning
Issues addressed: 0
- SonarCloud QG passed with no issues (neutral observation, not actionable)
Tier 1 blockers: 0
- template-drift fails but is explicitly non-required per AGENTS.md
Files changed: none
Status: No changes needed — PR is mergeable
```

@don-petry
don-petry enabled auto-merge (squash) September 24, 2026 03:22
@donpetry-bot

Copy link
Copy Markdown
Contributor

Advisory bots were rate-limited; auto-approval is withheld until they recover. pr-review-sweep will re-review this PR after 2026-09-24T04:23:17Z.

@cubic-dev-ai cubic-dev-ai Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed

Tip: instead of fixing issues one by one fix them all with cubic

Re-trigger cubic

Comment thread scripts/lib/maintainer-comment-gate.sh Resolved
Comment thread scripts/lib/maintainer-comment-gate.sh Resolved
Comment thread scripts/maintainer-resolve-comment.sh Resolved
Comment thread scripts/maintainer-resolve-comment.sh Resolved
Comment thread scripts/maintainer-resolve-comment.sh Resolved
Comment thread tests/test_reviewer_sources.bats Resolved
Comment thread AGENTS.md Resolved

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: cd07490020

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread scripts/lib/maintainer-comment-gate.sh Resolved
Comment thread scripts/maintainer-resolve-comment.sh Resolved
Comment thread scripts/maintainer-resolve-comment.sh Resolved
@don-petry
don-petry disabled auto-merge September 24, 2026 03:28

@cubic-dev-ai cubic-dev-ai Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 8 files

Requires human review: Auto-approval blocked because this review re-detected 7 unresolved issues already reported by Cubic.

Fix all with cubic | Re-trigger cubic

Comment thread scripts/maintainer-resolve-comment.sh Resolved
Comment thread AGENTS.md Resolved
Comment thread scripts/lib/maintainer-comment-gate.sh Resolved
@donpetry-bot

donpetry-bot commented Sep 24, 2026 •

Copy link
Copy Markdown
Contributor
Superseded by automated re-review at d63e05bd050d74a603fb0240c1c9575dbd738011 — click to expand prior review.

Review — fix requested (cycle 1/3)

The automated review identified the following issues. Please address each one:

Findings to fix

Automated review — NEEDS HUMAN REVIEW

Risk: MEDIUM
Reviewed commit: cd0749002055a7da40e3d239f828e2e81b6bf10b
Cascade: triage → audit (triage: haiku 4.5 [sonnet 5] → deep: opus 4.8 [sonnet 5] + duck: o4-mini → audit: fable 5)

Summary

Security audit finds no exploitable defect: the escape hatch grants no capability beyond what write-access users already have via GitHub's native hide-as-Resolved UI (which the gate already accepts), and it adds __typename==Bot verification plus a mandatory recorded --reason audit reply — a net auditability improvement. The deep review's major authorization finding is downgraded to a minor hardening recommendation (restrict the bot path to info_status_pattern sources), and codeant's Critical marker finding is a confirmed false alarm. Escalating solely because the template-drift CI gate is FAILURE — verified environmental (drift in petry-projects/repo-template stubs this PR does not touch and cannot fix) — and the audit mandate requires all gates green to approve; once that check is green or waived, this PR is safe to approve from a security standpoint.

Downstream impact

This change is consumed by 8 downstream repo(s) that pin the affected reusable workflow / lib / prompt. Impacted consumers:

Impacted shared surfaces:
  - .github/workflows/dev-lead-reusable.yml
  - .github/workflows/pr-review.yml

Impacted consumers (8, fetching up to 10):
  - petry-projects/.github (pins .github/workflows/dev-lead-reusable.yml)
      .github/workflows/dev-lead.yml
  - petry-projects/.github-private (pins .github/workflows/dev-lead-reusable.yml,.github/workflows/pr-review.yml)
      .github/workflows/dev-lead.yml
      .github/workflows/pr-review-canary.yml
      .github/workflows/pr-review-trigger.yml
      .github/workflows/test-dev-lead.yml
  - petry-projects/ContentTwin (pins .github/workflows/dev-lead-reusable.yml,.github/workflows/pr-review.yml)
      .github/workflows/dev-lead.yml
      .github/workflows/pr-review.yml
  - petry-projects/TalkTerm (pins .github/workflows/dev-lead-reusable.yml)
      .github/workflows/dev-lead.yml
  - petry-projects/bmad-bgreat-suite (pins .github/workflows/dev-lead-reusable.yml,.github/workflows/pr-review.yml)
      .github/workflows/dev-lead.yml
      .github/workflows/pr-review.yml
  - petry-projects/broodly (pins .github/workflows/dev-lead-reusable.yml)
      .github/workflows/dev-lead.yml
  - petry-projects/google-app-scripts (pins .github/workflows/dev-lead-reusable.yml,.github/workflows/pr-review.yml)
      .github/workflows/dev-lead.yml
      .github/workflows/pr-review.yml
  - petry-projects/markets (pins .github/workflows/dev-lead-reusable.yml,.github/workflows/pr-review.yml)
      .github/workflows/dev-lead.yml
      .github/workflows/pr-review.yml

Findings

  • minor: The registered-bot resolve path authorizes dispositioning any reviewer-sources.tsv login's comment, including finding-producing bots (codeant-ai, graphite-app). Downgraded from the deep review's major: this grants NO new capability — any write-access user can already minimize any comment as RESOLVED via GitHub's native hide-comment UI, which the gate accepts identically — and this path is strictly more audited (authenticated viewer, __typename==Bot, mandatory --reason posted as a marked reply before minimize, fail-closed on every branch). Hardening recommendation, non-blocking: restrict the bot path to sources declaring an info_status_pattern. (scripts/maintainer-resolve-comment.sh:433)
  • info: Asymmetry: the gate's auto-clear classifier matches on bare login + body pattern only, with no author __typename check (unlike maintainer-resolve-comment.sh, which added one for exactly this impersonation class). A hypothetical USER account literally named 'sonarqubecloud' posting a matching body would be auto-cleared. Not exploitable in practice: the only effect is making the attacker's own comment non-blocking, equivalent to not commenting at all; real SonarCloud failing/finding-bearing comments still block independently. Defense-in-depth: include author.__typename in the gate snapshot and require Bot. (scripts/lib/maintainer-comment-gate.sh:119)
  • info: codeant Critical ('anyone can add the maintainer-resolve marker to skip the gate') is a confirmed false alarm: a marker only self-excludes a comment the actor authored or can edit, granting nothing beyond staying silent; the RESOLVED-minimization requirement for all other comments is untouched. Same class as the pre-existing dev-lead/pr-review-agent/dependency-advisory markers. (scripts/lib/maintainer-comment-gate.sh:52)
  • info: The bot-path reply is posted before minimize permission is exercised: a caller with comment-but-not-write access would leave the audit reply and then fail the minimize. Harmless (such a caller can post comments anyway) and the ordering is intentionally fail-closed for the audit trail; noting for completeness only. (scripts/maintainer-resolve-comment.sh:476)
  • info: Injection review clean: reply body built with printf %s and sent via gh api -F body=@file (no shell/JSON interpolation); the repos/{owner}/{repo}/issues/{n}/comments endpoint components come from strictly-anchored regexes over the GraphQL-returned comment URL (trusted origin, no path traversal); the jq test() pattern is loaded from the in-repo reviewer-sources.tsv, same review-gated trust domain as code, with linear (non-backtracking-explosive) regexes. No secrets, crypto, GitHub Actions workflow, or dependency changes in the diff; gitleaks, CodeQL, actionlint, and agent-shield are all green.
  • minor: template-drift check is FAILURE, verified environmental via the job log: it reports drift of agent-shield.yml, copilot-setup-steps.yml, and dependency-audit.yml stubs in petry-projects/repo-template against the standards baseline — none touched by this PR, and this PR cannot resolve it (fix is re-seeding repo-template via scripts/seed-repo-template.sh). This is the sole reason for the escalate decision; the diff itself is security-approved.

Reviewed by the PR-review cascade (triage: haiku 4.5 [sonnet 5] → deep: opus 4.8 [sonnet 5] + duck: o4-mini → audit: fable 5). Reply if you need a human review.

Additional tasks

  1. Resolve all unresolved review thread comments from other reviewers
  2. Ensure all CI checks pass after your changes
  3. Rebase on the target branch if behind
  4. Do NOT modify files unrelated to the findings above

The review cascade will automatically re-review after new commits are pushed.

@don-petry

Copy link
Copy Markdown
Collaborator Author

Dispositioning the Qodo status comment above: it is an operational trial-ended / billing notice ("Qodo reviews are paused because your trial has ended"), not a code-review finding — there is nothing in the diff to change and the state is resolved by workspace billing, outside this PR. Recording it as informational so it does not hold the approval gate.

@don-petry

Copy link
Copy Markdown
Collaborator Author

Dispositioning the CodeRabbit status comment above: it is an auto-generated review rate-limit notice ("Review limit reached … Next included review available in 3 minutes"), carrying no finding about this PR. Recording it as informational so it does not hold the approval gate.

@don-petry

Copy link
Copy Markdown
Collaborator Author

Dev-Lead — fix-reviews (partial)

A commit was pushed, but not every requested change was applied. Per requested item:

  • scripts/lib/maintainer-comment-gate.sh:103 — applied
  • scripts/maintainer-resolve-comment.sh:158 — applied
  • scripts/maintainer-resolve-comment.sh:434 — applied
  • scripts/lib/maintainer-comment-gate.sh:161 — not applied
  • scripts/maintainer-resolve-comment.sh:169 — applied
  • scripts/maintainer-resolve-comment.sh:399 — not applied
  • scripts/maintainer-resolve-comment.sh:439 — applied
  • scripts/lib/maintainer-comment-gate.sh:74 — not applied
  • scripts/lib/maintainer-comment-gate.sh:74 — not applied
  • scripts/lib/maintainer-comment-gate.sh:169 — not applied
  • scripts/maintainer-resolve-comment.sh:412 — applied
  • scripts/maintainer-resolve-comment.sh:394 — applied
  • scripts/maintainer-resolve-comment.sh:439 — applied
  • tests/test_reviewer_sources.bats:163 — applied
  • AGENTS.md:111 — applied
  • tests/dev-lead/unit/test_maintainer_resolve_comment.bats:161 — not applied
  • scripts/lib/maintainer-comment-gate.sh:169 — not applied
  • scripts/maintainer-resolve-comment.sh:439 — applied
  • scripts/maintainer-resolve-comment.sh:384 — applied

The unaddressed items above still need work.

@don-petry
don-petry enabled auto-merge (squash) September 24, 2026 03:53
@don-petry

Copy link
Copy Markdown
Collaborator Author

Dev-Lead — fix-bot-comment (no-changes)

Agent reasoning
Issues addressed: 0
- Quality Gate Passed (informational): No actionable defects — this is a neutral status comment
Files changed: None
Skipped (informational): 1 (SonarCloud passing status)
```
**No changes required.** The PR has zero Tier 1 blockers and the SonarCloud comment contains no specific, actionable defects. All reviews indicate the work is sound.

@don-petry
don-petry enabled auto-merge (squash) September 24, 2026 12:04

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: d63e05bd05

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread scripts/maintainer-resolve-comment.sh Resolved
@don-petry
don-petry disabled auto-merge September 24, 2026 12:09
don-petry added a commit that referenced this pull request Sep 24, 2026
The existing-reply check used test("…id=" + $id + "[^>]*-->"), so for node
IC_abc it also matched a reply marked id=IC_abc123. The trailing [^>]* absorbed
the rest of the other id. The script then skipped posting THIS comment's
required audit reply, but still minimized the comment RESOLVED, leaving no
recorded reason (Codex P2 on PR #1920).

Move the match into a pure helper, mrc_existing_reply_ids. The node id must now
be a whole token: preceded by whitespace and followed by whitespace or the
closing "-->". The viewer-only filter (anti-forgery) and fail-closed-on-bad-JSON
behaviour are unchanged. New tests cover the exact match, the prefix collision
(red on the old pattern), the suffix collision, a non-viewer forgery, and
unparseable input.

Refs #1918

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@don-petry

Copy link
Copy Markdown
Collaborator Author

Dispositioning the SonarCloud status comment above: it is a clean Quality Gate passed notice (0 new issues, 0 accepted, 0 security hotspots), not a code-review finding — there is nothing in the diff to change. Recording it as informational so this passing-gate status does not hold the approval gate.

@don-petry
don-petry force-pushed the dev-lead/issue-1918-20260924-0304 branch from 4374cd2 to c9c10d5 Compare September 24, 2026 12:14
@don-petry

Copy link
Copy Markdown
Collaborator Author

Dev-Lead — fix-reviews (applied)

Changes committed and pushed. Requested items addressed:

  • scripts/maintainer-resolve-comment.sh:480 — applied

@don-petry
don-petry enabled auto-merge (squash) September 24, 2026 12:14
@sonarqubecloud

Copy link
Copy Markdown

@don-petry
don-petry disabled auto-merge September 24, 2026 12:16
@don-petry

Copy link
Copy Markdown
Collaborator Author

Dev-Lead — fix-bot-comment (no-changes)

Agent reasoning
Issues addressed: 0
Files changed: none
Skipped (informational): 1 — SonarCloud quality gate passed; no actionable findings
```
No changes required. The PR is ready.

@don-petry
don-petry enabled auto-merge (squash) September 24, 2026 12:16
@donpetry-bot

donpetry-bot commented Sep 24, 2026 •

Copy link
Copy Markdown
Contributor
Superseded by automated re-review at c9c10d5201257b9de2a1a4b57b5f520f6abc9828 — click to expand prior review.

Review — fix requested (cycle 2/3)

The automated review identified the following issues. Please address each one:

Findings to fix

Automated review — NEEDS HUMAN REVIEW

Risk: MEDIUM
Reviewed commit: d63e05bd050d74a603fb0240c1c9575dbd738011
Cascade: triage → audit (triage: haiku 4.5 [sonnet 5] → deep: opus 4.8 [sonnet 5] + duck: o4-mini → audit: fable 5)

Summary

Substantively, the security posture of PR #1920 is sound: the resolver bot-path is well hardened (__typename==Bot required, bots barred from the self path, registry pinned to the repo-local TSV, mandatory --reason posted as a marked reply before minimize, fail-closed reply-list/parse handling, viewer-bound reply dedup per CWE-345), and the gate classifier fails closed with a zero-count-pinned SonarCloud pattern so failing or finding-bearing statuses still block. The two remaining authenticity gaps (gate-side missing Bot actor-type check; body-trusted maintainer-resolve marker) are real but bounded — each only lets an account exempt ITS OWN comment, suppressing no other author's finding — and are formally tracked in open follow-up issue #1921 with all 31 review threads now resolved (the deep tier's unresolved-threads signal is stale). Escalating anyway because approval cannot safely bind: the head advanced twice during this audit (d63e05b -> 4374cd2 -> c9c10d5, a TOCTOU on the verdict SHA), bats/unit/cubic checks are still IN_PROGRESS at head, and mergeStateStatus is BLOCKED with reviewDecision REVIEW_REQUIRED.

Downstream impact

This change is consumed by 8 downstream repo(s) that pin the affected reusable workflow / lib / prompt. Impacted consumers:

Impacted shared surfaces:
  - .github/workflows/dev-lead-reusable.yml
  - .github/workflows/pr-review.yml

Impacted consumers (8, fetching up to 10):
  - petry-projects/.github (pins .github/workflows/dev-lead-reusable.yml)
      .github/workflows/dev-lead.yml
  - petry-projects/.github-private (pins .github/workflows/dev-lead-reusable.yml,.github/workflows/pr-review.yml)
      .github/workflows/dev-lead.yml
      .github/workflows/pr-review-canary.yml
      .github/workflows/pr-review-trigger.yml
      .github/workflows/test-dev-lead.yml
  - petry-projects/ContentTwin (pins .github/workflows/dev-lead-reusable.yml,.github/workflows/pr-review.yml)
      .github/workflows/dev-lead.yml
      .github/workflows/pr-review.yml
  - petry-projects/TalkTerm (pins .github/workflows/dev-lead-reusable.yml)
      .github/workflows/dev-lead.yml
  - petry-projects/bmad-bgreat-suite (pins .github/workflows/dev-lead-reusable.yml,.github/workflows/pr-review.yml)
      .github/workflows/dev-lead.yml
      .github/workflows/pr-review.yml
  - petry-projects/broodly (pins .github/workflows/dev-lead-reusable.yml)
      .github/workflows/dev-lead.yml
  - petry-projects/google-app-scripts (pins .github/workflows/dev-lead-reusable.yml,.github/workflows/pr-review.yml)
      .github/workflows/dev-lead.yml
      .github/workflows/pr-review.yml
  - petry-projects/markets (pins .github/workflows/dev-lead-reusable.yml,.github/workflows/pr-review.yml)
      .github/workflows/dev-lead.yml
      .github/workflows/pr-review.yml

Findings


Reviewed by the PR-review cascade (triage: haiku 4.5 [sonnet 5] → deep: opus 4.8 [sonnet 5] + duck: o4-mini → audit: fable 5). Reply if you need a human review.

Additional tasks

  1. Resolve all unresolved review thread comments from other reviewers
  2. Ensure all CI checks pass after your changes
  3. Rebase on the target branch if behind
  4. Do NOT modify files unrelated to the findings above

The review cascade will automatically re-review after new commits are pushed.

@donpetry-bot donpetry-bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review — APPROVED ✓

Risk: MEDIUM
Reviewed commit: c9c10d5201257b9de2a1a4b57b5f520f6abc9828
Review mode: triage-approved (single reviewer)

Summary

Re-review at settled head c9c10d5: the prior cascade escalation was purely procedural (head moved mid-audit; bats/unit/cubic still running) and both conditions are now cleared — head matches $PR_HEAD_SHA and every completed check is green. Substance was already cleared by the deep+audit tiers: the data-driven info-status classifier fails closed (unreadable registry auto-clears nothing), the SonarCloud pattern pins the passed headline plus both zero counts so a finding-bearing pass still blocks, and the resolver bot path requires registry membership, __typename==Bot, and a mandatory --reason reply before minimizing. The only diff since the prior review is the boundary-delimited node-id dedup fix plus its regression test, which the prior audit had already examined and called benign. Residual minor hardening (gate-side Bot actor-type check, body-trusted marker model) is tracked in open follow-up #1921 as accepted.

Linked issue analysis

Closes #1918. All five acceptance criteria are substantively met: (1) clean bot status comments are auto-cleared deterministically via the info_status_pattern column in reviewer-sources.tsv (schema v3), keyed by author login + body pattern with no LLM; the pattern requires Quality Gate passed AND [0 New issues] AND [0 Security Hotspots] per the issue's pattern-precision note; (2) failed/unknown/finding-bearing comments still block — the gate fails closed to clearing nothing on an unreadable registry, and tests cover passed-clears / failed-blocks / unknown-blocks including a passed-with-2-new-issues case; (3) maintainer-resolve-comment.sh gains a registered-bot path requiring __typename==Bot + registry membership + --reason posted as a marked reply before minimize, while another human's comment is still refused; (4) AGENTS.md documents the escape hatch next to the #1911 entry; (5) regression tests cover the deadlock loop. The rollout requirement is met: the PR body names the required pr-review/v1-next channel cut and the exact cut-release.sh commands.

Findings

  • Resolved (was major): head-SHA TOCTOU — the branch has settled; headRefOid c9c10d5 equals the review target.
  • Resolved (was major): in-progress checks — bats, unit, and cubic are now COMPLETED/SUCCESS; every completed check at head is green.
  • Carried as accepted residual (minor): gate-side info-status auto-clear keys on login + body pattern without a Bot actor-type check, and the maintainer-resolve reply extends the pre-existing body-trusted marker model. Both are bounded (an account can only exempt its own comment) and tracked in open issue #1921.
  • New commits since prior review (d63e05b→c9c10d5): only the audit-reply node-id dedup boundary fix (id must be followed by whitespace or -->, preventing prefix collisions) plus a regression test — already reviewed and rated benign by the prior audit.
  • Note: the mcp__github__run_secret_scanning MCP tool is not available in this environment; the gitleaks CI check (Secret scan) is green.
  • Note: the CodeRabbit commit status is stale-PENDING; it is not a required check (required: SonarCloud, CodeQL, agent-shield/AgentShield, dependency-audit/Detect ecosystems, duplicate-decl-gate — all green), its rate-limit notice was dispositioned, and the registry classifies coderabbitai as advisory_gate=no (not part of the approval-wait set).

CI status

All completed checks green at c9c10d5, including all 5 required checks (SonarCloud, CodeQL, agent-shield/AgentShield, dependency-audit/Detect ecosystems, duplicate-decl-gate), plus bats, unit-tests, shellcheck, gitleaks, CodeQL analyze (actions/python), SonarCloud Code Analysis, Graphite, and cubic. Non-required CodeRabbit status is stale-PENDING (rate-limited vendor; excluded from the approval-wait set). No failures. 0 unresolved review threads.


Reviewed automatically by the PR-review agent (single-reviewer mode: fable 5). Reply if you need a human review.

@don-petry
don-petry merged commit 1d6b38b into main Sep 24, 2026
71 of 72 checks passed
@don-petry
don-petry deleted the dev-lead/issue-1918-20260924-0304 branch September 24, 2026 12:24

@donpetry-bot donpetry-bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review — APPROVED ✓

Risk: MEDIUM
Reviewed commit: c9c10d5201257b9de2a1a4b57b5f520f6abc9828
Review mode: triage-approved (single reviewer)

Summary

Confirmation re-review at the same SHA a prior cycle approved. Implements #1918: clean bot status comments (SonarCloud's re-posted 'Quality Gate passed') auto-clear from the maintainer-comment gate via a data-driven classifier in reviewer-sources.tsv (schema v3, info_status_pattern column), and maintainer-resolve-comment.sh gains a registered-info-status-bot path requiring --reason with an audit reply. Design is fail-closed at every branch and thoroughly bats-tested. Triage's low-risk call is confirmed; risk is MEDIUM only because the change touches approval-gate logic.

Linked issue analysis

Closes #1918 (now closed). All acceptance criteria are substantively addressed:

  • AC1 — clean bot status never blocks: deterministic, LLM-free classifier in maintainer-comment-gate.sh keyed off the reviewer-source registry (scripts/lib/reviewer-sources.tsv, new info_status_pattern column) — patterns live in data, not hard-coded logins, exactly as the issue suggested. The SonarCloud pattern pins the **Quality Gate passed** headline AND the [0 New issues] / [0 Security Hotspots] lines.
  • AC2 — findings still block: Quality Gate failed, a passed gate with nonzero new issues/hotspots, an unclassifiable body, a bot with no pattern, and every human comment all still block (each covered by a bats test, including the 10 New issues prefix-match trap).
  • AC3 — maintainer escape hatch: maintainer-resolve-comment.sh accepts a registered info-status bot's comment, requires --reason posted as a marked audit reply before the minimize, requires a GitHub App actor (__typename == Bot) so a person with a bot-matching login is refused, and still refuses other humans and finding-producing bots — preserving the #1910 self-scope restriction.
  • AC5 — whole-loop regression: an approving review plus a later clean SonarCloud pass stays cleared (tested), so the approve→dismiss→re-review cycle burn this issue documents cannot recur for clean passes.
    Docs (AGENTS.md, maintainer-comment-gate.md, implementation.md) and the consumer manifest / downstream-impact / validator were updated consistently for the new .tsv surface source.

Findings

No blocking findings. Notable strengths verified in this pass:

  • Fail-closed everywhere: an unreadable registry degrades to {} (nothing auto-cleared, never clearing the unclassifiable); the gate's jq classifier only drops a comment when the author is registered AND the body matches its pattern; malformed snapshots still return 2.
  • Resolve-script hardening: repo-local registry pinned (a REVIEWER_SOURCES_MANIFEST override cannot widen authorization); reply-dedup lookup is scoped to the invoking viewer so a forged maintainer-resolve marker cannot suppress the audit reply (CWE-345, tested); full-node-id + delimiter matching prevents prefix-id confusion; failed comment-list reads abort before any write; reply is posted before the minimize so a failed reply fails the whole action closed.
  • Loop safety: the maintainer-resolve marker was added to the gate's agent-marker set, so the audit reply never becomes a fresh blocker (round-trip tested).
  • Secret scan: the run_secret_scanning MCP tool is not available in this run; the gitleaks CI check passed. No secrets or credentials appear in the diff (patterns/logins/docs only).

CI status

All required and repo-defined checks green: SonarCloud, agent-shield / AgentShield, shellcheck, bats, unit-tests, CodeQL (actions + python), Agent Security Scan, actionlint, gitleaks, holdout-guard, and all validator/guard jobs SUCCESS (irrelevant ecosystem audits SKIPPED). The only non-completed item is the third-party CodeRabbit advisory status stuck PENDING — its rate-limit notice was dispositioned as informational and its stale review was dismissed; it is not a required check and does not gate merge. No unresolved review threads; no unanswered human-reviewer questions.


Reviewed automatically by the PR-review agent (single-reviewer mode: fable 5). Reply if you need a human review.

@donpetry-bot donpetry-bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review — APPROVED ✓

Risk: MEDIUM
Reviewed commit: c9c10d5201257b9de2a1a4b57b5f520f6abc9828
Review mode: triage-approved (single reviewer)

Summary

Implements issue #1918: a data-driven info-status classifier so clean bot status comments (chiefly SonarCloud's re-posted 'Quality Gate passed') never deadlock the approval gate, plus a maintainer escape hatch to disposition a registered info-status bot's comment with a mandatory --reason audit reply. All prior-cycle findings are resolved at this head; triage's low-risk clearance is confirmed as accurate for content that is well-tested and fail-closed throughout.

Linked issue analysis

Linked issue #1918 (closes) is substantively addressed on all three acceptance criteria:

  • AC1 — clean bot status never blocks, deterministically, no LLM: reviewer-sources.tsv (schema v3) gains an info_status_pattern column; maintainer-comment-gate.sh auto-clears a comment only when the author is a registered source AND the body matches its pattern. The SonarCloud pattern pins the 'Quality Gate passed' headline plus the literal [0 New issues] and [0 Security Hotspots] lines. Patterns live in data, not hard-coded logins, exactly as the issue requested.
  • AC2 — findings still block: 'Quality Gate failed', a passing gate with non-zero counts ('[10 New issues]' cannot match '[0 New issues]' because the bracket is pinned), an unclassifiable body, a bot with no pattern, and every human comment all still block. An unreadable registry degrades to '{}' — nothing is auto-cleared.
  • AC3 — maintainer escape hatch: maintainer-resolve-comment.sh now accepts a registered info-status bot's comment (author must be a GraphQL Bot actor, so a person with a matching login is refused), requires --reason, and posts a marked audit reply before minimizing. Another human's comment and finding-producing bots (codeant-ai, graphite-app) remain refused, preserving the #1910 self-scope restriction.

Findings

No blocking findings. Prior-cycle (d63e05b) findings are resolved at this head:

  • Reply-dedup lookup is now viewer-scoped, so a forged maintainer-resolve marker from another login cannot suppress the audit reply (CWE-345) — covered by a dedicated bats test.
  • The dedup marker match requires the FULL node id followed by a delimiter, so IC_...fake12 cannot satisfy a lookup for IC_...fake1 — covered by a dedicated bats test.
  • A failed or unparseable comments-list lookup aborts before posting or minimizing (fail closed) — covered by a dedicated bats test.

Security review of the new surface: the registry path is pinned inside the resolve script so a REVIEWER_SOURCES_MANIFEST override cannot widen authorization; the audit reply is posted BEFORE the minimize so a failed reply fails the whole action closed; the gate's classifier only ever REMOVES a blocker for a finding-free clean-pass body, and every failure mode (unreadable registry, jq failure, empty output) degrades to blocking. 14 files, +816/−89, with docs (AGENTS.md, gate + implementation docs), consumer-manifest/downstream-impact wiring for the new .tsv surface, and ~20 new bats tests spanning classifier precision, authorization, idempotency, and fail-closed paths.

Secret scan: the run_secret_scanning MCP tool is not available in this environment; the gitleaks CI check passed and the diff introduces no credentials.

CI status

All checks green: shellcheck, bats, unit-tests, actionlint, CodeQL (actions + python), SonarCloud quality gate, gitleaks secret scan, agent-shield, and all validate-* / guard checks SUCCESS; remaining entries SKIPPED by design. The only queued entry is pr-auto-review / check-and-dispatch — the orchestrator running this very review. No unresolved review threads; no unanswered human-reviewer questions (all owner-login comments carry dev-lead automation markers).


Reviewed automatically by the PR-review agent (single-reviewer mode: fable 5). Reply if you need a human review.

@donpetry-bot

Copy link
Copy Markdown
Contributor

pr-review approved on PARTIAL advisory evidence: 5/7 required advisory bots reported before the gate's head-age-timeout fallback proceeded. Recorded for the miss-rate metric (#1596).

don-petry added a commit that referenced this pull request Sep 25, 2026
Resolve the reviewer-sources.tsv conflict with main. Since this branch was cut,
#1920 (issue #1918) added a seventh column, info_status_pattern, and bumped
schema_version to 3. Take main's rows unchanged and add cubic-dev-ai as a
seven-column row, with "-" for both check_run_name and info_status_pattern.
cubic reports through PR reviews, not a check run, and has no known-clean
status-comment pattern. Its rationale is unchanged.

Verified: 235/235 in the reviewer-sources, reviewer-report, advisory-gate,
intent-reviews, maintainer-comment-gate and miss-rate suites; shellcheck clean.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

2 participants