Repository navigation
feat: implement issue #1918 — Approval gate can never clear bot status comments — SonarCloud re-posts 'Quality Gate passed' on every push and #1911's resolve script is self-scoped, so the #1894 queue is still deadlocked - #1920
Conversation
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
Dev-Lead — waiting on PR blockers (intent: review-changes)PR: #1920 |
|
Note @don-petry I reviewed this PR and no code changes were needed, but I can't mark it done yet: required checks |
|
Advisory bots were rate-limited; auto-approval is withheld until they recover. pr-review-sweep will re-review this PR after 2026-09-24T04:20:22Z. |
|
No description provided. |
There was a problem hiding this comment.
Code Review
This pull request introduces a data-driven mechanism to prevent clean informational status comments from registered reviewer bots (such as SonarCloud's 'Quality Gate passed') from deadlocking PR approvals, and extends the maintainer resolution script to allow manual resolution of bot comments with a mandatory reason. The review feedback highlights several opportunities to improve shell script safety and efficiency, including avoiding command substitutions in conditional lists under 'set -e', adding error handling for 'mktemp', using pure Bash substring checks and parameter expansions instead of loops and subshells, and utilizing jq's 'sub' function for cleaner regex replacements.
…th requires a Bot actor Review follow-up for #1918 (PR #1920). 1. The sonarqubecloud info_status_pattern was the bare words "Quality Gate passed". A SonarCloud gate can PASS while still reporting new issues or security hotspots, depending on its conditions. Those are findings, and #1918 AC1 says a status comment clears only when it has "no new issues". The pattern now pins the headline AND both zero-count lines: \*\*Quality Gate passed\*\*[\s\S]*\[0 New issues\][\s\S]*\[0 Security Hotspots\] The leading "[" stops "[10 New issues]" from matching. Gate tests now use SonarCloud's real comment shape and add blocking cases for 2 new issues, 1 hotspot, "10 New issues", and bare headline text. The four new cases fail on the previous pattern and pass on this one. 2. maintainer-resolve-comment.sh authorized the registered-bot path on login alone. It now also reads author.__typename and requires "Bot", so a person whose login matches a registered bot's is refused (exit 3) like any other human. A shim-backed test covers User → refused with no write and Bot → authorized. The User test fails on the previous code. AGENTS.md is updated to match. 139/139 in the gate, resolve, registry and scorecard suites; shellcheck and markdownlint clean. Refs #1918 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Dev-Lead — fix-bot-comment (no-changes)Agent reasoning |
|
Advisory bots were rate-limited; auto-approval is withheld until they recover. pr-review-sweep will re-review this PR after 2026-09-24T04:23:17Z. |
There was a problem hiding this comment.
All reported issues were addressed
Tip: instead of fixing issues one by one fix them all with cubic
Re-trigger cubic
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: cd07490020
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
All reported issues were addressed across 8 files
Requires human review: Auto-approval blocked because this review re-detected 7 unresolved issues already reported by Cubic.
Fix all with cubic | Re-trigger cubic
Superseded by automated re-review at
|
|
Dispositioning the Qodo status comment above: it is an operational trial-ended / billing notice ("Qodo reviews are paused because your trial has ended"), not a code-review finding — there is nothing in the diff to change and the state is resolved by workspace billing, outside this PR. Recording it as informational so it does not hold the approval gate. |
|
Dispositioning the CodeRabbit status comment above: it is an auto-generated review rate-limit notice ("Review limit reached … Next included review available in 3 minutes"), carrying no finding about this PR. Recording it as informational so it does not hold the approval gate. |
Dev-Lead — fix-reviews (partial)A commit was pushed, but not every requested change was applied. Per requested item:
The unaddressed items above still need work. |
Dev-Lead — fix-bot-comment (no-changes)Agent reasoning |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: d63e05bd05
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
The existing-reply check used test("…id=" + $id + "[^>]*-->"), so for node
IC_abc it also matched a reply marked id=IC_abc123. The trailing [^>]* absorbed
the rest of the other id. The script then skipped posting THIS comment's
required audit reply, but still minimized the comment RESOLVED, leaving no
recorded reason (Codex P2 on PR #1920).
Move the match into a pure helper, mrc_existing_reply_ids. The node id must now
be a whole token: preceded by whitespace and followed by whitespace or the
closing "-->". The viewer-only filter (anti-forgery) and fail-closed-on-bad-JSON
behaviour are unchanged. New tests cover the exact match, the prefix collision
(red on the old pattern), the suffix collision, a non-viewer forgery, and
unparseable input.
Refs #1918
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Dispositioning the SonarCloud status comment above: it is a clean Quality Gate passed notice (0 new issues, 0 accepted, 0 security hotspots), not a code-review finding — there is nothing in the diff to change. Recording it as informational so this passing-gate status does not hold the approval gate. |
4374cd2 to
c9c10d5
Compare
Dev-Lead — fix-reviews (applied)Changes committed and pushed. Requested items addressed:
|
|
Dev-Lead — fix-bot-comment (no-changes)Agent reasoning |
Superseded by automated re-review at
|
donpetry-bot
left a comment
There was a problem hiding this comment.
Automated review — APPROVED ✓
Risk: MEDIUM
Reviewed commit: c9c10d5201257b9de2a1a4b57b5f520f6abc9828
Review mode: triage-approved (single reviewer)
Summary
Re-review at settled head c9c10d5: the prior cascade escalation was purely procedural (head moved mid-audit; bats/unit/cubic still running) and both conditions are now cleared — head matches $PR_HEAD_SHA and every completed check is green. Substance was already cleared by the deep+audit tiers: the data-driven info-status classifier fails closed (unreadable registry auto-clears nothing), the SonarCloud pattern pins the passed headline plus both zero counts so a finding-bearing pass still blocks, and the resolver bot path requires registry membership, __typename==Bot, and a mandatory --reason reply before minimizing. The only diff since the prior review is the boundary-delimited node-id dedup fix plus its regression test, which the prior audit had already examined and called benign. Residual minor hardening (gate-side Bot actor-type check, body-trusted marker model) is tracked in open follow-up #1921 as accepted.
Linked issue analysis
Closes #1918. All five acceptance criteria are substantively met: (1) clean bot status comments are auto-cleared deterministically via the info_status_pattern column in reviewer-sources.tsv (schema v3), keyed by author login + body pattern with no LLM; the pattern requires Quality Gate passed AND [0 New issues] AND [0 Security Hotspots] per the issue's pattern-precision note; (2) failed/unknown/finding-bearing comments still block — the gate fails closed to clearing nothing on an unreadable registry, and tests cover passed-clears / failed-blocks / unknown-blocks including a passed-with-2-new-issues case; (3) maintainer-resolve-comment.sh gains a registered-bot path requiring __typename==Bot + registry membership + --reason posted as a marked reply before minimize, while another human's comment is still refused; (4) AGENTS.md documents the escape hatch next to the #1911 entry; (5) regression tests cover the deadlock loop. The rollout requirement is met: the PR body names the required pr-review/v1-next channel cut and the exact cut-release.sh commands.
Findings
- Resolved (was major): head-SHA TOCTOU — the branch has settled; headRefOid c9c10d5 equals the review target.
- Resolved (was major): in-progress checks — bats, unit, and cubic are now COMPLETED/SUCCESS; every completed check at head is green.
- Carried as accepted residual (minor): gate-side info-status auto-clear keys on login + body pattern without a Bot actor-type check, and the maintainer-resolve reply extends the pre-existing body-trusted marker model. Both are bounded (an account can only exempt its own comment) and tracked in open issue #1921.
- New commits since prior review (d63e05b→c9c10d5): only the audit-reply node-id dedup boundary fix (id must be followed by whitespace or -->, preventing prefix collisions) plus a regression test — already reviewed and rated benign by the prior audit.
- Note: the mcp__github__run_secret_scanning MCP tool is not available in this environment; the gitleaks CI check (Secret scan) is green.
- Note: the CodeRabbit commit status is stale-PENDING; it is not a required check (required: SonarCloud, CodeQL, agent-shield/AgentShield, dependency-audit/Detect ecosystems, duplicate-decl-gate — all green), its rate-limit notice was dispositioned, and the registry classifies coderabbitai as advisory_gate=no (not part of the approval-wait set).
CI status
All completed checks green at c9c10d5, including all 5 required checks (SonarCloud, CodeQL, agent-shield/AgentShield, dependency-audit/Detect ecosystems, duplicate-decl-gate), plus bats, unit-tests, shellcheck, gitleaks, CodeQL analyze (actions/python), SonarCloud Code Analysis, Graphite, and cubic. Non-required CodeRabbit status is stale-PENDING (rate-limited vendor; excluded from the approval-wait set). No failures. 0 unresolved review threads.
Reviewed automatically by the PR-review agent (single-reviewer mode: fable 5). Reply if you need a human review.
donpetry-bot
left a comment
There was a problem hiding this comment.
Automated review — APPROVED ✓
Risk: MEDIUM
Reviewed commit: c9c10d5201257b9de2a1a4b57b5f520f6abc9828
Review mode: triage-approved (single reviewer)
Summary
Confirmation re-review at the same SHA a prior cycle approved. Implements #1918: clean bot status comments (SonarCloud's re-posted 'Quality Gate passed') auto-clear from the maintainer-comment gate via a data-driven classifier in reviewer-sources.tsv (schema v3, info_status_pattern column), and maintainer-resolve-comment.sh gains a registered-info-status-bot path requiring --reason with an audit reply. Design is fail-closed at every branch and thoroughly bats-tested. Triage's low-risk call is confirmed; risk is MEDIUM only because the change touches approval-gate logic.
Linked issue analysis
Closes #1918 (now closed). All acceptance criteria are substantively addressed:
- AC1 — clean bot status never blocks: deterministic, LLM-free classifier in
maintainer-comment-gate.shkeyed off the reviewer-source registry (scripts/lib/reviewer-sources.tsv, newinfo_status_patterncolumn) — patterns live in data, not hard-coded logins, exactly as the issue suggested. The SonarCloud pattern pins the**Quality Gate passed**headline AND the[0 New issues]/[0 Security Hotspots]lines. - AC2 — findings still block:
Quality Gate failed, a passed gate with nonzero new issues/hotspots, an unclassifiable body, a bot with no pattern, and every human comment all still block (each covered by a bats test, including the10 New issuesprefix-match trap). - AC3 — maintainer escape hatch:
maintainer-resolve-comment.shaccepts a registered info-status bot's comment, requires--reasonposted as a marked audit reply before the minimize, requires a GitHub App actor (__typename == Bot) so a person with a bot-matching login is refused, and still refuses other humans and finding-producing bots — preserving the #1910 self-scope restriction. - AC5 — whole-loop regression: an approving review plus a later clean SonarCloud pass stays cleared (tested), so the approve→dismiss→re-review cycle burn this issue documents cannot recur for clean passes.
Docs (AGENTS.md, maintainer-comment-gate.md, implementation.md) and the consumer manifest / downstream-impact / validator were updated consistently for the new.tsvsurface source.
Findings
No blocking findings. Notable strengths verified in this pass:
- Fail-closed everywhere: an unreadable registry degrades to
{}(nothing auto-cleared, never clearing the unclassifiable); the gate's jq classifier only drops a comment when the author is registered AND the body matches its pattern; malformed snapshots still return 2. - Resolve-script hardening: repo-local registry pinned (a
REVIEWER_SOURCES_MANIFESToverride cannot widen authorization); reply-dedup lookup is scoped to the invoking viewer so a forgedmaintainer-resolvemarker cannot suppress the audit reply (CWE-345, tested); full-node-id + delimiter matching prevents prefix-id confusion; failed comment-list reads abort before any write; reply is posted before the minimize so a failed reply fails the whole action closed. - Loop safety: the
maintainer-resolvemarker was added to the gate's agent-marker set, so the audit reply never becomes a fresh blocker (round-trip tested). - Secret scan: the
run_secret_scanningMCP tool is not available in this run; the gitleaks CI check passed. No secrets or credentials appear in the diff (patterns/logins/docs only).
CI status
All required and repo-defined checks green: SonarCloud, agent-shield / AgentShield, shellcheck, bats, unit-tests, CodeQL (actions + python), Agent Security Scan, actionlint, gitleaks, holdout-guard, and all validator/guard jobs SUCCESS (irrelevant ecosystem audits SKIPPED). The only non-completed item is the third-party CodeRabbit advisory status stuck PENDING — its rate-limit notice was dispositioned as informational and its stale review was dismissed; it is not a required check and does not gate merge. No unresolved review threads; no unanswered human-reviewer questions.
Reviewed automatically by the PR-review agent (single-reviewer mode: fable 5). Reply if you need a human review.
donpetry-bot
left a comment
There was a problem hiding this comment.
Automated review — APPROVED ✓
Risk: MEDIUM
Reviewed commit: c9c10d5201257b9de2a1a4b57b5f520f6abc9828
Review mode: triage-approved (single reviewer)
Summary
Implements issue #1918: a data-driven info-status classifier so clean bot status comments (chiefly SonarCloud's re-posted 'Quality Gate passed') never deadlock the approval gate, plus a maintainer escape hatch to disposition a registered info-status bot's comment with a mandatory --reason audit reply. All prior-cycle findings are resolved at this head; triage's low-risk clearance is confirmed as accurate for content that is well-tested and fail-closed throughout.
Linked issue analysis
Linked issue #1918 (closes) is substantively addressed on all three acceptance criteria:
- AC1 — clean bot status never blocks, deterministically, no LLM:
reviewer-sources.tsv(schema v3) gains aninfo_status_patterncolumn;maintainer-comment-gate.shauto-clears a comment only when the author is a registered source AND the body matches its pattern. The SonarCloud pattern pins the 'Quality Gate passed' headline plus the literal[0 New issues]and[0 Security Hotspots]lines. Patterns live in data, not hard-coded logins, exactly as the issue requested. - AC2 — findings still block: 'Quality Gate failed', a passing gate with non-zero counts ('[10 New issues]' cannot match '[0 New issues]' because the bracket is pinned), an unclassifiable body, a bot with no pattern, and every human comment all still block. An unreadable registry degrades to '{}' — nothing is auto-cleared.
- AC3 — maintainer escape hatch:
maintainer-resolve-comment.shnow accepts a registered info-status bot's comment (author must be a GraphQLBotactor, so a person with a matching login is refused), requires--reason, and posts a marked audit reply before minimizing. Another human's comment and finding-producing bots (codeant-ai, graphite-app) remain refused, preserving the #1910 self-scope restriction.
Findings
No blocking findings. Prior-cycle (d63e05b) findings are resolved at this head:
- Reply-dedup lookup is now viewer-scoped, so a forged
maintainer-resolvemarker from another login cannot suppress the audit reply (CWE-345) — covered by a dedicated bats test. - The dedup marker match requires the FULL node id followed by a delimiter, so
IC_...fake12cannot satisfy a lookup forIC_...fake1— covered by a dedicated bats test. - A failed or unparseable comments-list lookup aborts before posting or minimizing (fail closed) — covered by a dedicated bats test.
Security review of the new surface: the registry path is pinned inside the resolve script so a REVIEWER_SOURCES_MANIFEST override cannot widen authorization; the audit reply is posted BEFORE the minimize so a failed reply fails the whole action closed; the gate's classifier only ever REMOVES a blocker for a finding-free clean-pass body, and every failure mode (unreadable registry, jq failure, empty output) degrades to blocking. 14 files, +816/−89, with docs (AGENTS.md, gate + implementation docs), consumer-manifest/downstream-impact wiring for the new .tsv surface, and ~20 new bats tests spanning classifier precision, authorization, idempotency, and fail-closed paths.
Secret scan: the run_secret_scanning MCP tool is not available in this environment; the gitleaks CI check passed and the diff introduces no credentials.
CI status
All checks green: shellcheck, bats, unit-tests, actionlint, CodeQL (actions + python), SonarCloud quality gate, gitleaks secret scan, agent-shield, and all validate-* / guard checks SUCCESS; remaining entries SKIPPED by design. The only queued entry is pr-auto-review / check-and-dispatch — the orchestrator running this very review. No unresolved review threads; no unanswered human-reviewer questions (all owner-login comments carry dev-lead automation markers).
Reviewed automatically by the PR-review agent (single-reviewer mode: fable 5). Reply if you need a human review.
|
pr-review approved on PARTIAL advisory evidence: 5/7 required advisory bots reported before the gate's head-age-timeout fallback proceeded. Recorded for the miss-rate metric (#1596). |
Resolve the reviewer-sources.tsv conflict with main. Since this branch was cut, #1920 (issue #1918) added a seventh column, info_status_pattern, and bumped schema_version to 3. Take main's rows unchanged and add cubic-dev-ai as a seven-column row, with "-" for both check_run_name and info_status_pattern. cubic reports through PR reviews, not a check run, and has no known-clean status-comment pattern. Its rationale is unchanged. Verified: 235/235 in the reviewer-sources, reviewer-report, advisory-gate, intent-reviews, maintainer-comment-gate and miss-rate suites; shellcheck clean. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>



Problem
Approval gate can never clear bot status comments — SonarCloud re-posts 'Quality Gate passed' on every push and #1911's resolve script is self-scoped, so the #1894 queue is still deadlocked
From the issue: PR #1913 (implements #1908) was approved by pr-review twice, at 11:45Z on
e42886f9and 12:08Z on103d0513, with no blocking findings. Both approvals were dismissed withDismissing approval due to a PR issue comment lacking a verified disposition (#1813). The gate log (run 35858596981):Risk
Low — changes automation shell logic under scripts/, covered by shellcheck (--severity=warning) and the bats suite.
Test plan
Tests added/updated:
tests/dev-lead/unit/test_maintainer_comment_gate.bats,tests/dev-lead/unit/test_maintainer_resolve_comment.batstests/test_reviewer_sources.bats. Verification:bash scripts/dev-lead-lint.sh(shellcheck --severity=warning) ran pre-commit; the bats suite runs in CI.Rollback
Revert this PR. No non-revertible side effects (no tags, migrations, or external state).
Monitoring
This PR's Lint (shellcheck) and bats checks show pass/fail; watch subsequent dev-lead / pr-review runs for behavioral regressions.
Closes #1918
Rollout: a pr-review channel cut IS required after merge
maintainer-comment-gate.shruns inside pr-review, which in this repo executes from the channel tagpr-review/v1-next(.github/workflows/pr-review-trigger.yml). That tag is ateee2898e(2026-09-23 03:55Z) and does not contain #1911 either. Until the channel moves, this merge changes nothing at runtime, because the autocut only watches the reusable-workflow blob (#1592).After merge:
Review follow-up (cd07490)
**Quality Gate passed**and[0 New issues]and[0 Security Hotspots]. A passing gate that reports issues or hotspots still blocks, per Approval gate can never clear bot status comments — SonarCloud re-posts 'Quality Gate passed' on every push and #1911's resolve script is self-scoped, so the #1894 queue is still deadlocked #1918 AC1.maintainer-resolve-comment.shrequiresauthor.__typename == "Bot", so a person whose login matches a registered bot's is refused.Summary by CodeRabbit