Skip to content
32 changes: 32 additions & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -104,6 +104,38 @@ minimizes **your own** comment `RESOLVED` — the exact signal the gate already
action), is idempotent, and fails closed if it cannot confirm the comment's author or the invoking
viewer. An `@mention` (FORCE_REVIEW) is the alternative human-in-the-loop bypass.

### Bot status comments never deadlock approval, and the maintainer escape hatch for them (#1918)
Comment thread
cubic-dev-ai[bot] marked this conversation as resolved.
Resolved

The comments that actually strand the #1894 queue are posted by **bots** — chiefly SonarCloud's
`Quality Gate passed` status comment, which SonarCloud **re-posts on every push**, so dispositioning
the old one never helps. #1910's self-scope (above) meant a maintainer could not clear a bot's comment
at all (this was the deadlock reported in #1911). Two changes close this:

- **A clean bot status comment is auto-cleared, deterministically, with no LLM and no dev-lead.**
`maintainer-comment-gate.sh` reads a data-driven classifier keyed off the reviewer-source registry
(`scripts/lib/reviewer-sources.tsv`, new `info_status_pattern` column). A comment authored by a
registered source whose body matches that source's pattern is treated as **addressed** and never
blocks. For `sonarqubecloud` the pattern pins the `**Quality Gate passed**` headline **and** the
`[0 New issues]` and `[0 Security Hotspots]` lines. A gate can pass while still reporting new issues
or hotspots, and those are findings, so the headline alone never clears a comment. The patterns live in **data,
not hard-coded logins** in the gate (`reviewer_sources_info_status_patterns`). It stays **fail
closed**: a `Quality Gate failed` comment does not match and still blocks, as does any bot comment
carrying findings, any comment from a bot with no pattern, and every human comment. An unreadable
registry degrades to `{}` — i.e. **nothing is auto-cleared** — never to clearing something it cannot
classify.
- **Maintainers can clear a *registered info-status bot's* comment when needed.**
`scripts/maintainer-resolve-comment.sh` accepts a comment authored by a bot that declares an
`info_status_pattern` in `reviewer-sources.tsv` — a clean-status re-poster like `sonarqubecloud`,
**not** a finding-producing reviewer such as `codeant-ai` or `graphite-app` (whose findings still
require dev-lead's verified-fix flow, so this path cannot `--reason` them away) — in addition to
your own. The author must be a GitHub App actor (`__typename == Bot`), so a *person* whose login
matches a registered bot's is refused like any other human. This path **requires `--reason`**, which
is posted as a reply carrying a `<!-- maintainer-resolve … -->` marker (pinned to the original
comment's node id, so the reply is not itself a new blocker and a retry does not post a duplicate)
**before** the original is minimized `RESOLVED`. It **still refuses another *human's* comment** —
that restriction is exactly why #1910 was self-scoped — and fails closed if it cannot read the
registry, the comment's author, or the invoking viewer.

### The org automation-PR cap is real and near-silent

There is a hard ceiling of **50 concurrent open non-Dependabot automation PRs org-wide**
Expand Down
2 changes: 1 addition & 1 deletion docs/pr-review-agent/downstream-impact.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ When enabled, for each reviewed PR `scripts/review-one-pr.sh`:
`scripts/lib/downstream-impact.sh`) — bash + jq over the consumer manifest,
**no network**. Matching is single-hop and exact-path (issue #653):
- a changed reusable-workflow path a consumer pins exactly → a **direct** surface;
- a changed `scripts/lib/*.sh` / `prompts/*` path listed in a reusable's
- a changed `scripts/lib/*.sh` / `scripts/lib/*.tsv` / `prompts/*` path listed in a reusable's
`surface_sources` → that **reusable** surface → its consumers.
2. **Fetches** each impacted consumer's referencing workflow file(s) via `gh`
(`assemble_downstream_impact`) and writes a human-readable
Expand Down
12 changes: 9 additions & 3 deletions docs/pr-review-agent/implementation.md
Original file line number Diff line number Diff line change
Expand Up @@ -99,9 +99,15 @@ gates that withhold approval until an external finding is accounted for:
for advisory review bots (Gemini, Copilot, SonarCloud, Codex) to submit before
approving.
- **Maintainer issue-comment gate** (`scripts/lib/maintainer-comment-gate.sh`,
#1290) — withholds approval while the latest maintainer *issue comment*
(`gh pr comment` / the main comment box, which creates no review thread) postdates
the last push. It **fails closed**. See
#1290/#1813/#1918) — withholds approval while any PR *issue comment*
(`gh pr comment` / the main comment box, which creates no review thread) lacks a
**verified disposition**. The old push-timestamp model (a push postdating the
comment cleared it) was retired in #1813 because GitHub now returns the head push
timestamp as `null`; "addressed" now means the comment was minimized with
classifier `RESOLVED`. Every issue comment — human *and* bot — must carry one,
with two narrow exceptions: our own automation-marked comments, and clean
informational status comments from registered reviewer bots (e.g. SonarCloud's
"Quality Gate passed", #1918). It **fails closed**. See
[maintainer-comment-gate.md](maintainer-comment-gate.md) for why a review thread
blocks merge but a plain PR comment previously did not, and how the block clears.

Expand Down
123 changes: 82 additions & 41 deletions docs/pr-review-agent/maintainer-comment-gate.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,16 @@
# Maintainer issue-comment gate (issue #1290)
# Maintainer issue-comment gate (issues #1290, #1813, #1918)

> **Redesigned in [#1813](https://github.com/petry-projects/.github-private/issues/1813)
> and [#1918](https://github.com/petry-projects/.github-private/issues/1918).** The
> original gate (#1290) judged a comment *addressed* by comparing timestamps — a push
> at/after the comment cleared it. GitHub now always returns the head push timestamp as
> `null`, so that model failed closed on every open PR (the #1813 outage), and a push
> was only ever a proxy for whether the finding was actually read and acted on. The
> gate no longer reads push time at all: "addressed" now means a **verified
Comment thread
don-petry marked this conversation as resolved.
Resolved
> disposition** (the comment minimized with classifier `RESOLVED`), and **every** issue
> comment — human *and* bot — must carry one, with two narrow exceptions described
> below. The push-timing paragraphs that follow describe the **superseded** model and
> are retained only for the review-thread sibling gate, which still uses push time.

This document explains why a maintainer finding posted two different ways can have
opposite mechanical force, and how the pr-review **maintainer issue-comment gate**
Expand Down Expand Up @@ -32,58 +44,87 @@ deliberately modeled on the advisory-bot gate
([`advisory-review-gate.sh`](../../scripts/lib/advisory-review-gate.sh), #457/#458),
which likewise defers approval until a signal is incorporated.

> pr-review withholds its **automated approval** while the **latest maintainer
> issue comment postdates the last push**.
> pr-review withholds its **automated approval** while **any non-agent PR issue
> comment lacks a verified disposition** (is not minimized with classifier `RESOLVED`).

Because pr-review is the code-owner approver, withholding its approval means the PR
does not satisfy the code-owner review requirement and therefore **does not
auto-merge** — closing the "does not block" half at the approval boundary.

### What counts as a "maintainer issue comment"

An issue comment is treated as a maintainer finding **unless** it is clearly one of
ours or an already-gated bot:

- Comments carrying one of our automation markers — `<!-- pr-review-agent … -->`,
`<!-- persona:… -->`, `<!-- dev-lead … -->`, `<!-- dependency-advisory -->` — are
ours, never a finding. (This marker-based exclusion matters because several of
these workflows — dev-lead and the dependency-advisory pass — post as the human
owner `don-petry`, the *same account* a human maintainer uses, so login alone
cannot separate the two. dev-lead's own rate-limit acknowledgments and its
`@coderabbitai resolve` nudge carry a `<!-- dev-lead … -->` marker for exactly
this reason.)
- Comments from the agent's own login (`BOT_USER`, default `donpetry-bot`) are
excluded.
- Comments from advisory/review bots and generic automation (Gemini, Copilot,
SonarCloud, Codex, CodeRabbit, `github-actions`, `dependabot`) are excluded —
advisory bots are handled by the advisory-bot gate.

Everything else is treated as a human maintainer finding. This is the fail-closed
default: an *unknown* author blocks rather than slips through.
### What counts as a comment that must be dispositioned

Since #1813 the scope is **every** PR issue comment — from human maintainers **and**
bots alike (`codeant-ai`, `qodo-code-review`, `auto-rebase-conflict`, and so on). No
author is exempt for being a bot: a conflict report is a finding, a trial-ended notice
has an operational consequence. A comment blocks approval until it is minimized
`RESOLVED`, **unless** it is one of the two narrow exceptions:

- **Our own automation's replies.** Comments carrying one of our automation markers —
`<!-- pr-review-agent … -->`, `<!-- pr-review-claim … -->`, `<!-- persona:… -->`,
`<!-- dev-lead … -->` (including the `<!-- dev-lead:comment-disposition … -->`
reply), `<!-- dependency-advisory -->`, and `<!-- maintainer-resolve … -->` — are
ours, never a finding, so the agent never has to answer itself. (This marker-based
exclusion matters because several of these workflows — dev-lead and the
dependency-advisory pass — post as the human owner `don-petry`, the *same account* a
human maintainer uses, so login alone cannot separate the two.) Comments from the
agent's own login (`BOT_USER`, default `donpetry-bot`) are likewise excluded.
- **A registered clean *info-status* bot comment (#1918).** A comment authored by a
reviewer source that declares an `info_status_pattern` in the reviewer-source
registry (`scripts/lib/reviewer-sources.tsv`), and whose body matches that pattern,
is a clean status report carrying no finding and is treated as **addressed**. The
canonical case is SonarCloud's `Quality Gate passed` comment, which it **re-posts on
every push**; its pattern pins the `**Quality Gate passed**` headline **and** the
`[0 New issues]` and `[0 Security Hotspots]` lines, so a `Quality Gate failed`
comment, or one still reporting new issues or hotspots, does **not** match and still
blocks. An unreadable registry degrades to "clear nothing", never to clearing
something it cannot classify.

Everything else — a bot comment with a finding, a bot with no registered pattern, an
*unknown* author, or any human comment — blocks. This is the fail-closed default: an
author or body the gate cannot positively clear blocks rather than slips through.

### When the block clears

A maintainer issue comment is considered **addressed** — and the gate stops
blocking — when a commit has been pushed at or after it. In practice:

- **dev-lead (or the author) pushes a fix.** The head commit's `committer.date`
advances past the comment, so the latest maintainer comment no longer postdates
the last push → the gate clears on the next review.
A comment is considered **addressed** — and the gate stops blocking — when it is
minimized with classifier `RESOLVED`. In practice:

- **dev-lead dispositions it.** dev-lead researches each undispositioned comment,
posts exactly one evidence-backed reply carrying
`<!-- dev-lead:comment-disposition id=<id> disposition=<…> -->`, and the harness
verifies that disposition and then minimizes the **original** comment `RESOLVED`
(GraphQL `minimizeComment`). A push alone no longer clears anything.
- **A maintainer clears it without dev-lead (#1910/#1918).** When dev-lead is
suppressed, rate-limited, cancelled, or never dispatched, a maintainer can run
[`scripts/maintainer-resolve-comment.sh`](../../scripts/maintainer-resolve-comment.sh)
(see its `--help`). It minimizes **their own** comment `RESOLVED`, or — for a
*registered info-status bot* whose finding-free status stranded the queue (chiefly
SonarCloud, which re-posts `Quality Gate passed` on every push) — that bot's
comment, requiring a `--reason` posted as a `<!-- maintainer-resolve … -->` reply
before the minimize. It refuses another human's comment and a finding-producing bot
such as `codeant-ai` or `graphite-app` (whose findings still require dev-lead's
verified-fix flow), and fails closed if it cannot confirm the comment's author, its
actor type (`__typename == Bot`), or the invoking viewer.
- **A registered clean info-status bot comment auto-clears (#1918)** — see the scope
section above; no disposition action is needed for it.
- If you only want to *chat* (e.g. "LGTM") without requesting a change, leave an
**approving review** rather than a plain comment — a review is not a maintainer
finding, and it also positively signals "no changes needed".
**approving review** rather than a plain comment — a review is not an issue comment,
and it also positively signals "no changes needed".

### Fails closed

The [#1290](https://github.com/petry-projects/.github-private/issues/1290)
acceptance criterion is that an inability to determine whether a finding was
addressed must **not** read as "no findings" — that is exactly the bug that shipped
the regression. So:

- A maintainer comment with an **undeterminable push time** (GraphQL unreachable) →
**blocks** (return 1).
- A **malformed / unparseable** PR snapshot → **fails the PR** (return 2 → exit 1)
so a scheduled run retries, rather than approving blind.
The [#1290](https://github.com/petry-projects/.github-private/issues/1290) /
[#1813](https://github.com/petry-projects/.github-private/issues/1813) acceptance
criterion is that an inability to determine whether a finding was addressed must
**not** read as "no findings" — that is exactly the bug that shipped the regression.
So:

- Any non-agent comment **without** a verified `RESOLVED` disposition → **blocks**
(return 1).
- A **malformed / unparseable** PR snapshot → **fails the PR** (return 2 → exit 1),
reported with a distinct verdict reason so an undeterminable gate state can never
recur disguised as a legitimate hold (#1813 AC8), so a scheduled run retries rather
than approving blind. The gate makes **no** `gh`/network calls — it reuses the
snapshot the caller already fetched — so there is no push-time lookup left to fail.

### Bypass

Expand Down
6 changes: 5 additions & 1 deletion scripts/lib/consumer-manifest.json
Original file line number Diff line number Diff line change
Expand Up @@ -152,7 +152,9 @@
"scripts/lib/git-identity.sh",
"scripts/lib/maintainer-comment-gate.sh",
"scripts/lib/maintainer-review-thread-gate.sh",
"scripts/lib/pr-worktree.sh"
"scripts/lib/pr-worktree.sh",
"scripts/lib/reviewer-sources.sh",
"scripts/lib/reviewer-sources.tsv"
Comment thread
don-petry marked this conversation as resolved.
Resolved
],
".github/workflows/pr-review.yml": [
"prompts/cascade-action.md",
Expand All @@ -168,6 +170,8 @@
"scripts/lib/maintainer-review-thread-gate.sh",
"scripts/lib/review-cycle.sh",
"scripts/lib/review-registry.sh",
"scripts/lib/reviewer-sources.sh",
"scripts/lib/reviewer-sources.tsv",
"scripts/lib/token-metrics.sh"
]
}
Expand Down
2 changes: 1 addition & 1 deletion scripts/lib/downstream-impact.sh
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@
# source-graph expansion. Two and only two match kinds:
# (a) a changed reusable-workflow path that a consumer ref pins exactly
# (after normalising the manifest's provider prefix) -> direct surface;
# (b) a changed scripts/lib/*.sh or prompts/* path listed in a reusable's
# (b) a changed scripts/lib/*.sh, scripts/lib/*.tsv, or prompts/* path listed in a reusable's
# surface_sources -> that reusable surface -> the reusable's consumers.
#
# A consumer ref is provider-prefixed (e.g. ".github-private/.github/workflows/
Expand Down
Loading
Loading