Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions docs/pr-review-agent/maintainer-comment-gate.md
Original file line number Diff line number Diff line change
Expand Up @@ -116,6 +116,10 @@ minimized with classifier `RESOLVED`. In practice:
`::warning::… fixed-unverified:<reason>`. The harness then re-answers with the
branch commit whose diff removes the token, when one exists. Otherwise the next
pass re-answers the disposition, which is not settled.
When the push guard rebased the cited commit during the pass, that sha never
reaches the branch. The harness first edits the reply so its marker cites the
rebased successor that landed (#2156), then judges that commit by the same
rules.
- **A maintainer clears it without dev-lead (#1910/#1918).** When dev-lead is
suppressed, rate-limited, cancelled, or never dispatched, a maintainer can run
[`scripts/maintainer-resolve-comment.sh`](../../scripts/maintainer-resolve-comment.sh)
Expand Down
24 changes: 24 additions & 0 deletions prompts/dev-lead/fix-bot-comment.md
Original file line number Diff line number Diff line change
Expand Up @@ -55,6 +55,7 @@ Analyze the bot's findings and address each actionable issue:
2. Locate the referenced files and line numbers using Read/Grep/Glob tools
3. Apply targeted fixes using Edit/Write tools
4. **Reply to each fixed thread with the specific change** — the harness resolves the addressed bot threads afterward (see below); do not resolve them yourself
5. **Disposition the bot's issue comment when the finding came from one** — a review thread and a PR issue comment are answered differently; see [Fixing a finding posted as a PR issue comment](#fixing-a-finding-posted-as-a-pr-issue-comment-2261)

### Replying to threads from this bot

Expand Down Expand Up @@ -109,6 +110,27 @@ The claim payload is schema `v1` — one comment per reply, with a full 40-char

**Do not resolve the thread yourself.** You must not call the `resolveReviewThread` (or `unresolveReviewThread`) GraphQL mutation under any circumstance — resolution is done **only** by the harness (`dev-lead-fix-reviews.sh`), whose deterministic guards are the authoritative merge gate. The harness resolves every bot thread you addressed with an our-account addressed-marker reply, plus any thread from this bot marked `isOutdated: true`. Your reply and its marker are your only lever on resolution.

### Fixing a finding posted as a PR issue comment (#2261)

Every registered reviewer bot (`scripts/lib/reviewer-sources.tsv`), not only CodeRabbit, can post a finding as a PR **issue comment** instead of a review thread: CodeAnt's `## CodeAnt Nitpicks`, a Qodo or CodeRabbit summary, a SonarCloud quality-gate comment. The comment that triggered this pass is one, and `${COMMENT_NODE_ID}` is its node id. An issue comment has **no** review thread, so nothing in *Replying to threads from this bot* applies to it.

**The addressed-marker and the claim are for review threads only. They do not disposition an issue comment.** The maintainer-comment gate clears an issue comment only on a verified `dev-lead:comment-disposition` reply. A "Fixed in …" PR comment that ends with `<!-- dev-lead:addressed -->` and a claim leaves the bot's comment undispositioned, the retries run out, and the PR can never be approved (PR #2205). Never put the addressed-marker or a claim in a PR comment.

When you fix a finding from an issue comment, **commit the fix locally first**, then post **exactly one** reply on the PR. Name the file(s) you changed and how the change addresses the finding, and end the reply with **exactly one** marker: `disposition=fixed` with the full 40-char sha of that commit.

```bash
# First run the target and already-dispositioned check under "Non-actionable bot
# notices" below: it guards every disposition, not only `informational`.
node_id='${COMMENT_NODE_ID}'
[ -n "$node_id" ] || { echo "no triggering comment node id — not dispositioning" >&2; exit 1; }
sha=$(git rev-parse HEAD) # read AFTER committing the fix
gh pr comment "${PR_NUMBER}" --repo "${REPO}" --body "Fixed in scripts/foo.sh: replaced the unpinned curl|bash install with a SHA-verified binary download.

<!-- dev-lead:comment-disposition id=${node_id} disposition=fixed sha=${sha} -->"
```

One comment gets one reply and one marker, however many findings it lists. Say how each finding was handled, and use `fixed` if you changed code for any of them. If you changed no code, end the same single reply with `answered`, `invalid` or `out-of-scope ref=#<n>` and the reason. `informational` is only for a notice that carries no finding (see below). The harness verifies the sha against the pushed head and minimizes the bot's comment RESOLVED; do not call `minimizeComment` yourself. *Cite the commit that fixed it* below is the rule the sha must pass, and `scripts/lib/comment-disposition-verify.sh` is the normative parser. If the same bot also opened review threads, answer those in their threads with the addressed-marker and claim, and still post the one disposition reply for the comment.

## Deferring a valid finding that is out of scope (#2045)

When a thread from this bot is a real finding that does not belong in this PR, do not leave a bare skip note: that thread can never be resolved and blocks merge. Defer it to the repo's **single** `dev-lead: deferred review findings` tracking issue (never one issue per finding):
Expand Down Expand Up @@ -230,6 +252,7 @@ Some bot comments are pure **operational notices**, not code findings: a rate-li
- Do not suppress bot rules without a documented reason
- Do not modify the bot's configuration files
- For every thread you fix, post a reply naming the specific change, ending with the addressed-marker — never reply-less
- For a finding you fix from a PR issue comment, post exactly one reply on the PR ending with exactly one `<!-- dev-lead:comment-disposition id=<comment node id> disposition=fixed sha=<40-hex> -->` marker. The addressed-marker and the claim are for review threads only: they never disposition an issue comment, so never put them in a PR comment (#2261)
- **Never resolve a thread yourself**: do not call the `resolveReviewThread` (or `unresolveReviewThread`) mutation. Resolution is the harness's job — it resolves the addressed and outdated threads from `${ACTOR}`; your only lever is the addressed-marker on your reply
- Stay within the scope of the pull request's changed files where possible
- Commit your fixes locally so your claims can cite them, but **never push**. The CI workflow pushes after you finish, verifies the push landed, and retracts any claim that did not land (#2013)
Expand All @@ -242,6 +265,7 @@ After applying fixes, output a summary:
Bot: ${ACTOR}
Issues addressed: N
- <issue description>: <fix applied> [replied + addressed-marker]
- <issue description>: <fix applied> [issue comment: disposition=fixed sha=<sha>]
- <issue description>: outdated — replied (harness resolves)
Files changed: <list of files>
Skipped (informational): <count>
Expand Down
71 changes: 61 additions & 10 deletions scripts/dev-lead-fix-reviews.sh
Original file line number Diff line number Diff line change
Expand Up @@ -141,6 +141,10 @@ github_knows_commit() {
# - earlier: a commit missing from a still-shallow checkout, one GitHub knows
# (pushed, then rewritten), or one GitHub cannot be asked about is
# left as is.
# A reply that also carries the review-thread addressed-marker or claim (#2261)
# is handled here in the same edit, so it has one writer: a re-point moves a
# claim citing the same commit along with the disposition, and a retraction
# removes the addressed-marker and the claim with it.
# Best-effort: returns 1 on a listing/PATCH failure; the caller folds it in.
sweep_fixed_dispositions() {
local ref="${1:-}" window="${2:-pass}"
Expand Down Expand Up @@ -196,14 +200,20 @@ sweep_fixed_dispositions() {
fi
if [ -n "$rebased" ] && git merge-base --is-ancestor "$rebased" "$ref" 2>/dev/null \
&& new_body=$(cl_rewrite_disposition_sha "$body" "$rebased"); then
# #2261: a claim in the same reply citing the same commit moves with it.
if [ "$(acv_parse_claim "$body" 2>/dev/null | jq -r '.sha // ""' 2>/dev/null)" = "$sha" ]; then
new_body=$(cl_rewrite_claim_sha "$new_body" "$rebased") || new_body=$(cl_rewrite_disposition_sha "$body" "$rebased")
fi
if gh api -X PATCH "repos/${REPO}/issues/comments/${id}" -f body="$new_body" >/dev/null 2>&1; then
remapped=$((remapped + 1))
echo "::notice::\`fixed\` reply ${id} on PR #${PR_NUMBER}: ${sha} was rebased onto a foreign commit and landed as ${rebased} — disposition updated, not retracted (#2156)"
continue
fi
echo "::warning::could not update \`fixed\` reply ${id} on PR #${PR_NUMBER} to the rebased commit ${rebased} — retracting it instead (#2156)"
fi
new_body=$(cl_retract_disposition_body "$body" "$reason")
# #2261: the addressed-marker and claim go too, or the quoted original would
# keep a claim citing the unlanded commit. Without either, the body is unchanged.
new_body=$(cl_retract_disposition_body "$(cl_strip_claim_markers "$body" || printf '%s' "$body")" "$reason")
if gh api -X PATCH "repos/${REPO}/issues/comments/${id}" -f body="$new_body" >/dev/null 2>&1; then
retracted=$((retracted + 1))
echo "::warning::retracted \`fixed\` disposition reply ${id} on PR #${PR_NUMBER} (${reason}; cited ${sha:-<none>}, remote head ${ref:-<unknown>}) (#2156)"
Expand Down Expand Up @@ -3490,6 +3500,10 @@ save_pending_replies() {
# cl_retract_body, which strips both markers so it can never resolve a thread.
# This pass's `fixed` disposition replies on issue comments are swept against the
# same reference head (sweep_fixed_dispositions, #2156).
# Claim replies are swept on both surfaces (#2261): review-thread replies, and the
# PR issue comments the model posts when it answers a bot's issue comment the
# same way. A PR comment that also carries a `fixed` disposition is left to
# sweep_fixed_dispositions alone, so one reply is never rewritten by two sweeps.
#
# The reference head is the remote head. When there is no upstream (rc 1) on an
# `ok` outcome, the local HEAD is what was pushed; in every other case — including
Expand Down Expand Up @@ -3534,10 +3548,44 @@ retract_unlanded_claims() {
echo "::warning::retract_unlanded_claims: could not parse the review comments on PR #${PR_NUMBER} — this pass's claim replies were NOT verified (#2013)"
return 1
}
local rebased remapped=0
while IFS=$'\t' read -r id sha; do

# #2261: the model can also post its addressed/claim reply as a PR ISSUE comment
# (PR #2205: "Fixed in …" answering a bot's issue comment). That reply is not in
# the review-comment listing above, so its claim was never checked and kept
# citing a commit that was never pushed. Sweep the same selection (our account,
# since the pass start, carrying the marker or a claim) over the issue comments.
# A reply that also carries a `fixed` disposition is left out: it belongs to
# sweep_fixed_dispositions above, which handles its claim in the same edit.
# An unreadable listing is reported as a failure: those replies were NOT
# verified. The thread replies above are still swept.
local issue_raw="" issue_comments="" issue_rows=""
if ! issue_raw=$(gh api --paginate "repos/${REPO}/issues/${PR_NUMBER}/comments?per_page=100" 2>/dev/null) \
|| ! issue_comments=$(jq -s 'add // []' <<<"$issue_raw" 2>/dev/null) \
|| ! issue_comments=$(cl_without_fixed_dispositions "$issue_comments") \
|| ! issue_rows=$(cl_select_pass_claims "$issue_comments" "$bot_user" "$PASS_START_ISO"); then
echo "::warning::retract_unlanded_claims: could not list the issue comments on PR #${PR_NUMBER} — claim replies this pass posted as PR comments were NOT verified (#2261)"
issue_comments=""
issue_rows=""
failed=1
fi
# One row per reply: `<surface>\t<id>\t<sha>`. <surface> is the REST collection
# the reply lives in (pulls = review-thread reply, issues = PR issue comment),
# which is also the path segment its PATCH uses.
rows=$(
[ -z "$rows" ] || awk '{ print "pulls\t" $0 }' <<<"$rows"
[ -z "$issue_rows" ] || awk '{ print "issues\t" $0 }' <<<"$issue_rows"
)

local rebased remapped=0 surface listing what
while IFS=$'\t' read -r surface id sha; do
[ -z "$id" ] && continue
body=$(printf '%s' "$comments" | jq -r --arg id "$id" 'first(.[] | select((.id | tostring) == $id)) | .body // ""' 2>/dev/null || echo "")
listing="$comments"
what="claim reply"
if [ "$surface" = "issues" ]; then
listing="$issue_comments"
what="PR-comment claim reply"
fi
body=$(printf '%s' "$listing" | jq -r --arg id "$id" 'first(.[] | select((.id | tostring) == $id)) | .body // ""' 2>/dev/null || echo "")
if [ -z "$sha" ]; then
reason="unverifiable"
else
Expand All @@ -3563,21 +3611,24 @@ retract_unlanded_claims() {
new_body=$(cl_rewrite_claim_sha "$body" "$rebased") || new_body=""
if acv_claim_in_pass "$base" "$on_ref" "$in_base" >/dev/null \
&& [ -n "$new_body" ]; then
if gh api -X PATCH "repos/${REPO}/pulls/comments/${id}" -f body="$new_body" >/dev/null 2>&1; then
if gh api -X PATCH "repos/${REPO}/${surface}/comments/${id}" -f body="$new_body" >/dev/null 2>&1; then
remapped=$((remapped + 1))
echo "::notice::claim reply ${id} on PR #${PR_NUMBER}: ${sha} was rebased onto a foreign commit and landed as ${rebased} — claim updated, not retracted (#2032)"
echo "::notice::${what} ${id} on PR #${PR_NUMBER}: ${sha} was rebased onto a foreign commit and landed as ${rebased} — claim updated, not retracted (#2032)"
continue
fi
echo "::warning::could not update claim reply ${id} on PR #${PR_NUMBER} to the rebased commit ${rebased} — retracting it instead (#2032)"
echo "::warning::could not update ${what} ${id} on PR #${PR_NUMBER} to the rebased commit ${rebased} — retracting it instead (#2032)"
fi
fi
fi
# cl_retract_body removes only the addressed-marker and the claim. A
# non-`fixed` `dev-lead:comment-disposition` marker in the same reply is
# kept: the disposition resolver verifies that one on its own (#2261).
new_body=$(cl_retract_body "$body" "$reason")
if gh api -X PATCH "repos/${REPO}/pulls/comments/${id}" -f body="$new_body" >/dev/null 2>&1; then
if gh api -X PATCH "repos/${REPO}/${surface}/comments/${id}" -f body="$new_body" >/dev/null 2>&1; then
retracted=$((retracted + 1))
echo "::warning::retracted claim reply ${id} on PR #${PR_NUMBER} (${reason}; cited ${sha:-<none>}, remote head ${ref:-<unknown>}) (#2013)"
echo "::warning::retracted ${what} ${id} on PR #${PR_NUMBER} (${reason}; cited ${sha:-<none>}, remote head ${ref:-<unknown>}) (#2013)"
else
echo "::error::could not retract claim reply ${id} on PR #${PR_NUMBER} (${reason}) — a false 'Fixed' reply may remain (#2013)"
echo "::error::could not retract ${what} ${id} on PR #${PR_NUMBER} (${reason}) — a false 'Fixed' reply may remain (#2013)"
failed=1
fi
done <<< "$rows"
Expand Down
41 changes: 35 additions & 6 deletions scripts/lib/claim-landing.sh
Original file line number Diff line number Diff line change
Expand Up @@ -96,7 +96,8 @@ cl_push_landed_verdict() {

# cl_select_pass_claims <comments_json> <bot_user> <since_iso>
# <comments_json> is the REST `pulls/{pr}/comments` array ({id, user.login, body,
# created_at}). Emits one `<id>\t<sha>` row per comment that (a) our account
# created_at}), or the `issues/{pr}/comments` array, which has the same fields
# (#2261). Emits one `<id>\t<sha>` row per comment that (a) our account
# authored (bot_user or its [bot]-suffixed / stripped form), (b) was created at or
# after <since_iso> (the pass start), and (c) still carries the addressed-marker or
# a claim comment. <sha> is the parsed claim SHA, or empty when the claim is
Expand Down Expand Up @@ -322,6 +323,20 @@ cl_escape_dashes() {
printf '%s' "${t//--/-\\u002d}"
}

# cl_strip_claim_markers <body>
# <body> without its addressed-marker and claim comment, and without trailing
# whitespace. The claim comment ends at its FIRST `-->` (as acv_parse_claim reads
# it), so a different marker later on the same line, such as a
# comment-disposition, is kept, and a claim whose JSON spans lines is removed
# whole (#2261). Returns 1 when the body cannot be processed. Pure.
cl_strip_claim_markers() {
jq -Rsr --arg marker "$_DEV_LEAD_ADDRESSED_MARKER" '
gsub($marker; "")
| gsub("<!--[[:space:]]*dev-lead:claim(?:.|\\n)*?-->"; "")
| sub("\\s+$"; "")
' <<<"${1:-}" 2>/dev/null
}

# cl_retract_body <body> <reason>
# The retracted form of a claim reply. Removes the addressed-marker and the claim
# comment (so neither the thread gate nor another bot can treat it as a fix),
Expand Down Expand Up @@ -349,11 +364,7 @@ cl_retract_body() {
fi

local stripped
stripped=$(jq -Rsr --arg marker "$_DEV_LEAD_ADDRESSED_MARKER" '
gsub($marker; "")
| gsub("<!--[[:space:]]*dev-lead:claim[^\\n]*-->"; "")
| sub("\\s+$"; "")
' <<<"$body" 2>/dev/null) || stripped=""
stripped=$(cl_strip_claim_markers "$body") || stripped=""
local quoted
quoted=$(printf '%s\n' "$stripped" | sed 's/^/> /')

Expand Down Expand Up @@ -567,6 +578,24 @@ cl_select_fixed_dispositions() {
done <<<"$rows"
}

# cl_without_fixed_dispositions <comments_json>
# <comments_json> (the REST `issues/{pr}/comments` array) without the comments
# that carry a `disposition=fixed` marker, by the same test
# cl_select_fixed_dispositions selects them with. Those replies belong to the
# `fixed`-disposition sweep, which also handles a claim they carry; the claim
# sweep reads the rest, so no reply is rewritten by both (#2261). A payload
# that is JSON but not an array is echoed unchanged: the claim selector reads it
# as "no replies", as it does for the review-comment listing. Returns 1 only
# when the payload is not JSON. Pure.
cl_without_fixed_dispositions() {
jq -c '
if type == "array" then
map(select(((objects | .body // "")
| test("<!-- dev-lead:comment-disposition [^\\n]*disposition=fixed\\s")) | not))
else . end
' <<<"${1:-}" 2>/dev/null
}

# cl_retract_disposition_body <body> <reason>
# The retracted form of a `fixed` disposition reply (#2156): every
# comment-disposition marker is removed (so the minimizer can never act on it), a
Expand Down
Loading
Loading