Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 11 additions & 0 deletions .github/workflows/dev-lead-reusable.yml
Original file line number Diff line number Diff line change
Expand Up @@ -142,6 +142,11 @@ jobs:
# what caused the donpetry-bot regression (#1316); the manifest is now the
# sole source of truth. See scripts/lib/resolve-persona-identity.sh.
BOT_USER: 'don-petry'
# Repo-named test command for the pre-push regression guard
# (scripts/lib/test-regression-guard.sh, #2013). Mapped from a repository
# variable — NOT a workflow_call input — so no channel-pin sequencing is
# needed. Empty => the guard auto-discovers (npm/make/pytest/bats).
DEV_LEAD_TEST_CMD: ${{ vars.DEV_LEAD_TEST_CMD }}
# Empty by default so dev-lead-intent.sh derives the trusted set from the
# reviewer-source registry (scripts/lib/reviewer-sources.tsv), the single
# source of truth — no hardcoded list to drift from the gate/scorecard
Expand Down Expand Up @@ -447,6 +452,12 @@ jobs:
fi ;;
esac

# bats is not preinstalled on ubuntu-latest; the test-regression guard's
# `bats --recursive tests` fallback needs it (#2013). Same install as lint.yml.
- name: Install bats
if: env.INTENT_TYPE == 'fix-reviews' || env.INTENT_TYPE == 'fix-bot-comment' || env.INTENT_TYPE == 'review-changes' || env.INTENT_TYPE == 'human-pr'
run: sudo apt-get install -y bats

# ── fix-ci ────────────────────────────────────────────────────────────
- name: Run fix-ci
if: env.INTENT_TYPE == 'fix-ci'
Expand Down
3 changes: 3 additions & 0 deletions .github/workflows/lint.yml
Original file line number Diff line number Diff line change
Expand Up @@ -216,6 +216,9 @@
tests/dev-lead/unit/test_comment_disposition_verify.bats \
tests/dev-lead/unit/test_conflict_integrity.bats \
tests/dev-lead/unit/test_addressed_claim_verify.bats \
tests/dev-lead/unit/test_claim_landing.bats \
tests/dev-lead/unit/test_test_tamper_guard.bats \
tests/dev-lead/unit/test_test_regression_guard.bats \
tests/dev-lead/unit/test_fewshot_assemble.bats \
tests/dev-lead/unit/test_net_diff_guard.bats \
tests/dev-lead/unit/test_verify_approval_review.bats \
Expand Down Expand Up @@ -298,7 +301,7 @@
timeout-minutes: 5
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
# NOTE: persist-credentials is intentionally left at the default (true) —

Check warning on line 304 in .github/workflows/lint.yml

View workflow job for this annotation

GitHub Actions / Lint

304:9 [comments-indentation] comment not indented like content
# vci_resolve_reusable needs `git fetch origin` to authenticate against this
# private repo when resolving same-repo channel tags. Setting
# persist-credentials: false would silently push every such caller into the
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,94 @@
# 0011. Write-mode personas on a shared branch: verify what landed, retract what did not

## Status

proposed

## Context

ADR-0005 makes write access an opt-in gate: dev-lead's write surfaces are armed
by the `dev-lead` label. It does not say how a write-mode persona should behave
on a branch that **other actors are pushing to at the same time**, such as a
maintainer, a Claude session, or the auto-rebase bot.

petry-projects/.github#1220 showed what goes wrong
([#2013](https://github.com/petry-projects/.github-private/issues/2013)). While
a human session was driving the same branch:

- dev-lead posted "Fixed in …" replies for changes that never reached the
branch. Its claim named the PR's **first** commit: the head the model read
with `git rev-parse HEAD` *before* committing anything, because the prompt
told it to cite that SHA and also told it not to commit.
- CodeRabbit marked a thread "Confirmed as addressed" on the strength of a
reply describing `15a919e`, which *was* pushed. A wrong fix cleared a review
gate: it added a new test and broke an existing one without editing it, and
the persona never ran the suite against the pre-pass baseline.
- A bot suggestion ("update the older test so the suite can pass") was applied
literally. That rewrote an existing test and inverted deliberate behavior.

The no-clobber push (#1311/#1607) already prevented dev-lead from overwriting
another actor's commit. Everything *around* the push was still the model
describing itself. The #1692 claim check ran only when the harness resolved a
thread, and the pre-pass SHA passed it trivially. Nothing ever took a posted
claim back. That breaks ADR-0004's "Fail Loud, Never Fake".

## Decision

We will make every write-mode persona sharing a branch follow five rules, each
enforced by harness checks with bats tests (pure classifiers where applicable,
per ADR-0004), not by prompt text alone:

1. **Incorporate or stand down, never force.** Before pushing, the persona
reconciles with the *true* remote head. It rebases a foreign commit in, or
stops and escalates when that cannot be done cleanly. A foreign commit is
never discarded (`push_no_clobber`, `scripts/lib/git-push-guard.sh`).
2. **"Pushed" is a verdict, not an assumption.** After the push, the harness
compares the pre-pass head, the pushed SHA and the re-fetched remote head
(`cl_push_landed_verdict`, `scripts/lib/claim-landing.sh`). Any result other
than `landed` counts as a failed push.
3. **A claim must name a commit this pass produced and that landed.** The claim
SHA must be on the reference head and **not** contained in the pre-pass
head (`acv_claim_in_pass`, `scripts/lib/addressed-claim-verify.sh`). The
harness checks this at thread-resolution time against the PR head. It
checks it again after every push outcome against the **remote** head. Every
claim reply posted during the pass that fails is **retracted**: both markers
are stripped and a retraction notice is added, so the reply can never
authorize resolution and other bots cannot read it as a fix. The outcomes
covered are a rejected or unverified push, a guard abort, an engine
failure, no commit, and a stale SHA.
4. **An existing test is not the persona's to rewrite.** A fix pass that
changes or deletes an existing test line, or adds a skip, without an
explicit cited `Test-Change-Justification:` trailer is refused before the
push and escalated to a human (`scripts/lib/test-tamper-guard.sh`). A bot
suggestion that contradicts an existing test goes to a human.
5. **A red suite blocks the push.** Before the push, the harness runs the
repo's test command on the pass's result. A test that passed on the
pre-pass head and fails now refuses the push and escalates like the tamper
guard (`scripts/lib/test-regression-guard.sh`). A failure that already
existed on the pre-pass head does not block. When no test command can be
determined, the run summary says the suite was not run; it never implies
green.

The boundary a check can assert: **no dev-lead claim reply survives a pass
unless the claim's commit was produced by that pass and is on the remote
head.**

## Consequences

- A rejected, aborted or failed pass leaves retracted replies, not false
"Fixed" ones. The thread stays open and the next pass re-verifies.
- The fail-closed direction costs some true claims. When the push guard
rebases dev-lead's commits onto a foreign commit, the SHAs the model cited
no longer exist on the remote, so those claims are retracted even though the
fix landed. Re-mapping rebased SHAs is a possible follow-up. A stuck thread
is a nuisance; a wrongly cleared gate is a defect.
- The model must commit locally before it replies, so its claims can cite a
real commit. The prompts now say so. A pass that leaves its changes
uncommitted gets them committed by the harness, and its claims (citing the
pre-pass head) are retracted.
- Legitimate test changes need one extra line, the justification trailer. In
exchange, a bot-driven pass can no longer silently invert a tested behavior.
- Still open: whether a persona should *back off entirely* from a branch with
recent non-persona pushes, or require an opt-in label to push to a branch it
did not create. This ADR covers correctness of what is claimed. It does not
cover whether to participate at all.
13 changes: 9 additions & 4 deletions prompts/dev-lead/fix-bot-comment.md
Original file line number Diff line number Diff line change
Expand Up @@ -81,11 +81,11 @@ gh api graphql -f query='
))))'
```

For each thread you fixed, first **reply with the specific change** — name the file(s)/function(s) you touched and how the change addresses the finding (one or two concrete sentences; never just "done"). End the reply with **two** HTML comments: the addressed-marker `<!-- dev-lead:addressed -->` **and** a machine-readable claim `<!-- dev-lead:claim {…} -->` (#1692). The harness now verifies the claim against the pushed diff before resolving — a marker without a verifiable claim leaves the thread **unresolved**. Stamp both **only** on a genuine addressed reply, never on a skip note. Pass the body as a GraphQL variable so quotes and newlines are safe:
For each thread you fixed, first **commit the fix locally** (`git add -A && git commit -m "fix(bot): <what>"`, never push; if an earlier commit this pass already captured this fix the tree is clean — skip the `git commit` and cite that commit's SHA, since `git commit` on a clean tree fails), then **reply with the specific change** — name the file(s)/function(s) you touched and how the change addresses the finding (one or two concrete sentences; never just "done"). End the reply with **two** HTML comments: the addressed-marker `<!-- dev-lead:addressed -->` **and** a machine-readable claim `<!-- dev-lead:claim {…} -->` (#1692). The harness now verifies the claim against the pushed diff before resolving — a marker without a verifiable claim leaves the thread **unresolved**. Stamp both **only** on a genuine addressed reply, never on a skip note. Pass the body as a GraphQL variable so quotes and newlines are safe:

```bash
# Replace THREAD_NODE_ID with the id value from the query above.
# Get the full 40-char head SHA the fix rides on with: git rev-parse HEAD
# Commit the fix FIRST, then read the SHA of that commit with: git rev-parse HEAD
gh api graphql \
-f query='mutation($tid: ID!, $body: String!) { addPullRequestReviewThreadReply(input: {pullRequestReviewThreadId: $tid, body: $body}) { comment { id } } }' \
-f tid="THREAD_NODE_ID" \
Expand All @@ -95,7 +95,11 @@ gh api graphql \
<!-- dev-lead:claim {\"v\":1,\"sha\":\"3cc4132fd4b4692aa20865f8b68ea8e21de604b8\",\"files\":[\"scripts/foo.sh\"]} -->"
```

The claim payload is schema `v1` — one comment per reply, with a full 40-char `sha` (`git rev-parse HEAD`) and a non-empty JSON array of repo-relative `files` exactly as they appear in the diff. The normative schema and parser live in `scripts/lib/addressed-claim-verify.sh`.
The claim payload is schema `v1` — one comment per reply, with a full 40-char `sha` and a non-empty JSON array of repo-relative `files` exactly as they appear in the diff. The normative schema and parser live in `scripts/lib/addressed-claim-verify.sh`.

**Commit before you claim (#2013).** `sha` must be a commit **this pass produced**: run `git rev-parse HEAD` *after* you commit the fix. Citing the head you started from never verifies; it is the stale-claim defect from petry-projects/.github#1220. After the push, the harness checks the remote head. If the push was rejected or not incorporated, a guard refused it, or the pass failed, **every claim reply you posted this pass is retracted**. If you make no commit for a thread, post no addressed-marker or claim on it.
Comment thread
cubic-dev-ai[bot] marked this conversation as resolved.
Resolved

**Never rewrite an existing test to make a bot suggestion pass (#2013).** Run the **full** test suite. A previously-passing test that turns red is a reason to question the change, not the test. A bot suggestion that contradicts an existing test is not applied, and that includes "update the older test so the suite can pass". Reply explaining the conflict, **without** the addressed-marker, and leave it for a human. Changing or deleting an existing test line, or adding a `skip`, requires a cited `Test-Change-Justification: <why, with a reference>` commit trailer. Without one, the harness's test-tamper guard refuses to push and escalates. Adding a new test never needs it. Because `git commit -m` takes one message line, pass the trailer as a second `-m` paragraph: `git commit -m "fix(bot): <what>" -m "Test-Change-Justification: <why, citing #<issue>, a commit SHA, or the review URL>"`. The justification must be at least 20 characters and contain such a reference.

**Do not resolve the thread yourself.** You must not call the `resolveReviewThread` (or `unresolveReviewThread`) GraphQL mutation under any circumstance — resolution is done **only** by the harness (`dev-lead-fix-reviews.sh`), whose deterministic guards are the authoritative merge gate. The harness resolves every bot thread you addressed with an our-account addressed-marker reply, plus any thread from this bot marked `isOutdated: true`. Your reply and its marker are your only lever on resolution.

Expand Down Expand Up @@ -203,7 +207,8 @@ Some bot comments are pure **operational notices**, not code findings: a rate-li
- For every thread you fix, post a reply naming the specific change, ending with the addressed-marker — never reply-less
- **Never resolve a thread yourself**: do not call the `resolveReviewThread` (or `unresolveReviewThread`) mutation. Resolution is the harness's job — it resolves the addressed and outdated threads from `${ACTOR}`; your only lever is the addressed-marker on your reply
- Stay within the scope of the pull request's changed files where possible
- Do not commit or push — the CI workflow handles git operations after you finish
- Commit your fixes locally so your claims can cite them, but **never push**. The CI workflow pushes after you finish, verifies the push landed, and retracts any claim that did not land (#2013)
- Never edit an existing test to match your change. A conflicting bot suggestion goes to a human. An existing-test change needs a `Test-Change-Justification:` trailer (#2013)

## Output Format

Expand Down
Loading
Loading