You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
A dev-lead PR a few commits behind main is held for a human: Test-Deletion Guard and the test-tamper guard both read base-side test changes as the PR's #2254
A dev-lead PR whose branch is a few commits behind main gets held for a human by two guards that both misread changes that happened on main as changes made by the PR. Neither is a real finding, and together they stop the PR.
The branch was cut at 63c03b4f; main had moved 3 commits to b6bf0f2e. The PR itself adds two test files' worth of cases and deletes nothing (git diff origin/main...HEAD -- tests: 124 insertions, 0 deletions).
dev-lead's test-tamper guard then refused dev-lead's own fix. A review-changes pass rebased onto main to clear the first failure. The tamper guard reported "Existing test changed without justification" for tests/dev-lead/unit/test_dev_lead_retry_budget_hold.bats and tests/test_persona_pr_pregate.bats. Those files changed on main, not in the PR. dev-lead did not push, disabled auto-merge, and the PR was labelled needs-human-review three minutes after it opened.
A plain "update branch" (merge main into the branch) cleared the first guard. The label had to be removed by hand.
Acceptance criteria
Test-Deletion Guard compares the PR head against the merge base with the base branch, so a file added to the base after the branch point is not reported as deleted. Test: a branch behind a base that gained a test file passes.
dev-lead's test-tamper guard ignores test changes that a rebase or merge of the base branch brings in. Only changes in the pass's own commits relative to the updated base count. Test: a pass that only rebases onto a base with changed tests is clean; a pass that rebases and also edits an existing test is still flagged.
A rebase or update-branch pass that changes no file of its own does not apply needs-human-review or disable auto-merge.
Thanks for the detailed report and for including the run links and the PR #2252 evidence, which makes the failure easy to follow. To confirm the scope of the fix, can you say whether the tamper guard in the rebase case compares against the pre-rebase base SHA or the current tip of the base branch? Knowing which ref it uses will show whether a merge-base comparison, as in the acceptance criteria, is enough or whether the pass needs to record the base SHA before it rebases.
Issue:#2254 — A dev-lead PR a few commits behind main is held for a human: Test-Deletion Guard and the test-tamper guard both read base-side test changes as the PR's
Scope
Make both guards judge only the PR's (or the pass's) own test changes. Changes that main gained after the branch point, or that a rebase or merge of main brings in, no longer count.
Implementation Checklist
scripts/test-deletion-guard.sh (new): move the guard logic out of the workflow. It diffs the PR head against merge-base(base, head) instead of the base tip. If no merge base resolves, it fails closed.
.github/workflows/test-deletion-guard.yml: call the script (same env, same ack-test-deletion label semantics).
scripts/lib/test-tamper-guard.sh: ttg_scan_pass takes an optional 4th argument, the updated merge base (pass result vs. origin/<base>). When it differs from the pre-pass merge base, the pass brought base-side changes in. For each touched test file, the guard rebuilds what a plain rebase or merge would give: a 3-way git merge-file of pre-pass head, old merge base and new merge base. It then judges only the pass's diff against that expected content. A line still counts as "existing" if it is on the updated base and was not added by the PR. Skips are counted the same way. A test file that main added after the branch point is protected too.
scripts/dev-lead-fix-reviews.sh: resolve the updated merge base (git merge-base HEAD origin/<base>) next to the existing one and pass it to ttg_scan_pass. If it cannot be resolved, the old strict behaviour stays (fail closed).
Tests to Write
tests/test_deletion_guard.bats: a branch behind a base that gained a test file passes; a PR that really deletes a test fails; the ack-test-deletion label passes it; a missing merge base fails closed.
tests/dev-lead/unit/test_test_tamper_guard.bats: a pass that only rebases onto a base with changed tests is clean; rebase plus an edit to an existing test is tampered; rebase plus an edit to a test file main added is tampered; rebase plus a skip is tampered; a merge of the base (instead of a rebase) is clean; an unresolvable updated merge base gives unknown.
tests/dev-lead/unit/test_fix_reviews.bats: a review-changes pass that only rebases onto a moved main (whose test changed) is pushed, with no needs-human-review label and no auto-merge disable (AC 3). Rebase plus an existing-test edit is still refused.
Test Command
bats tests/test_deletion_guard.bats tests/dev-lead/unit/test_test_tamper_guard.bats tests/dev-lead/unit/test_fix_reviews.bats, plus bash .dev-lead/scripts/dev-lead-lint.sh
The implementation for issue #2254 is durable: commits are pushed and a pull request is open. This record is backed by verifiable artifacts you can confirm in one click.
Review happens on the PR. Acceptance criteria and test results are evidenced by the PR's diff and CI checks — not asserted here. If a later stage fails, this claim is retracted in place.
Activity
github-actions commentedon Oct 11, 2026
Thanks for the detailed report and for including the run links and the PR #2252 evidence, which makes the failure easy to follow. To confirm the scope of the fix, can you say whether the tamper guard in the rebase case compares against the pre-rebase base SHA or the current tip of the base branch? Knowing which ref it uses will show whether a merge-base comparison, as in the acceptance criteria, is enough or whether the pass needs to record the base SHA before it rebases.
don-petry commentedon Oct 11, 2026
Dev-Lead Implementation Plan
Issue: #2254 — A dev-lead PR a few commits behind main is held for a human: Test-Deletion Guard and the test-tamper guard both read base-side test changes as the PR's
Scope
Make both guards judge only the PR's (or the pass's) own test changes. Changes that
maingained after the branch point, or that a rebase or merge ofmainbrings in, no longer count.Implementation Checklist
scripts/test-deletion-guard.sh(new): move the guard logic out of the workflow. It diffs the PR head againstmerge-base(base, head)instead of the base tip. If no merge base resolves, it fails closed..github/workflows/test-deletion-guard.yml: call the script (same env, sameack-test-deletionlabel semantics).scripts/lib/test-tamper-guard.sh:ttg_scan_passtakes an optional 4th argument, the updated merge base (pass result vs.origin/<base>). When it differs from the pre-pass merge base, the pass brought base-side changes in. For each touched test file, the guard rebuilds what a plain rebase or merge would give: a 3-waygit merge-fileof pre-pass head, old merge base and new merge base. It then judges only the pass's diff against that expected content. A line still counts as "existing" if it is on the updated base and was not added by the PR. Skips are counted the same way. A test file thatmainadded after the branch point is protected too.scripts/dev-lead-fix-reviews.sh: resolve the updated merge base (git merge-base HEAD origin/<base>) next to the existing one and pass it tottg_scan_pass. If it cannot be resolved, the old strict behaviour stays (fail closed).Tests to Write
tests/test_deletion_guard.bats: a branch behind a base that gained a test file passes; a PR that really deletes a test fails; theack-test-deletionlabel passes it; a missing merge base fails closed.tests/dev-lead/unit/test_test_tamper_guard.bats: a pass that only rebases onto a base with changed tests is clean; rebase plus an edit to an existing test is tampered; rebase plus an edit to a test filemainadded is tampered; rebase plus a skip is tampered; a merge of the base (instead of a rebase) is clean; an unresolvable updated merge base givesunknown.tests/dev-lead/unit/test_fix_reviews.bats: areview-changespass that only rebases onto a movedmain(whose test changed) is pushed, with noneeds-human-reviewlabel and no auto-merge disable (AC 3). Rebase plus an existing-test edit is still refused.Test Command
bats tests/test_deletion_guard.bats tests/dev-lead/unit/test_test_tamper_guard.bats tests/dev-lead/unit/test_fix_reviews.bats, plusbash .dev-lead/scripts/dev-lead-lint.shdon-petry commentedon Oct 11, 2026
Dev-Lead: Implementation Complete — PR #2300
The implementation for issue #2254 is durable: commits are pushed and a pull request is open. This record is backed by verifiable artifacts you can confirm in one click.
3a6732d50b5a8ad09b0c0eede23c6522d933d2c2main: 9 file(s), +410/-30 linesReview happens on the PR. Acceptance criteria and test results are evidenced by the PR's diff and CI checks — not asserted here. If a later stage fails, this claim is retracted in place.