Skip to content

dev-lead merges zero-diff PRs that close their issues — 10 confirmed across 5 repos since 2026-07-17 (latest 2026-09-09) #1620

Closed
1 of 1 issue completed

Description

@don-petry

Summary

dev-lead is merging pull requests that change no files at all. They carry a feat: implement issue #N title, post a "Dev-Lead: Implementation Complete" comment asserting the work is "durable: commits are pushed and a pull request is open", pass CI (there is nothing to fail), merge, and auto-close their issue via Closes #N.

The finding is not fixed. Nothing was delivered. The issue is closed.

Confirmed instances

Nine found before the scan was cut short; several repos were not reached, so treat this as a floor, not a total.

PR Merged Issue closed Subject
markets#441 2026-08-31T21:14:19Z #418 ruleset-bypass-orgadmin-code-quality
markets#456 2026-08-31T21:06:35Z #448 ruleset-bypass-dependabot-code-quality
markets#377 2026-08-18T15:29:06Z #376 gitignore_baseline
markets#357 2026-08-02T16:12:04Z #329 stub-surface-drift
broodly#464 2026-08-04T21:04:39Z #451 SonarCloud: JavaScript
broodly#413 2026-07-17T16:57:03Z #411 dev-lead-stub-*
TalkTerm#373 2026-07-20T05:01:49Z #372 stub-surface-drift
incubator#79 2026-07-28T18:57:53Z #78 SonarCloud: shell script hygiene
incubator#55 2026-07-21T18:49:31Z #54 SonarCloud: shell script hygiene

Reproduce:

gh api "repos/petry-projects/<repo>/pulls?state=closed&per_page=60&sort=updated&direction=desc" \
  --jq '.[]|select(.merged_at!=null)|select(.head.ref|startswith("dev-lead/"))|.number' \
| while read n; do
    cf=$(gh api "repos/petry-projects/<repo>/pulls/$n" --jq '.changed_files')
    [ "$cf" = "0" ] && echo "$n files=0"
  done

This is not limited to compliance issues — SonarCloud remediation is affected identically.

Mechanism

The signature is consistent. markets#438 (still open, against #437) shows it mid-flight:

7 commits, changed_files: 0
  0ace9cdb  feat: implement issue #437 — Compliance: gitignore_baseline
  0d5676f9  chore: dev-lead update (review-changes) [skip ci-relay]
  a209a140  Merge branch 'main' into dev-lead/issue-437-20260821-1309
  afc34fbf  Merge branch 'main' into dev-lead/issue-437-20260821-1309
  29eb6a4c  Merge branch 'main' into dev-lead/issue-437-20260821-1309
  cb9bc016  Merge branch 'main' into dev-lead/issue-437-20260821-1309
  5e27066a  Merge branch 'main' into dev-lead/issue-437-20260821-1309

An implementation commit, a review-changes commit, then repeated merges from main — and a net diff of nothing against base. markets#441 demonstrates the transition directly: it carried a real +3/−2 change to .github/scripts/tests/apply-code-quality-ruleset.bats when inspected earlier on 2026-08-31, and merged hours later with changed_files: 0. The content is being lost after implementation, most plausibly by a resolve-in-favour-of-main on the repeated merges, or by a force-push from a stale base.

AC1 — refuse to open or update a PR with an empty diff

Before opening a PR (and after any review-changes or rebase/merge push), compare against the base and abort if the net diff is empty:

if [ -z "$(git diff --name-only "origin/${BASE}...HEAD")" ]; then
  echo "::error::dev-lead produced no net change against ${BASE} — refusing to open/update PR"
  exit 1
fi

Fail loudly and label the issue dev-lead:needs-human rather than opening an empty PR.

AC2 — a completion claim must be backed by a diff

The "Implementation Complete" comment asserts durability. It must not be posted when changed_files == 0. The comment template should state the file count and net line change it is claiming.

AC3 — do not let the merges eat the work

Establish why repeated Merge branch 'main' commits erase the implementation commit's content. If conflict resolution is defaulting to --theirs/ours in favour of main, that is the bug. Prefer rebase-onto-main with an explicit conflict stop over repeated merges.

AC4 — a merge gate

An empty diff should not be mergeable. Add an advisory-or-required check that fails when a PR's net diff against base is empty, so this cannot reach main even if AC1 is bypassed.

AC5 — remediate the closed issues

The nine issues above are closed and unremediated. They need to be reopened or refiled once AC1–AC3 land. markets#418 and markets#448 have already been reopened manually with live-state evidence.


Note on verification. The PR file list in the GitHub UI and gh pr view can show the implementation commit's changes while the net diff against base is empty. Use the branch compare instead:

gh api repos/petry-projects/<repo>/compare/main...<branch> --jq '.files|map(.filename)'

Related, in petry-projects/.github: #1036 (audit closes live findings), #1037 (ruleset findings routed to a code agent), #1038 (apply-repo-settings.sh never reaches the ruleset self-heal).

Filed from a manual compliance sweep, 2026-09-01.


AC REVIEW 2026-09-07 — evidence refreshed, AC5 made actionable

No recurrence in the last week

Re-scanned the most recent ~12 merged dev-lead/* PRs in markets, broodly, TalkTerm, google-app-scripts, bmad-bgreat-suite and .github. The only zero-diff merges are still the two from 2026-08-31 (markets#441, markets#456).

This is not evidence the defect is fixed. Nothing has changed in the engine, and the volume of compliance PRs dropped sharply over the same period — the vendored apply-repo-settings callers were disabled on 09-01 and the ruleset findings stopped producing PRs. A quiet week with fewer PRs is not a passing test. The ACs stand as written.

AC5 — made actionable

The original AC5 said the nine issues "need to be reopened or refiled." Current disposition, so nobody has to re-derive it:

Issue State Action
markets#418 open reopened 2026-09-01 with live-ruleset evidence (code-quality bypass_actors: 0); re-closed by the audit 09-01, and left closed pending .github#1036 — now fixed, so it should be re-verified
markets#448 open same — reopened with evidence, dependabot-automerge-petry bypass still absent
markets#376 closed re-verify gitignore_baseline against live state before refiling
markets#329, TalkTerm#372 closed stub-surface-drift — re-verify; the stub-pin check has a known false-positive mode (.github-private#1571/#1674), so confirm the drift is real before refiling
broodly#451, broodly#411, incubator#78, incubator#54 closed SonarCloud findings — re-verify against current analysis

Do not bulk-reopen. Each needs a live-state check first; reopening a finding that has since been genuinely fixed re-creates the churn this issue exists to stop.

Detection query for the backfill

for r in <repos>; do
  gh api "repos/petry-projects/$r/pulls?state=closed&per_page=100&sort=updated&direction=desc" \
    --jq '.[]|select(.merged_at!=null)|select(.head.ref|startswith("dev-lead/"))|.number' \
  | while read n; do
      [ "$(gh api "repos/petry-projects/$r/pulls/$n" --jq '.changed_files')" = "0" ] \
        && echo "$r#$n"
    done
done

Note this must read .changed_files on the PR object — the file list in the UI and gh pr view can show the implementation commit's changes while the net diff against base is empty. The authoritative check is the branch compare:

gh api repos/petry-projects/<repo>/compare/main...<branch> --jq '.files|length'

AC1 placement

Worth stating explicitly: the guard belongs after any merge-from-main or rebase step, not only before the initial PR open. markets#441 carried a real +3/−2 diff when inspected mid-flight on 2026-08-31 and merged with changed_files: 0 hours later — the content was lost by the repeated Merge branch 'main' commits, long after the PR was opened. A pre-open check alone would not have caught it.


⚠️ STILL LIVE — 10th instance found 2026-09-10 (recurred 2026-09-09)

petry-projects/incubator#103 reproduces the signature exactly and is not in the table above.
It merged 2026-09-09T12:16:51Z, nine days after the "9 confirmed" scan, so this issue is not stale
and not self-healed
— the #1621/#1691/#1692 integrity work did not close this path.

Field Value
PR incubator#103 — feat: implement issue #94 — SonarCloud: shell script hygiene
Branch dev-lead/issue-94-20260811-1809
Merged 2026-09-09T12:16:51Z
Diff changed_files: 0, +0/-0, 4 commits
Issue closed incubator#94 CLOSED at 2026-09-09T12:16:52Z (1 second after merge)

Commit shape — the AC3 merge-eats-work signature, unchanged:

49fcc254  feat: implement issue #94 — SonarCloud: shell script hygiene
6250d6d9  fix(reviews): address review comments [skip ci-relay]
bd82c773  Merge branch 'main' into dev-lead/issue-94-20260811-1809
72320d1b  Merge branch 'main' into dev-lead/issue-94-20260811-1809

The PR body additionally carries a CodeAnt-generated description asserting delivered behaviour
("Make incubation gate checks report their results reliably") that does not exist in the diff —
so the misleading-completion-claim surface in AC2 is wider than the dev-lead comment alone.

Scan run 2026-09-10 over markets, broodly, TalkTerm, incubator (merged dev-lead/* branches,
changed_files == 0) reproduced all 9 originals plus incubator#103. Note markets#438, cited above as
mid-flight, is now closed unmerged with changed_files: 0.

For the record, the two agent repos are clean: a 500-PR sweep of .github-private
(2026-05-21 → 2026-09-10) found only #1481 (2026-08-07), #549, #481; .github's only zero-diff
merges are chore: sync … stub(s) PRs that close no issue. The pathology is confined to the
downstream repos dev-lead serves
, which is where AC1/AC4 need to land.


Plan 2026-09-13: sliced so each part fits one action budget

The 10th instance is incubator#103: merged 2026-09-09T12:16:51Z with changed_files: 0, closing incubator#94 one second later. The 2026-09-07 quiet week was not a fix. Five ACs across two repos will not fit one 2700s pass, so this issue is now a tracking parent:

Slice ACs Owner Status
#1786 AC1 + AC2: dev-lead refuses to open, update, or claim a PR with an empty net diff, checked after merge-from-main pushes too dev-lead released
slice 2 AC4: an empty-net-diff merge gate in a fleet reusable, advisory first dev-lead (petry-projects/.github) filed after #1786 merges
slice 3 AC3: root cause of why repeated Merge branch 'main' commits erase the implementation steering (investigation) after #1786
AC5 remediate the closed issues, each with a live-state check first; no bulk reopen steering after slices 1–3

Do not implement from this parent.

Activity

  1. github-actions commented on Sep 1, 2026

    @github-actions
    Contributor

    Thank you for this thorough and well-documented report — the reproduction script, the mid-flight example, and the confirmed instance table make this immediately actionable. The mechanism section is compelling, but before we prioritize AC3 (the merge strategy fix) over AC1 (the pre-open diff guard): do you know whether the content loss in cases like markets#441 was caused by a conflict resolved in favour of main, or by a force-push from a stale base — and is there a git reflog or workflow run log that shows which happened?

  2. changed the title [-]dev-lead merges zero-diff PRs that close their issues — 9 confirmed across 5 repos since 2026-07-17[/-] [+]dev-lead merges zero-diff PRs that close their issues — 10 confirmed across 5 repos since 2026-07-17 (latest 2026-09-09)[/+] on Sep 10, 2026
  3. don-petry commented on Sep 13, 2026

    @don-petry
    CollaboratorAuthor

    Re-verified 2026-09-13 — still live, root cause located. Releasing to dev-lead.

    This is the most damaging item in the held set, so I re-confirmed it rather than taking the original scan on trust.

    It is still happening

    Re-scanned every dev-lead feat: implement issue … PR merged fleet-wide since 2026-08-25, fetching changedFiles per PR (gh pr list does not populate that field — a list-level filter silently returns nothing, which is how a scan can look clean while this is ongoing):

    ZERO-DIFF incubator#103   2026-09-09T12:16:51   feat: implement issue #94 — SonarCloud: shell script hygiene
    ZERO-DIFF markets#456     2026-08-31T21:06:35   feat: implement issue #448 — Compliance: ruleset-bypass…
    ZERO-DIFF markets#441     2026-08-31T21:14:19   feat: implement issue #418 — Compliance: ruleset-bypass…
    

    incubator#103: changedFiles: 0, +0/-0, merged 2026-09-09T12:16:51Z. incubator#94 closed at 12:16:52Z — one second later. Four days ago, with this issue already open and held.

    Root cause — the guard exists, but not on this path

    flag_noop_pr (#1340) does exactly the right thing: it detects a net-zero base…HEAD diff, posts a deduped human-attention comment, adds needs-human-review, disables auto-merge, and suppresses the EXIT-trap auto-merge restore so a self-cancelling PR cannot be silently made mergeable again.

    It lives only in scripts/dev-lead-fix-reviews.sh. git grep -l flag_noop_pr at main returns that one file. The PRs in the table above are produced by the issue-implementation path — scripts/dev-lead-fix-issue.sh — which has no equivalent guard. So the intent path can do what #1340 made the review path incapable of.

    That also explains the shape of the evidence: these are all feat: implement issue #N PRs, never fix-reviews output.

    Why this outranks the rest of the held set

    A zero-diff merge is not a no-op. It closes its issue via Closes #N, so a finding that was never fixed is recorded as fixed. For the compliance issues in the table (ruleset-bypass-*), the idempotent audit will re-open them on the next cycle and the cycle repeats — but for anything not regenerated by an idempotent scan, the work is silently lost and the audit trail says otherwise. That is the exact failure mode the clean-cycle promotion gate in .github#647 depends on not existing.

    Acceptance criteria

    • Apply the no-op guard to the issue-implementation path. Reuse flag_noop_pr — extract it and its NOOP_MARKER_PREFIX into a shared lib both dev-lead-fix-reviews.sh and dev-lead-fix-issue.sh source. Do not write a second implementation; two copies of one invariant is how this repo grows [Incident] Trunk corruption: main's scripts carry 3–6x duplicated function definitions (merged via #1420, #1292) — surgical restoration required #1485-class drift.
    • The guard must run before the PR is made mergeable: disable auto-merge and suppress the EXIT-trap restore, exactly as the review path does.
    • A zero-diff issue PR must never close its issue. Verify the Closes #N reference is removed or the PR is left unmergeable — state in the PR which mechanism you used.
    • Regression test: drive the issue path to a net-zero diff and assert (a) no merge, (b) auto-merge disabled, (c) the issue stays OPEN, (d) exactly one flag comment on a repeat pass.
    • Audit the three PRs above: incubator#94, markets#448, markets#418 were closed without a fix. Re-open them or say why each is safe to leave closed. Do not skip this — it is the actual damage.

    Also clearing needs-human-review: applied by issue-triage at 2026-09-01T00:34, at filing time, not by a human decision (#1778).

  4. don-petry commented on Sep 13, 2026

    @don-petry
    CollaboratorAuthor

    @petry-projects/qa-lead — asking for test-design input on the regression coverage for this one, since the failure mode is specifically "CI passed and the work was never done".

    Context: dev-lead's issue-implementation path can merge a PR with changedFiles: 0, which then auto-closes its issue via Closes #N. Verified live — incubator#103 merged 2026-09-09T12:16:51Z at +0/-0, and incubator#94 closed one second later. The flag_noop_pr guard from #1340 exists only in scripts/dev-lead-fix-reviews.sh, not in scripts/dev-lead-fix-issue.sh.

    Why this needs test-design input rather than just a test. A zero-diff PR passes every check we have, and it passes them honestly — there is nothing to lint, nothing to break, no coverage to drop. Every gate in the pipeline is measuring properties of a diff, and the diff is empty. The pathology is invisible to the entire existing gate set by construction.

    What I would like your view on:

    1. Where does the assertion belong? A unit test around the guard proves the guard works. It does not prove the guard is reachable from the issue path, which is precisely the defect — the code exists and is simply never called there. What is the right level to pin "this path cannot merge an empty diff"?
    2. Is there a general invariant worth asserting fleet-wide — e.g. no PR that closes an issue may merge with zero changed files — rather than one guard per intent path? There are several intents (issue, fix-ci, fix-reviews, fix-bot-comment, rebase, review-changes) and I would rather not discover this a third time on the fourth path.
    3. How should the negative case be exercised? Driving the real issue path to a genuine net-zero diff is awkward to set up; a fixture that fakes it risks testing the fixture. Your call on which is worth the cost.
    4. Detection for what already merged. Three zero-diff merges since 2026-08-25 closed three issues that were never fixed. Is a standing check worth having — a scheduled scan for merged Closes #N PRs with changedFiles == 0 — or is that redundant once the guard is in place?

    One practical note if you or anyone writes the scan: gh pr list --json changedFiles does not populate that field. A list-level filter on it silently matches nothing, so a scan can look clean while this is actively happening. It has to be gh pr view per PR. That cost me a false negative earlier today.

  5. don-petry commented on Sep 13, 2026

    @don-petry
    CollaboratorAuthor

    QA Lead — test-risk advisory

    Risk tier: HIGH — an unguarded production path lets Closes #N PRs merge at zero diff, silently closing real defects as fixed; confirmed 10x across 5 repos over ~2 months.

    What I'd shore up (highest leverage first):

    • Assert reachability, not just correctness. A unit test on flag_noop_pr proves the guard works in isolation — it does not prove dev-lead-fix-issue.sh (or any intent script) ever calls it. Pin this with an integration-level test per intent entrypoint: drive each script to a net-zero diff and assert the guard fires from that script's own code path, not via a shared helper's test suite.
    • Prefer one fleet-wide invariant over N guards. Six intents (issue, fix-ci, fix-reviews, fix-bot-comment, rebase, review-changes) each risk this independently. Centralize the zero-diff check as a single pre-merge gate all intents route through (or a required check dev-lead-*.sh scripts can't bypass), then test that gate once plus one "is it wired in" smoke test per script.
    • Exercise the negative path against real git state, not a mocked diff — a fixture that fakes changed_files risks testing the fixture, and this defect's whole signature is "the guard exists, the call site doesn't." Script a repo fixture that reaches genuine net-zero via merge, and assert the PR path refuses/labels rather than merges.
    • Detection for already-merged damage is worth it and cheap: a scheduled gh pr view scan (not pr list --json changedFiles, per the reporter's note — that field is unpopulated at list scope) for merged Closes #N PRs with 0 changed files. This is a backstop, not a substitute for AC1.

    Escalate? yes — real defect (false completion signal across repos), not test-infra noise; needs a maintainer decision on the centralized-invariant approach before more intent paths ship.

    Advisory only — I comment, I do not change code. Ground: BMAD Test Architecture (bmad-tea v1.19.0). Opt out on this item with the qa-lead:hands-off label.

  6. added a commit that references this issue on Sep 13, 2026
  7. added a commit that references this issue on Sep 21, 2026
  8. don-petry commented on Oct 21, 2026

    @don-petry
    CollaboratorAuthor

    Closing after the 2026-10-21 backlog triage.

    Why: Fixed: dev-lead refuses to open/claim a PR with empty net diff.
    Evidence: PR #1801 (#1786); main:scripts/dev-lead-fix-issue.sh:496

    Reopen if this still reproduces on main.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugBug reportsdev-leadFor dev-lead agent pickup

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions