Skip to content

fix(auto-merge): retain pending evaluations on every run - #872

Merged
devantler merged 2 commits into
mainfrom
claude/auto-merge-retire-queue-opt-in-277
Nov 7, 2026
Merged

devantler merged 2 commits into
mainfrom
claude/auto-merge-retire-queue-opt-in-277

Conversation

@devantler

@devantler devantler commented Nov 4, 2026 •

Copy link
Copy Markdown
Contributor

🤖 Generated by the Agentic Engineer (interactive Claude Code session)

Why

#277 is the last step of the queued auto-merge rollout. When several pull-request events arrived close together, the single concurrency queue could drop the evaluation that was waiting, and that evaluation could be a disarm (#286). Three repositories (ksail, homebrew-tap, .github) have run with the retained queue since early October. #741 made it the default for direct and organization-required runs on 2026-10-25, and #286 is closed. What was left was the temporary activation plumbing.

What

  • Workflow-level queue is now max unless actor-enforced cancellation applies. GitHub forbids a retained queue together with cancellation, so the cancellation guard is unchanged.
  • Both mutation jobs (disarm-untrusted-update and auto-merge) use queue: max.
  • The QUEUE_PENDING_EVALUATIONS variable is no longer read.
  • Interface: the queue-pending-evaluations input stays declared, with the same type and default. It is now marked Deprecated and ignored. This follows AGENTS.md's rule to keep workflow_call inputs backward-compatible and to keep an alias where feasible. A caller that still passes the input keeps validating instead of failing with "input not defined". The behaviour change is that workflow_call callers which omit the input, or pass false, now retain pending evaluations as well. That is the rollout's goal.
  • test-enable-auto-merge-author-gate.sh now expects the new queue expressions. It also asserts that nothing outside the input declaration reads the retired input or variable, and that the input stays declared as deprecated. CI keeps its omitted, explicit-false and explicit-true callers, so a later hard removal would break this repository's CI before it broke a consumer.
  • The README section and the inputs table are updated, and the variable row is removed.

Consumer census

  • gh search code across the organization: no repository passes queue-pending-evaluations. Only this repository's own CI does. The single external enable-auto-merge.yaml mention (homebrew-tap) is a comment, not a call.
  • Repository variables across all 25 non-archived repositories: QUEUE_PENDING_EVALUATIONS is set only in ksail, homebrew-tap and .github, and always to true, the October opt-in. No repository opted out, so nothing loses a deliberate false. The organization-level variable is unreadable with this token (HTTP 403), so that one stays unknown. The three repository variables are now dead config and can be deleted at any time.

Validation

  • test-enable-auto-merge-author-gate.sh passes here and fails against main's workflow (RED before GREEN).
  • test-auto-merge-dry-run.sh, test-ci-metadata-events.sh and test-enable-auto-merge-pentad-gate.sh pass. shellcheck is clean. zizmor's only finding, dangerous-triggers, is also on main. actionlint 1.7.12 reports nothing new: its queue key warning predates this change and comes from an older schema.

Fixes #277

🤖 Generated with Claude Code

The queued evaluation default has run on required runs since #741 and on
three opted-in repositories since early October. Retire the temporary
activation plumbing: every non-cancelling run and both mutation jobs now
keep every pending evaluation, and the QUEUE_PENDING_EVALUATIONS opt-out
is no longer read. The queue-pending-evaluations input stays declared and
ignored so callers that still pass it keep validating. Actor-enforced
cancellation keeps the single queue that GitHub requires.

Fixes #277

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@devantler

devantler commented Nov 4, 2026 •

Copy link
Copy Markdown
Contributor Author

🤖 Generated by the Agentic Engineer (interactive Claude Code session)

@codex review

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ⚠️ Failed 2026-11-04T19:09:10.974305Z
We couldn’t complete this review. Comment @codex review to try again.
bab2f5a Manual request
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, add credits to your account and enable them for code reviews in your settings.

@github-code-quality

github-code-quality Bot commented Nov 4, 2026 •

Copy link
Copy Markdown

Code coverage overview

Languages: Go, C#

Go / code-coverage/test

main 17ee709 claude/auto-merge-re... e149e6a +/-
50% 50% 0%

C# / code-coverage/dotnet

main 17ee709 claude/auto-merge-re... e149e6a +/-
100% 100% 0%

Go / code-coverage/go

main 17ee709 claude/auto-merge-re... e149e6a +/-
50% 50% 0%

ℹ️ Calculations represent line coverage. Results may change as calculations are updated.


Updated November 04, 2026 22:10 UTC

… the workflow

Capture the workflow read before searching it, so a yq failure fails the
test instead of matching nothing, and drop a stale "when opted in" from
the README now that every run queues.

Fixes #277

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@devantler devantler left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 Generated by the Agentic Engineer (interactive Claude Code session)

Self-review (fallback — CodeRabbit, Codex and Cursor Bugbot unavailable)

Reviewed commit: e149e6a

  • CodeRabbit: rate-limited across the organization since 2026-11-03T09:04Z (coordinator lane-health reading). No review requested.
  • Codex: requested at the first head. It answered "You have reached your Codex usage limits for code reviews" at 2026-11-04T19:09:17Z.
  • Cursor Bugbot: hit its usage limit at 2026-11-04T00:29Z, with no delivery evidence since. Not requested.

An independent reviewer agent reviewed this PR at bab2f5a20966be763a5616c2cdf81f9ad4b475a7: correctness and security first, then this repository's review guidelines. It ran the PR's tests and mutation fixtures in a detached worktree. I fixed or documented every finding, then re-reviewed the delta to e149e6a3957a244ddb596ac90bb49f9d17d93a45 myself:

  • nit, fixed: the stale "When opted in" README sentence.
  • nit, fixed: the retired-read check now captures the workflow read first, so a yq failure fails the test instead of matching nothing.
  • Confirmed: no combination of inputs, variables or events gives cancel-in-progress: true with queue: max. Cancellation needs the actor-enforced condition, which forces single. A mutation back to the old expression fails the test.

Verdict: no P0/P1 findings

@devantler

Copy link
Copy Markdown
Contributor Author

🤖 Generated by the Agentic Engineer (interactive Claude Code session)

Tried as a user at e149e6a395, in this PR's own CI run 41899116961, by calling the reusable workflow the way consumers do:

  • [Test] Enable Auto-Merge omits the deprecated input. Its dry-run passed.
  • [Test] Enable Auto-Merge - Actor Trust Enabled passes queue-pending-evaluations: false. Its dry-run passed. Actor enforcement keeps cancellation, so this run takes the single queue.
  • [Test] Enable Auto-Merge - Pending Queue Enabled passes queue-pending-evaluations: true. Its dry-run passed, so a caller that still sends the deprecated input keeps validating.
  • Queue Slot 1 and Queue Slot 2 passed. They exercise the production mutation queue with concurrent slots. Queue Slot 3 was cancelled, not failed: an edit to the PR body started a newer ci.yaml run for this PR ("a higher priority waiting request for ci-872 exists"). The newer run is in progress at the same head.
  • Author Gate and Pentad Gate passed.

🤖 Generated with Claude Code

@devantler

Copy link
Copy Markdown
Contributor Author

🤖 Generated by the Agentic Engineer (interactive Claude Code session)

Status: held as a draft until CI finishes at e149e6a395. The local review round at this head passes local-review-verdict.sh (GREEN self@e149e6a3957a244ddb596ac90bb49f9d17d93a45). The review has no open findings, and the PR has no unresolved threads and no conflict. The first CI run at this head failed nothing: its only non-green jobs were cancelled when a PR-body edit started a newer run, which is still running. Promote once every required check is green at this head.

🤖 Generated with Claude Code

@devantler
devantler marked this pull request as ready for review November 7, 2026 11:06
@devantler
devantler merged commit 6aa4884 into main Nov 7, 2026
450 of 452 checks passed
@devantler
devantler deleted the claude/auto-merge-retire-queue-opt-in-277 branch November 7, 2026 11:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Roll out queued auto-merge evaluations and retire the temporary opt-in

1 participant