Auto-merge gate can silently skip an evaluation: adopt concurrency.queue: max

Why

GitHub Actions shipped a queue key for concurrency, and it closes a gap this repo is currently exposed to.

cancel-in-progress: false does not mean "every run eventually executes." It protects the run that is already executing; the run waiting behind it is still discarded when a newer one arrives:

So on a burst of three events in one group, run 1 executes, run 2 is silently cancelled, run 3 executes. There is no failure, no annotation, and nothing in the run list to say a run was dropped.

`queue`	Meaning
`single` (default)	"At most one job or workflow run can be `pending` in the concurrency group."
`max`	"Up to 100 jobs or workflow runs can be `pending` in the concurrency group." — FIFO

queue: max with cancel-in-progress: true is a validation error, so it only ever applies where cancelling is already switched off.

Announcement: https://github.com/orgs/community/discussions/12835#discussioncomment-16602100
Reference: https://docs.github.com/en/actions/reference/workflows-and-actions/workflow-syntax#concurrency

Proven in this org already: devantler-tech/agent-skills has run queue: max in its release.yaml since 2026-07-11, with every Release run green since. This is propagating a fix we have already validated, not a trial.

Why it matters here specifically

.github/workflows/enable-auto-merge.yaml sets a job-level group:

concurrency:
  group: enable-auto-merge-${{ github.event.pull_request.number || github.event.issue.number || github.run_id }}
  cancel-in-progress: false

The group is per-PR, and this workflow is deliberately triggered by a wide event set — pull_request, pull_request_review, issue_comment, merge_group — precisely so the gate re-evaluates when reviewer results land after the original push. Those events routinely arrive in quick succession on one PR: a synchronize, then a CodeRabbit review, then a Codex result comment.

Under the default queue: single the middle evaluation is cancelled. The file's own comments state the requirement that makes this serious:

A dropped evaluation is therefore not just a missed arming — it can be a missed disarm, leaving a PR armed on evidence that has since been withdrawn. That is a correctness defect in a fail-closed gate.

This is also the highest-leverage instance in the portfolio: this is a reusable workflow consumed across repos, so one fix propagates to every consumer.

What

Add queue: max to the enable-auto-merge job's concurrency block so every triggering event gets its own evaluation, FIFO.

Audit the other workflows in this repo while here — lint.yaml and validate-go-project.yaml use cancel-in-progress: true, which is correct for superseded CI and must stay as-is.

Acceptance criteria

The enable-auto-merge job carries queue: max; cancel-in-progress: false is retained.
A burst of PR events on one PR produces one evaluation run per event, none cancelled.
cancel-in-progress: true workflows are left unchanged.

Part of devantler-tech/monorepo#2453 · Rough size: XS

Blocker: #277 | upstream | last-verified 2026-10-18: #277 is still open (rollout not complete); as of 2026-10-10 the three repository opt-ins still read true; this repository has now run two eligible evaluations under its opt-in (two Dependabot pull requests on 2026-10-09, both auto-merge jobs succeeded, nothing cancelled in 127 complete runs), but each pull request had exactly one, so no burst of real evaluations has been observed; the blocker issue records the decision to rely on the native three-job queue check plus this steady state, and this issue closes when every repository under the gate has the opt-in

🔗 Related PRs

devantler-tech/actions#412 - ci: skip the suite on the release-commit push to dodge the self-pin tag race [merged]
devantler-tech/actions#427 - fix: resolve first-party self-references at the same commit [merged]
devantler-tech/actions#575 - fix(auto-merge): harden mutable review evidence [merged]
devantler-tech/actions#594 - fix(validate-go-project): raise govulncheck timeout to survive cold-cache scans [merged]
devantler-tech/actions#599 - fix(auto-merge): complete ineligible required runs [merged]

📝 Issue Planner

_{Check the box below or use the @coderabbitai plan command to generate an implementation plan and prompts that you can use with your favorite coding assistant.}

Create Plan

🧪 Issue enrichment is currently in open beta.

You can configure auto-planning by selecting labels in the issue_enrichment configuration.

To disable automatic issue enrichment, add the following to your .coderabbit.yaml:

issue_enrichment:
  auto_enrich:
    enabled: false

💬 Have feedback or questions? Drop into our discord!

This is already implemented — do not build it again

I picked this up as the oldest unblocked bug and found the code already in place. Recording that here so the next run does not re-derive it, which is the waste pattern this portfolio keeps paying for.

It shipped in 81ada4a, "add opt-in retention of queued evaluations" (#1151). Verified at the current tip of the default branch: all three concurrency blocks in the auto-merge workflow carry the key — the workflow-level one and both job-level ones — and cancel-in-progress: false is retained on the job-level pair.

Against the acceptance criteria

The job carries the key, with cancellation still off — met, with one deliberate refinement worth noting. It is not the literal constant this issue asked for; it is an expression that resolves to the queued behaviour when opted in and to the default otherwise. That is this portfolio's feature-flag-first convention rather than a shortfall: the change lands latent and activation is a separate, reversible step.
Workflows that cancel in progress are left unchanged — met. The three that cancel are untouched, which is also required, since the two keys are a validation error together.
A burst of events produces one run per event, none cancelled — not met, and cannot be while the opt-in is off. This is the only criterion still open.

What actually remains

Activation, then the behavioural proof. There are two ways in, and they differ in who can do them:

A repository variable, which is a settings change and so the maintainer's.
An input on the call, which a caller repository can set in its own workflow — squarely inside what an agent may change.

The second is the interesting one, because it means this does not have to wait on a settings toggle. A single caller could opt in, the burst behaviour could be observed there, and the rollout could widen from that evidence.

Suggestion

Re-scope this issue to activation and verification rather than implementation, or close it and open a small successor for the rollout. I have not changed either, since which caller goes first is a judgement about blast radius that belongs with whoever picks the rollout up.

Unassigning myself and releasing the claim — there is no code to write here.

Decision: activate with a canary on `devantler-tech/actions`, then roll out org-wide

I checked the one route I could have taken myself, and it is not available.

The caller-input route does not exist. No repository calls enable-auto-merge.yaml through workflow_call. It runs in ksail, platform, monorepo and elsewhere as a required workflow, and that ruleset is managed in the UI rather than declaratively in .github/deploy/. A direct required-workflow run has an empty inputs context, so the only switch it reads is the QUEUE_PENDING_EVALUATIONS variable. Adding a workflow_call caller just to set the input would run the gate twice on every event, which is worse than leaving it off.
Nothing declares org Actions variables in Git. .github/deploy/ manages rulesets, repositories, teams and labels, but not variables. So setting the variable is a settings change, and settings changes belong to the maintainer.

Recommended order, smallest blast radius first:

Set the repository variable QUEUE_PENDING_EVALUATIONS=true on devantler-tech/actions only. This is the canary: the workflow's own PRs see the most review churn.
Trigger a burst on one PR there (push, then a CodeRabbit review, then a Codex comment) and confirm every evaluation runs, with none cancelled. That is the one criterion still open.
Set it at the organisation level and delete the repository variable.

Rollback at any point is deleting the variable. It changes only how many pending evaluations are kept (up to 100 instead of 1). Cancellation behaviour is unchanged, and GitHub rejects queue: max together with cancel-in-progress: true, so the two cannot combine.

I answered you in: https://devantlertech.slack.com/archives/D0B1JG2EWLW/p1790013722580349?thread_ts=1789987000.073659&cid=D0B1JG2EWLW

Correction: the recorded burst protocol cannot fail, so it cannot verify anything

Picked this up again and re-checked the premise and the plan against main at 0e07438 before building anything.

The key is real. GitHub's workflow-syntax and concurrency docs define concurrency.queue with single (default, at most one pending run, a newer one replaces it) and max (up to 100 pending, FIFO), and reject queue: max together with cancel-in-progress: true. The implementation is on main (all three concurrency blocks), so there is still no code to write.

The protocol in the previous comment would report a PASS whatever the queue setting is. Three issue_comment events from devantler on a PR here never share a concurrency group:

Workflow level: a comment run only joins the shared per-PR state lane when it is a deleted review-bot comment on a trusted-bot PR with merge gates enforced. Every other comment run's group ends in its own github.run_id, so each run has a private group.
Mutation jobs (the only other lane): they run only when eligibility classifies the event as eligible. A comment is eligible only when its author is a trusted review bot (CodeRabbit or Codex) and the PR author is a trusted bot. A devantler comment is never eligible, so both mutation jobs are skipped and never enter the lane.

So "three runs, none cancelled" is guaranteed by construction. The same applies to any devantler-authored PR, which covers every agent PR in this repository.

What a valid burst needs

Two or more eligible evaluations must be pending in the same group at once. In this repository that means one non-draft PR authored by a trusted bot (Dependabot, Renovate, etc.) and either:

several pull_request lifecycle events close together (for example a close and reopen repeated while the first evaluation is still running). This is only meaningful while actor-trust enforcement is off here: with it on, those events are deliberately cancel-in-progress: true with queue: single, so a cancelled middle run would be expected, not a defect; or
with merge gates enforced, a CodeRabbit review followed by a Codex result comment on that bot PR, both landing inside the first evaluation's runtime. This is the scenario the issue was written about.

PASS = every run in that burst has a job that waited on the group (its started_at is after the previous job's completed_at) and none is cancelled. Runs that never overlapped prove nothing, as the 2026-09-22 sample already showed.

I have not triggered either burst. Both mean manipulating a live Dependabot PR, and whether actor-trust and merge-gate enforcement are on in this repository has to be read from its settings first, which I cannot see from here. Leaving this open for the canary verification. Claim released.

Generated by Claude Code

Canary exposure measured: zero eligible evaluations so far

I read the job list of every enable-auto-merge.yaml run on this repository since the variable was set (2026-09-21T18:01:32Z → 2026-09-23T20:50Z): 155 runs, 131 issue_comment and 24 pull_request_review, and not one of them ran a mutation job. Only eligibility executed in each.

That is why the earlier "zero cancellations" reading proves nothing: nothing ever entered the queued lane. The reason is who authored the PRs in the window. Every review-bot event landed on a botantler-1 PR, which is not in TRUSTED_BOT_AUTHORS, and the reviews on the two Dependabot PRs came from botantler-1, which is not in TRUSTED_REVIEW_ACTORS. So the canary has had no exposure, and waiting passively will not change that on its own.

Verification plan, recorded so the next run can execute it without re-deriving: the next time a Dependabot PR is open and non-draft here, post a CodeRabbit review request and a Codex review request on it back to back. Both reviewers are trusted review actors and the PR author is a trusted bot, so both result comments are eligible evaluations in the same per-PR mutation group. PASS = every resulting mutation job either ran or waited (its started_at is after the previous job's completed_at), and none is cancelled.

No code change is needed; this stays open for that measurement.

Current canary readback on 2026-09-24: the repository variable QUEUE_PENDING_EVALUATIONS=true remains present. ENFORCE_ACTOR_TRUST and ENFORCE_MERGE_GATES have no repository entries and do not appear in the complete applicable organization-variable listing. The only open Actions PR is devantler-tech/actions#1335, authored by devantler, so it is not an eligible trusted-bot canary. No burst or settings mutation was performed, and this remains unverified.

One correction to the latest proposed protocol: requesting CodeRabbit and Codex back to back conflicts with the consumer contract's rule that only one review-provider request may be active at a time. Do not manufacture a second review after a successful first review just to produce queue traffic. A future canary should use an eligible, real trusted-bot PR and a permitted event sequence that demonstrably overlaps in the actual mutation concurrency group. Record pending/start/completion timestamps and eligibility for every event; green eligibility-only runs still prove nothing about retention.

Canary re-measured 2026-10-01: the actions-repo canary has retired, and it never got any exposure

devantler-tech/actions: 400 enable-auto-merge.yaml runs between 2026-09-23T20:52Z and 2026-09-25T21:12Z (total_count 400, so the list is complete). There were 345 issue_comment runs and 55 pull_request_review runs. All 1,200 jobs were eligibility successes, plus auto-merge and disarm-untrusted-update skipped. Not one evaluation reached the queued mutation lane. There have been no runs since 2026-09-25T21:12Z, which matches the actions → .github consolidation. The QUEUE_PENDING_EVALUATIONS=true variable there now guards nothing.
devantler-tech/.github (the workflow's home now): enable-auto-merge.yaml reads QUEUE_PENDING_EVALUATIONS only for direct runs whose workflow_ref is this file, and only from a repository or organization variable. This repository has no repository variables. The organization variable listing needs admin:org, which this run does not have, so whether the queue is armed here is UNKNOWN, not "no". There have been 883 runs here since 2026-09-25.

Next step: set the repository variable QUEUE_PENDING_EVALUATIONS=true on .github, or confirm the organization-level one, so the canary runs where the traffic now is. That is a persistent settings change, so it is not taken from an unattended run. Then repeat the job-level measurement above against .github's runs, using organic trusted-bot traffic only (per the 2026-09-24 correction). No code change is needed.

Verified after protected merge of #741 as signed commit d3f31a258a5df8c715fb2a6625e41a1d953e8756, published in v13.1.1 and retained in current released main. Its post-merge CI is successful. Direct and required calls now default both mutation paths to queue: max while retaining cancel-in-progress: false; explicit workflow-call false remains compatible. Cancelling validation workflows were unchanged.

A fresh native census covers all 23 active repositories. The active inherited required-workflow rule selects 22 and binds the maintained workflow at main. The actual QUEUE_PENDING_EVALUATIONS setting is true in three selected repositories, absent in the other 19 and explicitly false in none. The complete organization-variable UI shows no organization override, with an existing variable as a positive control. The current merged source has no job environment introducing an additional setting scope. A fresh complete workflow census finds no external workflow-call consumers to retain the old caller default.

This closes the production exposure using the two-part acceptance decision already recorded on #277: the two real eligible pilot evaluations 37903680988 and 37903955045 are freshly re-read as successful, while native CI 40136759588 retains and completes all three jobs in its real shared queue. No new organic burst is claimed. #277 remains open for the separate temporary input/variable retirement; that planning work no longer leaves any selected repository on a single pending slot.

Navigation Menu

Repository navigation

Auto-merge gate can silently skip an evaluation: adopt `concurrency.queue: max` #286

Why

Why it matters here specifically

What

Acceptance criteria

This is already implemented — do not build it again

Against the acceptance criteria

What actually remains

Suggestion

Decision: activate with a canary on `devantler-tech/actions`, then roll out org-wide

4 remaining items

Correction: the recorded burst protocol cannot fail, so it cannot verify anything

What a valid burst needs

Canary exposure measured: zero eligible evaluations so far

Canary re-measured 2026-10-01: the actions-repo canary has retired, and it never got any exposure

Metadata

Assignees

Labels

Type

Fields

Projects

Milestone

Relationships

Development

Auto-merge gate can silently skip an evaluation: adopt concurrency.queue: max #286

Description

Why

Why it matters here specifically

What

Acceptance criteria

Activity

coderabbitai commented on Jul 25, 2026

devantler commented on Sep 20, 2026

This is already implemented — do not build it again

Against the acceptance criteria

What actually remains

Suggestion

devantler commented on Sep 21, 2026

Decision: activate with a canary on devantler-tech/actions, then roll out org-wide

devantler commented on Sep 21, 2026

4 remaining items

devantler commented on Sep 22, 2026

Correction: the recorded burst protocol cannot fail, so it cannot verify anything

What a valid burst needs

devantler commented on Sep 23, 2026

Canary exposure measured: zero eligible evaluations so far

devantler commented on Sep 24, 2026

devantler commented on Oct 1, 2026

Canary re-measured 2026-10-01: the actions-repo canary has retired, and it never got any exposure

devantler commented on Oct 25, 2026

Metadata

Metadata

Assignees

Labels

Type

Fields

Projects

Milestone

Relationships

Development

Issue actions

Auto-merge gate can silently skip an evaluation: adopt `concurrency.queue: max` #286

Decision: activate with a canary on `devantler-tech/actions`, then roll out org-wide