Skip to content

feat: implement issue #1045 — canary-rollout can't promote the #482 reusables: agent dispatch input hardcoded to [dev-lead] + no scheduled promote path - #1046

Closed
don-petry wants to merge 3 commits into
mainfrom
dev-lead/issue-1045-20260703-1229
Closed

don-petry wants to merge 3 commits into
mainfrom
dev-lead/issue-1045-20260703-1229

Conversation

@don-petry

Copy link
Copy Markdown
Collaborator

Closes #1045

Implemented by dev-lead agent. Please review.

@don-petry
don-petry requested a review from a team as a code owner July 3, 2026 12:41
Copilot AI review requested due to automatic review settings July 3, 2026 12:41
@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, you can upgrade your account or add credits to your account and enable them for code reviews in your settings.

@coderabbitai

coderabbitai Bot commented Jul 3, 2026 •

Copy link
Copy Markdown

Warning

Review limit reached

@don-petry, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 54 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro

Run ID: 0cabe989-327a-4baf-a092-f7958481e2c1

📥 Commits

Reviewing files that changed from the base of the PR and between 3a971b6 and 2dbf49f.

📒 Files selected for processing (5)
  • .github/workflows/canary-rollout.yml
  • AGENTS.md
  • scripts/canary-rollout.sh
  • scripts/canary_agent_options.sh
  • tests/canary_rollout.bats
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch dev-lead/issue-1045-20260703-1229

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@don-petry

Copy link
Copy Markdown
Collaborator Author

Dev-Lead — review-changes (no-changes)

No changes were needed for this PR.

@don-petry
don-petry enabled auto-merge (squash) July 3, 2026 12:42

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces automated canary rollout capabilities, adding a fleet-wide gated promotion command (promote-all) to scripts/canary-rollout.sh and a drift-guard script (scripts/canary_agent_options.sh) to ensure workflow dispatch options remain synchronized with the registry. Comprehensive documentation and BATS tests have also been added. The feedback suggests using the optional chaining operator (?) in jq within cmd_promote_all to safely handle missing or null objects and prevent script crashes under set -e.

Comment thread scripts/canary-rollout.sh Resolved
@don-petry
don-petry disabled auto-merge July 3, 2026 12:44

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR unblocks completing the #482 canary rollout by making canary-rollout operate on all registry agents (not just dev-lead) and adding an automated scheduled promotion path aligned with issue #1045.

Changes:

  • Add registry validation + a new fleet-wide promote-all path to the canary rollout orchestrator.
  • Introduce scripts/canary_agent_options.sh to generate/verify workflow_dispatch agent choice options from standards/canary-rings.json, with Bats drift-guard tests.
  • Update canary-rollout.yml and AGENTS.md to reflect registry-derived dispatch options and scheduled gated promotion, including deployment recording via a promotions journal.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
tests/canary_rollout.bats Adds coverage for unknown-agent validation, registry/workflow option drift checks, and promote-all behavior.
scripts/canary-rollout.sh Implements _require_agent, promotion journaling, and the new promote-all orchestrator command.
scripts/canary_agent_options.sh New helper to list/check registry-driven agent input options for the workflow.
AGENTS.md Documents the canary rollout dispatch + scheduled promotion behavior and the drift-guard mechanism.
.github/workflows/canary-rollout.yml Expands agent choice options, switches scheduled runs to promote-all, and records deployments from a journal.

Comment thread scripts/canary-rollout.sh Resolved
Comment thread tests/canary_rollout.bats Resolved
@don-petry

Copy link
Copy Markdown
Collaborator Author

Dev-Lead — fix-reviews (applied)

Changes committed and pushed.

@don-petry
don-petry enabled auto-merge (squash) July 3, 2026 12:48
@don-petry
don-petry disabled auto-merge July 3, 2026 12:49
coderabbitai[bot]
coderabbitai Bot previously approved these changes Jul 3, 2026
@don-petry

Copy link
Copy Markdown
Collaborator Author

Dev-Lead — review-changes (applied)

Changes committed and pushed.

coderabbitai[bot]
coderabbitai Bot previously approved these changes Jul 3, 2026
@don-petry
don-petry disabled auto-merge July 3, 2026 12:53
@don-petry

Copy link
Copy Markdown
Collaborator Author

Dev-Lead — review-changes (no-changes)

No changes were needed for this PR.

@don-petry

Copy link
Copy Markdown
Collaborator Author

Auto-rebase failed — merge conflict — this branch has conflicts with main that must be resolved.

dev-lead will attempt to resolve this automatically. If it cannot, a follow-up comment will explain what needs manual attention.

To resolve manually instead:

git fetch origin
git merge origin/main
# resolve conflicts, then:
git add .
git commit
git push

@don-petry
don-petry disabled auto-merge July 3, 2026 13:37
@don-petry
don-petry dismissed coderabbitai[bot]’s stale review July 3, 2026 13:40

The merge-base changed after approval.

@don-petry
don-petry force-pushed the dev-lead/issue-1045-20260703-1229 branch from 7e6b8dc to 2dbf49f Compare July 3, 2026 13:40
@don-petry

Copy link
Copy Markdown
Collaborator Author

Dev-Lead — rebase (applied)

Rebase completed and pushed.

@don-petry
don-petry enabled auto-merge (squash) July 3, 2026 13:40
@sonarqubecloud

sonarqubecloud Bot commented Jul 3, 2026

Copy link
Copy Markdown

@don-petry
don-petry disabled auto-merge July 3, 2026 13:41
@don-petry

Copy link
Copy Markdown
Collaborator Author

Dev-Lead — fix-bot-comment (no-changes)

Agent reasoning
Issues addressed: 0
- Quality Gate passed with zero new issues, security hotspots, or code quality concerns
Files changed: None required
Skipped (informational): 0
```
The PR is clean with respect to code quality. No fixes needed.

@don-petry
don-petry enabled auto-merge (squash) July 3, 2026 13:42
@donpetry-bot

Copy link
Copy Markdown
Contributor

Review — fix requested (cycle 1/3)

The automated review identified the following issues. Please address each one:

Findings to fix

Automated review — NEEDS HUMAN REVIEW

Risk: MEDIUM
Reviewed commit: 2dbf49f12a1ff342a6035fcb9c88b1067d569f6b
Review mode: triage-approved (single reviewer)

Summary

Implements #1045: registry-derived agent dispatch options with a drift guard (scripts/canary_agent_options.sh + bats), a gated fleet-wide scheduled promote-all (never --override, hard-stops on REGRESSION, honours gate.control.hold), and a promotions journal so each real promotion records a GitHub Deployment. Code is sound and prior bot-review findings were applied, but the PR is currently CONFLICTING with main and cannot merge — escalating for conflict resolution and re-review.

Linked issue analysis

Closes #1045. Both findings are substantively addressed: (1) the hardcoded [dev-lead] agent choice list is replaced with the full registry-derived, sorted list, generated/drift-guarded by scripts/canary_agent_options.sh (check wired into bats) plus runtime validation via _require_agent; (2) the 4h schedule now runs gated promote-all — one ring per agent per tick iff the #548 gate is PROMOTE, skipping gate.control.hold agents, never auto-overriding, hard-stopping on REGRESSION — matching the owner directive quoted in the issue. AGENTS.md documents the new model.

Findings

Blocking:

  1. Merge conflict — mergeable: CONFLICTING / mergeStateStatus: DIRTY against main (re-verified after the 13:40 UTC rebase; main appears to have moved again). The PR cannot merge, and conflict resolution will produce new commits requiring re-review. This is the sole blocker.

Non-blocking / verified:

  • Prior review feedback applied and all 3 review threads resolved: jq optional chaining (.agents?), and cmd_promote now returns 1 on BLOCKED+REGRESSION so promote-all genuinely hard-stops (scripts/canary-rollout.sh) — with matching bats coverage.
  • Behavior change is real but intentional and well-guarded: scheduled runs flip from read-only evaluate-all to promote-all with DRY_RUN=false (real tag moves via the App token). No auto---override, hold switch honoured, REGRESSION aborts the fleet loop.
  • No workflow injection: deployment-recording heredoc interpolates values from the script's own journal (registry keys/SHAs), not untrusted PR input. if: always() on that step is correct — it records promotions that completed before a mid-fleet hard-stop.
  • Secret scan: run_secret_scanning MCP tool unavailable in this environment; gitleaks CI check passed and no credentials appear in the diff.
  • Minor (informational): cmd_promote_all invokes cmd_promote in an if ! condition, which suppresses errexit inside it; explicit return 1 paths cover the important failures, so acceptable.

CI status

All substantive checks SUCCESS at 2dbf49f: Lint, ShellCheck (x2), bats, unit-tests, CodeQL (actions+python), gitleaks, AgentShield, Agent Security Scan, SonarCloud quality gate, holdout-guard, template-drift, gh-aw-compile, validate-agent-profiles, Test-Deletion Guard, CodeRabbit. Two Dev-Lead Agent orchestration jobs (dispatch, ci-relay) show CANCELLED — superseded orchestration runs, not code validation. Dependency-audit sub-jobs SKIPPED (no matching ecosystems).


Reviewed automatically by the PR-review agent (single-reviewer mode: fable 5). Reply if you need a human review.

Additional tasks

  1. Resolve all unresolved review thread comments from other reviewers
  2. Ensure all CI checks pass after your changes
  3. Rebase on the target branch if behind
  4. Do NOT modify files unrelated to the findings above

The review cascade will automatically re-review after new commits are pushed.

@don-petry

Copy link
Copy Markdown
Collaborator Author

Closing as superseded. promote-all and the all-agent workflow_dispatch input (the #1045 deliverable) are already on main. This PR now conflicts against the landed version. (Tracking: #1045 can be closed if nothing outstanding remains.)

@don-petry don-petry closed this Jul 4, 2026
auto-merge was automatically disabled July 4, 2026 17:40

Pull request was closed

@don-petry don-petry added the dev-lead:hands-off Exclude from dev-lead agent automation label Jul 4, 2026
don-petry added a commit that referenced this pull request Sep 1, 2026
…stable — orphaned v1-* channel family (#1592) (#1653)

* chore(dev-lead): repin stub from dev-lead/v1-stable to dev-lead/v139-next (#1592)

The `dev-lead/v1-*` channel family is orphaned. dev-lead is on major 139
(highest release v139.5.0), so `_resolved_channel` derives `v<major>-<tier>`
and promotion moves the v139 line — while every caller stub still pinned a
`v1-*` tag that nothing advances. Result: this repo has been executing
40f6c23 (2026-08-18) while main moved on.

Also a tier correction. standards/canary-rings.json makes this repo `next`'s
ONLY member (`members: ["$host"]`, order 0), but the stub pinned `v1-stable`
— the OUTERMOST tier. The canary was running backwards: the repo that hosts
dev-lead and is supposed to dogfood each candidate first was instead on the
last tier to receive it, and `next` had no consumer at all (which is why
"next == main" is a meaningless health signal — see #1049).

Effect: this repo moves from 40f6c23 (08-18) to the current candidate and
becomes the first to execute each new one, as the ring order intends.

Landed FIRST and by hand, ahead of the other six repos, so the host is
exercising the new channel before the fleet follows it.

Verified before pinning:
  - dev-lead/v139-{next,ring0,ring1,stable} seeded and resolving
  - `canary-rollout evaluate dev-lead` reports
      candidate (v139-next) = 3e5f5f3  cut=2026-09-01T09:34:34Z
      next->ring0  SOAKING  dwell=2h/4h  sample=78/3  cum_fail=0
    i.e. a resolvable cut date (the #1046 fix, live) and no false REGRESSION
  - YAML parses; the NOSONAR(S7637) marker on the `uses:` line is preserved

Refs #1592, #657, #1046, #1049

* chore(freeze): regenerate dev-lead caller-stub baseline for the v139-next pin

The caller-stub-freeze guard (#1255, epic #1052 Part B) freezes each ring-0
self-host stub's `on:` + `uses:`/`with:` block byte-for-byte, precisely so a
channel-pin change cannot slip through: a forwarding change to a
channel-pinned self-host stub is exercised by nothing in PR CI and only
breaks post-merge (#1034).

The guard did its job — it caught this repin. Regenerating the baseline in
the same reviewed diff is the sanctioned path the script documents, via
`bash scripts/caller_stub_freeze.sh --update`.

The regenerated diff is minimal and contains only the intended change:
`uses:` and `agent_ref:` move from dev-lead/v1-stable to dev-lead/v139-next.
pr-review-trigger.block and ci-failure-analyst.block regenerated
byte-identically (no change), confirming nothing else was disturbed.

* fix: repin to dev-lead/v139-stable, not v139-next — SC2 self-host guard

Correcting this PR's first commit. I repinned to `v139-next` on the reasoning
that standards/canary-rings.json makes this repo `next`'s only member, so it
should dogfood each candidate first. That reasoning was wrong, and
tests/test_sc2_self_review_channel.bats (#1624) exists precisely to catch it:

  "a deliberately-broken in-development version of this repo's agentic
   review/merge duty MUST NOT be able to block the PR that fixes its own
   breakage ... although this repo sits in ring `next`, dev-lead.yml pins a
   stable-tier channel. A broken `next` therefore cannot, by construction,
   gate its own fix ... nothing stops a well-meaning PR from repinning
   dev-lead to @dev-lead/v1-next and silently restoring the self-hosting
   circular dependency."

Ring membership and the stub pin are different things. This repo IS `next`'s
member — that is where `next` is sampled for the gate's health signal — while
the stub deliberately EXECUTES stable so a bad candidate cannot brick the
engine's ability to fix itself. The `next` canary duty in this repo belongs
to pr-review-trigger.yml, which pins pr-review/v1-next on purpose and is
explicitly excluded from the SC2 stable duty.

So the correct change here is the v1 -> v139 family move only, with the tier
unchanged: dev-lead/v1-stable -> dev-lead/v139-stable. Both resolve to
40f6c23 today, so this is a pure rename with zero behaviour change — it
moves the pin onto the channel family promotion actually advances, and
nothing else.

Verified:
  src_channel_tier dev-lead/v139-stable -> "stable"  (parser handles v-scoped)
  src_assert_self_review_stable          -> PASS
  caller_stub_freeze.sh --check          -> dev-lead.yml ALIGNED
  YAML parses; NOSONAR(S7637) marker preserved

No test was modified.

Refs #1624, #503, epic #495, #1592

* test(freeze): make the dev-lead caller-stub assertions version-agnostic

Two tests in tests/caller_stub_freeze.bats hardcoded the literal pin
`dev-lead/v1-stable` and so broke on the v1 -> v139 channel-family move.
Neither encodes an invariant about WHICH channel is pinned — they test the
extractor and the drift mechanism — so the literals are incidental. Made
them shape-based instead of updating one literal for another, so the next
channel move does not break them again.

This mirrors what tests/test_sc2_self_review_channel.bats deliberately does:
it parses the tier from the pin "never a hardcoded version, so cutting a new
release does not break it". That file's invariant (dev-lead.yml must pin a
stable TIER) is real and is left untouched — it correctly caught the earlier
mis-repin to v139-next on this branch.

1. "extract: dev-lead block ..." now matches the SHAPE of a dev-lead channel
   pin rather than a literal version, and additionally asserts the forwarded
   `agent_ref` EQUALS the `uses:` pin — a mismatch means the reusable and the
   scripts it checks out come from different channels, which is a real defect
   the old literal-pair assertion only caught by coincidence.

2. "acceptance: editing a frozen block ..." derived its simulated edit from
   `sed 's#@dev-lead/v1-stable#@dev-lead/next#'`. After the move that pattern
   matches nothing, so the "edit" became a no-op and the DRIFTED assertion
   was vacuously false. It now derives the current pin from the stub and
   rewrites that, so the test exercises a real edit whatever is pinned.

Verified locally: both tests pass. Tests 5 and 14 fail on this machine with
`CALLER_FREEZE_STUBS[@]: unbound variable` under bash 3.2 — confirmed
pre-existing by running the same file on unmodified origin/main, where they
fail identically. CI (ubuntu, bash 5) is the real gate.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

dev-lead:hands-off Exclude from dev-lead agent automation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

canary-rollout can't promote the #482 reusables: agent dispatch input hardcoded to [dev-lead] + no scheduled promote path

3 participants