Repository navigation
Open
Labels
initiativeEpic / initiative tracking issueEpic / initiative tracking issue
Description
Story
As a maintainer who decided no human is needed on the happy path,
I want the pipeline to promote a candidate that passed every gate to the head of its (family, tier) chain and walk it to stable,
so that release-to-default time is bounded only by the policy's shadow and soak windows.
Acceptance Criteria
- A pure decision function combines the stage results (pricing, compatibility/liveness preflight, quality gate, shadow, canary cost/latency) into PROMOTE, PAUSE(reason) or WAIT. Any PAUSE trigger from the ADR wins; INSUFFICIENT data → WAIT, never PROMOTE.
- On PROMOTE, the pipeline opens a PR that makes the candidate the first model of its (family, tier) chain in
scripts/lib/engine-models.sh, keeping the previous default as the next fallback hop. Only the chain head changes,scripts/check-model-pins.shpasses, andET_BASELINE_MODELand the eval judge are untouched. The PR merges once CI is green. - After merge, the channel advances
next→ring0→ring1→stablewithscripts/cut-release.sh, under the ring dwell and sample rules. At each transition the generic canary report (--tier <tier>) is evaluated against the policy tolerances (cost +10% USD; latency p50 +10%, p95 +25%; spike cap 3×). - On PAUSE, the onboarding issue gets the reason and a for-human-analysis label, and the pipeline cuts and merges nothing further for that rollout. The kill switch stops promotion at any stage.
- Negative-control test (offline): a stubbed known-bad candidate whose quality gate returns exit 1 (and, separately, whose canary cost is +30%) ends in PAUSE with no PR opened and no channel moved. Bats asserts this.
- Every stage result and the decision are appended to the onboarding issue.
Tasks / Subtasks
- Write the pure decision function with bats, including the negative control (AC: test issue from agent #1, feat: add Copilot engine support via REVIEW_ENGINE toggle #5)
- Open and merge the chain-head PR on PROMOTE (AC: Go-live improvements for PR review agent #2, feat: feature ideation framework — adversarial proposer/challenger/synthesizer #6)
- Advance the rings with per-transition canary evaluation (AC: Add @claude delegation, auto-merge, and rebase handling #3, feat: feature ideation framework — adversarial proposer/challenger/synthesizer #6)
- Implement pause, label and kill-switch handling (AC: Optimize review: small-PR and incremental fast paths #4, feat: feature ideation framework — adversarial proposer/challenger/synthesizer #6)
Dev Notes
- Supersedes the human-only promotion guardrail per the ADR from the policy story. A merged engine change is inert until the channel tag moves (Script-only changes never reach the fleet: autocut watches the reusable blob, but agents execute scripts/ pinned at agent_ref #1592), so ring advancement is part of this story, not optional.
- Ring dwell (
next→ring04h,ring0→ring18h,ring1→stable12h), the 14-day baseline window and the 3× spike cap live inrelease/registry.ymland the cross-repostandards/canary-rings.json. Read them; do not copy them. - Cost evidence method (from
docs/initiatives/sonnet-5-default-go-no-go.md): re-price the incumbent's records withcost_usdunder the candidate model; compare in USD, not tokens. - Depends on the generic canary report ([Phase 2] Canary go/no-go report: compare deep-tier cost, cache-read cost and latency by model from token-usage artifacts #1951 / PR feat: implement issue #1951 — [Phase 2] Canary go/no-go report: compare deep-tier cost, cache-read cost and latency by model from token-usage artifacts #1953).
- Keep decisions in pure functions (ADR-0004);
shellcheckall scripts; SHA-pin any action (AGENTS.md). - Reviewability: land it as an ordered sequence if the diff grows. First the pure decision function and the negative-control bats; then the chain-head PR and merge wiring; then ring advancement and pause handling. Each part must be reviewable on its own.
- Needs the merge- and tag-push-capable credential listed in the epic untracked prerequisites.
Project Structure Notes
Pipeline decision library plus wiring in the onboarding workflow; reuses scripts/cut-release.sh and the canary workflows.
References
- scripts/lib/engine-models.sh
- scripts/cut-release.sh
- release/registry.yml
- .github/workflows/pr-review-canary.yml
- scripts/lib/auto-merge.sh
- scripts/check-model-pins.sh
- docs/initiatives/sonnet-5-default-go-no-go.md
- docs/architecture/adr/0002-channel-tag-release-and-rings.md
- issue [Phase 2] Canary go/no-go report: compare deep-tier cost, cache-read cost and latency by model from token-usage artifacts #1951
- issue Script-only changes never reach the fleet: autocut watches the reusable blob, but agents execute scripts/ pinned at agent_ref #1592
Likely target surface
scripts/lib/.github/workflows/
Story prepared by the BMAD Scrum Master (Bob) for epic #2113. Status: ready-for-dev.
Metadata
Metadata
Assignees
Labels
initiativeEpic / initiative tracking issueEpic / initiative tracking issue
Activity