You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
[Acceptance] Re-run Haiku 5.5 through the finished pipeline and prove a known-bad candidate pauses it (negative control) #2132
As a maintainer accepting the automated pipeline,
I want Haiku 5.5 re-run end to end through Phases A–D, plus a known-bad candidate that must pause the pipeline,
so that we have evidence that the pipeline promotes a good model with zero human actions and stops a bad one before it reaches any ring.
Acceptance Criteria
The pipeline is force-onboarded (detection workflow_dispatch with model_id=claude-haiku-5-5, incumbent claude-haiku-4-5-20251001) and runs every stage. Its decision and stage results on the onboarding issue match the hand-run evidence (A/B exit 0, shadow bars met, cost and latency within tolerance), and the number of human actions on the happy path is recorded as 0.
Because Haiku 5.5 is already the chain head after the hand rollout, the promotion step is recorded as a no-op (no PR opened, no channel moved). Any other outcome is reported as a defect.
Negative control: a known-bad candidate (a configured stub or deliberately incompatible model id that fails preflight or the quality gate) is force-onboarded. The pipeline pauses with the expected reason, opens no chain-head PR and moves no channel.
A summary comment on the epic records both runs against the pipeline success metrics (0 human actions; 0 wrongly promoted).
Maintainer decision (discussion Enable Haiku 5.5 #2095): Haiku 5.5 is done by hand, then re-run through the pipeline as its acceptance test, together with a negative control.
The offline negative-control test lives in the auto-promotion story; this is the live run.
Spend counts toward the $25 triage cap per model.
Hands-off: it spends tokens and is the maintainer's acceptance decision.
Activity