Repository navigation
Warn on gate-target mismatch when recording a formulation #214
Copy link
Copy link
Closed
Description
Important
Problem
The formulation validator only checks that target is a string, so the agent can lock a formulation whose target contradicts the problem it belongs to — a real CZ-gate workspace shipped with target X, and the formulation tab faithfully rendered an "x-gate" chip for a CZ run. Nothing at record time even hints at the mismatch.
Approach
Add a cheap gate-token mismatch heuristic at formulation record time, surfaced through the existing soft-warning channel in the formulate tool's reply — a warning, never a block, because problem names are free-form and workspaces get repurposed.
Approaches Considered
- Soft warning via mismatch heuristic (chosen) — catches the observed failure with zero false-positive cost; the agent self-corrects in the same turn.
- Hard validation failure — rejected: slugs are free-form; repurposed/renamed workspaces would be blocked on legitimate targets.
- Prompt-only fix in the score — rejected: prompts drift and give no durable guardrail; this failure already slipped past the current prompt.
Scope
- In: token extraction from problem name/slug; comparison against normalized target; warning appended to the formulate tool response alongside existing soft warnings; unit tests.
- Out: blocking validation — free-form names make false positives certain.
- Out: reconciling against the actual solve script — separate issue (linked below).
- Out: UI changes — the chip is correct once the data is correct.
Assumptions / Open Qs
- Assumes a small closed set of recognizable gate tokens (X, Y, Z, H, CZ, CX/CNOT, CCZ, iSWAP, …) is enough; names without a recognizable token never warn.
- Open: should the warning also fire on trajectory-type mismatch (name says "state-prep", formulation says gate)?
Acceptance Criteria
- Recording a formulation with target
Xin a problem whose name contains acztoken returns a warning naming both values. - Recording a matching target (e.g.
CZin acz-*problem) produces no warning. - Problem names with no recognizable gate token (e.g.
transmon-pulse-design) never warn, whatever the target. - The warning is soft: the entity is still recorded and the stage still completes.
- Unit tests cover token extraction, aliasing (CNOT/CX), and the no-token case.
Key Decisions
- Reuse the existing formulation soft-warning path (the same channel that carries spec §3.2 warnings) rather than a new mechanism.
- Normalize both sides before comparing: lowercase, strip
-gate/gatesuffixes, alias table for common synonyms. - Compare against the problem name and slug; the system entity is not consulted (it doesn't know the gate).
Source
- Found while diagnosing a wrong "x-gate" chip in the formulation tab: workspace
cz-gate-transmon-2(2026-07-24) hastarget: "X"recorded by the formulate tool per its event log; 17 sibling CZ workspaces recordCZcorrectly, so the miss was one agent flub with no guardrail. - Companion issue: Reconcile the formulation entity against what the solve actually ran #215 (formulation ↔ solve reconciliation) — covers everything after launch; this issue covers the capture window.
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
No labels
Activity