Operator queue: acknowledging a platform-minted alert leaves it responded forever — the write-back re-injects it into the agent's queue file and the #1631 guard rejects it on every boot #2372

Summary

A platform-minted operator-queue alert — any row whose id carries one of _RESERVED_ID_PREFIXES (base-image-stale-, queue-flood-, poison-, sync-failing-, git-bloat-, skill-not-found-, db-backup-, …) — that an operator acknowledges in the Operations UI lands in status='responded' and can never leave it. The only responded → acknowledged transition in the codebase is the agent's file acknowledgement, and for these ids that path is structurally dead. The ~5s write-back loop then reconstructs the row into the agent's ~/.trinity/operator-queue.json whenever it is absent (so the entry cannot be removed), and the ingestion side rejects that very entry on every backend process start with

WARNING Rejecting operator-queue request '<id>' from '<agent>': ids with a reserved platform prefix are minted only by the platform (#1631)

— i.e. the platform blames the agent for an entry the platform wrote. The row stays responded (Clear All only hides terminal rows, and responded is deliberately kept) until the retention sweep deletes it at max(OPERATOR_QUEUE_RESPONDED_MIN_RETENTION_DAYS, operator_queue_retention_days) = 90 days by default, per alert.

Context

Observed on a local dev instance at dev b7bac09d with the #1816 stale-base-image alert for trinity-system (base-image-stale-trinity-system-<ts>): acknowledged on 2026-07-31, still responded and still re-injected into the system agent's queue file three weeks later; the WARNING fired twice (once per uvicorn worker) at every boot.

Mechanics (src/backend/services/operator_queue_service.py, src/backend/db/operator_queue.py):

1. Trigger — the Operations "Acknowledge" action is respondToItem(id, 'acknowledged', '') in src/frontend/src/stores/operatorQueue.js, i.e. POST /api/operator-queue/{id}/respond. respond_to_item is type/origin-agnostic and moves pending → responded. No API route, MCP tool or UI action reaches mark_acknowledged; only the sync loop calls it, from an agent file entry.
2. Write-back — _write_responses_to_agent takes get_responded_items_for_agent(agent) (every responded row — no type or origin filter) and appends a reconstructed entry for any row missing from the file ("Reconstruct items missing from the file (e.g. after container restart)"). Reproduced: delete the entry from the file → 6 s later Wrote 1 responses and 0 terminal-status flips back to trinity-system and it is back.
3. Ingestion — _sync_agent rejects any file entry whose id carries a reserved prefix and continues before the status == "acknowledged" branch, so even an agent that flips the entry to acknowledged cannot close it; the rejection is logged once per (agent, id) per process — i.e. on every boot.
4. Retention — _operator_queue_prune_predicate deletes responded rows only past max(responded_retention_days, retention_days), so the stuck state lasts 90 days by default.

Net effect: an operator action that looks terminal is not; the platform writes into an agent's queue file an entry that its own guard then refuses and logs as an impersonation attempt; and the only way out today is direct DB surgery (OperatorQueueOperations().mark_acknowledged(agent, request_id) plus hand-editing the agent file) — which is how the observed instance was cleaned up.

Acceptance Criteria

• Acknowledging (responding to) a platform-minted alert — a row whose request_id carries a _RESERVED_ID_PREFIXES prefix — lands it in a terminal status (acknowledged, acknowledged_at set) with no agent-side action required; such a row never sits in responded.
• The write-back never reconstructs a platform-minted row into an agent's ~/.trinity/operator-queue.json. Regression test: a responded reserved-prefix row + an empty queue file → the file stays empty across a sync cycle. The skip predicate reuses the same _RESERVED_ID_PREFIXES constant the ingestion guard uses (one definition, no second copy — the bug: retention prunes have no blast-radius guard — a mistyped window deletes most of a table within 5 minutes #1644/bug: platform operator-queue alert emitters bypass #1632 ingestion caps — skill-not-found is agent-triggerable (flood residual, gates pull default-ON) #1677 convention).
• Backend boot no longer logs the bug: operator_queue.id is a global primary key — cross-agent id collision silently swallows requests #1631 "reserved platform prefix" WARNING for entries the platform itself wrote back; the line is reserved for genuine agent impersonation.
• Existing stuck rows heal: responded rows whose request_id carries a reserved prefix are moved to acknowledged once (SQLite + Alembic migration per Rule Fix git pushing bug #9, or a one-shot step in the cleanup sweep). Add Clear All button on the Ops page #1017's terminal write-back (still-pending-only flips, never appended) is not widened.
• An acknowledged platform alert is clearable from the Resolved tab via Clear All (it is terminal now) and ages out under the normal operator_queue_retention_days window, not the responded floor.
• docs/memory/architecture.md (Operator Queue section) states the rule: platform-minted alerts have no agent audience — an operator acknowledgement is their terminal event, and they are never written back to an agent file.

Technical Notes

• Smallest fix shape: (a) at the sink, _write_responses_to_agent (or get_responded_items_for_agent) filters out rows whose request_id starts with a reserved prefix; (b) respond_to_item (or the router) writes status='acknowledged' + acknowledged_at when the row is platform-minted — or add an explicit admin POST /api/operator-queue/{id}/acknowledge that the UI's Acknowledge button calls for type == 'alert' items; (c) a one-shot heal for rows already stuck.
• The WS operator_queue_acknowledged event and the frontend store key on the row uuid (bug: operator_queue.id is a global primary key — cross-agent id collision silently swallows requests #1631), so a platform-side acknowledge should broadcast the uuid exactly as mark_acknowledged's caller does.
• The sink and the guard are both OSS (operator_queue_service.py), so the fix lands in the public tree; the private submodule's operator-queue consumers are unaffected.
• Related: bug: operator_queue.id is a global primary key — cross-agent id collision silently swallows requests #1631 (reserved-id guard), bug: operator-queue create path has no rate limit or size caps — flooding/operator-fatigue surface (blocking pull default-ON) #1632 (ingestion caps), Add Clear All button on the Ops page #1017 (Clear All / terminal write-back), Add operator_queue retention sweep to cleanup_service #1142 (retention sweep), bug: platform operator-queue alert emitters bypass #1632 ingestion caps — skill-not-found is agent-triggerable (flood residual, gates pull default-ON) #1677 (platform-only emitters), bug: trinity-system never adopts a rebuilt base image — ensure_deployed short-circuits on 'already running' #1816 (base-image-stale- producer).