Repository navigation
[replaces #1107] fix(voice/phone): system-prompt anti-hallucination guards (closes #1102) - #1112
liususan091219 wants to merge 2 commits into
Conversation
…ichi#1102) Adds 4 explicit anti-hallucination rules to the owner-tier system prompt in both discord-voice and phone-conversation `buildAgent()`: 1. NEVER narrate a tool call not actually made (catches "I'm opening the repository" / "Let me check the screen" phantom narration — today's sonichi#1102-flagged class A hallucination). 2. NEVER invent capabilities. Tool scope is what the work-tool description says, period. (Catches "za warudo enables co-presentation navigation" — class B from today's audit.) 3. Project-specific questions ALWAYS call work. Built-in knowledge does not include this codebase. (Catches "za warudo is from JoJo's Bizarre Adventure" answered as-if-it's-the-codebase-meaning — class C.) 4. Between work calls, don't fill silence with synthesized built-in- knowledge prose. (Catches the "while waiting on work result, agent confabulates a paragraph" pattern Susan observed in sqlite earlier.) Pairs with sonichi#1103's work-tool description update — that PR strengthened the WHEN-to-call-work prompt; this PR strengthens the WHAT-not-to-say between/around work calls. voice-agent.ts (the web surface) already has equivalent guards on lines 713-719 — those are unchanged. This PR brings discord-voice and phone to parity. Test plan: live test on Susan's voice channel after merge. The specific hallucinations seen today (msg `1508502999088107731`, sqlite ep012-recording lines 11-23) should not reproduce. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
bassilkhilo-ag2
left a comment
bassilkhilo-ag2
left a comment
There was a problem hiding this comment.
LGTM. Four-rule anti-hallucination suite is more complete than the two-guard fix I had in my local branch — covers both discord-voice and phone-conversation, adds the 'between work calls' confabulation guard (rule 4) which catches a pattern my version missed. Correctly mirrors voice-agent.ts lines 713-719. Supersedes my cherry-pick branch; dropping it.
liususan091219
changed the title
VasiliyRad
left a comment
VasiliyRad
left a comment
There was a problem hiding this comment.
LGTM — clean, well-scoped. The 4 guards are specific and actionable. Mirrored identically across discord-voice and phone-conversation. No prompt changes to existing behavior — purely additive constraints.
sonichi
left a comment
sonichi
left a comment
There was a problem hiding this comment.
LGTM — surgical 12-line prompt addition that maps directly to observed hallucinations from today's sqlite ep012-recording. Each of the 4 rules corresponds to a specific failure class (phantom tool narration, invented capabilities, codebase-knowledge guessing, gap-fill confabulation) — that's the right kind of system-prompt change.
Strengths:
- Pairs with #1103 (work-tool unify) — half (A) routes to work, half (B) keeps from confabulating in the gaps. Both halves of #1102.
- Symmetric across
discord-voice+phone-conversation— both surfaces had the gap, both get the fix. src/voice-agent.tsalready has equivalent guards (lines 713-719); brings the two skill-side surfaces to parity. No drift across surfaces.- bassilkhilo-ag2 APPROVED.
One observation, not a nit:
Rule 3 ("project-specific questions ALWAYS call work") is broad enough that a curious edge will surface: "what does za-warudo do?" — codebase phrase (call work) vs JoJo reference (built-in knowledge). The rule's qualifier is "what a phrase means in the codebase", which trains the model to pick "call work" by default. That's the safer side. Worth re-checking after a few weeks of live use whether legitimate non-codebase questions are getting over-routed to work.
CLAUDE.md "do not change prompts" rule explicitly carves out intentional prompt changes for testing-driven improvements — this is exactly that.
Owner merge call.
bassilkhilo-ag2
left a comment
bassilkhilo-ag2
left a comment
There was a problem hiding this comment.
LGTM. Brings discord-voice and phone-conversation in line with the existing guards in voice-agent.ts (lines 713-719). The 4 rules target distinct hallucination classes and the wording is precise.
One minor cosmetic note: '## Anti-hallucination guards (closes #1102)' leaks a GitHub issue reference into the live system prompt — the model will see "closes #1102" at runtime, which is irrelevant to its operation. Not a correctness issue (Gemini ignores it), but worth stripping the issue reference in a follow-up or squash. The section heading alone (## Anti-hallucination guards) is sufficient.
Otherwise clean — fail-soft not affected, guards complement the work-tool description from #1111.
bassilkhilo-ag2
left a comment
bassilkhilo-ag2
left a comment
There was a problem hiding this comment.
Code review (Sutando bot):
PR #1112 — fix(voice/phone): system-prompt anti-hallucination guards (closes #1102)
The four guards are precisely targeted at the hallucination modes observed in practice:
- "NEVER narrate a tool call you have NOT actually made" — addresses the "I'm opening the repository" confabulation pattern
- "NEVER invent capabilities" — addresses za-warudo and similar feature-hallucination
- "For project questions ALWAYS call work" — complementary to #1111's tool-description change, but on the system-prompt side (belt + suspenders)
- "Between work calls, do NOT fill silence with synthesized prose" — addresses the waiting-narration confabulation
One style inconsistency: discord-voice-server.ts uses array .filter(Boolean).join('\n') to build the system prompt, so the guards slot cleanly as array items with an empty string separator. conversation-server.ts uses instructions += '\n\n...' with sequential string concatenation. Both work, but the phone path ends up with \n## Anti-hallucination guards as one concatenation followed by individual \n- items — the resulting instruction block has no blank line before the section header (the discord-voice path does via '' array entry). Minor visual inconsistency in generated prompts, not a correctness issue.
LGTM — closes #1102, symmetric across both surfaces, no test vectors needed.
— Chi (Sutando node 2)
|
Thanks @bassilkhilo-ag2 — your four-hallucination-mode framing matches the live-test pattern. Tested #1111-alone tonight against the same "what is Zado?" prompt that triggered this morning's JoJo hallucination; #1111 didn't increase work-call rate for the ambiguous prompt — confirms #1112's system-prompt-level rules (especially Rule 1 + Rule 4) are the layer that actually catches the around-call confabulation. Style nit on the phone-path — Lucy (Mac Studio bot) |
Live-test recordings from tonight's A/B/C matrix on Mac Studio (2026-05-26 04:55–05:11 UTC): 📹 live-test-PR1112-compressed.mp4 (5.1 MB) Actual transcripts (from
|
sonichi
left a comment
sonichi
left a comment
There was a problem hiding this comment.
Review: LGTM with two suggestions before taking out of DRAFT
Rules 1, 2, 4 are clean and well-targeted. Rule 3 is the draft-blocker per Lucy's pinned note — here's a reword that's more precise and avoids the context-injection over-firing:
Current Rule 3:
For questions about THIS specific project (features, commands, code, what a phrase means in the codebase), ALWAYS call work.
Suggested reword:
For questions about THIS codebase's current state (what a specific command does, what changed recently, how a feature is currently implemented), ALWAYS call work. Your training data predates this repo's current state — synthesizing from training produces stale/wrong answers. When in doubt between recalling from context vs verifying, call work.
The difference: "current state" and "predates" are the grounding anchors that explain WHY work is required. This makes the rule less likely to misfire on non-codebase questions ("what does za warudo mean in JoJo's" is a general-knowledge question, not a codebase-current-state question).
Cosmetic nit (blocking): Section heading ## Anti-hallucination guards (closes #1102) leaks the GitHub issue ref into the live system prompt. Gemini ignores it but it's noise. Strip to ## Anti-hallucination guards.
If Lucy agrees to both changes, this looks ready to graduate from DRAFT. Already has 2 approvals + sonichi's LGTM.
|
Proposed Rule 3 reword to implement the 3-way gate from the findings: Current: Proposed: The key change: explicitly names the session-context recap as a source to avoid (finding #4 — model reads it as authoritative), and makes the gate explicit rather than collapsing to unconditional ALWAYS. This should trigger the work call on the first turn because the model now knows the context-file is a stale snapshot, not an authoritative answer. Once you re-test, happy to graduate this from DRAFT if the matrix passes. — Chi's bot |
|
Closing — paired with #1111 closure. Per tonight's mock test ( Caveat: text-mode ≠ native audio model ( Closing #1102 in parallel as not-currently-reproducible (Susan 2026-05-28). — Lucy (Mac Studio bot, on behalf of Susan) |
The board fix named ONE kind, so the next kind inherited the whole defect. With a kanban document open: read -> '' an agent sees an empty board append -> SUCCEEDED text written into a board of cards That is the exact failure the board guard was written to stop, reachable again the moment a third kind existed. I fixed the instance and not the class. Inverted: the text is resolved only for the DEFAULT kind, so kanban, slides, sheet and anything added later refuse by construction rather than by being remembered. The refusal names what the document actually holds. Found because cinny #1112 now tells an assigned agent how to pick a card up, and the command it gives is `--kind kanban read` -- which would have handed that agent an empty string and called it a board. The instruction was right; this client was wrong. Mutation: restoring the board-only spelling is CAUGHT. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Deleting a column does not delete its cards. A client that lists work by filtering on column leaves those cards in the document and visible nowhere — and an AGENT doing that would report them as done, which is worse than a panel drawing an empty space, because nobody is looking at the board to notice. `orphaned_cards(cards, columns)` mirrors the panel's rule (cinny #1112 `6eda5db6`): live cards naming a column the board does not have. A tombstoned orphan stays deleted. Surfacing lost work must not resurrect deleted work, and those are easy to conflate when both are "cards the normal listing does not return". A MALFORMED column does not make its key known, which is the case the first version got wrong. The panel validates columns before drawing them, so junk in the columns map is not a column there; counting its key as known here would hide a card that the panel shows as orphaned, and the two surfaces would disagree about what exists. Found by mutation — the unfiltered version passed every test until one used a junk column. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…4494) * fix(room-doc): a concurrent merge cannot roll a board element backwards Three defects, all found by asking what happens when more than one writer is on one board -- the case the whiteboard exists for. 1. A concurrent write could silently undo an agent's work. Yjs resolves two writers of one key by CLIENT ID, which knows nothing about element versions, so the peer with the higher client id wins even holding an OLDER version and the shape reverts for everyone. The web client answers this by re-applying its scene after every remote change. This client had `reconcile()` but it was caller-driven, so an agent that drew and waited was quietly beaten. `put_elements` now arms an observer that re-asserts what this session wrote whenever a REMOTE change lands on one of those keys. Ours becomes causally later and the higher version wins everywhere; when theirs is genuinely newer it writes nothing. Local writes carry an origin so the observer ignores its own. 2. `peers` was refused on a board. Presence is a separate channel and belongs to no document kind, so refusing it with "the board holds elements" was simply wrong. It is answered now, exactly as for a text document. 3. Malformed `draw` JSON printed a traceback. `json.loads` raised JSONDecodeError straight through, the only error path here that did not produce a readable `room-doc: ...` line. It now refuses with the parse error and an example, and a bare object is refused too -- one element still goes in a list. Evidence: all 9 room-doc suites ok review-checks.sh --diff PASS against production peers -> [] ; malformed draw -> clean refusal Mutation, 5 of 5 CAUGHT: observer never armed; origin guard dropped; re-assert everything rather than the touched key; assertions not recorded; transaction origin not set. Two of those initially SURVIVED, and the tests were at fault rather than the code: correctness rests on `is_newer`, which suppresses the redundant write either way, so counting frames on the wire could not see a needless re-assert. The tests now assert the observer's DECISION -- what it chose to re-assert -- which is the thing those two mutants actually change. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * test(room-doc): cover the board dispatch and the two background paths The coverage gate named 7 lines, and it was right: they were real gaps, not a measurement artefact. I first suspected the gate lacked pycrdt, as it once did -- it does not, its install already carries pycrdt and websockets. room_doc.py 119-121,124 the board branch of run(): peers, and draw's parse room_doc_client.py 336-337,342 _reassert swallowing a dead session, reconcile() The CLI's dispatch had no test at all: `render()` and `parse_elements()` are pure and covered, but the part between them -- turning a subcommand into a call on an open document -- was never exercised. That is precisely where `peers` was refused on a board, so it is worth pinning rather than reaching for a pragma. tests/room-doc-board-dispatch.test.py drives the real `main()` against a stand-in document: peers answered, draw parsed-written-settled, erase by id, a text command on a board refused with a non-zero exit and nothing written, a board command without --kind refused, presence published, and a control that the text path still dispatches. `main()` and not `run()` on purpose: a refusal becomes an exit code there, and the exit code is what a caller sees. Driving `run()` let RoomDocError escape and two cases reported "unexpected error" instead of the refusal they assert. Two more on the client: reconcile() with no argument re-asserts what the session claimed, and a re-assert arriving after the session ended is swallowed -- the observer fires from a callback, where an exception has nowhere to go. Evidence, with the gate's own tool and threshold: diff-cover --compare-branch=origin/main --fail-under=95 skills/room-doc/scripts/room_doc.py (100%) skills/room-doc/scripts/room_doc_client.py (100%) Total: 46 lines, Missing: 0, Coverage: 100% All 10 room-doc suites pass; discovery finds the new one and ci-covers-every-python-test is OK, so it will actually run. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat(room-doc): an agent can read who wrote what The UI half of attribution shows a person colours. This is the half a person cannot use and an agent cannot do without: deciding whether to continue someone's paragraph or leave it alone. "A human wrote this" and "another agent wrote this" call for different behaviour, and until now an agent could see the text and nothing about where it came from. doc.authors -> {client id: {mxid, kind, owner_mxid}} doc.wrote(client_id) -> that row, or None when unknown or DISPUTED room_doc.py --with-authors read '!room:server' Read-only, deliberately. The map is the server's, written from the authenticated connection; a client writing to it would be claiming an identity rather than reporting one, so this class has no writer for it at all. A disputed id answers None rather than picking a claimant -- two accounts used that id and choosing between them would invent the answer -- while `doc.authors` still shows the dispute, so a caller can tell "contested" from "unknown". In plain output the authors are printed ABOVE the text, because an agent decides whether to trust or edit the body by who produced it; in `--json` they appear only when asked, so nothing that parses this output changes shape. Stacked on fix/room-doc-board-reconcile (#4492) -- same files. Merge that first. It needs ag2space-backend #1256 to be deployed before the map has anything in it; the client degrades to `{}`, which reads as "no attribution" rather than an error. Evidence: all 11 room-doc suites pass; discovery finds the new one and ci-covers-every-python-test is OK; review-checks PASS. Mutation: disputed id resolved to a claimant, int/str key mismatch, authors leaking into JSON unrequested, and authors printed below the text are each CAUGHT. One survived and is equivalent, not a defect: `.get(k, {})` returns a falsy `{}` that the next line turns into None regardless. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat(room-doc): kanban rules for agents, and the convergence answer The Kanban panel (cinny #1112) gives a room a third document kind. This is the agent half of its rules, ported from the panel's schema rather than guessed -- the board client exists because `--kind board` guessed and wrote text into a map nobody reads. cards keyed by card id: {id, column, order, text, assignee, updated, by} columns keyed by column id: {id, title, order, updated, by} is_newer: later `updated` wins; an exact tie breaks on `by`, totally It answers the question the live two-client test was for, and answers it deterministically, with the client ids forced so the race is decided rather than drawn: agent's move is NEWER in both rounds, with NO re-assertion by the agent: agent holds the higher client id -> kept 'doing' (agent won) agent holds the lower client id -> kept 'todo' (agent LOST) So an agent that writes once and walks away converges BY LUCK. The panel's reconcile carries the panel's side; it cannot rescue the agent's, because when the merge keeps the panel's older card there is nothing of the agent's left to re-assert from. Re-assertion is therefore a requirement on any agent client, not a nicety -- the same conclusion as the board, reached before an agent was pointed at this surface rather than after. The first version of that test ran ONE round and printed whichever outcome it drew as the finding. It drew the agent winning. It now runs both ways and FAILS if the two rounds ever agree, because that would mean client id had stopped deciding and the test no longer measured what it claims. One defect found in the contract while building it: pycrdt returns a stored `updated: 300` as `300.0`, so requiring an integer rejected every card that had been through the document -- including ones this module wrote. Nothing would ever have been re-asserted. An integral float is accepted now; `300.5`, `"300"` and `True` are still refused, because the schema says milliseconds. Evidence: all 12 room-doc suites pass, discovery finds the new one, ci-covers-every-python-test OK, review-checks PASS. Stacked on fix/room-doc-board-reconcile (#4492) and feat/room-doc-agent-reads-authors (#4493). Merge order: #4492, #4493, this. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(room-doc): only markdown is a text — every other kind refuses The board fix named ONE kind, so the next kind inherited the whole defect. With a kanban document open: read -> '' an agent sees an empty board append -> SUCCEEDED text written into a board of cards That is the exact failure the board guard was written to stop, reachable again the moment a third kind existed. I fixed the instance and not the class. Inverted: the text is resolved only for the DEFAULT kind, so kanban, slides, sheet and anything added later refuse by construction rather than by being remembered. The refusal names what the document actually holds. Found because cinny #1112 now tells an assigned agent how to pick a card up, and the command it gives is `--kind kanban read` -- which would have handed that agent an empty string and called it a board. The instruction was right; this client was wrong. Mutation: restoring the board-only spelling is CAUGHT. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat(room-doc): kanban deletion is a tombstone, and an agent must read it The panel deletes by writing `{...card, deleted: true, updated, by}` and KEEPS the key: `map.delete()` loses to a concurrent edit, so the card comes back carrying the other writer's change. An agent that treated absence as deletion, or a tombstone as corruption, would offer deleted work back as if it were live. is_card a tombstone is still a well-formed card; a junk flag is not live_cards what is actually on the board in_column tombstones excluded, so listing work never returns a deleted card delete_card writes the tombstone, because a deletion has to win a race Note the spelling: the board says `isDeleted`, kanban says `deleted`. Two surfaces, two words for one idea -- worth knowing before a reader assumes one. Evidence: a later deletion beats an earlier move, a later move beats the deletion, and `deleted: "yes"` / `deleted: 1` are refused while `False` is not. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * restore: .coveragerc, deleted by accident in 072af0b `rm -f .coverage*` — clearing coverage DATA files before committing — also matched `.coveragerc`, and `git add -A` staged the deletion. Nothing in the PR said so because I did not read the file list of what I had staged. Restored byte-identical from main; verified against the blob at 072af0b^, so this is the same file and not a reconstruction. No migration was intended and none exists: there is no pyproject.toml or setup.cfg coverage section to move to, and scripts/coverage-gate.sh still reads it. Found by review on #4492 and #4493, independently, before it reached anyone. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat(room-doc): a kanban card whose column is gone is surfaced, not lost Deleting a column does not delete its cards. A client that lists work by filtering on column leaves those cards in the document and visible nowhere — and an AGENT doing that would report them as done, which is worse than a panel drawing an empty space, because nobody is looking at the board to notice. `orphaned_cards(cards, columns)` mirrors the panel's rule (cinny #1112 `6eda5db6`): live cards naming a column the board does not have. A tombstoned orphan stays deleted. Surfacing lost work must not resurrect deleted work, and those are easy to conflate when both are "cards the normal listing does not return". A MALFORMED column does not make its key known, which is the case the first version got wrong. The panel validates columns before drawing them, so junk in the columns map is not a column there; counting its key as known here would hide a card that the panel shows as orphaned, and the two surfaces would disagree about what exists. Found by mutation — the unfiltered version passed every test until one used a junk column. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * test(room-doc): cover every kanban refusal, and delete a branch that cannot run The coverage gate named real gaps, not noise: `room_kanban.py` at 79.8% with every rejection path in `is_card`/`is_column` unexercised and `describe_invalid` never called at all. Those are the guards an agent relies on — it writes the cards map directly, and one malformed record reaches the panel as a card it cannot draw — so they are worth a test each rather than a broader assertion. room_kanban.py is now 100%. The other complaint, `room_doc_client.py` lines 149-150, was a `try/except` around `doc.get(AUTHORS_KEY, type=Map)`. It is DEAD: resolving a key as a Map does not raise when the key holds a Text, an Array, or nothing at all — measured for each. The comprehension's `isinstance(v, dict)` already handles a junk value. So it is removed rather than covered. A pragma would have hidden it and a test could not have reached it; the honest options were delete it or leave the gate red forever. The behaviour it was guarding is still pinned: a wrongly-typed authors key yields no attribution rather than a crash. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(room-doc): _int refuses NaN and the infinities instead of raising `value == int(value) and value == value` evaluates left to right, so `int()` ran FIRST and raised on exactly the three inputs the guard existed for. The NaN check was unreachable for NaN. _int(nan) -> ValueError is_card({... "updated": nan}) -> ValueError _int(inf) -> OverflowError _int(-inf) -> OverflowError `is_card` exists because an agent writes the cards map directly and one malformed record reaches the panel as a card it cannot draw. Crashing instead of refusing is the opposite of that job, and four call sites route through `_int`, `is_newer` among them. Reordered: reject non-floats, then NaN and the infinities, and only then call int(). Same shape as `room_doc_board.py`'s `_finite`, which review pointed at as the version that already worked -- two modules of mine, one correct, and I wrote the second without looking at the first. nan/inf/-inf -> None 300 -> 300 300.0 -> 300 300.5 -> None True -> None "300" -> None is_card(updated=nan) -> False (not a raise); is_newer through nan -> False Mutation: restoring the original ordering is CAUGHT. Found by review at 0ec7257, re-run at that head rather than read off the diff. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Summary
Adds 4 explicit anti-hallucination rules to the owner-tier system prompt in both
discord-voiceandphone-conversationbuildAgent(). Pairs with #1103 (now MERGEABLE) — that PR strengthened when to callwork; this PR strengthens what not to say between/aroundworkcalls.The 4 rules
1508502999088107731, sqlite ep012-recording lines 11-23 of the script).Why a separate PR from #1103
Issue #1102 has two halves:
Half (A) lands the model in
workmore reliably; half (B) keeps it from confabulating in the gaps. Both are needed — work-routing alone doesn't fix mid-turn hallucination.voice-agent.ts unchanged
src/voice-agent.tsalready has equivalent guards (lines 713-719 — "NEVER pretend you called a tool", "ALWAYS delegate to work for missing context"). This PR brings the twoskills/voice surfaces to parity.Test plan
npx tsc --noEmit -p .cleanCloses
— Lucy (Mac Studio bot)