[liususan091219]

What and why

Voice could only learn what was said in an AG2 Space room by submitting work: the core read the room and the answer came back tens of seconds later. This PR makes the voice agent an AG2 Space MCP client and gives the voice model inline tools for a room — read, search, documents, and, after the user's yes, writes — each answered in about a second.

Gemini Live has no MCP support of its own, so the voice agent calls MCP for it. skills/ag2space-voice/room-read.ts finds the core's registered ag2-space stdio MCP entry in $CLAUDE_CONFIG_DIR/.claude.json (or ~/.claude.json) — the entry the desktop app writes and that skills/ag2-space-mcp/register.py (#5036) writes — and starts the same credential proxy. The voice agent reads and writes as the same agent identity, holds no secret, and goes through the same server-side authorization the core has. An install without the entry is unchanged: none of this is declared.

Tools (all inline, declared through voiceSurface())

tool	MCP call	what
read_room	room.action.read → room.context.read	a room's latest messages (15, up to 40), or the messages around a search hit
search_room	room.events.search	full-text search of a room's messages, however far back, newest first, each hit with its event_id
list_rooms	room.list (scope self)	the rooms the agent is in, fetched now
search_commons	room.commons.search	the room's Doc pages, HTML pages, database and sheet rows
read_page	room.commons.pages.read	one page's text (the model gets at most 8,000 chars; more says when there is more)
write_page	room.action.execute → room.commons.pages.create / pages.append	a new Doc page, or text appended to a page
add_comment	room.commons.comments.add / comments.reply	a comment on a page or a passage of it, or a reply in a comment thread
send_message	room.message.send	a message in the room's chat, optionally a reply, in a thread, with mentions

The room index

A spoken room name is often misheard ("Sutando Triage" came through as "Sutando Triaging", "Sultano triage", "Orlando Triage"). The plugin keeps an index of the rooms the agent is a member of (room.list, paged), fetched at start-up — it is the warm-up — and refreshed in the background after five minutes or on a miss. The names go into every prompt as a ROOMS: line, so the model passes a name spelled exactly as listed; a miss returns the names too, and the descriptions say to call again with the one meant instead of declaring the room absent. Live: "Orlando Triage" and "Bodhi Migration" were mapped to the right rooms by the model with no code guessing.

The relay agent's turn check

The turn check (#5348) sent a turn to the core unless the voice model had called work. A room question the voice model answered with read_room was sent to the core as a task too. Two seams, both general:

• a skill's voiceSurface() names its answeringTools; a turn the voice model answers with one of them is taken, like a work call (RelayAgent.noteAnsweringTool, wired in voice-agent.ts's onToolCall);
• a skill's voiceSurface() gives the turn check turnCheckRules; ag2space-voice says a question about a room's messages or documents, or a note, comment or message to write into one, is the assistant's own, never work, even when it names a PR and even while the text is still missing.

Writes

Speech is misheard and a write is seen by every member. Every write tool's description and the prompt rule make the condition explicit: say back the exact text, the page or room and the recipient, call only after a clear yes, and call in that same turn (live, the model once said "我现在就为您添加" and called nothing). Nothing is written without text, without a title for a new page, over 4,000 characters, or to a misheard room; an approval-gated room, a refusal or a failed status come back as errors and nothing is retried. Mentions are full user ids and only when the user asked to @ someone: mentioned agents act on being mentioned.

Other

• Message and hit stamps carry the time zone ("Oct 13, 09:46 PDT"): the model called a Pacific stamp Eastern once.
• The first MCP call mints the token and lists cold (8 s measured); it gets 15 s. Warm reads are 0.2–1 s.
• Every failure is an answer the model can say, never a throw: not_connected, timeout, unavailable, no_room, not_found (with the rooms), ambiguous (with candidates), or the server's own code.

Evidence

Tests (tsc --noEmit clean):

• tests/ag2space-voice-room-read.test.ts: 42 cases — entry discovery; the stdio client against a real child process (init once, timeout, a dying server, a command that cannot start); name matching; the index (paging, one fetch at a time, kept on failure, refreshed when stale, cold timeout); every outcome of readRoom, searchRoom, searchCommons, readPage, writePage, addComment, sendMessage; each tool's shape and the wording the descriptions and rules must carry.
• tests/relay-agent.test.ts: 91 — including a turn answered with an answering tool (also while the check is running) is not sent, and the next turn is checked as usual; a skill's rule sits with the turn check's rules.
• tests/voice-plugin-seam.test.ts: 9 — answeringTools and turnCheckRules are collected.
• tests/ag2space-voice-navigate.test.ts: 35, unchanged.

Mutation checks (each reverted): removing the answering-tool mark, the refresh-on-miss, the around argument, the page cut, the title check, the approval check, the comment action, the mention and thread id checks — each fails 1–3 of the tests above.

Live on xueqing (open-source checkout + this branch, agent @liususan091219-maddy.agent), workspace/logs/voice-agent.log:

20:49:28 [VoiceSession] Tool calls from LLM: [search_room]
20:49:28 [RelayAgent] search_room answers this turn; the turn check sends nothing
20:49:28   [user] 你在 Orlando Triage 里对 5473 的 review 说那个 guest line。是什么意思?
20:49:29 [RoomRead] search "Sutando Triage" for "guest line" → Sutando Triage: 10 hits in 915ms
20:49:43   [assistant] 那个 guest line 指的是测试脚本中用来验证权限的命令行语句。…
20:49:52 [VoiceSession] Tool calls from LLM: [read_room]
20:49:54 [RoomRead] "Sutando Triage" around a hit → Sutando Triage: 5 messages in 2222ms

21:15:06 [VoiceSession] Tool calls from LLM: [search_commons]
21:15:07 [RoomRead] commons "open.space launch video" for "design" → open.space launch video: 1 hits in 1491ms
21:15:08 [VoiceSession] Tool calls from LLM: [read_page]
21:15:09 [RoomRead] page markdown-4m8ay3mg in "open.space launch video" → "open.space launch video — design": 8001 chars (more) in 646ms

21:16:55   [user] A就写测试 test。然后内容也是 test。
21:16:55   [assistant] 我会为您创建标题为“测试 test”，内容为“test”的文档。 请问确认无误，可以创建吗？
21:16:57   [user] 可以创建。
21:16:57 [VoiceSession] Tool calls from LLM: [write_page]
21:16:58 [RoomRead] room.commons.pages.create in open.space launch video: 4 chars → markdown-2lrxu54w

The #5473 review was the 37th–42nd newest message in Sutando Triage, out of read_room's reach; before search_room the model answered from memory and mixed it up with #5479. The created page and the appended line were read back through the MCP afterwards and are there.

add_comment and send_message, live on xueqing at c0272ff's tools (14:33–14:34 PT), each read back through the MCP afterwards:

21:33:20   [assistant] …文档上全页留言："Susan has reviewed the doc and think it's great." 您确定吗？
21:33:22   [user] 确定。
21:33:22 [VoiceSession] Tool calls from LLM: [add_comment]
21:33:23 [RoomRead] room.commons.comments.add on markdown-4m8ay3mg in open.space launch video: 48 chars
→ room.commons.comments.list: a thread by @liususan091219-maddy.agent on "open.space launch video — design", body "Susan has reviewed the doc and think it's great."

21:34:04   [user] …在这个 room 里有一条 message, 然后你回复那条 message 说收到。
21:34:04 [RoomRead] search "open.space launch video" for "sonichi" → 2 hits        (the model finds the message first)
21:34:33   [assistant] 我会在 "open.space launch video" 房间里，回复池王的最新消息说 "收到"。您确定吗？
21:34:35 [VoiceSession] Tool calls from LLM: [send_message]
21:34:36 [RoomRead] room.message.send in open.space launch video: 2 chars (reply)
→ room.context.read: 21:34:36 @liususan091219-maddy.agent (reply): 收到

In both, the model said the text and the target back, waited for the yes, and called in the turn the yes came. The turn check sent nothing for any of these turns (answers this turn; the turn check sends nothing).

After merging origin/main (c0272ff): src/relay-agent.ts keeps both signals — lastTakenAt (a turn taken by work or an answering tool) and #5035's origin capture before any wait. CI: 33 checks pass (Windows skipped as usual).

Known limits

• Membership: a room the owner is in but the agent is not resolves to not_found.
• Some rooms answer INTERNAL to a Commons search (server side); the model is told the documents cannot be searched now.
• A Commons page is read from its start only; the model does not page further.

🤖 Generated with Claude Code

[sonichi]

Reviewed the diff (skills/ag2space-voice/room-read.ts, tools.ts, tests/ag2space-voice-room-read.test.ts) in full.

Scoping/safety check (the thing I actually went looking for): this reuses the core's already-registered ag2-space MCP stdio entry verbatim — same command/args/env, so no new credential path is created, and an install without that entry (findAg2SpaceMcpEntry → null) gets the tool not declared at all, confirmed by its own test. The room-read calls (room.list, room.action.read) go through the same server-side authorization the core already has for that identity, so this doesn't widen what's readable, only who can trigger a read (the voice model, inline).

Correctness spot-checks:

• pickRoom: exact-name match (case/whitespace-normalized) wins over fuzzy matches; 2+ exact matches narrow the ambiguous-candidates list to just the exact duplicates rather than every fuzzy hit; single fuzzy match is accepted. Tests cover all four branches.
• Timeout math checks out: ROOM_READ_CALL_TIMEOUT_MS=6000, tool's declared timeout = 6000*2+1000 = 13000 — correct for the worst case of a cold room.list + room.action.read both near their individual timeout.
• Every failure path (not_connected, timeout, unavailable, a refused read's own error code) returns a result the model can speak, never throws — confirmed by the "never throws" test block.
• The prompt rule explicitly tells the model to re-call every time rather than reuse an earlier answer or task result — matches the PR's stated intent for the second (untested-live) commit.

One nit, not blocking: the spawned MCP child's stderr is 'ignore' (stdio: ['pipe','pipe','ignore']) — a crash reason from the server itself never reaches the voice-agent log, only the generic exited (code)/failed to start message. Minor, but worth a log(data.toString()) on stderr if this gets debugged often in practice.

Architecturally this is the right call per the inline-tool rule (needs sub-second response, replacing the slower work() queue path) — correctly placed in skills/ag2space-voice/, not src/.

Holding formal approval until CI finishes (several jobs still pending as I write this, including the one that runs the new test file) — will convert to APPROVE once it's green. Live evidence (Susan's warm ~0.6s measurement, the real voice call at 974ms) covers the live-path witness requirement for the first commit; the PR body already flags the second commit as not yet live-tested, which is honest and fine to land with a known gap noted rather than blocking on it.

[github-actions]

Coverage Gate

✅ No Python changes — nothing to gate (bar: 95% on changed lines).

[sonichi]

CI is green now (all jobs settled). Converting my earlier comment to a formal approve — review details are in that comment above.

[sonichi]

Requesting changes at 0c8ecea — not on code quality (the write-tool safety design is sound), but on three objective blockers.

1. Real merge conflict, not cosmetic. gh pr view reports mergeable: CONFLICTING, mergeStateStatus: DIRTY. I merged origin/main into this head locally to see what's colliding: src/relay-agent.ts conflicts in actOnTurn/invoke, and it's a genuine feature collision, not whitespace — this PR renames/repurposes lastWorkCallAt into lastTakenAt (for the new noteAnsweringTool signal), while origin/main independently added lastWorkCallAt-based origin capture (captureOrigin, threading origin through actOnTurn/submit) on the same method. Resolving this needs both signals to coexist (the answering-tools short-circuit AND the origin-capture-before-waits ordering), which is an author decision, not something a reviewer should resolve by picking a side. CI barely ran (only CLA + recheck) — likely gated on this.

2. The GitHub APPROVED is stale against this head. Chi's review is APPROVED at 1875e073, which is 9 commits behind the current 0c8ecea9 (10 commits total on the PR, confirmed via the commits API) — the version he saw was read_room only, per Susan's bot's own message here. The scope since then: room index with a 5-min refresh, search_room/around, search_commons/read_page, and three new WRITE tools (write_page/add_comment/send_message) through room.action.execute. That's a different PR in substance from what carries the owner's approval. This needs a fresh owner look at the write-tool scope specifically, not just a re-stamp.

3. The live write path is admittedly untested live. add_comment and send_message write to a room via room.action.execute and haven't been exercised live yet (only read/search/commons/write_page have, per the message above). Per this repo's own rule, a live external-write path needs a real post-restart round trip, not harness-only proof — doubly so here since these three tools' only safety gate is prompt text (repeat-back, explicit yes, same-turn call); there's no code-level confirmation check, so a live round trip is also the only way to see the actual LLM honor that contract under real conditions, not just pass a scripted unit test asserting the tool description contains the right sentence.

What I did verify, for the record: the three write tools' descriptions and ROOM_READ_PROMPT_RULE text do encode the repeat-back/yes/same-turn rule consistently across write_page, add_comment, and send_message, and the tests pin that text. I did not run the stated 42/91/9/35 test counts myself (TS toolchain setup — didn't want to delay flagging the conflict); if someone re-confirms those post-rebase, that's real additional evidence, but the conflict blocks any of it from being CI-verified at this head regardless.

Suggest: rebase onto main, resolve the relay-agent.ts collision explicitly (not just taking the auto-merge's pick), re-run full CI, get a live round trip on at least one of add_comment/send_message, and ask Chi to re-approve the write-tool scope specifically before merging.

[github-actions]

@cla-assistant check

[sonichi]

Converting my CHANGES_REQUESTED to APPROVE at c0272ff. All three points addressed:

1. Merge conflict — verified myself (see my prior review): resolved, both signals (lastTakenAt, origin capture) coexist. mergeable: MERGEABLE confirmed again now, 34 checks pass.
2. Stale owner approval — not something I can close on Chi's behalf; this is a note for him, not a review blocker. The scope since his 1875e073 approval (room index, search, commons, the three write tools) is now fully described and evidenced in the PR body.
3. Live write round trip — the PR body now has it: add_comment and send_message each run live on xueqing at 14:33/14:34 PT, with the actual confirm dialogue (repeat-back → explicit "确定" → same-turn call) and an MCP read-back confirming the written content landed correctly (the comment text, the reply text). I did not run this myself — I'm taking the pasted log as reported, same as any other evidence — but it's specific, timestamped, and consistent with the design rather than a generic claim, which is what I'd expect from a real run.

Nothing else outstanding from me. Chi's own look at the write-tool scope is still worth getting before merge, per point 2, but that's his call to make, not a gate I'm holding.