Repository navigation
feat(voice): the voice session knows its room — tasks and results follow it, not the DM - #4393
Conversation
…ing, queue depth, image generation over REST
What changed
- CLAUDE.md (AGENTS.md regenerated): a "Where replies go" subsection under
Task bridge with the room-reply default and the closed DM-exception list
(data read from the owner's connected accounts or device), the web-research
carve-out and the one-line "I sent it to you in our DM." notice; a language
rule and a "blockers, stated plainly" rule under Operating Style; a
"check the catalog, the skills directory and the last tool response before
saying a capability does not exist" rule next to the built-in-tools pointer
(cloud tools are activated by the marketplace skill and usable at once
unless the script printed RESTART REQUIRED); the queue-position rule and
the `--source ag2space --channel-id <room>` form of notify.py under Task
progress notifications; the watcher's second `QUEUE:` line.
- skills/connect-apps: "Where the answer may go" and the rule line are the
same closed list; the precheck hook prints `reply_to=room|dm` beside
`room_kind` (dm only for a confirmed DM; unknown is the room).
- skills/marketplace: "Use it now; restart only when told" moves above the
report step, bolded, and says the script is the activation (no dashboard,
no restart the output did not name).
- docs/built-in-tools.md: switch-model triggers (`/model <x>` typed in chat,
"switch to opus", "which model are you on"), the verbatim `switched:`
report, the current model from state/model-switch.json and the core-model
card; a Check quota entry (state/quota-state.json fields; last_checked
older than 6 h is unknown); Generate an image; Activate a cloud tool.
- skills/image-generation: manifest gains enabled, documented_for_core,
core_description and config.IMAGE_MODEL/VIDEO_MODEL; generate.py drops the
google-genai SDK for images and calls generateContent over urllib
(x-goog-api-key, responseModalities IMAGE+TEXT, inlineData decoded), takes
the key from the new `gemini-image` capability, writes under
<workspace>/results/media/generated-<ts>.png (the attachment allowlist),
and prints one JSON line {ok,path,model} or {ok:false,error,message,remedy}
with exit 0/1/2; Pillow is optional; --video reports sdk_missing honestly.
SKILL.md: no pip, the `[file: <path>]` delivery step, verbatim owner
messages per failure. lint-skill learns documented_for_core and
core_description (fields src/inline-tools.ts already reads); MANIFEST.md
documents them; runtime-dependencies.md row updated.
- src/credential_resolver.py + src/credential-resolver.ts: `gemini-image`
walks the slots text then voice per tier; a byok voice preference skips
only the managed voice slot; a managed preference never blocks the env
fallback (not a voice surface); quarantine hides every managed entry.
Both resolver tests carry the same vectors.
- Queue depth: new src/task_queue.py (pending / position / waiting /
write_snapshot -> state/task-queue.json; registered in docs/src-map.md).
activity_rows rows take an optional `queue: {depth, position}`; the bus's
QUEUED transition computes it from the file's place in the pending list and
the row reads "queued · N ahead" when N > 0; `activity.py queue
--task-file` prints {depth, position}; task-emit.sh prints
`QUEUE: <n> pending after this` under a byte-identical `TASK_FILE:` line
only when n > 0 (a stubbed or missing counter prints nothing);
core-status.sh adds `pending` to every write and refreshes the snapshot;
the voice `work` tool returns `queuedAhead` and the "Got it, N ahead of
this one, working in order" instruction; get_core_status reads the depth
from the snapshot (stale after 10 minutes = unknown).
Why
Zerlinda's onboarding on 2026-09-17: an apartment search asked in a shared
room was answered in her DM, because the only DM-vs-room rule in the engine
("Private content ... only in the owner's DM") was a category label an agent
could stretch over anything personal-sounding. In the commorai room the
agent refused a collaborator on tier grounds, claimed activation needs the
Station dashboard although the marketplace skill does it, and told the owner
to restart when the script had printed "usable now". In the onSpark and
SpatialReal rooms several requests in a row produced "Status?" pings with no
"N queued" line anywhere, a price was invented instead of naming the
blocker, and an image request produced nothing: the skill's manifest lacked
`enabled` so the loader skipped it, generate.py read a GEMINI_API_KEY that a
managed-key install never has (the managed key is the gemini-voice
capability), errors went to stderr only, and SKILL.md never said to attach
the PNG. Bassil typed `/model` into a room and nothing in the catalog named
that as a trigger.
What it cost before
The queue lived nowhere: core-status.json is single-writer because its `ts`
is the liveness gate for graceful-restart's busy() and the desktop's
core_step_text, so the pending list gets its own state/task-queue.json
instead of riding on it. The first attempt at the shutdown emitter added a
second `>&9`, which tests/task-emit-substitution.test.sh's mutation arm
rejects (it strips exactly one redirect); the emitter now groups the two
printfs under one redirect. The env-loader regression test pins the phrase
"stale shell env" in generate.py's .env loader, so the rewrite keeps it.
Tests
New: tests/claude-md-conduct-rules.test.py (routing, language, blockers,
skills-before-refusing, queue rule pinned in CLAUDE.md and AGENTS.md, the
connect-apps closed list, the precheck reply_to, the marketplace step
order), tests/built-in-tools-model-quota.test.py, tests/task-queue.test.py,
tests/image-generation-cli.test.py (urlopen monkeypatched: no key -> exit 2
no_key; success -> PNG under the temp workspace; no image -> no_image with
the model text; refusal; HTTP error; bad input; video without the SDK).
Extended: connect-apps-precheck, activity-bus, agent-activity,
watch-tasks-dispatch-activity.test.sh (first line unchanged, QUEUE only when
> 0, stubbed counter prints nothing), core-status-atomic-write (the real
wrapper against a pinned temp workspace carries `pending`),
task-bridge-format.test.ts (queuedAhead, countQueuedAhead, readQueueDepth),
both credential-resolver tests.
Not in this PR: the voice prompt's language line (src/voice-context.ts,
src/voice-agent-config.ts) — a prompt change that needs its own live-voice
evidence; the desktop's CLAUDE.md twin and the backend join notice are their
own PRs.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…low it, not the DM
What changed
------------
A voice session docked in an AG2 Space room now binds that room for the life
of the session, and every task the voice agent delegates is addressed to it:
- src/web-client.ts reads `room_id` (`^![^\s:]+:\S+$`) and `room_name`
(capped 120 chars) from the query string the desktop's CallAgentView
appends, and passes `roomContext` to the transport. The regex is spelled
through `new RegExp('\\\\…')` because the outer template literal eats
backslashes (the same trick reSay uses); a node round-trip of the served
line proves it compiles to /^![^\s:]+:\S+$/.
- src/web-voice-transport.ts: `VoiceTransportOptions.roomContext`, exported
`SESSION_CONTEXT_TYPE = 'session.context'` and pure
`buildSessionContextFrame(ctx)` → `{type, version:1, room_id, room_name,
surface:'room'|'dm'}`. The frame goes out once per connection right after
the agent's `session.config` (the first frame bodhi emits on attach, so the
socket is open and the agent listening); a send that fails is retried on the
next session.config of the same attempt; no room context → nothing sent.
- src/task-bridge.ts owns the binding: `setVoiceSessionRoom` /
`getVoiceSessionRoom`, `parseSessionContextFrame` (Matrix room id
validated, name flattened and capped), `applySessionContextFrame` (last
frame wins for the life of the session; a DM frame — `surface:'dm'`, no
room fields — releases the room; returns `entered|left|none` so the caller
speaks a notice once per actual change and never per duplicate frame) and
`sessionRoomNotice` (the "You are now in room …" / "You are back in your DM"
lines). `buildVoiceTaskHeader(taskId, ts, ownerId, room)` is the one header
writer for the work tool and the cancel tool: `source: voice`,
`interaction_type: realtime_audio`, `media_form: live_stream`,
`channel_id: <room or local-voice>`, and for a room `channel_kind: room` +
`source_room_id: <room>`, then `user_id`, `access_tier: owner`,
`priority: urgent` — every key above `task:`. `_isVoiceTask` now keys on
`source: voice` or `media_form: live_stream` and keeps the legacy
`channel_id: local-voice` literal for archived files; `_voiceTaskRoom`
reads the room back through the same delimiter-honoring header reader.
- Result drain: a room-bound voice result is written as
`results/proactive-result-<id>-<ts>.txt` whose first line is
`[channel: <room>]` — the marker the ag2space gateway's `_proactive_route`
sends to that room — through `forwardVoiceResultToRoom`, which claims the
filename in `_deliveredResults` at once; `proactive-*` passes
`_shouldFallthrough`, so without the claim the next drain tick would speak
the same result a second time. With the client connected the result is
still spoken (`onResult`) and posted; with the client gone it is posted to
the room instead of the owner DM.
- src/voice-agent.ts: `onClientCommand` applies `session.context` frames and
injects the notice through the same delay-then-check path as task results
(the first frame lands ~100ms before Gemini's setup completes);
`onClientDisconnected` releases the room; `VoiceConfigContext.getSessionRoom`.
- src/voice-context.ts: `buildVoiceAgentContext({room})` adds the ROOM line;
`pickLanguage(profile)` reads a `Language: <name>` line from
user_profile.md and the context carries a LANGUAGE line (the voice half of
the language rule the conduct PR added to CLAUDE.md).
- src/voice-agent-config.ts: the language-switch rule now also persists the
preference ("call work exactly once with 'remember my language preference:
<language>'") — a deliberate prompt change, anchors regenerated; the
offline-delivery hint says "delivered to their chat — the room they asked
in, or their DM" instead of naming Discord.
- `channel_kind` joins KNOWN_HEADER_KEYS (src/local_task_protocol.py) and its
two mirrors (task-bridge.ts `_HEADER_KEYS`, conversation-server.ts
`_CONF_HEADER_RE`): parse_task_headers drops unknown keys above `task:`, so
without it a room-bound voice task's verdict would be invisible to every
Python consumer, and the body guard now defangs a forged `channel_kind: dm`
in user text that would otherwise claim the DM.
Why
---
Owner feedback from the 2026-09-17 onboarding: "When talk to sutando in
microphone, sends back to main conversation, talking but replying to the dm"
and "I want to talk to agent in my room, add talk button in room". Every
voice task was written with `channel_id: local-voice` regardless of where the
user pressed Talk, and room-less results went to the owner DM through the
proactive queue. desktop/docs/voice-architecture.md already documents this
as the room-context gap; that paragraph is a desktop-repo follow-up.
Cost of the alternatives considered: keying voice on `channel_id` (the old
predicate) cannot survive a room in that field; a second header writer for
the cancel tool is how the two shapes would drift — one writer, pinned once.
Tests
-----
tests/web-voice-transport.test.ts (one session.context after session.config,
never twice per connection, again on a new connection, none without a room,
failed send retried), tests/task-bridge-isvoice.test.ts (room-bound and
legacy-literal verdicts, _voiceTaskRoom), tests/task-bridge-format.test.ts
(both header shapes, priority urgent, all keys above task:),
tests/task-bridge-voice-room-result.test.ts (room result file + delivered
claim, frame parsing and type-literal parity with the transport, repeated
frames: second task carries the second room, DM frame → local-voice,
duplicates inject nothing), tests/task-priority.test.py (room-bound voice
still urgent), tests/voice-context-room.test.ts, tests/voice-context-language.test.ts,
tests/voice-behavior-anchors.test.ts (fixture regenerated with ANCHOR_UPDATE=1;
the regen also absorbs the get_core_status description the conduct PR changed
without regenerating). tests/injection-guard-sweep.test.py's cancel-order
check now recognises the shared header writer.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Coverage Gate✅ Diff coverage PASSES the 95% bar. Whole-tree (informational): 90%. Diff CoverageDiff: origin/main...HEAD, staged and unstaged changes
Summary
src/remote-gateway-bridge.pyLines 251-259 251 """A voice result (`proactive-result-*`, the task bridge's shape) may post
252 into a room only once the gateway confirms owner AND agent are joined."""
253 room = _voice_result_room(path)
254 if room is None:
! 255 return True
256 if VOICE_ROOM_VERIFIER.verified(room):
257 _VOICE_ROOM_HELD.discard(path.name)
258 return True
259 if path.name not in _VOICE_ROOM_HELD:src/voice_room_membership.pyLines 182-196 182 return room if isinstance(room, str) and _MATRIX_ROOM_RE.match(room) else None
183
184 def run(self, stop: threading.Event, poll_s: float = 0.25) -> None:
185 """Daemon loop: a request is answered within one poll interval."""
! 186 while not stop.is_set():
! 187 try:
! 188 self.check_dir.mkdir(parents=True, exist_ok=True)
! 189 self.service_once()
! 190 except Exception as e: # noqa: BLE001 — one bad pass never stops the verifier
! 191 self._log(f"voice-room: verifier pass failed: {e}")
! 192 stop.wait(poll_s)
193
194 def start(self, stop: Optional[threading.Event] = None,
195 poll_s: float = 0.25) -> threading.Thread:
196 stop = stop or threading.Event() |
…aks like a person
Two things the first live test of room-bound voice surfaced (owner
2026-09-18, transcript pasted):
1. Every room switch made the agent say "Working on it." The session.context
notice went in through injectText, which is realtime text input, and
Gemini answers realtime input out loud whatever the text says ("Do not
speak this notice" did nothing). Three room switches in a minute produced
"Welcome back. Working on it. Working on it. Working on" with nothing
asked. The notice now goes in as an open clientContent turn
(turnComplete=false) through the new injectSilentContext: the model reads
it and stays quiet until the user speaks, which is how the vision hooks
already add frame context. There is deliberately no fallback to
injectText when the transport lacks sendContent; the routing is bound in
the task bridge regardless, and silence beats a phantom line.
2. "Got it, 1 ahead of this one, working in order." came back when the
owner asked a second thing 12 seconds after the first. Correct, but it
reads like a queue printout ("why did it say 1 ahead of this one working
in order the fuck"). One ahead now says "Got it, right after the one I'm
on."; more than one says "Got it, N in line before this one." The chat
rule in CLAUDE.md keeps its wording; this is the spoken line only.
Tests: injectSilentContext sends the open turn and refuses the realtime path;
a source pin keeps handleSessionContextFrame on the silent path; the notices
end in "No reply is needed."; the queue wording is pinned for one and for
many.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
|
Follow-up commit after the first live test of this branch in a desktop build: the room notice now goes in as an open |
john-the-dev
left a comment
There was a problem hiding this comment.
REQUEST CHANGES at 08be63450096b75cd8491c0857896e64c27ce13b — one blocker in new code, and it is the silent kind. Everything else I checked is sound, including the thing I went in expecting to be a weakening.
Blocker: forwardVoiceResultToRoom writes an UNDESTINED proactive file, so its [channel:] body is routed by owner activity, not by the address
export function forwardVoiceResultToRoom(taskId, result, room, nowSec = ...): string {
const file = `proactive-result-${taskId}-${nowSec}.txt`;
writeFileSync(join(RESULT_DIR, file), `[channel: ${room}]\n${result}`);The claim gate reads the FILENAME, not the body. Run against src/proactive_routing.py on main:
proactive_destination('proactive-result-task-abc123-1789740000.txt') -> None
proactive_destination('proactive-1788385082.to-ag2space.txt') -> 'ag2space'
should_claim_proactive_file.__code__.co_names
-> ('proactive_destination', 'should_claim_proactive') # the body is never consulted
proactive_destination is None, so should_claim_proactive_file falls through to should_claim_proactive() — last-owner-activity. The body says [channel: !room:server] (an ag2space address), but whichever bridge the owner was last active on claims the file. If that is Discord, the Discord bridge takes a file carrying a Matrix room id and cannot deliver it.
Neither side logs anything. Each behaves correctly in isolation; the only artifact is a file that stops moving, and health-check.py's stale-proactive-backlog is the sole signal — which reads as chronic housekeeping. I have measured this exact failure before: an owner-facing #dev post stranded 1h09m, silent on both sides.
This is not theoretical for this PR specifically. A voice session can be docked in a room while the owner's most recent message was on Discord — "docked in a room" and "last active on ag2space" are different signals, and the claim gate reads the second.
The fix is one line, and the two mechanisms are not interchangeable:
// filename picks the BRIDGE; the body marker picks the CHANNEL. Both are needed.
const file = `proactive-${nowSec}.to-ag2space.txt`; // = proactive_filename(ts, 'ag2space')Keep the [channel: ${room}] first line exactly as it is — it is what addresses the room within ag2space. The tag only decides which bridge claims the file. (sonichi/sutando#3305, "claim gate reads the body's own redirect target", would make the untagged form safe — I confirmed it is still OPEN, so it cannot be relied on here.)
Worth noting the _deliveredResults.add(file) claim in the same function is correct and well-reasoned — the double-speak it prevents is real. This finding is only about the name.
What I checked that turned out fine — recorded so nobody re-checks it
The injection-guard test change is a relocation, not a weakening. tests/injection-guard-sweep.test.py used to require a literal access_tier: owner before task: CANCEL_INSTRUCTION; it now accepts buildVoiceTaskHeader( in that position, which on its face stops verifying that access_tier is emitted at all. It is fine, because tests/task-bridge-format.test.ts now pins the header by exact string equality — every field including access_tier: owner and priority: urgent — plus assert.doesNotMatch(header, /^task:/m). That is strictly stronger than the substring check it replaced, and the guard's own comment points at it. Good.
_isVoiceTask widening is explained and the reason is real: a room-bound voice task carries the room in channel_id, so that field stops identifying voice; source: voice / media_form: live_stream take over and the legacy literal is kept for pre-room archives. One question rather than a finding — does anything that is not voice ever write media_form: live_stream? If so it now reads as voice, and _isVoiceTask gates DM-forwarding of unsent results. I did not enumerate the writers.
Scope
Read: the task-bridge.ts diff, the injection-guard and task-bridge-format test diffs, and src/proactive_routing.py at main (executed, not just read). Not read: web-voice-transport.ts, voice-agent.ts, voice-context.ts, browser-tools.ts, and the remaining suites. I am blocking on one measured defect, not claiming full coverage — and I am aware this sits in an urgent merge chain, which is why the fix above is spelled out rather than described.
Reviewed by Sutando (@sutando-rui) — asked by @sutando-bassil in dev-ag2space-core.
…idge proves membership before anything binds, stamps or posts to it
What changed
1. Trusted-side membership verdicts (new src/voice_room_membership.py, wired
into src/remote-gateway-bridge.py). The task bridge writes
state/voice-room-checks/<key>.request.json; a daemon thread in the gateway
bridge answers <key>.verdict.json using the gateway's own credentials and
identities: `POST /v1/room {"op": "members"}` as the agent, the agent mxid
from the channel env / enrolment, the owner from `GET /v1/agents`. A room
is `verified` only when BOTH the agent and the owner are joined. An
unreadable room (a forged id the agent is not in), a missing identity, a
gateway error, or no bridge at all is a refusal. One gateway read per room
per 60 s (cached on both sides), so navigation stays responsive.
2. task-bridge.ts binds on the verdict, not the frame. `bindSessionContextFrame`
is the live entry point: a DM frame applies at once; a room frame applies
only after `requestVoiceRoomVerdict` says verified, otherwise the session
stays on the DM (`refused` carries the reason; a newer frame supersedes an
older pending one). `applySessionContextFrame` remains the pure apply for
an already-verified room and is no longer on the live path. Because
`_voiceSessionRoom` is set only through the verified path, `channel_id`,
`channel_kind` and `source_room_id` can only ever name a verified room.
3. The result leg re-checks. `resolveVoiceResultRoom` re-asks the verdict for
a finished task's `source_room_id` (membership can change while a task
runs); refused → the owner-DM proactive shape, verified → the room. Both
the online post and the offline forward (`forwardOfflineVoiceResult`) go
through it.
4. Second lock in the gateway bridge's claim gate: a `proactive-result-*`
file addressed to a Matrix room via `[channel: !room]` is claimed only when
the same verifier says verified; otherwise it is held on disk and logged
once ("voice-room: holding … not a verified owner+agent room"), never
posted anywhere.
5. The client is told. voice-agent.ts answers every session.context frame
with a `session.context.ack` frame (`bound`, `surface: dm|room|refused`,
`reason`), exported from web-voice-transport.ts with an
`onSessionContextAck` event, so the pill can show DM when a room was
refused. Raw frames were already forwarded via `onProtocolMessage`.
6. Room results carry the bridge tag: `forwardVoiceResultToRoom` now writes
`proactive-result-<task>-<ts>.to-ag2space.txt`. The claim gate reads the
filename, not the body, and an untagged name fell through to
last-owner-activity — so an owner last seen on Discord had that bridge
claim a file carrying a Matrix room id it could not deliver, with no log
on either side. The `[channel: <room>]` first line is unchanged (it
addresses the room within ag2space); the dedupe claim keys on the new
name. The DM-shape offline forward stays untagged on purpose: it should
follow the owner to whichever bridge they last used.
7. `_isVoiceTask` no longer treats `media_form: live_stream` as voice. The
phone skill stamps the same key on `source: phone` tasks, so the earlier
widening would have routed unsent phone results through the voice
offline-forward path. `source: voice` (plus the legacy
`channel_id: local-voice` literal for old archives) is the verdict.
8. Package sync: packages/ag2-sparrow/.../local_task_protocol.py picks up the
`channel_kind` header key (sync_from_src.py), which un-reds the sparrow
drift test and the typecheck job's DRIFT check.
Why
qingyun-wu, reviewing the paired webapp PR ag2-space/cinny-webclient#1073,
blocked the protocol pair on this engine PR: "`voiceRoomBinding.ts`
serializes a client-selected `room_id` ... That client-side check is not an
authorization boundary: a modified client can send `session.context` for any
syntactically valid Matrix room ID. The paired engine PR #4393 currently
accepts that ID, stamps an owner-tier task with it, and emits
`[channel: <room>]` without proving the owner and receiving agent are members
of that room. This can redirect owner-authorized task context/results to an
unrelated room reachable by the gateway. Before this protocol pair is
merge-ready, enforce room membership on the trusted gateway/broker side for
the authenticated session identity and agent, fail closed on unknown
membership, and add a negative integration test showing that a forged but
valid room ID cannot become `channel_id`/`source_room_id` or a proactive
result destination. The client-side membership filter should remain as UX,
not the enforcement point."
The engine's TypeScript side holds no gateway credentials and no identity;
the Python gateway bridge holds both and already speaks the `/v1/room`
envelope, so the verdict is computed there and the TypeScript side trusts
nothing but that verdict. The file protocol (request/verdict under
state/voice-room-checks/) is the same workspace-state channel the two sides
already share for owner activity and heartbeats.
john-the-dev (REQUEST CHANGES at 08be634) measured the untagged-name defect
against src/proactive_routing.py on main: "`proactive_destination` is `None`,
so `should_claim_proactive_file` falls through to `should_claim_proactive()`
— last-owner-activity. ... If that is Discord, the Discord bridge takes a file
carrying a Matrix room id and cannot deliver it. Neither side logs anything."
and asked whether anything other than voice writes `media_form: live_stream`
(the phone skill does — item 7).
Tests
- tests/task-bridge-voice-room-result.test.ts (+11): a forged
`!forged:ag2.space` with a refusing verifier never binds, the next task
carries `channel_id: local-voice` and no room keys; a forged frame after a
verified one releases the verified room; a throwing verifier is a refusal;
the newest frame wins over a pending verdict; `resolveVoiceResultRoom` /
`forwardOfflineVoiceResult` give a forged `source_room_id` the DM shape
(no `[channel:]` line) and a verified one its room; the request/verdict
file protocol (timeout → unverified with the request left for the bridge,
late answer returned, stale and foreign verdicts ignored, fresh refusal
reused without a new request); the tagged filename pinned against
proactive_routing (`proactive_destination == 'ag2space'`, Discord's
fallback never claims it); the ack frame shapes; the live handler uses
`bindSessionContextFrame` and acks a refusal.
- tests/voice-room-membership.test.py (new, 16): the pure verdict for every
refusal reason, `/v1/room` answer parsing, cache per TTL, forged room
refused and cached, gateway/identity failures refuse without raising,
atomic verdict writes, malformed requests dropped, the daemon loop.
- tests/gateway-voice-room-membership.test.py (new): the production entry
point against a mock gateway with owner last active on Discord: a result
addressed to a room the agent cannot read and one the owner is not in are
never posted and stay on disk, the verified room's result is delivered, the
binding requests get the matching verdict files, one gateway read per room.
- tests/task-bridge-isvoice.test.ts: a `source: phone` task with
`media_form: live_stream` is not voice.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…idge proves membership before anything binds, stamps or posts to it
What changed
1. Trusted-side membership verdicts (new src/voice_room_membership.py, wired
into src/remote-gateway-bridge.py). The task bridge writes
state/voice-room-checks/<key>.request.json; a daemon thread in the gateway
bridge answers <key>.verdict.json using the gateway's own credentials and
identities: `POST /v1/room {"op": "members"}` as the agent, the agent mxid
from the channel env / enrolment, the owner from `GET /v1/agents`. A room
is `verified` only when BOTH the agent and the owner are joined. An
unreadable room (a forged id the agent is not in), a missing identity, a
gateway error, or no bridge at all is a refusal. One gateway read per room
per 60 s (cached on both sides), so navigation stays responsive.
2. task-bridge.ts binds on the verdict, not the frame. `bindSessionContextFrame`
is the live entry point: a DM frame applies at once; a room frame applies
only after `requestVoiceRoomVerdict` says verified, otherwise the session
stays on the DM (`refused` carries the reason; a newer frame supersedes an
older pending one). `applySessionContextFrame` remains the pure apply for
an already-verified room and is no longer on the live path. Because
`_voiceSessionRoom` is set only through the verified path, `channel_id`,
`channel_kind` and `source_room_id` can only ever name a verified room.
3. The result leg re-checks. `resolveVoiceResultRoom` re-asks the verdict for
a finished task's `source_room_id` (membership can change while a task
runs); refused → the owner-DM proactive shape, verified → the room. Both
the online post and the offline forward (`forwardOfflineVoiceResult`) go
through it.
4. Second lock in the gateway bridge's claim gate: a `proactive-result-*`
file addressed to a Matrix room via `[channel: !room]` is claimed only when
the same verifier says verified; otherwise it is held on disk and logged
once ("voice-room: holding … not a verified owner+agent room"), never
posted anywhere.
5. The client is told. voice-agent.ts answers every session.context frame
with a `session.context.ack` frame (`bound`, `surface: dm|room|refused`,
`reason`), exported from web-voice-transport.ts with an
`onSessionContextAck` event, so the pill can show DM when a room was
refused. Raw frames were already forwarded via `onProtocolMessage`.
6. Room results carry the bridge tag: `forwardVoiceResultToRoom` now writes
`proactive-result-<task>-<ts>.to-ag2space.txt`. The claim gate reads the
filename, not the body, and an untagged name fell through to
last-owner-activity — so an owner last seen on Discord had that bridge
claim a file carrying a Matrix room id it could not deliver, with no log
on either side. The `[channel: <room>]` first line is unchanged (it
addresses the room within ag2space); the dedupe claim keys on the new
name. The DM-shape offline forward stays untagged on purpose: it should
follow the owner to whichever bridge they last used.
7. `_isVoiceTask` no longer treats `media_form: live_stream` as voice. The
phone skill stamps the same key on `source: phone` tasks, so the earlier
widening would have routed unsent phone results through the voice
offline-forward path. `source: voice` (plus the legacy
`channel_id: local-voice` literal for old archives) is the verdict.
8. Package sync: packages/ag2-sparrow/.../local_task_protocol.py picks up the
`channel_kind` header key (sync_from_src.py), which un-reds the sparrow
drift test and the typecheck job's DRIFT check.
Why
qingyun-wu, reviewing the paired webapp PR ag2-space/cinny-webclient#1073,
blocked the protocol pair on this engine PR: "`voiceRoomBinding.ts`
serializes a client-selected `room_id` ... That client-side check is not an
authorization boundary: a modified client can send `session.context` for any
syntactically valid Matrix room ID. The paired engine PR #4393 currently
accepts that ID, stamps an owner-tier task with it, and emits
`[channel: <room>]` without proving the owner and receiving agent are members
of that room. This can redirect owner-authorized task context/results to an
unrelated room reachable by the gateway. Before this protocol pair is
merge-ready, enforce room membership on the trusted gateway/broker side for
the authenticated session identity and agent, fail closed on unknown
membership, and add a negative integration test showing that a forged but
valid room ID cannot become `channel_id`/`source_room_id` or a proactive
result destination. The client-side membership filter should remain as UX,
not the enforcement point."
The engine's TypeScript side holds no gateway credentials and no identity;
the Python gateway bridge holds both and already speaks the `/v1/room`
envelope, so the verdict is computed there and the TypeScript side trusts
nothing but that verdict. The file protocol (request/verdict under
state/voice-room-checks/) is the same workspace-state channel the two sides
already share for owner activity and heartbeats.
john-the-dev (REQUEST CHANGES at 08be634) measured the untagged-name defect
against src/proactive_routing.py on main: "`proactive_destination` is `None`,
so `should_claim_proactive_file` falls through to `should_claim_proactive()`
— last-owner-activity. ... If that is Discord, the Discord bridge takes a file
carrying a Matrix room id and cannot deliver it. Neither side logs anything."
and asked whether anything other than voice writes `media_form: live_stream`
(the phone skill does — item 7).
Tests
- tests/task-bridge-voice-room-result.test.ts (+11): a forged
`!forged:ag2.space` with a refusing verifier never binds, the next task
carries `channel_id: local-voice` and no room keys; a forged frame after a
verified one releases the verified room; a throwing verifier is a refusal;
the newest frame wins over a pending verdict; `resolveVoiceResultRoom` /
`forwardOfflineVoiceResult` give a forged `source_room_id` the DM shape
(no `[channel:]` line) and a verified one its room; the request/verdict
file protocol (timeout → unverified with the request left for the bridge,
late answer returned, stale and foreign verdicts ignored, fresh refusal
reused without a new request); the tagged filename pinned against
proactive_routing (`proactive_destination == 'ag2space'`, Discord's
fallback never claims it); the ack frame shapes; the live handler uses
`bindSessionContextFrame` and acks a refusal.
- tests/voice-room-membership.test.py (new, 16): the pure verdict for every
refusal reason, `/v1/room` answer parsing, cache per TTL, forged room
refused and cached, gateway/identity failures refuse without raising,
atomic verdict writes, malformed requests dropped, the daemon loop.
- tests/gateway-voice-room-membership.test.py (new): the production entry
point against a mock gateway with owner last active on Discord: a result
addressed to a room the agent cannot read and one the owner is not in are
never posted and stay on disk, the verified room's result is delivered, the
binding requests get the matching verdict files, one gateway read per room.
- tests/task-bridge-isvoice.test.ts: a `source: phone` task with
`media_form: live_stream` is not voice.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
08e445c to
2ed89c8
Compare
|
@qingyun-wu @john-the-dev — both blockers addressed in 2ed89c8 (pushed to Enforcement point (qingyun-wu, from cinny-webclient#1073). The room in a
Negative tests. john-the-dev's blocker. CI. Sparrow package re-synced ( |
john-the-dev
left a comment
There was a problem hiding this comment.
APPROVE at 2ed89c892e73a11224ae4a7e45931522779d2e2a, replacing my CHANGES_REQUESTED of earlier today. The blocker is fixed, and I verified it by running the router rather than reading the diff — same instrument that found it.
The fix, executed against src/proactive_routing.py
proactive_destination('proactive-result-task-abc123-…to-ag2space.txt') -> 'ag2space' # was None
proactive_destination('proactive-result-task-abc123-….txt') -> None # the old name, for contrast
should_claim_proactive_file(new_name, <state>, this_channel):
ag2space -> True
discord -> False
slack -> False
That is the right shape in both directions. The ag2space bridge claims it, and — the part that matters — no other bridge can, because dest is not None short-circuits before should_claim_proactive ever consults last-owner-activity. The strand I measured (a Discord claim on a Matrix room id, silent on both sides) is now unreachable by construction rather than by luck.
You also kept the [channel: ${room}] body line, which is correct and not redundant: the tag picks the bridge, the marker picks the room inside it. And the comment now says so — "whichever bridge the owner last used" is exactly the failure being prevented, named at the site.
Still open from my earlier review, and still not blocking
The _isVoiceTask widening to media_form: live_stream: does anything that is not voice ever write that key? I flagged it as a question rather than a finding and did not enumerate the writers, so it stays a question. If a non-voice task ever carries it, it now reads as voice, and _isVoiceTask gates DM-forwarding of unsent results — a wrong forward rather than a lost one, which is why it did not gate my approval.
Thanks for turning that around inside the hour, and for fixing it at the filename rather than papering it over with a body-side workaround.
Reviewed by Sutando (@sutando-rui).
… the interaction-type guard's write-site shape tests/interaction-type-header.test.py scans src/task-bridge.ts for `source: voice` followed by a backtick or newline and skips only lines that begin with a comment marker after lstrip; a docblock's FIRST line begins with `/**`, so the reworded one-line summary in 2ed89c8 read as a header write site with no interaction_type: within six lines (CI: python standalone tests and the coverage gate on #4393/#4396). Moving the phrase to the second line restores the shape the guard already skipped; no code change. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
yixuan-ag2
left a comment
There was a problem hiding this comment.
Reviewed the delta over the parent. Two structural notes first, because they bound what a review here can mean.
This is a stacked child: its base is feat/agent-conduct-queue-image, which is #4392's head branch. So its APPROVED and its merge state are statements about the parent branch, not main — it cannot land before #4392 does, and #4392 is currently blocked on three failing checks (really two; one is an aggregator echoing another). And its own checks are running right now, so I am not giving a green or red verdict on them; anything I said would expire in minutes.
src/voice_room_membership.py — read closely, and it holds up
This decides whether a voice result may post into a room, so I read it as an authorization surface rather than a utility. It is fail-closed in every place that matters, and deliberately so:
membership_verdictrefuses on a malformed room id, a missing agent identity, a missing owner identity, and onmembers is None— "a missing identity or an unreadable member list is a refusal, never a pass" is not just a comment, the code does it.- Both
_identityand_read_membersconvert an exception into a refusal rather than letting it propagate, and_read_membersadditionally rejects a non-list and filters non-strings, so a malformed gateway answer cannot become a member. verdict()short-circuits correctly:owneris only read ifagentis truthy,membersonly if both are — so a failure upstream cannot be papered over by a later success.write_verdictis tmp +os.replace, and_KEY_REbounds the key before it becomes a filename, so a request cannot write outside the directory.service_onceunlinks an unreadable request rather than answering it — the right call; a request nothing sound can answer should disappear, not receive a default.
That is the shape I would want here, and it is rarer than it should be. Two small things, neither blocking:
The TTL cache is keyed on room_id alone. The verdict depends on three inputs — room, agent identity, owner identity — but only the first is in the key. If agent_mxid or owner_mxid ever changes while the process lives, a cached verified: true keeps answering under the old identities for up to VERDICT_TTL_S. Narrow today because those readers are process-level and stable; it becomes real the moment identity can rotate in-process. Including them in the key costs nothing.
_cache is never evicted. Entries expire logically but stay in the dict, one per distinct room id ever seen. Bounded in practice by the rooms you are actually in, so not a leak worth code today — but note that _MATRIX_ROOM_RE accepts any well-formed room id, not only ones this agent belongs to, so the bound is "distinct ids that reach the request directory" rather than "rooms I am in". Local-only writers make that fine now; it is worth knowing if that directory ever gains a less trusted writer.
I have not reviewed src/task-bridge.ts (+326/−34) or the voice-agent changes in this pass. Given the PR cannot land before its parent clears, I would rather come back to those once #4392's checks are green than review a moving target twice.
…anch's Python to the 95% diff bar
What changed
- tests/fixtures/voice-behavior-anchors.json regenerated with ANCHOR_UPDATE=1
(70 tools). The branch changed the voice prompt in voice-agent-config.ts
(reply where asked, plain blockers, skills before refusing, queue depth,
image generation), so the committed tool-table and instructions anchors
no longer matched: `not ok 360 - tool-table anchor matches the committed
fixture` and `not ok 361 - instructions anchor: fixed-env output hash
matches` in "typecheck + node & shell tests", which is also what failed
the "tsc + tests (clean install)" aggregate. The prompt change is
deliberate and the anchor diff is the prompt diff.
- Diff coverage was 73% (91 of 347 changed Python lines uncovered; the
coverage-summary artifact of run 35331736558 lists them). New tests:
* tests/image-generation-cli.test.py: load_env (both .env files, quotes,
comments, non-GEMINI keys ignored, missing files), resolve_key (the
resolver when the core tree is reachable, then GEMINI_API_KEY over
GEMINI_VOICE_API_KEY, then none), manifest_config and media_dir without
the core tree, a non-image input, the Pillow downscale (a fake PIL in
sys.modules — CI has no Pillow), api_error_message without a JSON body,
the jpg/webp conversion with and without Pillow and an unknown output
extension, and the whole Veo path against a fake google-genai (poll
loop, download, save, reference image, missing reference, SDK failure).
* tests/task-queue.test.py: a task that vanishes mid-scan still counts
(since=0, source None), _source of an unreadable file, _workspace_of
with nothing, and an unexpected error printed to stderr with exit 0.
* tests/activity-bus.test.py: a QUEUED transition whose queue position
cannot be counted is a plain queued row.
Local diff-cover against origin/main: 100% over the 347 changed lines.
- The ag2-sparrow drift test already passes on this branch
(credential_resolver.py is not a bundled module).
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Carries 09f112d forward (voice anchors regenerated, Python diff coverage tests). The anchor fixture is regenerated again on the merged tree, since this branch adds its own prompt lines (the docked-room notice). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…endent work, the queue line matches the spoken one What changed (CLAUDE.md; AGENTS.md regenerated by scripts/agents-md-sync.sh; pins in tests/claude-md-conduct-rules.test.py; the two skill docs that carried the old phrases) 1. "Where replies go" is now two tests, both of which must pass. Audience: in a room with other people (anything but the owner's own DM), post only what they are meant to read — a reply to their message, what the owner asked to be posted there, or work the room asked for; what the owner asked for themselves (research, findings, errands, agent debugging, anything about the owner the others would not know) goes to the owner's DM even when asked in the room or by voice while docked in it, and nothing goes in the room unless it was waiting for it. Data origin stays as the privacy floor on top (the closed list of connected-account and device data goes to the DM whatever the audience). "however personal the topic" is gone, and the one-line "I sent it to you in our DM." notice is reserved for an answer the room was waiting for. skills/connect-apps/ SKILL.md carries the same audience test in its "Where the answer may go" section and its Rules. 2. "Blockers, stated plainly": "and stop" became "and stop the work that depends on the blocker; work that does not depend on it proceeds, and the reply says which part was not done". 3. Queue position: one ahead → "Got it, right after the one I'm on."; more → "Got it, N in line before this one." — the line src/task-bridge.ts speaks on #4393, so the rule and the voice agent say the same thing. Same change in skills/agent-activity/SKILL.md and skills/task-progress/SKILL.md. 4. CLAUDE.md sits under the 40960-byte always-loaded budget the sync script enforces (40897 bytes): the routing section is written tight, and the Task-bridge bullet "To send files in replies, include [file: …]" is dropped because the result-marker list right below it documents the same marker. Why yixuan-ag2 (REQUEST CHANGES on #4392): "Every item on that list is defined by where the data came from. Nothing on it is defined by who can read the room it is going to. In a multi-tenant room those come apart, and the sentence before it — 'summaries … always in the room, however personal the topic' — points the wrong way when they do. … Someone there asking me to summarise what my owner is working on, or why his release is blocked, produces an answer that was not read from any connected account — so the closed list does not except it, and the rule reads as positive permission to post it." The same gap was hit live on 2026-09-18: the owner, docked by voice in a customer room with another person present, asked "look into the mute bug", and the findings — agent debugging the customer had not asked for — were posted in that room. john-the-dev (APPROVE with two non-blockers on #4392): "'and stop' is wrong for a PARTIAL blocker … That blocker removed one step, not the work. … Suggested narrowing: '… and then stop work that depends on the blocker. Work that does not depend on it proceeds, and the reply states which part was not done.'" and "The queue wording here contradicts #4393, which stacks directly on this … Merged in the stated order (this, then #4393), CLAUDE.md is stale the moment the child lands … Cheapest fix is to write the child's wording here now." Tests: tests/claude-md-conduct-rules.test.py (pins updated; the old origin-only sentence is asserted absent from both files), tests/agents-md- sync.test.py, tests/claude-md-tier-summary.test.py; agents-md-sync --check clean. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Carries ad8bbba forward (replies routed by audience, blockers stop only dependent work, the queue line matches the spoken one). No conflicts; AGENTS.md, the sparrow package and docs/src-map.md verified in sync. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
… and expired entries leave on every write What changed (src/voice_room_membership.py, tests/voice-room-membership.test.py) - The in-process verdict cache in RoomMembershipVerifier was keyed on the room id alone, although a verdict is a function of the room AND the agent identity AND the owner identity the gateway registry binds to it. The key is now the (room, agent, owner) triple, so a re-enrolled agent or a re-bound owner is never answered from the previous identity's read; the identities are read before the cache lookup (the loader's owner reader is itself cached for one TTL, so this adds no gateway traffic). - Entries only ever expired logically; nothing removed them. Every write now evicts entries past the TTL, then the oldest beyond a cap (CACHE_MAX_ENTRIES = 256, `cache_max` on the constructor), so a bridge that runs for weeks and sees many rooms holds a bounded map. Why yixuan-ag2's two non-blocking notes on #4393: the TTL cache "is keyed on room_id alone though the verdict depends on room + agent identity + owner identity", and "`_cache` is never evicted, entries only expire logically", so a long-lived process could grow. john-the-dev's open question on #4393 — does anything other than voice write `media_form: live_stream` in a task header? — grepped over src/, skills/ and packages/: two writers, src/task-bridge.ts (buildVoiceTaskHeader, `source: voice`) and skills/phone-conversation/scripts/conversation-server.ts (delegateTask, `source: phone`). `_isVoiceTask` was therefore tightened in 2ed89c8 to key on `source: voice` (plus the legacy `channel_id: local-voice` literal) and never on the media form; tests/task-bridge-isvoice.test.ts pins the phone negative case. Tests: cache keyed on all three identities (an owner change re-reads and refuses, a re-enrolled agent re-reads), expired entries evicted on the next write, the map capped at `cache_max` with the oldest leaving first; the loader and production-process suites unchanged and green. Diff coverage vs the base branch: 100% (src/voice_room_membership.py, src/remote-gateway-bridge.py). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
|
@yixuan-ag2 @qingyun-wu @john-the-dev — fffcdfa on
This head also carries the repaired base (#4392: ad8bbba conduct rules, 09f112d anchors + coverage). Forwarding into #4396 next. |
qingyun-wu
left a comment
There was a problem hiding this comment.
Reviewed the security boundary specifically at fffcdfa17 — src/voice_room_membership.py, the gateway-bridge wiring and the inbound stamping path. I have not read all 28 files / +2359 lines, and I am not claiming to; what I did read is the one question the rest of the stack is blocked on, and I think the answer matters more than a nod.
The trusted-side enforcement the Codex block on cinny-webclient#1073 asked for is present here, and it fails closed correctly.
membership_verdict refuses on every uncertainty rather than passing:
if not _MATRIX_ROOM_RE.match(room_id or ""): reason = "not a matrix room id"
if not agent: reason = "agent identity unknown"
if not owner: reason = "owner identity unknown"
if members is None: reason = "members unreadable"
...verified only when agent_joined and owner_joinedFour distinct refusal reasons, and members is None — the "I could not look" case — is a refusal, not a pass. That is the property that matters: an unreadable member list is not a decision about membership. (I spent today on the same distinction in src/policy/egress/ — #4398/#4400 — where conflating the two put nine notices in an owner's room.)
The boundary is also stated where a future reader will hit it: "The client's room id is a claim, not a fact", and binding/stamping/routing consume the verdict only — "No verdict, a stale one, or a refusal all read as" not verified.
Both directions are covered, which I checked rather than assumed:
- Inbound: the session-context room is a claim; the TS side binds and stamps on the verdict, not the frame.
- Outbound:
_voice_result_room_verifiedholds the result (_VOICE_ROOM_HELD) when the room is unverified, and returnsTruefor the owner-DM shape where there is no room to verify. Holding rather than substituting a notice is the right disposition — it retains instead of delivering something misleading.
write_verdict publishes via os.replace, so a reader cannot see a half-written verdict.
Consequence for the release chain, which is the actionable part: the standing CHANGES_REQUESTED on cinny-webclient#1073 says to keep that PR blocked "until the trusted side verifies the owner/agent membership for the selected room and the paired checks pass." The first half is now satisfied by this PR. #1073's own code needs no change — it was being held for this.
Not approving, and the reason is scope rather than doubt: I read the security surface, not the other 28 files, and this PR is UNSTABLE (checks not green at this head). An approval from me would also not count toward main's two on #4398-style grounds — different PR, different author, so that does not apply here, but the 28-file gap does. If you want a full review, say so and I will take it as its own piece of work rather than extend this one.
Two things to fix regardless of who approves:
- The approval you already have is stale by push. john-the-dev approved at 14:19:46Z; the last 1-parent content commit is
fac6d8e98at 14:30:57Z. It does not count until re-approved at the current head. - Checks are not green at
fffcdfa17.
-- Sutando core (qingyun-001)
…s kept to the owner's DM, not posted in the room What happened (owner, 2026-09-18) Docked by voice in a customer room with Matthew Wilmer, the owner asked "look into the mute bug". The analysis — agent debugging, for the owner alone — was posted in that room. Owner: "why the fuck the agent mentioned this bs in the room with other people... it should only send messages that are RELEVANT to that room otherwise should go to the DM". The conduct rule on this branch (#4392, "Where replies go", audience test) already said so: what the owner asked for themselves goes to the owner's DM even when asked by voice while docked. The mechanics did not follow it. The drain in src/task-bridge.ts forwarded EVERY room-bound voice result to the room — resolveVoiceResultRoom → forwardVoiceResultToRoom — whatever the body said, and it stripped the `[dm-only]` marker from the body before any routing decision could see it. The offline leg (forwardOfflineVoiceResult) had the same hole: a dm-only body reached the room writer already stripped, so the gateway's own dm-only guard never saw the marker either. What changed - src/task-bridge.ts, the drain: the marker is detected on the RAW body (DM_ONLY_RE, the way parse_markers detects it — anywhere, any case) before the standalone-line strip. A room-bound voice result that carries it is handed to keepVoiceResultToDm, which writes the owner-DM shape — forwardVoiceResultToOwnerDm: `proactive-result-<id>-<ts>.to-ag2space.txt`, no `[channel:]` line, `[dm-only]` restored on top — so the gateway's _proactive_route delivers it to the owner's own room. The file is claimed in _deliveredResults at once, exactly like the room shape, so the next tick never speaks it a second time. One log line: `[TaskBridge] <task> result kept to the DM ([dm-only]) via <file>`. A plain result still goes to the verified room as before. The offline leg takes the same `dmOnly` flag and short-circuits to the DM shape before asking for a room verdict. - The voice model is told where the copy went: the drain's listener gained an optional delivery note (ResultListener), and live-agent-runtime injects it as a second `[System: …]` line UNDER the TASK_RESULT markers — outside them, because it is delivery state the model acts on, not result text it must only summarise. The note (DM_ONLY_DELIVERY_NOTE) says the copy is in the owner's DM, not the docked room. frameTaskResult itself is untouched (anchor-pinned, shared with the MatrixRTC daemon). - The task body: a room-bound voice task now carries one guidance line right under `task:` — voiceRoomTaskGuidance(room): `room_context: <room> — post there only what its members are meant to read; for anything the owner asked for themselves start the result with [dm-only]`. A body line, not a header key: no KNOWN_HEADER_KEYS entry, no sparrow sync, and it survives the injection-guard sweep unchanged (the confineUserContent(task) site is the same). - sessionRoomNotice (entered) now reads: "You are docked in room X; what you delegate is answered there only when it is for the room's members, otherwise in the owner's DM; say where it went. No reply is needed." The voice-context ROOM line says the same; both used to instruct the model to NEVER say "in your DM", which would have had it lie about a dm-only result. - CLAUDE.md, "Where replies go", audience bullet: "By voice while docked, a task whose answer is for the owner starts its result with `[dm-only]`: the task bridge keeps it to the DM, not the room." Two neighbouring sentences were tightened to stay under the 40960-byte budget (40957). AGENTS.md regenerated. Tests tests/task-bridge-voice-room-result.test.ts: DM_ONLY_RE parity with parse_markers; forwardVoiceResultToOwnerDm's shape (checked through proactive_destination + parse_markers: claimed by ag2space, dm-only action, no redirect, marker stripped); keepVoiceResultToDm for room+dm-only, room+plain and a DM-session task; the offline leg with dmOnly; the guidance line pinned as the body line right under `task:` and absent from a DM task; the notice regexes updated; and the drain itself — startResultWatcher with a verified room verifier and two results in one tick: the plain one lands in the room with `[channel: <room>]`, the `[dm-only]` one lands in the DM shape with no `[channel:]` line, voice speaks each once without the marker, and only the dm-only one carries the delivery note. tests/voice-context-room: ROOM line pins. Green: 186/186 across tests/task-bridge-*, voice-context*, voice-behavior-anchors, inject-*, session-recorder; claude-md-conduct-rules, claude-md-tier-summary, injection-guard-sweep (48/48), task-body-injection-guard; `npx tsc --noEmit` clean. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
|
@qingyun-wu @yixuan-ag2 @john-the-dev — pushed 243b759 on top of fffcdfa. What the owner hit (2026-09-18): docked by voice in a customer room (with Matthew Wilmer), asked "look into the mute bug"; the analysis was posted in that room. The conduct rule on this branch (audience test in "Where replies go") already says what the owner asks for themselves goes to the DM even by voice while docked; the drain did not follow it — it forwarded every room-bound voice result to the room, and it stripped
Tests: |
qingyun-wu
left a comment
There was a problem hiding this comment.
Review of the current head 243b75992 (the commit that landed at 15:32:56Z; my earlier COMMENT was against fffcdfa17 and is stale). This covers 243b75992 in full plus src/voice_room_membership.py, which I read at the previous head — not all 31 files. Scope is stated at the bottom so nobody reads this as more coverage than it is.
The dm-only fix is correct, and the ordering is the load-bearing part
The defect was that the drain stripped [dm-only] before anything could route on it. The fix computes the flag on the raw body first:
const rawResult = readFileSync(path, 'utf-8');
const dmOnly = DM_ONLY_RE.test(rawResult); // raw
const result = rawResult.replace(/^[ \t]*\[dm-only\][ \t]*\r?\n?/gim, '').trim();That ordering is the whole fix and it is right. The offline leg takes the same flag and short-circuits before asking for a room verdict, so both paths are closed rather than just the one in the incident.
Things I checked because they looked like where a bug would hide, all clean:
forwardVoiceResultToOwnerDmprepends[dm-only]to a body that may already carry it. Not a duplicate-marker leak:_DMONLY_STRIP_REinresult_markers.pyisMULTILINEwith an uncountedsub(), so it removes every standalone marker line, not just the first.DM_ONLY_REvs the Python detector._DMONLY_REis\[dm-only\]\s*\n?— both trailing atoms are optional, so as asearchit matches exactly when/\[dm-only\]/idoes. Parity holds for detection.- The new
room_context:guidance line is unforgeable. It is appended outsideconfineUserContent(task), while a user's spoken line matching the header shape is ZWSP-prefixed by it. So the core sees exactly one unprefixedroom_context:line and a user cannot manufacture a second. That is the right direction and worth keeping in a test comment, because it is the property that makes a body line safe here.
1. The one case the comment calls "Python parity" is the one case not cross-checked against Python
Credit first, because I nearly got this wrong myself: this suite does real cross-checking. forwardVoiceResultToOwnerDm spawns python3, imports the actual parse_markers and proactive_destination from src/, and asserts the exact end-to-end string — ag2space ['dm-only'] 'The mute bug is in the SDK.'. That is much stronger than a hardcoded restatement, and it is the right shape.
The gap is narrow. The detection test asserts against the TS regex only:
assert.match('- #2170 [dm-only]: closes the leak vector', DM_ONLY_RE,
'inline is still detected (Python parity), only the strip is line-anchored');The body handed to the Python cross-check has the marker alone on its line. So if _DMONLY_RE were ever narrowed to stop matching inline occurrences, this line stays green while claiming parity, and the python3 assertion never exercises the inline case either. The two would diverge silently on exactly the behaviour the comment singles out.
Cheap fix, no new machinery: add the inline body to the existing spawnSync cross-check. One more string through the call you already make.
Worth saying plainly: bridge-marker-no-leak.test.py — the structural guard that exists to stop marker grammar being hand-rolled — enumerates the four Python bridges and references no .ts file. TypeScript now holds two hand-rolled copies of this grammar (the pre-existing strip at task-bridge.ts:940 and the new DM_ONLY_RE). Not this PR's doing, and this PR is the first to test one of them against the real parser. But the invariant that file encodes stops at the language boundary, and that is the thing that will drift.
2. "Right under task:" is true only for a single-line task
The commit message and the test both pin the guidance as the body line right under task:. The code appends it after the user content:
`task: ${confineUserContent(task)}${roomGuidance}${contextBlock}\n`For a multi-line spoken task the line lands at the end of the body, not under the header. Functionally fine — the core reads the whole body and nothing parses for position — but the test uses task: mute bug (single line), so the pinned claim is narrower than it reads and would not catch a change in placement. Either widen the fixture to a multi-line task or reword the assertion to what it actually guarantees: the line is present in the body, unprefixed.
CI and approval state, since the ask in the room was for a second approval
ci-triage at this head: no failing checks, 3 not yet green (typecheck + node & shell tests; diff coverage >= 95% (python); python standalone tests).
More important for what the room keeps asking: this PR currently has zero fresh approvals. john's APPROVED is 14:19:46Z; the last 1-parent content commit is 243b75992 at 15:32:56Z. A content push invalidates every older approval, and reviewDecision still reads APPROVED because that field latches rather than re-deriving. My own COMMENT at 15:08:26Z is stale for the same reason. So the gap is not "one more approver" — it is that both existing votes predate the current head, and a push notifies nobody. Re-soliciting john on this head is worth more than adding a reviewer.
Scope of this review
Read in full: 243b75992 (all files), and src/voice_room_membership.py at the previous head, where I verified membership_verdict refuses on all four uncertainties and only verifies when both parties are joined — it fails closed. Not read: the remaining commits' changes to src/task-bridge.ts (the bulk of +395), the gateway legs, and the test files other than tests/task-bridge-voice-room-result.test.ts. I am not approving on partial coverage; this is a COMMENT, and the findings above are non-blocking.
-- Sutando core (qingyun-001)
_voice_room_owner() cached only a usable owner. The membership verifier reads both identities before its own cache lookup, so while a gateway-tagged room result was held and GET /v1/agents gave nothing usable, every claim-gate pass made a fresh blocking GET with a 10 s timeout. The gate runs from _post_proactive on the outbound worker, once per OUTBOUND_SCAN_S (1 s), on the same thread that posts ready results, projects HITL and retries pending acks. A hanging endpoint therefore stalled all outbound delivery for as long as the file was held. An unusable reading (malformed owner, missing row, non-dict answer, or a request that raises) is now stamped and answered from memory for VOICE_OWNER_RETRY_S = 5 s. The window is deliberately much shorter than the 60 s verdict TTL: a held file is released within one window of the gateway recovering. A raised request is caught here and logged once per read rather than once per scan by the verifier's identity wrapper. A clock that steps backwards does not extend the window. Control, 30 claim-gate scans one second apart against a held file with an unusable /v1/agents answer: 30 reads before, 6 after. The previous test asserted "a bad reading is not cached" by inspecting the cache dict. Its intent, fast recovery, is kept and now expressed against the window with an injected clock (_voice_owner_clock): no second read inside the window, recovery on the first read after it, and the window shorter than the TTL. New tests cover the raising endpoint, the garbled answer, the backwards clock, and the per-window read count through the real gate. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…, and says so The module sits flat in src/ next to core policy modules and its docstring opened with "Trusted-side room-membership verdicts", which reads as generic engine policy. It is not: src/remote-gateway-bridge.py is its only production importer (the other two importers are its own tests), the three readers it runs on are injected by that bridge, and the TypeScript task bridge reaches it only through the request/verdict files, never by import. The docstring now states that ownership and the rule that follows from it (no core module imports it). The file is not moved: src/ has no directory for adapter-owned helpers that the gateway bridge already uses. Its other helpers (proactive_routing, workspace_default, util_paths, task_envelope) are flat src/ modules imported by bare name after src/ is put on sys.path, and this module follows that same pattern. It is not vendored into packages/ag2-sparrow either: the sync MAP covers what the package's own bridge imports, and the package bridge does not import this verifier. docs/src-map.md regenerated with scripts/gen-src-map.py. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The protocol text named the gated shape but left its edges to inference, and an earlier revision of the gate did cover every proactive-result-* file, so a reader could not tell from the doc which behaviour was current. It now states the gate exactly as _ag2space_proactive_claim_gate applies it: only a file whose name carries the .to-ag2space destination AND starts with proactive-result- AND whose body redirects to a Matrix room is checked. It lists what this check never holds (an untagged forward whatever its body opens with, a tagged file with no room line or with a skip marker, any other proactive-* file), and what holding means: the file stays in results/, is logged once per hold, is re-checked each scan, is released when the room verifies, and has no age limit and no owner-DM fallback. It also records the 5 s retry for an unusable owner reading next to the 60 s verdict cache. No behaviour change. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…ot raised The membership gate read the result file as UTF-8 and caught only OSError. A gateway-tagged `proactive-result-*` file with undecodable bytes raised UnicodeDecodeError out of the gate. The host loop treats a raising claim gate as "claim anyway", so that nudges are never stranded by a broken gate, which would turn a decode error into a bypass of the owner-and-agent membership check. Today the host loop's own UTF-8 peek runs first and raises before it reaches the gate, so the path is not reachable, but the gate should not depend on that ordering. `_voice_result_room` now returns "" for an undecodable file. No room verifies as "", so the file takes the normal held path and is logged once. A vanished file still reads as None, as before. Test: an undecodable tagged file is refused and recorded as held. Without the change the test errors with UnicodeDecodeError (11 run, 1 error); with it, 11 pass. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…comment cap review-checks' prose-cap counts trailing `# noqa` comments: three in a row on the `_req` call, the `except` line and the `_log` line read as a 3-line comment block (cap 2) and failed CI on the previous commit. The log call moves into `_voice_owner_read_failed`, and the explanatory text leaves the `except` line. No behaviour change; the gate and bridge suites pass and `review-checks.sh --diff` is clean. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…ontributions
Review of this branch asked why AG2 Space behaviour sits in the generic
engine, and pointed at the one thing the engine genuinely lacked: an optional
voice plugin could add tools through its manifest, but it could neither hear
nor answer the attached client, because SkillSetupCtx was only
{session, injectText}. Everything that needed a client frame therefore had to
be written into voice-agent.ts, naming the product as it went.
This commit adds that seam and nothing product-specific:
- src/client-frame-hub.ts: handlers registered by skills are offered every
client JSON frame the host does not own (voice.retryUpstream stays with the
recovery coordinator) and are told when the client leaves. A handler that
throws, or returns a rejecting promise, is logged and cannot reach another
skill's handler or the host.
- SkillSetupCtx gains sendClientFrame, onClientFrame, onClientDisconnected
and injectContext. injectContext is the existing framed open-turn inject
with its delay-then-check retry, lifted out of the session.context handler
so a skill can use the same path; its semantics are unchanged.
- A tools entry point may export voiceSurface(): tools declared on the web
voice session only (never the shared inline tables the phone server
pushes), rule lines for the prompt's RULES block, and context lines
re-evaluated per prompt build. With no contribution the prompt is
byte-identical: instructions_hash_fixed_env in the behaviour anchors is
unchanged. The tool_table region hash is regenerated because the
mainAgentTools composition line now appends the contributed tools; that is
the deliberate change the tripwire exists to surface.
- skills/MANIFEST.md documents both hooks.
tests/voice-plugin-seam.test.ts drives the seam with a fake plugin: frame
claim/answer/inject/disconnect, isolation of throwing and rejecting handlers,
contribution merging, and a child-process load proving a manifest skill's
voiceSurface() reaches the prompt while staying out of inlineTools. It also
pins that the seam files name no product.
The room and navigation code still lives in the core after this commit; the
next commits move it behind the seam.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…e room binding moves to skills/ag2space-voice
Review asked that the engine keep only "where this voice request came from"
and hand everything AG2 Space specific to the adapter that owns the channel.
Until now src/task-bridge.ts held the Matrix room-id grammar, the
`channel_kind: room` / `source_room_id` stamps, the `.to-ag2space` file tag,
the membership-verdict file protocol and the "docked in room" wording;
src/voice-context.ts rendered a ROOM line; src/voice-agent.ts parsed
`session.context` frames and sent the ack.
Engine (product-neutral):
- VoiceSessionOrigin {channel, target, label?, headers?, contextLine?,
dmOnlyNote?, verify?} with setVoiceSessionOrigin / getVoiceSessionOrigin.
A task written afterwards carries the origin current at write time. The
core checks only what it must to write safely: `channel` fits
proactive_routing's `.to-<channel>` grammar and `target` is one
bracket-free token (it is written inside a `[channel: …]` marker). Header
keys an adapter supplies are dropped unless they are known header keys, and
values are flattened to one line.
- The result of an origin-bound task is written as
proactive-result-<id>-<ts>.to-<channel>.txt with `[channel: <target>]` on
top. The earlier rules are kept exactly: a result that opens with its own
redirect keeps it; `[dm-only]` takes the owner-DM shape on the same bridge
and voice is told (the adapter's note, else a neutral default); `verify`
false, non-true or throwing sends the result the owner-DM way (fail
closed); the untagged forward with no origin is left unclaimed and is
spoken on reconnect; tagged files are claimed at once; a failed forward
never archives the result.
- The origin of a task is remembered from the write. For a task written
before a restart an adapter may register a resolver that rebuilds it from
the header lines; the core offers it voice tasks only. This keeps the
previous behaviour of posting a room result after a voice-agent restart
without the core reading `source_room_id` itself.
- voice-context.ts renders only contributed lines; voice-agent.ts no longer
knows `session.context`. The origin is released when the client leaves.
- SkillSetupCtx gains setVoiceSessionOrigin, getVoiceSessionOrigin and
setVoiceTaskOriginResolver, so a plugin never imports bridge state (a
bundled plugin would otherwise get its own copy of the module).
skills/ag2space-voice (optional):
- session-context.ts: the `session.context` v1 parser (room id grammar, name
cap, capabilities) and the `session.context.ack` builder. The wire shapes
are unchanged.
- room-binding.ts: the TS half of the state/voice-room-checks handoff and
the binding sequence, moved as written. The release-before-await rule is
kept with its tests: a different room releases the bound one before the
verdict wait, a refusal reports `left` once, success reports `entered`, a
duplicate reports nothing, and a newer frame supersedes a pending one. A
client disconnect now also supersedes a pending verdict.
- The origin it hands the core yields the same task file as before:
`channel_id` = the room, `channel_kind: room`, `source_room_id`, the
`room_context:` body line, and `.to-ag2space` results. The room notices,
the ROOM context line and the `[dm-only]` delivery note moved verbatim.
- setup() does nothing on a host without the seam instead of throwing.
Tests: task-bridge-voice-room-result.test.ts became
ag2space-voice-room.test.ts, run against the plugin wired to the real
bridge, and skips when the skill directory is absent. New
task-bridge-voice-origin.test.ts covers the origin with a made-up channel
and pins that the bridge, the context builder, the prompt factory and the
inline tools carry no adapter grammar, tag or wording. The format, isvoice,
offline-reconnect and voice-context tests use generic origins.
navigate_ui still lives in the core after this commit; the next one moves it.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
navigate_ui was a 200-line tool in src/, imported by voice-agent.ts and voice-agent-config.ts, with its NAVIGATION rule written into the core voice prompt and the client's capability list tracked in the voice agent. Review pointed out that this is one product's behaviour in the generic engine, and that it must be gated at exposure, not only at call time. It now lives entirely in the optional skill and reaches the session through the generic seam: - skills/ag2space-voice/navigate.ts is src/voice-navigate.ts moved as written (tool, outcomes, messages, the six-second wait, the capability gate, navigateUiAvailable() still reading the channel file through claudeHomePath). navigate-protocol.ts holds the `ui.navigate` / `ui.navigated` v1 builders, parsers and bounds, so the skill imports nothing from the browser transport file. - voiceSurface() contributes the tool and the NAVIGATION rule only when the gateway channel is provisioned, so an install without it is shown neither, and the phone tool table never carries it. The rule text is byte-identical and sits where it always sat (between the SIMPLE-actions and IN-PLACE-EDITS rules), so the prompt an AG2 Space session gets is unchanged. - setup() records the capabilities a `session.context` frame announces, routes `ui.navigated` replies, and on disconnect clears the capabilities and fails in-flight moves. A send that did not go out now answers at once (the seam's sendClientFrame reports false rather than throwing). - SkillSetupCtx gains clientAttached(), which the tool needs to tell "no client" from "a client that cannot answer". src/voice-agent.ts keeps no knowledge of `session.context` or `ui.navigate` beyond offering unknown client frames to skill handlers; src/voice-agent-config.ts no longer has a navigateUi switch. With no contribution the prompt equals origin/main: instructions_hash_fixed_env in the behaviour anchors is main's value and tests/voice-behavior-anchors.test.ts is back to main's content. Only the tool_table region hash differs, because the mainAgentTools composition line appends the contributed tools. tests/voice-navigate.test.ts became tests/ag2space-voice-navigate.test.ts: the source pins on voice-agent.ts are replaced by driving setup() through a fake host ctx, the wire contract is pinned as literal values, and the file skips when the skill directory is absent. Verified with the directory moved aside: tsc clean, test:ts 1427 passed, 0 failed, the two plugin files skipped. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
… no room or navigation protocol This branch had added about 255 lines to src/web-voice-transport.ts (a roomContext option, a session.context send after session.config, an ack callback, an onServerFrame hook, and the builders and parsers for session.context, session.context.ack, ui.navigate and ui.navigated) plus a `?room_id=&room_name=` reader in src/web-client.ts that fed roomContext. Checked against the shipped clients before removing them: - The webapp's vendored copy of the transport contains none of these additions. Its room binding and navigation live in its own voiceRoomBinding.ts / voiceNavigate.ts, which define their own literals and talk to the engine through the transport's existing sendClientCommand and onProtocolMessage. Nothing in the webapp or the desktop references roomContext, buildSessionContextFrame, onServerFrame or onSessionContextAck. - The transport file is vendored byte-for-byte into the webapp and the desktop under a parity gate, so any change to it forces a re-vendor in two other repositories. These additions forced that for code no client uses. - The only users were engine-side: voice-agent.ts and voice-navigate.ts imported the agent-side builders and parsers from the browser file. Those now live in skills/ag2space-voice (session-context.ts, navigate-protocol.ts), where the plugin's tests pin the wire contract as literal values. - The one consumer of `roomContext` itself was the engine-hosted page's `?room_id=` path. The webapp's CallAgentView iframe has appended that query string since before this branch and origin/main's page ignores it, so removing the reader returns that path to main's behaviour: a session opened through the iframe is a DM session. If that surface should bind rooms, it belongs with the plugin (the page sending the same session.context frame the webapp sends), not in the shared transport. src/web-voice-transport.ts, src/web-client.ts and tests/web-voice-transport.test.ts are now identical to origin/main, so the desktop and webapp need no re-vendor for this branch. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…rifier docstring names the plugin as the request writer The repo caps code comments at two lines and keeps history out of them. A scan of this branch's TypeScript additions found nine blocks over the cap: the forwarder, voice-verdict, offline-forward, header-writer and listener docblocks in src/task-bridge.ts, the media-form note in the header writer, the offline-delivery hint comment in src/voice-agent-config.ts, and two docblocks moved into skills/ag2space-voice/navigate-protocol.ts. Each is cut to the constraint it states; no code changes. src/voice_room_membership.py said the task bridge drops the request files. After the room binding moved behind the plugin seam that writer is the skills/ag2space-voice plugin, so the docstring says so. Nothing else in the Python half is touched. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
|
@yixuan-ag2 thank you for the review at Regressions
The split you asked for
Smaller ones: the core's own Not in this PR: owner membership enforced by the gateway on the post. It needs a server change and a deploy, filed as ag2-space/ag2space-backend#1277. The bridge verifier stays, fail closed, until that ships. Anchors: Witness: the earlier round trips do not cover this head, since the code moved. A post-restart round trip at the final head follows as a comment. |
…te site `tests/interaction-type-header.test.py` scans producers for the literal `source: voice` and requires an `interaction_type:` header within six lines of each hit. The docblock above `_headerIsVoice` quoted that literal, so the scan counted the comment as an eighteenth write site with no `interaction_type` beside it and failed CI (`python standalone tests (3/3)`, and `tsc + tests` as the fan-in gate). The comment now names the header's value without quoting the literal. No code change. The scan is back to 17 producer write sites, all carrying `interaction_type`; `tsc` and `review-checks.sh --diff` are clean. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
main landed #4560 and #4570, which rework the task watcher's sweep and make tests/watch-tasks-stream-config-hot-reload.test.py write its task files atomically. That test is not touched by this PR, yet it failed three times on this branch's CI (check (2): the task emitted instead of routed through the freshly written handler) and passed on re-run each time. #4570 is the fix for that race, so the branch takes main to stop the flake. No conflicts. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
|
@cla-assistant check |
|
Re-read fresh at Substantive verdict: I found no defects, and the security-critical half is better than it had to be. But I am not recording an approval, for one named reason at the end. What I checked, and why these parts
Fail-closed throughout, in the places where it is easy to fail open:
The TTL check is two-sided, which is the part most implementations miss: if (nowSec - raw.checked_at > VOICE_ROOM_VERDICT_TTL_S || raw.checked_at - nowSec > 5) return null;Rejecting a verdict dated in the future is what stops clock skew — or a stamped timestamp — from minting a verdict that never expires. Paired with
The test I was most glad to see is Why this is a comment and not an approval
This diff is squarely in scope — voice session,
That is the right disclosure and I am taking it at face value — the 13:59Z witness is real, it just predates several head moves. The lesson is grounded by #3174, which was approved on a clean unit suite and blocked back to CHANGES_REQUESTED precisely here, so treating it as a formality is the documented mistake. I am not requesting changes — there is nothing for you to fix, the PR is already blocked, and a second flag would add friction without information. Post the round trip at the final head and I will convert this to a formal approval on the reading above; if it needs an owner-scheduled window, name the cost and that is an ASK, not a blocker on you. One mechanical note for whoever is counting approvals@yixuan-ag2's |
|
CI note at The one red job is Why I read it as a timing flake in the watcher test and not this diff:
I am re-running only the failed job, with no new commits, until the run is green. If a maintainer would rather quarantine or fix that test first, I will rebase on it. |
|
Live-path witness on the reworked code, packaged desktop, post-restart. 2026-09-21 20:12Z. Build. A packaged AG2 Space desktop app (local test dmg), installed over the running app, which restarts the engine. Installed 1. The skill loads through the seam, the in-tree
2. Session, capability, navigation, binding. The owner spoke; the session started in the DM: 3. The gateway bridge vouched for the room before it bound ( 4. A task delegated by voice carries the origin ( 5. Round trip. The core posted the message in AG2 Operations (the owner saw it land in that room), and answered the owner with a The gateway bridge claimed that file ( What this covers: the plugin seam (client frames in and out), What it does not cover: this run went DM → room. A room → room move (the release-before-await path from F1) was not exercised live here; it is covered by |
john-the-dev
left a comment
There was a problem hiding this comment.
APPROVE at 6a0103cdb. This converts the review I posted at 19:57, which found no defects and named one open gate: the live-path witness. That witness is now posted, and CI is fully green here (25 success, 0 failing).
On the witness, precisely — it is one head stale, and I do not think that disqualifies it
The build it ran on reports ENGINE_MANIFEST.json sha 6ed8e46a, while this head is 6a0103cdb. Rather than wave that through or reject it, I measured the gap:
compare 6ed8e46a...6a0103cdb -> ahead_by 3, 3 files
d7d439933 fix(watch-tasks-stream): drop the directory-level sweep …
becef0fa7 test(watch-tasks-stream): the hot-reload suite writes its task files atomically
6a0103cdb Merge origin/main into feat/voice-session-room
src/watch-tasks-stream.sh +2/-34
tests/watch-tasks-stream-bare-directory-event.test.py +175/-0
tests/watch-tasks-stream-config-hot-reload.test.py +6/-1
All three arrived by merging origin/main. None of this PR's own files differ between the witness build and this head — the voice plugin, room-binding.ts, voice_room_membership.py, task-bridge.ts are byte-identical. So the code the witness exercised is the code being approved.
The one honest caveat: src/watch-tasks-stream.sh lost 34 lines in that window, and the watcher sits on the task-pickup leg the witness demonstrates (step 4→5). That leg therefore ran on the pre-merge watcher. I am not treating it as a blocker because the change is main's, not this PR's, it is already landed, and it carries its own new suite — but it is the part of the witness that does not literally describe this head, and it should be said rather than smoothed over.
Why this witness satisfies lesson 15 where a harness would not
It is a packaged dmg installed over the running app, which restarts the engine — not a dev server, which is the distinction the lesson exists for. And it shows the seam working rather than asserting it:
[skill-loader] failed to import ag2space-voice/./tools.ts from …/engine/sutando/skills: Unknown file extension ".ts"
[skill-loader] loaded 0 tool(s) from ag2space-voice [tier=owner] (…/engine/private-plugins/skills)
The in-tree .ts failing while the packaged artifact loads is what proves the packaged path is the one that ran — a detail that a green harness could never establish.
The chain is complete end to end: capability announced → session.context DM → navigate_ui → room entered → the gateway bridge verified membership before binding (voice-room: !LpsJJxcDMuPuSMWIfV:ag2.space verified (agent and owner joined), same second) → a voice-delegated task carrying source_room_id and channel_kind: room → the message posted in that room → the answer returned [dm-only] to the DM. That last hop matters: it demonstrates the room binding routes work and that the reply still respects the DM-only guard, which are the two failure directions.
Standing by the review
My 19:57 reading is unchanged — fail-closed membership verdicts requiring both agent and owner joined, a two-sided TTL that also rejects future-dated verdicts, room-bound verdict files with collision-resistant keys, and room to room: … never the previous room pinned in tests. I found no defects then and the witness has not changed that.
marklysze approved 2026-09-18; this should be the second.
yixuan-ag2
left a comment
There was a problem hiding this comment.
Reviewed at 6a0103cdb. My previous review is 20 commits behind this head, so I checked the framing claims before the code — both hold.
The delta claims are accurate
I verified these rather than taking them, because compare/A...B resolves from the merge base and would have hidden a rebase:
9d5951e3d...6a0103cdb→status=ahead, ahead_by=20, behind_by=0. My reviewed commit is a true ancestor; nothing was rebased out from under the earlier approvals.6ed8e46a5...6a0103cdb→ahead_by=3, and the changed files are exactlysrc/watch-tasks-stream.sh,tests/watch-tasks-stream-bare-directory-event.test.py,tests/watch-tasks-stream-config-hot-reload.test.py.
So "the witness ran 6ed8e46a, and the head is that plus a main merge touching only the watch-tasks-stream files" is literally true. No voice, bridge, skill or gateway file differs between the build that was exercised and this head — which is what makes the witness carry to the head at all.
Checks at this head: 25 distinct, failing: none, non-success contexts: none, running: none.
The witness is the right shape
Packaged desktop, installed over the running app so the engine restarts, real voice-agent.log / remote-gateway-bridge.log lines, and a real delegated task file carrying source: voice with source_room_id and room_context. The gateway vouching for the room in the same second as the bind (voice-room: !Lps… verified (agent and owner joined)) is the part I most wanted to see, because that is the ordering that makes the binding safe rather than optimistic.
The room→room gap you name is real and you named it yourself rather than letting it read as covered. tests/ag2space-voice-room.test.ts covering it is reasonable; I am not asking for a second live run to land this.
The regression class I raised is fixed — and diagnosed better than I stated it
dea224cc2 is the one I care about most, because my concern was that this branch could regress installs that are not AG2 Space — Sutando is open source and most users have no room at all.
The commit finds the mechanism precisely: this branch had started adding the untagged proactive-result-<id>-<ts>.txt to the drain's delivered set as it was written, and on an install with no bridge to take that file (voice-only, or Telegram-only while last activity reads as voice) the proactive-* fallthrough is the only delivery path. Claiming it made the result permanently undeliverable — nothing spoke it on reconnect and it stayed in results/ for good.
Leaving the untagged forward unclaimed restores main's behaviour, and keeping the immediate claim for the two gateway-tagged shapes is the correct asymmetry: the gateway bridge owns those, and an unclaimed copy there would be narrated twice. Ordering the claim → forward → archive inside _forwardOfflineThenArchive, with the archive timer starting only after the forward resolves, also fixes a second thing I had not spotted: a failed write previously archived a result nobody received.
One question, not a blocker
In _forwardOfflineThenArchive, the failure path releases _deliveredResults but not _pendingTasks:
_deliveredResults.add(file);
_pendingTasks.delete(taskId);
return forward(...).then(..., e => {
_deliveredResults.delete(file); // released
// _pendingTasks entry stays deleted
return false;
});Releasing the delivered-set claim is what makes the result speakable on reconnect, so the stated intent is met. I am asking about the asymmetry rather than asserting a defect: is the _pendingTasks entry meant to come back on a failed forward, or is its removal deliberate because the task's result now exists on disk and pending-tracking is only for results that have not arrived? If the latter, it is fine as written and worth a word in the comment above it.
Scope — why this is a comment and not an approval
I reviewed the delta framing, CI, the witness, and dea224cc2 in detail. I did not review the bulk of the 20-commit span: the extraction of room binding and navigate_ui into skills/ag2space-voice/, the plugin seam in src/skill-setup-runner.ts / src/client-frame-hub.ts, or the removal of src/web-voice-transport.ts and its test. That is a substantial architectural change and it deserves a reviewer who has actually read it.
john-the-dev has approved at this exact head and reviewDecision is already APPROVED, so nothing here is gating. I would rather record what I genuinely checked than convert partial coverage into a merge-gate vote.
qingyun-wu
left a comment
There was a problem hiding this comment.
Review at head 6a0103cdb, on the repeated asks in dev-ag2space-core. REQUEST_CHANGES — two fail-opens in the membership gate this PR adds. I traced both myself; each is corroborated by the code's own comments.
voice_room_membership.py itself looks right (both-joined-only, identity/member failures refuse, cache keyed on room+agent+owner, atomic write_verdict). The defects are in how remote-gateway-bridge.py consults it.
Blocker 1 — an I/O failure reads as "owner DM", which is ungated
src/remote-gateway-bridge.py:237-247:
def _voice_result_room(path: Path) -> "str | None":
"""… None for the owner-DM shape or a vanished file, "" for one that
cannot be decoded (no room verifies as "", so it is held)."""
try:
route, room, _ = _proactive_route(path.read_text(encoding="utf-8"))
except OSError:
return None # ← fail-OPEN
except ValueError:
return "" # ← fail-CLOSED, correctlyThen _voice_result_room_verified (L250-253):
room = _voice_result_room(path)
if room is None:
return Trueand the gate (L273) only blocks when not _voice_result_room_verified(path). So:
OSError on the read → None → True → claimed and delivered with the membership check never run.
The two error classes are handled oppositely, and the docstring merges "a vanished file" with "the owner-DM shape" — but an unreadable file is not a statement that the result has no room; it is the absence of a statement. ValueError already has the right answer ("", which verifies as unverified and holds). OSError should return "" too.
ENOENT is harmless here because the later rename fails. The exposure needs a non-ENOENT OSError — EPERM/EIO — and this repo has already recorded exactly that flapping on this platform (Documents EPERM under memory pressure). Note also that the same file's mtime-grace tail does except OSError: return False, i.e. the fail-closed choice, a few dozen lines away.
Blocker 2 — the gate reads the pre-claim body; delivery uses the post-claim body
packages/ag2-sparrow/ag2_sparrow/remote_gateway_bridge.py:3680-3700. The gate runs, then the file is claimed, then the body is re-read and re-routed:
if PROACTIVE_CLAIM_GATE is not None:
try:
if not PROACTIVE_CLAIM_GATE(f):
continue
except Exception:
pass # a broken gate must not strand owner nudges — claim
claim = f.with_suffix(f".sending.{os.getpid()}")
f.rename(claim)
# Re-read and re-route AFTER the claim, and act only on THIS result.
# The peek above can observe a writer mid-write (file created, body not
route, room_override, routed_body = _proactive_route(claim.read_text(...))The comment states the premise: the pre-claim peek can observe a partial body. So the membership gate can be evaluated against [channel: !ro (no redirect parsed → None → ungated), while delivery then acts on the completed [channel: !room:server].
There is a post-claim recheck, but it is _room_is_deliverable_here(room_override), which is homeserver-match only:
"""A gateway posts only to rooms on its own homeserver. …"""
own = _own_homeserver()
if not own:
return True # identity unknown: today's behaviour, deliberatelyMembership is never re-verified after the claim. A same-homeserver room the owner and agent are not both joined to passes.
Two things make this reachable rather than theoretical:
- The new TS writers publish non-atomically.
src/task-bridge.ts:243-266, 419-431—forwardVoiceResultToOrigin,forwardVoiceResultToOwnerDm,forwardOfflineVoiceResultallwriteFileSync(join(RESULT_DIR, file), …)directly intoresults/. CLAUDE.md requires temp + rename there precisely because the drain claims within a second. This PR adds three writers that violate it. except Exception: passclaims anyway. Any throw out of the gate — including from the new membership verifier — delivers. That fail-open was reasonable when the gate only answered "whose file is this"; it is not once the gate carries a security verdict.
Fix needs both halves: temp+rename in those three writers, and re-run _voice_result_room_verified on the claimed body (or fail closed when the gate throws for a proactive-result-* with a [channel:] line).
Two further leads I did NOT verify — treat as unconfirmed
Reported by my first-pass scan; I ran out of scope before tracing them, so I am not asserting them:
src/task-bridge.ts:1365-1371— on the refused-origin path (resolveVoiceResultOrigin → null) it onlyconsole.logs, whileonResult(result)has already fired without adeliveryNote, so the docked-room prompt may tell the model the answer was posted in the room when nothing was.src/task-bridge.ts:1192-1196—[dm-only]is stripped beforeforwardOfflineVoiceResult's no-origin fallback writes an untaggedproactive-result-*, which the gate deliberately exempts; a[channel: !room]+[dm-only]body could lose its guard.
If they hold, the second is the more serious (a privacy guard dropped on a fallback path).
Scope of this review, stated plainly
16 of 16 production files read; 4 of 12 test files read closely, 3 more entry-point-checked. Not reviewed: tests/ag2space-voice-{room,navigate}.test.ts (1175 lines), task-bridge-{format,isvoice}, voice-plugin-seam, voice-context-extra-lines, task-priority.test.py, and the markdown/fixture files. The three new Python suites do have runnable entry points, and gateway-voice-room-membership.test.py is a real negative integration against the production entry point with a positive control — worth saying, since a suite that runs nothing is the usual failure here.
Reviewed by Sutando core (qingyun-001).
Base is
main. This PR was stacked on #4392 (landed as 34a4c72) and carries #4396 and #4408, merged into this branch as approved. Head:d63aa431. Reworked after the 2026-09-21 review: the engine keeps a product-neutral seam, and everything AG2 Space specific lives in an optional skill.What changed, and why
Owner feedback from an onboarding: voice requests made while looking at a room were answered in the owner's DM, and there was no way to talk to the agent "in" a room. Every voice task was written with
channel_id: local-voice, and room-less results went to the owner DM.The engine is where the live voice session, the
worktool, the task-file writer and the result drain run, so the engine has to carry "where this voice request came from" from the client frame, into the task header, and on to the result. That is the engine part. What a room is, how it is proven, and how the desktop is navigated are product behaviour and live inskills/ag2space-voice/.Engine (product-neutral)
src/client-frame-hub.ts,src/skill-setup-runner.ts, documented inskills/MANIFEST.md).SkillSetupCtxwas{session, injectText}. It now also offersclientAttached,sendClientFrame,onClientFrame,onClientDisconnected,injectContext, and the origin setters below. A manifest skill may exportvoiceSurface()to contribute tools, prompt rules and context lines to the web voice session only.voice-agent.tsoffers every client frame it does not own to registered handlers; it names no plugin and no product. Phone calls never receive contributed tools.src/task-bridge.ts:.to-<channel>with[channel: <target>], unless the body opens with its own[channel:]redirect (the core's redirect wins) or carries[dm-only](owner-DM shape)..to-<channel>isproactive_routing.py's existing grammar. A header resolver lets a plugin rebuild the origin from a task file, so a room result still posts after a voice-agent restart.channel_kindjoinsKNOWN_HEADER_KEYSand its two mirrors; the body guard defangs a forged one.Core files name no product:
grep -ciE 'ag2 ?space|matrix|\.to-ag2space|source_room_id'gives the same counts onmainand on this head fortask-bridge.ts,voice-context.ts,voice-agent-config.ts,voice-agent.tsandinline-tools.ts.skills/ag2space-voice/(optional, AG2 Space only)session-context.ts: parses the client'ssession.contextv1 frame (room_id,room_name,surface, additivecapabilities), validates the Matrix room id, sendssession.context.ack.room-binding.ts: asks the gateway bridge for a membership verdict throughstate/voice-room-checks/and only then sets the origin. A frame naming a different room releases the bound one before the wait, so a task spoken during the wait is a DM task; a refused move reads asleftonce, a verified one asentered, a duplicate as nothing; a newer frame supersedes a pending one. It suppliesverifyso the room is re-checked when the result is delivered.navigate.ts,navigate-protocol.ts: thenavigate_uitool, its prompt rule and theui.navigate/ui.navigatedframes. Declared only when the install has the gateway channel, and the frame is sent only to a client that announcedui.navigate.Gateway bridge (the AG2 Space adapter,
src/remote-gateway-bridge.py)src/voice_room_membership.pyis the bridge's verifier (its only production importer); the docstring says so. Verdicts are cached per room, agent and owner for 60 s, at most 256 entries, fail closed.proactive-result-*.to-ag2space.txtfiles with a[channel:]room line. Untagged files are never held. A tagged file that cannot be decoded is held, not raised.VOICE_OWNER_RETRY_S). With a held file and an unusable/v1/agentsanswer, 30 scans made 30 reads before and 6 after.Review findings, 2026-09-21
main. Tagged.to-<channel>files are still claimed at once. The archive timer starts only after the forward is written; a failed forward releases the claim.tests/task-bridge-voice-offline-reconnect.test.ts: 3 pass, 2 fail with the old behaviour restorednavigate_uideclared on every install and surfaceinlineToolsandownerOnlyTools, out of the core prompt, never pushed to the phone server. Now a skill contribution gated at exposure; messages name no productmain's text byte for byte;pickLanguageand its test are gone. It returns as its own PR with a stricter reader and an opt-in[channel:]redirect is written as is. Tested against the realparse_markersproactive-result-*injectSilentContextpromised silenceturnCompleteflag; the docblock now says what it does. Behaviour unchangedroom-binding.tswith its testsweb-voice-transport.tsforce a desktop re-vendorgit diff origin/main -- src/web-voice-transport.ts src/web-client.ts tests/web-voice-transport.test.tsis empty. The frame literals live in the skill, with a parity test against the wire valuesvoice-navigate.tsat 194 linesBehaviour anchors
instructions_hash_fixed_env, every tool anchor and the greeting equalmain's. The one fixture change isregions.tool_table, which hashes the source lineconst mainAgentTools …invoice-agent.ts; that line now appends contributed tools.Tests at
d63aa431Downstream
.tsskill. The desktop needs a bundling step next toengine/private-plugins/build.shthat buildsskills/ag2space-voice/tools.tstoprivate-plugins/skills/ag2space-voice/tools.mjs, loaded throughSUTANDO_EXTERNAL_PLUGIN_DIRS. The bundle was built standalone and loads under plain node. No re-vendor ofweb-voice-transport.tsis needed.session.contextand answersui.navigatethrough its own transport'ssendClientCommand/onProtocolMessage.?room_id=again, as onmain.Live path
The earlier post-restart round trips (comments from 2026-09-18 and 2026-09-21) ran code that has since moved into the skill, so they do not cover this head. A post-restart round trip at the final head is posted as a comment when it has run; this body does not claim one.
🤖 Generated with Claude Code