[bassilkhilo-ag2]

What changed, and why?

The browser voice transport played Gemini's 24 kHz PCM through one AudioContext created with plain new AudioContext(), i.e. at the device rate (48 kHz on Windows). Each chunk became an AudioBuffer at 24 kHz inside a 48 kHz context, and Chromium/WebKit resample that case per AudioBufferSourceNode with linear interpolation, which mirrors every component above a few kHz to 24 kHz − f. Every "s"/"t" in the agent's speech therefore came with a 16–20 kHz image only ~15 dB below the sibilant. Listeners who can hear that range (two Windows-laptop testers so far) perceive it as a short "beep" a few seconds into every answer.

Fix: playback now runs on its own AudioContext created at the configured output rate (24 kHz by default), so buffer and context rates match and the only conversion left is the browser's band-limited device resampler. The capture (microphone) context and the D7.5 recovery FSM are untouched. If the browser rejects the rate, playback falls back to the device rate with one warn debug line; a one-time warn marks any remaining buffer/context rate mismatch; a session.config that renegotiates the output rate retires the playback context so the next chunk plays at the new rate; the audio_health heartbeat's playback-clock evidence now reads the playback context.

Surfaced while testing #1789 (Windows support) on Windows 11, but the code is identical on main (git diff origin/main..pr-1789 -- src/web-voice-transport.ts is empty). This PR is independent of #1789; #1789 inherits it on rebase.

Review first

1. src/web-voice-transport.ts — playbackCtx + createPlaybackContext() / retirePlaybackCtx(), the connect() block, playChunk(), teardownAudio(), the session.config branch, buildStats() / heartbeat x[0].
2. tests/web-voice-transport.test.ts — FakeAudioContext now honours { sampleRate } and can reject it; new describe "playback runs on a dedicated AudioContext at the stream rate" (12 tests); one existing ledger test now indexes created[1] because buffer sources live on the playback context.
3. CHANGELOG-PENDING.md.

What user behavior or bug does this prove?

Agent speech in the web client on a 48 kHz output device (every Windows machine; many Macs) no longer carries 16–20 kHz imaging on sibilants — the "beep" is gone. Measured below.

Feature/documentation consistency

N/A — no user-visible feature, command or config changes; docs/ does not describe the transport's audio-graph internals.

Before/after evidence

1. Live measurement — what the speakers played vs the PCM the server sent

Method: the voice agent's outgoing PCM was dumped to disk (24 kHz int16, per chunk, with arrival timestamps) and the default render endpoint was captured with a WASAPI loopback recorder (48 kHz) while talking to the agent in the web client. Each utterance's capture was aligned to its source PCM by cross-correlation; over all 50 ms frames where the source is a sibilant (>90 % of energy above 1.5 kHz), the median band energy relative to the frame total. Windows 11, Chromium 152 (Edge/Chrome), Realtek speakers.

===== BEFORE (parent 5ca8e629 transport, shared 48 kHz context) =====
utterance 10:11:30Z len=6.1s align corr=0.71, 12 sibilant frames (50 ms)
   4-8k   speakers played:   -2.0 dB   source:   -0.9 dB   (median re total, over sibilant frames)
   12-16k speakers played:  -15.2 dB   source:  -35.5 dB   (median re total, over sibilant frames)
   16-20k speakers played:  -14.7 dB   source:  -66.2 dB   (median re total, over sibilant frames)
===== AFTER (this commit, dedicated 24 kHz playback context) =====
utterance 10:53:21Z len=2.4s align corr=0.92, 4 sibilant frames (50 ms)
   4-8k   speakers played:   -2.8 dB   source:   -2.7 dB   (median re total, over sibilant frames)
   12-16k speakers played:  -86.0 dB   source:  -29.2 dB   (median re total, over sibilant frames)
   16-20k speakers played:  -85.6 dB   source:  -67.1 dB   (median re total, over sibilant frames)
utterance 10:53:32Z len=2.9s align corr=0.73, 9 sibilant frames (50 ms)
   4-8k   speakers played:   -2.0 dB   source:   -1.2 dB   (median re total, over sibilant frames)
   12-16k speakers played:  -83.1 dB   source:  -32.5 dB   (median re total, over sibilant frames)
   16-20k speakers played:  -83.4 dB   source:  -65.9 dB   (median re total, over sibilant frames)
utterance 10:53:36Z len=2.9s align corr=0.84, 12 sibilant frames (50 ms)
   4-8k   speakers played:   -1.5 dB   source:   -0.9 dB   (median re total, over sibilant frames)
   12-16k speakers played:  -78.8 dB   source:  -37.8 dB   (median re total, over sibilant frames)
   16-20k speakers played:  -78.2 dB   source:  -65.8 dB   (median re total, over sibilant frames)

Before: 16–20 kHz at −14.7 dB on the speakers while the source has −66 dB there (a simulation of 2× linear interpolation on the source reproduces the speaker spectrum within ~1 dB in every band, which is how the cause was pinned). After: −78 to −86 dB — below the source's own noise floor, because the browser's device resampler is band-limited.

Listening A/B on the same 200 ms sibilant region: the tester heard the "beep" in the speaker capture and not in the source PCM played directly.

2. Unit tests — fail at the parent source, pass at HEAD

$ git checkout origin/main -- src/web-voice-transport.ts   # parent source, HEAD tests
$ npx tsx --test --test-force-exit tests/web-voice-transport.test.ts
    not ok 7 - bounded fallback: a wedged self-close handshake resolves after disconnectCloseTimeoutMs
    not ok 1 - scheduled/ended/cancelled split: barge-in cancellations never count as completion
    not ok 1 - connect() creates the capture context (no options) first, then the playback context at 24 kHz
    not ok 2 - the playback context follows opts.outputRate
    not ok 3 - rate honoured: chunks are scheduled on the playback context and no mismatch line is emitted
    not ok 4 - rejected rate: no-arg fallback, one NotSupportedError warn, live, and ONE mismatch line across two chunks
    not ok 5 - disconnect() closes the playback context along with the capture context
    not ok 6 - a suspended playback context is resumed by the next chunk
    not ok 7 - a closed playback context is recreated by the next chunk, still at the stream rate
    not ok 8 - session.config with a new output rate retires the playback context; the next chunk plays at the new rate
    not ok 9 - stats report the playback clock, not the capture clock
    not ok 10 - session.config without an outputSampleRate keeps the playback context
    not ok 11 - connect() survives a playback context that cannot be created; the next chunk retries
    not ok 12 - session.config confirming the same output rate keeps the playback context
# tests 117
# pass 102
# fail 14

$ git checkout HEAD -- src/web-voice-transport.ts
$ npx tsx --test --test-force-exit tests/web-voice-transport.test.ts
    not ok 7 - bounded fallback: a wedged self-close handshake resolves after disconnectCloseTimeoutMs
# tests 117
# pass 115
# fail 1

The one remaining failure (bounded fallback: a wedged self-close handshake…) is pre-existing and Windows-environmental — it fails identically on the parent with the parent tests; CI on Linux/macOS is the authoritative signal for it.

What tests did you run?

• npx tsx --test --test-force-exit tests/web-voice-transport.test.ts tests/web-voice-transport-delivery.test.ts — see above (the delivery file's two serve-path subtests fail on this Windows host with spawn npx ENOENT, also pre-existing).
• npm run typecheck → tsc --noEmit exit 0.
• npx eslint src/web-voice-transport.ts tests/web-voice-transport.test.ts → 0 errors (pre-existing no-explicit-any warnings in the test file only).
• bash scripts/review-checks.sh --diff <(git diff origin/main...HEAD) → hardcoded-paths + root-artifacts clean (prose-cap had no in-scope .py files).

Live path — real post-restart round trip (voice, Windows 11)

The web client was restarted on the fixed transport (/web-voice-transport.js now serves createPlaybackContext), the page hard-reloaded, and the same math questions asked as in the "before" session. data/conversation.sqlite, voice table, session session_1789293804655:

ts                   kind   text
2026-09-13 10:53:34  user   What's 2 + 5?
2026-09-13 10:53:34  agent  2 + 5 is 7. Anything else I can add up for you?
2026-09-13 10:53:38  user   What's 5 * 5?
2026-09-13 10:53:38  agent  5 times 5 is 25. What's next?
2026-09-13 10:53:46  user   What's the next line?
2026-09-13 10:53:46  agent  9 times 9 is 81. Anything else you'd like to multiply, or maybe something else entirely?
2026-09-13 10:53:52  user   What's 82 * 82?
2026-09-13 10:53:52  agent  82 times 82 is 6724. Need help with any other math?
2026-09-13 10:54:01  user   Was 99 39?
2026-09-13 10:54:01  agent  99 times 99 is 9801. Are there any other numbers you want me to multiply?
2026-09-13 10:54:09  user   What's 20% of 22?
2026-09-13 10:54:09  agent  20% of 22 is 4 4 Need any other calculations?

The tester reported no beep in this session; the three "AFTER" utterances in the measurement above are from it. Mic capture, barge-in and the avatar analyser behaved as before.

Edge cases / non-happy paths checked

• Browser rejects { sampleRate } → no-arg fallback, one warn, session goes live, playback continues (interpolated, as today) with exactly one mismatch warn per attempt.
• Playback context born/turned suspended → resumed on the next chunk; closed → recreated at the stream rate on the next chunk.
• session.config with a different outputSampleRate while chunks are draining → sources flushed, old context closed, analyser re-handed to the surface on the next chunk; same rate or absent rate → context kept (a device-rate fallback context is not churned by a confirming config).
• Playback context creation throwing on both forms in connect() → attempt still goes live with playbackCtx = null; playChunk retries lazily.
• disconnect() / teardown closes both contexts; the D7.5 fence ordering for the capture context is unchanged.

Worst-case disruption to existing users, and mitigation

• One extra AudioContext per voice session (capture + playback + the page's tool-cue context). Created inside the same user-gesture call stack as before; iOS/WebKit context limits are far above three.
• A browser/device that rejects a 24 kHz context gets exactly today's behaviour plus one debug line — no functional regression is possible on that path.
• The avatar's speaking ring reads an AnalyserNode that now lives on a 24 kHz context (bins span 0–12 kHz instead of 0–24 kHz), so its fixed threshold may trigger slightly more eagerly — cosmetic; looked normal in the live session.
• Residual, pre-existing and out of scope: a speech_speed ≠ 1.0 (playbackRate ≠ 1) still makes the browser interpolate per chunk; the measurement above is at unity speed.
• WebKit caveat: the lazy re-creation after a renegotiated rate happens outside a user gesture; Chromium's sticky activation covers it, WebKit was not exercised (the default 24 kHz path never hits it).
• No migrations, config, permissions or on-disk state. Rollback = revert the commit.

Known gaps / follow-ups

• The tool-invocation cue (feat(voice): audible tool-invocation cue (research vs core vs inline, by pitch) #2042) deliberately plays short 500–1080 Hz blips on every tool/core call; testers who report "beeps" may be hearing that feature as well. It is unrelated to this fix and untouched here; an opt-out would be a separate PR.
• The fallback warn in createPlaybackContext() is not latched per attempt (only reachable if both constructor forms throw repeatedly).

🤖 Generated with Claude Code

https://claude.ai/code/session_01TqRPjsrR2kmfcWwX1uAb8H

[github-actions]

Coverage Gate

✅ No Python changes — nothing to gate (bar: 95% on changed lines).

[sonichi]

Reviewed at head 16c87b0b, full diff + isolated worktree, nobody else has reviewed this yet.

The bug and the fix are real. Playback was sharing the capture AudioContext, which runs at the device's default rate (typically 48kHz) — but the agent's audio stream is 24kHz (or whatever the session negotiates). A rate mismatch between the buffer and the context makes the browser linearly interpolate every chunk, which is audible pitch/quality degradation, not a crash. The fix creates a dedicated playbackCtx at the stream's own rate, with a fallback to the device rate if the browser rejects the requested rate (NotSupportedError), and retires/recreates it if the server renegotiates the output rate mid-session (session.config).

Verified myself, not just trusting the tests:

• Read the full diff on both src/web-voice-transport.ts and the test file.
• Confirmed the analyser (visualizer feed) is correctly rewired to the new playbackCtx inside playChunk, not left pointing at the old shared context — this was the thing I was most worried would be silently broken by the split.
• Confirmed adoptCtx's state-change recovery (handleCtxStateChange) is scoped to audioCtx only (if (ctx !== this.audioCtx) return) and deliberately not extended to playbackCtx — reasonable scope boundary for this PR: losing playback is recoverable by the next chunk (suspended→resume, closed→recreate, both tested), losing capture breaks the conversation and already has its own recovery FSM.
• Ran the test suite myself: 117 tests, 116 pass, 1 pre-existing conditional skip (needs --expose-gc, unrelated to this change). The new describe block (12 tests) covers rate-honored, rejected-rate fallback with the fallback itself still visible in the debug trace, disconnect cleanup of both contexts, suspended/closed recreation, rate-renegotiation retiring and recreating the context (with a fresh analyser), stats reporting the playback clock not the capture clock, and a double-failure case (both capture and playback context creation fail once, then recover). These are genuine failure-mode tests, not happy-path only.
• git diff --check clean, CI 19/19 green, CLA signed, no host-specific paths.

Worst-case disruption: this can only improve the existing behavior (mismatched-rate interpolation happens today, unconditionally) — the fallback path when the browser rejects the requested rate reproduces today's exact behavior (device-rate context), so there's no new failure mode, only a removed one.

I RECOMMEND APPROVE #4240 — Bassil Khilo (bassilkhilo-ag2), fixing a real audio-quality bug in voice playback with a well-scoped, well-tested change.

Not casting a formal review (needs current owner authorization). No merge, approval, or restart action taken.

Stand: Echo Act IV Pro

[john-the-dev]

APPROVE at 16c87b0bd — first review on this. The diagnosis is right, the fix is the correct one, and the reasoning in the body is the kind I wish more audio bugs got. One finding below that I think matters more than its size suggests.

The diagnosis holds

A 24 kHz AudioBuffer played in a 48 kHz context is resampled per AudioBufferSourceNode by linear interpolation, and linear interpolation without an anti-imaging filter leaves images at fs − f. For a 24 kHz source that puts a sibilant's 4–8 kHz energy back at 16–20 kHz — audible to anyone whose hearing reaches there, inaudible to everyone else, which is exactly the shape of "two Windows testers hear a beep and nobody else does." Matching the context rate to the buffer rate removes the interpolation entirely rather than masking it.

ctx.createBuffer(1, f32.length, this.outputRate) inside a context constructed at outputRate is the whole fix, and the rest is the care around it: a constructor fallback with a warn, a one-time mismatch warn if it still diverges, retirePlaybackCtx() on a renegotiated session.config, and closing the new context on teardown. The analyser follows playback correctly — playChunk builds it against ctx and retirePlaybackCtx nulls it so it is rebuilt against the successor.

The finding: playback no longer has a state-change handler

adoptCtx() installs onstatechange, and handleCtxStateChange opens with:

if (ctx !== this.audioCtx) return; // stale context

Both adoptCtx call sites pass the capture context — :829 and :1461 (const ctx = this.audioCtx!). playbackCtx is never adopted. So once playback moves to its own context:

• ctxSuspendCount and ctxLastTransition no longer observe the playback context at all.
• The only thing that notices a suspended playback context is if (ctx.state === 'suspended') ctx.resume(); at the top of playChunk — which runs only when the next chunk arrives, and is not awaited.

Before this PR playback shared audioCtx, which was adopted, so a suspension on the context carrying playback was counted. That coverage is gone, and it is gone for the failure mode this PR is adjacent to: playback going silent.

I checked the suite — the suspension tests drive FakeAudioContext.created[0], which is the capture context, so nothing exercises a suspended playback context.

Why I am not blocking: the user-facing bug is real and this fixes it, the gap is observability rather than correctness, and holding a working audio fix for missing telemetry would be the wrong trade. But this.adoptCtx(this.playbackCtx) at creation, plus widening that identity check to accept either context and tagging which one transitioned, is small and worth doing soon.

Sharper half: the stats object now spans two contexts, labelled as one

ctxState:  this.audioCtx?.state ?? null,        // capture
ctxTimeMs: this.playbackCtx ? … : null,         // playback   <- changed

The heartbeat legend was updated for one of these (x [playback ctxTimeMs…]) and not the other — cs still reads "ctx state initial", and sc ctxSuspendCount is now capture-only without saying so.

That is a trap for whoever debugs the next audio incident: cs=r beside a stalled x[0] now means "capture is running and playback is not", which reads as a contradiction rather than a diagnosis. And "capture suspended while playback runs" has become a normal state that the field names do not admit. Renaming to captureCtxState / playbackCtxTimeMs, or reporting both states, would cost a line and remove the ambiguity permanently.

CI: 18 check-runs SUCCESS, 1 entry without a conclusion (license/cla, which reports on commits/<sha>/status rather than the rollup — worth confirming there rather than reading the rollup as pending).

[github-actions]

@cla-assistant check

[john-the-dev]

APPROVE — re-affirming at the current head. Stale by timestamp; the content is unchanged.

18 commits arrived since my approval, which looks alarming until you separate the PR's own work from main flowing in. I fingerprinted this PR's own diff (merge-base → head) at both heads:

• src/web-voice-transport.ts — identical
• tests/web-voice-transport.test.ts — identical
• CHANGELOG-PENDING.md — flagged as changed by my first pass, then not: the added line is byte-identical, and only the surrounding context lines moved because other entries landed above it from main.

Worth recording that correction, since it is a trap for anyone doing this check: hashing the whole patch counts context drift as a content change. The +/- lines are the right granularity, and by that measure this PR has not moved since I read it.

Everything else in those 18 commits is main — #4230, #4235, #4238, #4244, #4245, #4248, #4249, #4252, #4253, #4131 and friends, which I reviewed individually.

My original finding stands and is still non-blocking: playbackCtx is never passed to adoptCtx, so its suspensions are invisible to ctxSuspendCount/ctxLastTransition, and the stats object now reports capture state beside playback clock under one legend.

[sonichi]

Re-checked at the new head 3204576ec (a pure merge from main; my prior review was at 16c87b0b, before it).

git diff origin/main...HEAD is unchanged in substance from what I reviewed before (CHANGELOG-PENDING.md, src/web-voice-transport.ts, tests/web-voice-transport.test.ts — same 3 files, same shape). Re-ran the full test suite in an isolated worktree at this exact head: 116/117 pass (1 skip needs --expose-gc), and the control still holds — reverting src/web-voice-transport.ts to the origin/main version while keeping these tests fails session.config confirming the same output rate keeps the playback context (1 !== 2), so the new coverage exercises a real behavior change, not a tautology. scripts/review-checks.sh --diff clean (no hardcoded paths).

Nothing material moved since my first pass. Verdict stands: ready to merge on the bar (currently 1/2 standing approvals — @john-the-dev's — short by one).

[yixuan-ag2]

Read the diff and the surrounding transport at 3204576ec205. The change is right, and the part I went looking for a bug in is the part that is best covered.

What I actually checked, rather than the diff's own story. Splitting one AudioContext into two retires an invariant the old code stated out loud — "playback and capture share one clock" — so my question was what else depended on it.

• The analyser follows playback, correctly. It is created inside playChunk against ctx (now playbackCtx), connected to ctx.destination, and re-handed through this.ev.onAnalyser?.(...) on every recreation. It visualises the agent's audio, so playback is where it belongs, and retirePlaybackCtx() nulls it so the next chunk rebuilds it against the new context.
• Stats. it('stats report the playback clock, not the capture clock') — that is the consequence of the split I would most expect to be missed, and it is the one you tested.
• Teardown symmetry. I suspected the retire path flushed while the disconnect path did not, since only retirePlaybackCtx names flushPlayback() in the diff. Wrong: the teardown calls this.flushPlayback() before closing either context, so the cancelled/D7.1 bookkeeping holds on both paths. Checking beat publishing the suspicion.

The lazy-recreate path is genuinely lazy in the right places. connect() swallows a construction failure and leaves playbackCtx = null for playChunk to retry; createPlaybackContext() falls back to the device rate on NotSupportedError and says which happened; rateMismatchLogged latches so a degraded session warns once rather than per chunk. The test asserting one mismatch line across two chunks is what makes that latch a property rather than an intention.

Pair I'd point at if anyone reads only one thing: session.config with a new output rate retires the playback context alongside session.config confirming the same output rate keeps the playback context. A retire-on-config test alone would pass for code that retired unconditionally; the same-rate case is what makes if (this.outputRate !== prevOutputRate) load-bearing.

pr-checks.py rc 0 at this head: 18 distinct checks from 18 entries, 0 superseded, nothing failing, nothing running.

Scope, so this is not read as more than it is: I read web-voice-transport.ts and the new tests. I did not run the suite and did not exercise browser audio — every claim above is from the tree, and a rate-interpolation fix ultimately wants an ear on it, which I do not have.