Repository navigation
README: emphasize remote control in video caption - #32
Merged
Merged
Conversation
Per Susan's feedback — highlight that user controls Mac from phone. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
4 tasks
liususan091219
pushed a commit
to liususan091219/sutando-fork
that referenced
this pull request
May 28, 2026
Per Susan's feedback — highlight that user controls Mac from phone. Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
yixuan-ag2
added a commit
that referenced
this pull request
Aug 18, 2026
… surfaces) (#3131) Picks up the five merged bodhi PRs the voice context-budget design depends on (sonichi/bodhi_realtime_agent #33, #34, #31, #30, #32): - upstream send counters + getDiagnostics() (attempted/queued/skipped/threw, split raw/wire bytes, per-outcome timestamps; reset per transport generation) - typed connection-lifecycle events (attempt / setup-ok / setup-failed / attempt-close / generation-close, correlated by connectAttemptId) - onUsageMetadata (promptTokenCount et al), reachable from VoiceSession - mediaResolution passthrough (session-wide media token cost) - optional compressionConfig thresholds ({} = server defaults), sent as the int64-over-JSON strings the API expects Plus review hardening from the merge train: observer isolation on both new callback surfaces, reconnect replay counted AND moved off the deprecated `media` field onto the supported video/audio slots, and superseded-dial setup-failed fenced like every other stale-dial signal. Pin only; no sutando code change. Consumers land separately (Phase 1: ledger sampling, logicalSessionId, up=/ctx= health segments, shadow matrix). Evidence: Full TS suite at this pin: 1096 pass / 0 fail / 2 skipped. Live round trip through the PINNED artifact (not the SDK directly) against gemini-2.5-flash-native-audio-preview-12-2025 — connect with mediaResolution=LOW + compressionConfig:{} accepted; text in -> model audio out; every new surface read while doing it: connected in 304ms; lifecycle so far: attempt -> setup-ok text slot : {"attempted":1,"queued":1,"queuedRawBytes":26} generation : 1 audio chunks : 1 (first after 1442ms) usage samples : 1 promptTokenCount 375 -> 375 lifecycle full: attempt -> setup-ok -> generation-close LIVE ROUND TRIP: PASS Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
yixuan-ag2
added a commit
that referenced
this pull request
Aug 18, 2026
…ow matrix (#3132) * chore(deps): bump bodhi pin ca0f61de -> 0d506ead (B1-B5 observability surfaces) Picks up the five merged bodhi PRs the voice context-budget design depends on (sonichi/bodhi_realtime_agent #33, #34, #31, #30, #32): - upstream send counters + getDiagnostics() (attempted/queued/skipped/threw, split raw/wire bytes, per-outcome timestamps; reset per transport generation) - typed connection-lifecycle events (attempt / setup-ok / setup-failed / attempt-close / generation-close, correlated by connectAttemptId) - onUsageMetadata (promptTokenCount et al), reachable from VoiceSession - mediaResolution passthrough (session-wide media token cost) - optional compressionConfig thresholds ({} = server defaults), sent as the int64-over-JSON strings the API expects Plus review hardening from the merge train: observer isolation on both new callback surfaces, reconnect replay counted AND moved off the deprecated `media` field onto the supported video/audio slots, and superseded-dial setup-failed fenced like every other stale-dial signal. Pin only; no sutando code change. Consumers land separately (Phase 1: ledger sampling, logicalSessionId, up=/ctx= health segments, shadow matrix). Evidence: Full TS suite at this pin: 1096 pass / 0 fail / 2 skipped. Live round trip through the PINNED artifact (not the SDK directly) against gemini-2.5-flash-native-audio-preview-12-2025 — connect with mediaResolution=LOW + compressionConfig:{} accepted; text in -> model audio out; every new surface read while doing it: connected in 304ms; lifecycle so far: attempt -> setup-ok text slot : {"attempted":1,"queued":1,"queuedRawBytes":26} generation : 1 audio chunks : 1 (first after 1442ms) usage samples : 1 promptTokenCount 375 -> 375 lifecycle full: attempt -> setup-ok -> generation-close LIVE ROUND TRIP: PASS Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat(voice): P7 Tranche B — upstream ledger, lineage, ctx gauge, shadow matrix The health matrix's most common verdict on 2026-08-17 was insufficient-evidence (speech-without-model-event; upstream-hop-unobserved) — 28 times in one day. The right verdict, and a useless one: up=n/a on every line because the agent→Gemini hop had no counters. The bumped bodhi pin exposes them; this wires the sutando half, dark. Ledger (voice-audio-health.ts): - coverage becomes an ordered LedgerCoverage level with observesUpstreamSend(); the live snapshot stays 'session-only'. - samples bodhi's VoiceSession.getDiagnostics() on ledger ticks only (never the frame path; §D7.0b untouched). Absent or throwing ⇒ up=n/a — null means unobserved, never zero. - [Health] renders up={aQ/s,aKB/s,vQ/s,vKB/s,drop,buf:n/a} from per-tick deltas; a generation change renders that tick from zero rather than a negative rate. ctx={tok,age} appears once usage metadata is seen. - logicalSessionId/lineageState derived sutando-side from lifecycle events (design §1.1): minted on a handle-less setup-ok; resumed keeps it; suspected-sever (setup-failed with a handle, or handle-accepted 1008 close) is TERMINAL — only a fresh lineage exits it, by minting. Context occupancy belongs to the lineage and resets with it. - persisted rows (full and reduced) carry every evaluator input: counters, per-outcome timestamps, generation, lineage, contextTokens, echoSuppressed. Matrix (voice-health-matrix.ts): - structural guard BEFORE every delta row: a transport-generation change inside the window is insufficient-evidence — upstream counters reset per generation, so row 5's timestamps are as unusable across it as row 4's deltas. - row 4 upstream-send-dead: eligible ingress, zero attempts — a dead call path. Row 4b upstream-send-failing: attempts, zero queued, named by guard outcome. Both exclude echo-suppressed windows (suppression returns before sendAudio and would masquerade as a dead path). - row 5 renamed model-silent → post-sdk-silent, reachable only through the lossless-window proof: dAttempted >= dDelivered ∧ dQueued == dAttempted ∧ lastQueuedAt >= speech.lastAboveFloorAt. Aggregate counters cannot say WHICH frame queued; losslessness proves the speech-bearing ones necessarily did. A lossy window stays insufficient-evidence — at aggregate resolution it genuinely cannot distinguish "model went quiet" from "we dropped the speech". voice-agent.ts: - onUsageMetadata/onConnectionLifecycle wired into the VoiceSession config; getSessionDiagnostics injected into the ledger. - a SHADOW evaluator runs the 'session+egress' ruleset every health tick and logs [Matrix-shadow] on verdict change, telemetry-only. Canary 3a later compares live vs shadow on identical input; without the shadow, 3a has no baseline. The live verdict stream is byte-for-byte unchanged. - the [Health] line now carries vision={sent,defB,defG,disp} while streaming — whether the vision throttle engaged was previously unknowable from the log (deferredBudget existed but was never surfaced; investigation F2). Tests: 7 new ledger cases (lineage state machine incl. terminal suspected-sever, ctx latch+reset, up= rendering, generation-reset rendering, n/a on unobserved/throwing, replayable persisted row) and 8 new/updated matrix cases (rows 4/4b, structural guard, lossless/lossy/order-unproven, echo exclusion, live-at-session-only unchanged, row-5 rename). Full suite: 1109 pass / 0 fail / 2 skipped. tsc clean. prose-cap PASS. Stacked on voice/bodhi-pin-b1-b5 (#3131); merge order pin → this. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(voice): P7 Tranche B round 2 — seven codex findings on #3132, all confirmed - Matrix: an unobserved→observed diagnostics transition could FABRICATE post-sdk-silent — the generation guard fired only when both generations were non-null, while unobserved counters baselined as placeholder zeros, so generation-lifetime totals diffed as one window (the review repro'd it). Upstream deltas now require a valid window: observed baseline, same generation. Rows 4/4b gate on it; row 5 downgrades to insufficient-evidence 'upstream-baseline-unobserved'. - Matrix: MatrixResult gains the §1.5 facts — attemptedAudioAdvanced, queuedAudioAdvanced, losslessWindowWithSpeechQueued, transportGenerationChanged, echoSuppressedAdvanced — computed before any row can return, so every result carries an honest set. The watchdog integration reads facts, never verdict strings; verdict-independence needs them to exist. - Matrix: row 4b named the guard outcome from generation-lifetime totals, letting earlier skips masquerade as the current failure mode. The baseline now carries skippedNoSession/threw; reasons are window deltas. - Ledger: up= rendered cumulative totals over one dt after a missed diagnostics sample (900 queues → "30.0/s" out of thin air). A tick with no adjacent observed baseline renders up=n/a(baselining) and establishes the window; rates resume next tick. A generation change WITH an adjacent baseline still renders from zero — the new generation's counters span at most the window. - Ledger: §1.1's generation→lineage reconciliation records now exist — every lineage commit/re-label persists a reason:'lineage' row carrying connectAttemptId, transportGeneration, resolved logicalSessionId, final state, so offline analysis re-attributes earlier rows instead of trusting a stale in-row id. A failed handle-less dial commits nothing and emits nothing. - Ledger: the reduced persistence row was NOT replayable — flattened upstream fields, missing speech/heartbeat/clientTotals/lastModelEventAt. It now carries every matrix-evaluator input SNAPSHOT-SHAPED (§1.6 option 1) plus the recorded decision: verdict, facts, reasons (§1.6 option 2). A row still over budget strips only the episode list and says so. Replay is pinned: the evaluator produces an identical baseline, verdict, and facts from the full and reduced forms of the same tick. - voice-agent: promptTokensDetails now reaches the ledger and both persistence payloads — §1.4's modality split is what separates "video filled the context" from "conversation did". Held and reset per lineage like the scalar it details. Tests: 9 new/extended cases pin the two review repros (null→observed fabrication, missed-sample rate), 4b window-delta naming, facts honesty at every coverage, reconciliation records, the modality breakdown, and full/reduced replay equivalence. Full suite green; tsc clean; eslint 0 errors (14 pre-existing warnings untouched). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(voice): matrix facts are data-level — coverage- and verdict-independent Codex round 3, two findings on the round-2 commit, both confirmed: 1. Facts depended on coverage and on the verdict path: identical diagnostics produced all-false upstream facts under 'session-only' but true facts under 'session+egress', and a CONNECTING tick returned before facts were derived. §1.5's obligation is the opposite — the watchdog reads facts and must behave identically under every coverage mode and verdict, so a future coverage flip must not change watchdog behaviour. Facts are now computed FIRST, from data validity alone (adjacent same-epoch baseline, both generations observed and equal): facts say what the data shows; coverage gates only which VERDICTS may be claimed. The live verdict stream is unchanged — facts never drive verdicts. 2. upstreamWindowValid required only the BASELINE generation to be non-null. A typed sample with counters present but a null CURRENT generation would diff counters across generations and could emit post-sdk-silent. The window now requires both ends observed; the row-5 downgrade reason is renamed 'upstream-baseline-unobserved' → 'upstream-window-unobserved' since it covers both directions. Tests: the session-only facts test inverted into the §1.5 pin (same input, same facts, different claim across coverage modes); observed→null direction pinned; a new verdict-independence case proves an early reconnect-window return carries the same facts as the ACTIVE evaluation of identical input. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs(voice): name why the clamp cannot hide a silent reset sonichi asked whether a counter reset without a generation bump would render aQ/s:0.0 indistinguishably from 'the agent stopped sending'. It cannot: at the pinned bodhi commit the reset and the generation bump are adjacent lines in one synchronous block, and each is the only such site in the file. Recording the commit so the next reader re-checks the coupling instead of re-deriving it. * docs(voice): the try/catch guards the TDZ, not the typeof qingyun-wu's review nit, verified: typeof on a let/const inside its temporal dead zone throws ReferenceError rather than yielding 'undefined', so the typeof check cannot be what makes a tick racing construction safe. Behaviour was already correct — only the comment credited the wrong mechanism. * fix(voice): an unobserved upstream baseline must not read as a real zero john-the-dev's review finding. makeBaseline collapsed the four upstream counters to 0 when `upstream` was absent, while upstreamWindowObserved inferred the previous tick's observedness from transportGeneration being non-null. Those are independent fields off the same diag, so a baseline with a real generation and absent counters passed the guard and the next tick diffed against placeholder zeros — turning lifetime totals into a "window" and letting losslessWindowWithSpeechQueued go true on a window that was never measured. The counters are now `number | null` with no `?? 0`, and the guard checks the counter itself rather than using generation-nullness as a proxy. Unobserved is now unrepresentable as zero instead of relying on two fields moving together. Defence-in-depth at the current pin, not a live bug: bodhi's getDiagnostics (0d506ead:575) returns upstream and transportGeneration unconditionally, and `upstream` is always an object — so the two cannot diverge today. That is an invariant held in another repo, which is exactly why the guard should not depend on it. Also amends the structural-guard comment: it is gated on observesUpstreamSend, so below session+egress row 5 is protected by its own honesty branch, not by that guard. Control: the added test fails without the fix (39/40) and passes with it (40/40). * fix(voice): echoSuppressed carries the same unobserved-vs-zero defect john-the-dev's follow-up to 2af342d: that fix made the four upstream counters nullable but left echoSuppressed on `?? 0`, which fires when the whole `diag` is null — reachable by construction, since sampleDiagnostics() returns null whenever getSessionDiagnostics is absent or throws, and the option's own doc says "Absent or throwing => upstream stays unobserved". So an unobserved tick recorded 0, and the next observed tick turned a generation-lifetime total into a one-window delta on echoSuppressedAdvanced, which the watchdog reads. Same violation of this module's stated contract ("Null fields mean UNOBSERVED ... never zero"), one field over; it escaped because the nullability lived in the container rather than the field. echoSuppressed is now `number | null` on both SessionDiagnosticsSample and AudioHealthSnapshot, the `?? 0` is gone, and dEchoSuppressed is gated on both ends being observed — the same shape as its siblings. Also drops a dead assertion they caught in the previous regression test: `r.facts.upstreamWindowObserved` is a local const, not a MatrixFacts member, so `?? false` made it unfailable. Replaced with two real members. Control: the added test fails without the fix (40/41) and passes with it (41/41). --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Highlights that user controls Mac entirely from a phone call.