Repository navigation
Conversation
- blaze_stt_python: stream to wss /v1/stt/realtime (stt-stream-1.5), laid out like xai_asr_python with reconnect manager, finalize via trailing silence, timeline-mapped timestamps and session metadata - blaze_tts_python: stream PCM from wss /v1/tts/realtime (2.0-realtime) on AsyncTTS2BaseExtension, laid out like deepgram_tts, with preheat, cancel, TTFB metrics and fatal auth errors - use base_url for the endpoint param and add standard guarder configs and unit tests against in-process mock servers - add voice_assistant_blaze graph (Vietnamese STT, LLM prompt and TTS) and Blaze variables to .env.example
Findings[P1] Do not reconnect from the receive task that must be closed [P1] Report the ASR connection state transitions [P1] Finalize audio even when Blaze has not emitted a partial [P1] Preserve finalize request identity and serialize concurrent finalizes [P2] Drain retained audio when the connection becomes ready [P2] Record user audio only after the WebSocket send succeeds [P2] Measure the no-audio end interval from the actual start event [P2] Add the required independent Blaze TTFB hook [P2] Cover TTS terminal accounting and flush at the extension boundary ASR design review
Review is based on the PR diff and checked-out base guidance; I did not execute code or install dependencies from the PR head. |
blaze_stt_python, following the ASR design review guide and the long-lived WebSocket pattern of fish_asr_python: - run reconnect on its own task with capped backoff and unlimited retries; a socket close no longer cancels its own recovery - report connected after Blaze's ready message and disconnected on every connect failure or close; retries go through start_connection() - ignore callbacks from replaced sockets with a connection generation; a normal close emits no error; auth failures stay fatal and final - finalize on audio sent since the last final, serialize requests and echo each request's finalize_id and session in asr_finalize_end - drain kept audio when a new socket is ready, with per-frame metadata - add user audio to the timeline only after a successful send - add connection status, reconnect lifecycle and audio timeline tests, including recovery from a real server-side socket close blaze_tts_python: - measure the start-to-end interval from the emitted start event when a request ends without audio - add deterministic extension tests for success, partial-audio error, no-audio error and flush accounting - register a Blaze TTFB observer in the TTS guarder reference table
Findings[P1] Preserve newer audio when an earlier final arrives [P1] Redact credentials embedded in configured URLs [P2] Preserve buffered frame order after a failed drain [P2] Ignore empty binary frames when starting TTFB [P2] Cover interrupted accounting and connection cancellation at the extension boundary ASR design review
Review based on the current PR head and repository guidance; I did not execute or install code from the PR head. |
The TTS guarder found that a missing dump directory made the dump flush raise after tts_audio_end, skipping finish_request(); every later request then queued behind it. Create the dump directory and log dump flush errors instead of letting them escape finalization.
|
Thanks for the review. All findings are addressed in 1a96722, plus 8654c65 for an issue the TTS guarder found. [P1] Reconnect from the receive task. Fixed. Recovery runs on its own task ( [P1] Connection state transitions. Fixed. [P1] Finalize without a prior partial. Fixed. Pending input is now tracked as user audio sent since the last final, independent of partials ( [P1] Finalize identity and concurrency. Fixed. [P2] Drain retained audio on readiness. Fixed. Kept frames are drained right after the handshake, each with its own frame metadata, while [P2] Timeline after a successful send. Fixed. [P2] TTS no-audio interval. Fixed. When a request ends without audio, the start boundary is set when [P2] Blaze TTFB hook. Added [P2] TTS terminal accounting tests. Added Results
Blaze does not support Chinese, so there is no |
|
[P1] Bound the entire STT readiness handshake The timeout is restarted for every message. If a connected server sends progress, unknown, or binary messages more frequently than the timeout but never sends [P2] Treat HTTP authentication failures as fatal on STT connect A WebSocket upgrade rejected with HTTP 401 or 403 raises ASR design review
The TTS-specific issues and test gaps from the preceding review remain applicable at this head. Review based on head |
blaze_stt_python: - track pending speech with a watermark on the vendor stream clock, so a late final for an earlier utterance cannot acknowledge newer speech - keep sending real-time silence while finalize waits on an idle stream, since Blaze endpointing only advances while audio arrives - bound the whole handshake with one deadline; finalize no longer waits on a handshake and stop aborts one in progress - treat HTTP 401/403 on the upgrade as fatal with the real status - keep a failed buffered frame first so drained audio stays in order, and mark a closed client unusable immediately so recovery runs blaze_tts_python: - start TTFB and audio only on the first non-empty, whole-sample PCM - finalize an interrupted request before closing the socket, so the closed socket cannot turn it into an error both: - reject base_url values with credentials, a query or a fragment, without echoing them - add tests for each case, including real-client cancellation while waiting for the first byte and while reconnecting
|
Thanks for the follow-up reviews. Both rounds of findings are addressed in b72f30d. [P1] Delayed final acknowledging newer audio. Fixed with a per-utterance watermark on Blaze's stream clock (user audio plus injected silence, the clock [P1] Credentials in configured URLs. [P1] Unbounded readiness handshake. One deadline now covers connect, auth, and [P2] HTTP 401/403 on the upgrade. [P2] Failed drain order. A failed send now keeps that frame first, followed by the rest in order, with [P2] Empty binary frames and TTFB. TTFB and audio start now fire only on the first non-empty, whole-sample PCM payload ( [P2] Interrupted accounting and real-client cancellation. The flush test now asserts the interrupted request's exact Also found by the live check. On some runs Blaze needed more than the initial 800 ms of finalize silence. Once the input stream went idle its endpointing stopped advancing and no final arrived. While finalize waits on an idle stream, the extension now keeps sending real-time silence in 100 ms steps, and adds nothing while user audio still flows ( Results
|
|
[P1] Cancel the in-progress WebSocket open, not just the current socket If a flush arrives while [P1] Validate or redact the URL before logging the TTS configuration
[P1] Preserve session ownership when draining buffered ASR audio (merge blocker) When an outage buffers frames from sessions A and B, the drain sends both into the same vendor stream and repeatedly overwrites [P1] Do not let the coverage tolerance acknowledge new speech (merge blocker) After a final ending at 1000 ms, a new voiced frame ending at 1300 ms is treated as already covered because it falls inside the 400 ms tolerance. If finalize arrives before Blaze's first partial for that new utterance, [P2] Cover interleaved requests and replacement-connection cancellation The extension-local suite covers sequential requests and appended fragments, but not rapidly interleaved request IDs or concurrent cancellation against a replacement connection. Those are mandatory TTS transport and request-ownership cases, and the implementation has shared request and socket state. Add deterministic tests that interleave fragments for two IDs and hold an old connection open while cancellation and a new request overlap, asserting audio ownership, exactly-once ends, and that the replacement remains usable. ASR checklist
|
- stt: keep-buffer holds one session; a newer session supersedes kept audio - stt: label results by the session that owned that span of the stream - stt: end-time tolerance only covers speech sent before the final arrived - stt: abort a stalled handshake instead of waiting for close_timeout - tts: cancel invalidates a socket still opening or reconnecting - tts: a cancelled stream only discards its own socket - tts: validate config before logging; redact url parts and secret params
|
Thanks for the third review. All findings are addressed in 9e69c42. [P1] TTS flush while the socket is opening. While writing the overlap test I found a related bug. A cancelled stream that hit [P1] TTS config logged before validation. [P1] STT drain across sessions. Two parts:
Tests check the emitted JSON: [P1] Tolerance acknowledging new short speech. The 400 ms end-time tolerance now applies only to voiced frames sent before the latest final arrived. Voiced audio sent after a final is tracked separately and stays pending until a final's [P2] TTS coverage. Added Also fixed. Results
|
|
[P1] Keep different sessions out of the same Blaze utterance (merge blocker) The single-session policy only applies while frames are buffered. When connected, audio for session B is sent on the same Blaze stream immediately after session A, even if A has not finalized. Blaze can return one final spanning both sessions; assigning its [P1] Do not let a delayed final cover a short newer utterance (merge blocker) If A's final ending at 100 ms is delayed until after B's voiced 100–200 ms frame was sent, [P1] Do not classify low-level speech as an already finalized turn (merge blocker) The only indication of pending audio before a partial is the fixed mean-amplitude threshold of 300. For valid quiet PCM below that threshold, a finalize arriving before Blaze's first partial takes the no-pending-speech branch, sends no endpointing silence, and emits [P1] Serialize audio emission with the TTS flush boundary With dumping enabled, flush can run while [P2] Attribute partials to the active utterance, not the previous final After session A's final ends at 100 ms, endpointing silence can advance the stream to 900 ms before session B starts. The first B partial has no timestamp, but this code looks up the owner at [P2] Classify transient preheat failures as recoverable An HTTP 429 or 5xx during the WebSocket upgrade becomes ASR checklist
Reviewed against head |
Summary
Rebuilds the Blaze (blaze.vn) Vietnamese STT and TTS extensions on Blaze's realtime WebSocket APIs and aligns them with the standard extension layout. The previous versions used the batch HTTP APIs (job submit and download), which added several seconds per turn.
blaze_stt_pythonStructured like
xai_asr_python:config.py,const.py,recognition.py(WebSocket client),reconnect_manager.py,extension.pywss://api.blaze.vn/v1/stt/realtime(stt-stream-1.5)start_ms/duration_ms,metadata.session_idandvi-VN/en-USlanguage codesasr_finalize_endblaze_tts_pythonStructured like
deepgram_tts, onAsyncTTS2BaseExtension:config.py,blaze_tts.py(WebSocket client),extension.pywss://api.blaze.vn/v1/tts/realtime(2.0-realtime) over one reused socket, preheated at startShared
base_url, as in other extensions; all properties are declared in the manifestsvoice_assistant_blazegraph in the voice-assistant example: Vietnamese STT, a Vietnamese LLM prompt and greeting, and Vietnamese TTS.env.exampleTesting
tests/bin/start) inten_agent_build:0.7.14: STT 75 passed, TTS 46 passedblack --line-length 80clean;pylintshows only the base-class attribute warnings that other extensions (for examplefish_asr_python,deepgram_tts) also gettest_multi_language, which needsproperty_zh.jsontest_sequential_requests_ttfbpasses with the new Blaze reference.voice_assistant_blaze: Vietnamese speech in, Vietnamese answer outASR design review
config:after redactionstart_connection()reports connected only after vendor handshaketest_multi_languageneeds Chinese, which Blaze does not supportblackformat andpylintchecks passNotes
property_zh.jsonand the guarder's multi-language case does not apply.ttfb_hooks/blaze.py) to the TTFB reference table, with its own tests.