[0xShug0]

Summary

• Use the first generated chunk's audio codes and transcript to condition subsequent chunks in reference-free generation.
• Register that generated reference with the existing prepared-reference cache, allowing subsequent chunks to reuse its KV prefix. The first reference-conditioned chunk populates the KV prefix; later chunks can reuse it when capacity permits.
• Keep generated conditioning local to the request. Explicit voice cloning and single-chunk requests retain their existing behavior.
• Applies to offline and streaming generation; only model-local session code and its guide change.

Reproducer

Published Higgs Audio v3 TTS 4B Q8 GGUF, CPU, 12 threads, seed 42, text_chunk_size=40, no reference audio:

The lighthouse shines across the bay. Tonight the sea is calm and clear. We will sail home before sunrise.

This produces three chunks. Nemotron diarization detects two speakers on the unpatched base, with a change at the first chunk boundary (2.56 s), versus one speaker with anchoring. The default chunk budget is 1,024 characters, so this short reproducer requires the override. This is not a fix for reference-audio cloning.

Validation

Debug builds, existing GGUF without a spec override, fixed seed, sequential inference.

• CPU and CUDA A–B–A sessions: repeated A outputs are byte-identical within each version; the single-chunk B control is unchanged. Each new A starts without a reference KV cache hit.
• Prepared-reference cache hits occur on the conditioned chunks, with KV reuse from the third chunk. CPU prefill steps: 11 / 92 / 9, versus 11 / 92 / 91 before cache registration. CUDA: 11 / 82 / 9, versus 11 / 82 / 81.
• CPU third-chunk prefill in the repeated A request drops from approximately 633 to 155 ms. Whole-request savings are modest on this short case.
• Interleaved CUDA recheck, three runs per version: median warm A RTF 0.10425 → 0.10306, comparing the preceding PR revision against cache registration. No substantial speedup is claimed.
• Peak memory is effectively unchanged by cache registration: CPU peak RSS approximately 6,289 MiB; CUDA process peak VRAM 5,584 → 5,576 MiB. This does not eliminate the extra conditioning cost relative to unanchored main.
• CPU/CUDA offline ASR recovers every requested word, and Nemotron detects one speaker. CUDA streaming also recovers every requested word; streaming/offline durations both 6.56 s, waveform SNR 52.8 dB.
• KV reuse changes the waveform relative to the uncached revision; byte identity across those revisions is not claimed.

The measured consistency improvement is input/seed-dependent, not a claim that every unconditioned request changes speakers.