[5uck1ess]

Symptom

Long dialogue sent through tag-aware chunking produces chunks that open with a stray closing quote. On the dialogue passage from drama_stress.py in #920, 8 of 14 chunks at the default 200-codepoint budget begin like this:

" He tapped the map. "She said it's tomorrow or never. ...
"\n\nElena stared at him for a long moment. "So what do we do?"

The model then reads quoted and unquoted text the wrong way round from that boundary on. In one Kitten TTS2 run, two dialogue lines ("Please, don't start." "I'm not starting anything.") were dropped and another stretch was garbled. After the fix, the same seed reads them correctly.

Cause

split_tag_aware_units ends a word unit right after ., ! or ?. In money." He, the unit is therefore money., and the closing " becomes the first unit of the next sentence, and so of the next chunk. Default mode keeps ASCII punctuation attached to the preceding word, so " is never a unit of its own that could start the next chunk.

Fix

In tag-aware mode only, a sentence stop now absorbs the closing quotes and brackets that follow it directly: " ' ) ” ’ » 」 』 ）. A unit ending in a stop plus closers counts as a sentence end. Because ", ' and ’ also open words ('Tis, or a quote right after a stop with no space), those three are absorbed only when the run of closers ends at ASCII whitespace, the end of the text or [/<, or when an unambiguous closer follows them. The run is scanned once, so a long run of quotes stays linear. Default, Japanese and Endline modes are untouched; the new helpers are called only from split_tag_aware_units.

Tag-aware users: fish_drama3, kitten_tts2, voxcpm1, voxcpm2, maya1 and dots_tts (session defaults), omnivoice when text_chunk_size is set explicitly, plus any request that passes text_chunk_mode=tag_aware.

Validation

Build, from upstream main d3cfd519:

cmake -S . -B build-test -DCMAKE_BUILD_TYPE=Release -DENGINE_BUILD_TESTS=ON -DAUDIOCPP_MODEL_SET=custom -DAUDIOCPP_MODELS=fish_drama3
cmake --build build-test -j 12 && (cd build-test && ctest -j 8)
cmake -S . -B build-multi -DCMAKE_BUILD_TYPE=Release -DENGINE_ENABLE_CUDA=ON -DCMAKE_CUDA_ARCHITECTURES=86-real \
  -DAUDIOCPP_MODEL_SET=custom "-DAUDIOCPP_MODELS=fish_drama3;kitten_tts2;voxcpm1;voxcpm2;omnivoice;maya1"
cmake --build build-multi --target audiocpp_cli -j 10

Linux, RTX 3090 (shared with other services, so roughly 14 GB was free), CUDA. The "before" binary is the same build with only chunking.cpp reverted.

Unit tests. text_chunking_test gets exact-output cases. Each places the chunk boundary right after one closer, and there are opener cases (Wait.'Tis, Wait.’Tis, left."Hi,", left."'Tis.") that guard against over-absorbing; main already handled those. A 100,000-quote run checks the linear scan. The test fails on main ([He said "stop.][" Then left.]). Removing any single closer from the set also fails it. ctest passes 59 of 59.

Text level. Six long stress passages from #920's drama_stress.py (narration, dialogue, emotional with tags, action, whisper, numbers), budget 200:

• Default mode: byte-identical before and after on all six.
• Tag-aware: identical on the four passages without dialogue. These passages have no stop immediately followed by a closer; text that does have one, such as (It always was.), changes on purpose.
• The two dialogue passages keep their chunk counts. Chunks opening with a stray closer go from 8 to 0 and from 3 to 0.

Runtime, same text, Whisper-base WER, before → after:

Family	Dialogue passage	Action passage
kitten_tts2 (seed 0)	0.174 → 0.036 (two dropped lines restored)	0.032 → 0.032
omnivoice (seed 0, default duration chunking, so not on this path)	identical audio	identical audio
voxcpm1	identical audio (seed 0)	seeds 0–3: 0.023/0.046/0.172/0.054 → 0.034/–/0.032/0.037
voxcpm2 (CPU, seed 0)	0.017 → 0.017	not run
maya1, documented instruct, seeds 1–3	mean 0.287 → 0.272	mean 0.182 → 0.163
fish_drama3 Q4_K_S	stray-quote chunks 8 → 0	—

Maya1's WER is high on this text either way, and its six seed pairs split 4 better and 2 worse, within its seed-to-seed spread, so that's no clear change. VoxCPM2 on CUDA, and VoxCPM1 action seed 1 after the fix, failed to allocate the AudioVAE graph (9.6 GB and 20 GB) on this shared card; this happens before and after alike. Single-seed rows are single draws.

Known limitations: a ", ' or ’ followed by non-ASCII whitespace (for example U+3000) is not absorbed. dots_tts also defaults to tag-aware and was not run. The runtime runs used a build on 847812ec; the rebase onto d3cfd519 added the opener guard and the linear scan, and the six passages chunk identically. Metal and Vulkan were not tested.

Scripts and raw outputs: https://github.com/5uck1ess/audio.cpp-fork/tree/drama3-parity-main/tests/community_models/fish_drama3/parity (chunker_probe/, chunk_probe.cpp, pacing_score.py).

Related: #920 (Drama 3 long-form pacing). #978 covers the other cause found there, top_k=30 silence stalls in fish_drama3.

[0xShug0]

Merged! Thanks!