[Ryu1845]

Summary

Adds Zyphra's ZONOS2 as the community family zonos2: text to speech and voice cloning, offline and streaming, from the published Zyphra/ZONOS2-GGUF files. The runtime is written on audio.cpp framework modules and validated against Zyphra's ggml port Zyphra/zonos2.cpp.

• Backbone (backbone.cpp): 28-layer gated GQA transformer with QK-norm and a per-head temperature, dense SwiGLU layers 0-2 and 27, and 16-expert MoE layers 3-26 (top-1, top-2 on layer 26) whose MLP router carries an EDA state across MoE layers. The router MLP is model-local; the routing tail (MoeModule::build_routing) and the expert matmuls (MoeModule::build) are the framework's. The optional expert cache uses DiskExpertWeightStore for decode and StagedMoeRuntime for prefill, with one staged runtime per top-k. F16 KV cache, flash attention for decode.
• Sampler (sampler.cpp): port of the upstream Python sample_tts (penalty once per distinct code over generated frames only, top-k ties kept, softmax, top-p, min-p).
• DAC 44.1 kHz decoder (codec_decoder.cpp): framework Conv1dModule, ConvTranspose1dModule (col2im fast path, or unpadded + crop on the CPU) and Snake1dModule; windowed decoding with context frames for streaming.
• ECAPA speaker encoder (speaker_encoder.cpp): the framework mel front end plus the ECAPA graph, built for the exact frame count.
• Session / loader / spec: request and session options, voice cache, encode_reference_only, speaker_embedding_file, conditioning rows (speaker background, accurate mode, speaking rate, quality buckets), fade-out, long-form chunking, StreamingAudioController streaming.

No framework change.

Build

cmake -S . -B build/zonos2 -DCMAKE_BUILD_TYPE=Release -DAUDIOCPP_MODEL_SET=custom -DAUDIOCPP_MODELS=zonos2 \
  -DENGINE_ENABLE_OPENMP=OFF -DGGML_OPENMP=OFF
cmake --build build/zonos2 -j 8 --target audiocpp_cli audiocpp_server

For the parity tests add -DENGINE_BUILD_TESTS=ON -DENGINE_BUILD_MODEL_TESTS=ON -DAUDIOCPP_ZONOS2_MODEL_DIR=$PWD/models/ZONOS2-GGUF -DAUDIOCPP_ZONOS2_PARITY_DIR=/tmp/zonos2-oracle.

Run

python3 tools/model_manager_v2.py install zonos2_q8_0 --models-root models
M=models/ZONOS2-GGUF/zonos2-q8_0.gguf
build/zonos2/bin/audiocpp_cli --task tts  --family zonos2 --model $M --backend metal --text "Hello from audio.cpp." --out out.wav
build/zonos2/bin/audiocpp_cli --task clon --family zonos2 --model $M --backend metal --text "Hello from audio.cpp." --voice-ref voice.wav --out clone.wav
build/zonos2/bin/audiocpp_cli --task tts  --family zonos2 --model $M --backend metal --mode streaming --text "Hello from audio.cpp." --out stream.wav

Models

Package	Files	Size
zonos2_q8_0 (default)	zonos2-q8_0.gguf, dac.gguf, spk-encoder.gguf	8.82 GB

Parity

Oracle: bash tests/community_models/zonos2/make_oracle.sh ../zonos2.cpp/build models/ZONOS2-GGUF assets/resources/a.wav /tmp/zonos2-oracle (zonos2.cpp 0e08ea0). It also compiles oracle_decode.cpp against the zonos2.cpp build tree to record its own teacher-forced KV-cache decode.

zonos2 prompt oracle: PASS
zonos2 backbone parity: PASS cos_min=1.000000 argmax=1.0000 max_diff=0 steps=100 (f32 prefill pass: cos_min=0.999775 argmax=0.9670)
zonos2 backbone speaker parity: PASS cos_min=1.000000 argmax=1.0000 max_diff=0 steps=100 (f32 prefill pass: cos_min=0.994825 argmax=0.9505)
zonos2 backbone parity (disk experts): PASS max_diff_vs_resident=0 oracle_cos_min=1.000000 oracle_argmax=1.0000 steps=100
zonos2 decoder parity: PASS snr=59.06 seam_max=0 samples=153600 oracle_samples=153600 frames=300
zonos2 speaker parity: PASS cos=1.000000

The backbone is compared with zonos2.cpp's own prefill + KV-cache decode of the same rows: logits are bit-identical (CPU, 4 threads). The plan's first oracle was zonos2.cpp's single F32 prefill pass over the whole sequence (--validate); zonos2.cpp's own decode departs from that pass on near-tied codes (argmax 96.7% / 95.1%), exactly like ours, so the test now pins the decode path and reports the prefill pass for information.

Greedy (temperature=0, repetition_penalty=1): 2,000 frames identical to zonos2-cli --greedy; audio within 3 LSB of 16-bit PCM. Streaming equals offline exactly; speaker_embedding_file cloning equals --voice-ref cloning exactly; the expert cache equals the resident path exactly; our encode_reference_only embedding has cosine 0.9999997 with spk-encoder-cli's.

Quality

Qwen3-ASR-1.7B WER over 4 English prompts (numbers as words), speaker cosine from the ECAPA encoder against the reference clip, CPU:

	zonos2.cpp	audio.cpp
WER, TTS (4 seeds)	0.37%	4.10%
WER, clone (2 seeds)	0.00%	0.00%
Speaker cosine, clone	0.964	0.968
WER, TTS, Metal (2 seeds)	-	2.99%

The audio.cpp TTS number is one take of 16 (seed 2, shortest prompt) that repeated the sentence; the other 15 are word-perfect. The two runtimes share bit-identical logits but use different samplers (upstream Python's vs zonos2.cpp's), so seeded takes differ. A 1,284-character paragraph (7 chunks, Metal) gave 70.5 s of audio and 14.5% WER: one chunk dropped one sentence.

Performance

Apple M3, 16 GB, Q8_0, 14-word sentence; the machine was already swapping during these runs.

Backend	Mode	Frames/s	Wall	Audio	RTF	Peak footprint
CPU, 4 threads	TTS	26.6	27.8 s	5.42 s	5.13	9.3 GB
Metal	TTS	42.0	16.4 s	6.84 s	2.39	8.6 GB
Metal	Clone	46.3	13.9 s	6.05 s	2.30	8.8 GB
Metal	TTS, disk_experts	9.6	54.1 s	5.77 s	9.38	5.2 GB

zonos2-cli on the same machine: 25.7 frames/s CPU (4 threads), 47.6 frames/s Metal.

Server (CPU, lazy_load, streaming session): 5 POST /v1/audio/speech requests (3 plain, 2 voice_ref, one SSE with 15 speech.audio.delta events before speech.audio.done) all returned 200; peak memory footprint 10.1 GB. Request wall times (91-129 s) were dominated by swap on this machine, so the voice-cache speed-up is not visible in them.

Backends tested

CPU and Metal (Apple M3). CUDA and Vulkan untested.

Known limitations

• Not implemented: emotion directions, speaker blending, upstream long-form continuation, CFG, text normalization (the model reads raw UTF-8 bytes; write numbers as words).
• Sampled takes follow the upstream Python sampler, so fixed seeds do not reproduce zonos2.cpp's sampled audio; greedy matches.
• Long form is independent chunks; a chunk can drop or repeat a sentence.
• About 9 GB resident; on 16 GB machines under memory pressure, use zonos2.disk_experts (about 5 GB, much slower).

License

Weights: Apache-2.0 (Zyphra/ZONOS2-GGUF). The runtime was written for audio.cpp and numerically validated against Zyphra/zonos2.cpp.