From 966d7a3d2092e02d5b479f7a1aee5ed41c04ae19 Mon Sep 17 00:00:00 2001 From: KKTTSPJ <291752079+KKTTSPJ@users.noreply.github.com> Date: Sat, 17 Oct 2026 11:05:06 +0900 Subject: [PATCH 1/3] server: resolve Irodori-TTS speaker embeddings from voice_dir A request "voice" name can select a wav in the server's voice library, but an Irodori-TTS Speaker Inversion embedding could only be used by path (the speaker_embedding_file option) or from an embeddings/ folder next to the weights, which needs the model path to be that folder. With a GGUF file as the model path, and in the WebUI's voice list, the shared voice library was the natural place but did not take embeddings. For models whose family reads these files (irodori_tts), a "voice" name now resolves /.speaker.safetensors first and passes it as speaker_embedding_file; a wav of the same name is used otherwise, as before. GET /v1/audio/voices lists the embedding names for such models, so the WebUI offers them. Both server runtimes get the change. Other families, including vieneu_v3_turbo whose option of the same name reads comma-separated floats, are unchanged. Co-Authored-By: Claude Opus 5.5 --- CMakeLists.txt | 12 ++ app/server/README.md | 7 +- app/server/api.md | 4 +- app/server/parallel_runtime.cpp | 10 +- app/server/runtime.cpp | 10 +- app/server/server_protocol.cpp | 78 ++++++++- app/server/server_protocol.h | 10 ++ docs/models/irodori_tts.md | 1 + tests/unittests/test_server_voice_library.cpp | 163 ++++++++++++++++++ 9 files changed, 283 insertions(+), 12 deletions(-) create mode 100644 tests/unittests/test_server_voice_library.cpp diff --git a/CMakeLists.txt b/CMakeLists.txt index ddaf9e2f3..642671cf6 100644 --- a/CMakeLists.txt +++ b/CMakeLists.txt @@ -3443,6 +3443,18 @@ if (ENGINE_BUILD_TESTS OR ENGINE_BUILD_EXTENDED_TESTS OR ENGINE_BUILD_MODEL_TEST add_test(NAME server_runtime_selection_test COMMAND server_runtime_selection_test) set_tests_properties(server_runtime_selection_test PROPERTIES TIMEOUT 60) + add_executable(server_voice_library_test + tests/unittests/test_server_voice_library.cpp ${server_selection_sources}) + target_include_directories(server_voice_library_test PRIVATE + ${CMAKE_CURRENT_SOURCE_DIR}/app/server + "${AUDIOCPP_SERVER_TEST_GENERATED_DIR}" + "${CMAKE_CURRENT_BINARY_DIR}/generated") + target_link_libraries(server_voice_library_test PRIVATE engine_runtime ggml Threads::Threads) + if(WIN32) + target_link_libraries(server_voice_library_test PRIVATE ws2_32) + endif() + add_test(NAME server_voice_library_test COMMAND server_voice_library_test) + # Exercise real handlers with controlled, tiny model loaders. No native # installer or backend-specific model admission is needed by this test. get_target_property(server_lifecycle_sources audiocpp_server SOURCES) diff --git a/app/server/README.md b/app/server/README.md index 17e78bf5d..0651475a9 100644 --- a/app/server/README.md +++ b/app/server/README.md @@ -531,11 +531,13 @@ Relative `voice_dir` paths resolve against the config file's directory. The comm When a request sends `"voice"` that is not a configured model preset, the server checks `/.wav`; if the file exists it is loaded as the cloning reference, and the `` transcript from `prompt_text` is injected as `reference_text` unless the request already provides one. +For Irodori-TTS, the library can also hold Speaker Inversion embeddings as `.speaker.safetensors` (the files Irodori-TTS training writes). `"voice": ""` then passes the file as the `speaker_embedding_file` request option, in place of reference audio; see [Speaker Inversion Embeddings](../../docs/models/irodori_tts.md#speaker-inversion-embeddings). An embedding wins over a `.wav` of the same name. Other models do not use these files. + Server-side resolution of reference audio and voice names: 1. Explicit `voice_ref` replaces any preset reference audio and bypasses WAV-name lookup. 2. `voice` matching a configured model preset — preset wins. -3. `voice` matching a wav basename in `voice_dir` — voice-library clone. +3. `voice` matching a wav basename in `voice_dir` — voice-library clone. For Irodori-TTS, a `.speaker.safetensors` in `voice_dir` comes first and is used as the speaker embedding. 4. Otherwise, `voice` is passed as the model-native cached voice ID. If neither `voice` nor `voice_ref` is supplied, the configured default preset @@ -544,7 +546,8 @@ applies. Prefer one selection method per request: combining `voice` and is model-specific. Top-level `reference_text` overrides the preset/library text. [`GET /v1/audio/voices`](api.md#get-v1audiovoicesmodelid) lists library WAV -basenames, deduplicated against configured preset and cached voice names. +basenames, deduplicated against configured preset and cached voice names, and +for an Irodori-TTS model also the library's speaker embedding names. With native UI management enabled and no configured `voice_dir`, the server provides embedded demo recordings in a temporary reference library. Those names diff --git a/app/server/api.md b/app/server/api.md index 0aaff1561..9acd45de0 100644 --- a/app/server/api.md +++ b/app/server/api.md @@ -383,7 +383,9 @@ A browser cannot drive it: `fetch()` request streaming requires HTTP/2 and is ha Lists cached voice IDs, configured model preset names, and shared `voice_dir` WAV basenames. If the configured model path is a directory containing `embeddings/*.safetensors` (used by PocketTTS), their filename stems are also -listed. It does not enumerate speaker tensors packed inside a GGUF; use the +listed. For an Irodori-TTS model, `voice_dir` speaker embeddings +(`.speaker.safetensors`) are listed by ``. It does not enumerate +speaker tensors packed inside a GGUF; use the [model's documented built-in IDs](README.md#built-in-voices-and-discovery) for those. Pass `model` explicitly when several entries are configured. Omitting it selects diff --git a/app/server/parallel_runtime.cpp b/app/server/parallel_runtime.cpp index 11fe10edd..f56b03399 100644 --- a/app/server/parallel_runtime.cpp +++ b/app/server/parallel_runtime.cpp @@ -828,7 +828,8 @@ engine::runtime::TaskRequest ParallelServerState::build_speech_request(const Loa request_base_, config_.voice_dir, model.voice_presets, model.default_voice_preset, model.accepts_reference_text, model.accepts_speed, model.accepts_speaking_rate, - model.config.default_request_options}); + model.config.default_request_options, + uses_voice_library_speaker_embeddings(model.config.family)}); } struct ParallelServerState::TimedTaskResult { @@ -2336,8 +2337,10 @@ HttpResponse ParallelServerState::handle_voices(const HttpRequest & request) con } if (model_idx != SIZE_MAX) { selected = models_.at(model_idx).get(); } } + bool lists_speaker_embeddings = false; if (selected != nullptr) { const auto & model = *selected; + lists_speaker_embeddings = uses_voice_library_speaker_embeddings(model.config.family); std::shared_lock metadata_lock(model.metadata_mutex); for (const auto & [name, preset] : model.voice_presets) { (void) preset; @@ -2362,6 +2365,11 @@ HttpResponse ParallelServerState::handle_voices(const HttpRequest & request) con } } } + if (lists_speaker_embeddings) { + for (auto & name : voice_library_speaker_embedding_names(*config_.voice_dir)) { + voices.push_back(std::move(name)); + } + } } std::sort(voices.begin(), voices.end()); voices.erase(std::unique(voices.begin(), voices.end()), voices.end()); diff --git a/app/server/runtime.cpp b/app/server/runtime.cpp index 01d2737fb..32bac29eb 100644 --- a/app/server/runtime.cpp +++ b/app/server/runtime.cpp @@ -717,7 +717,8 @@ engine::runtime::TaskRequest ServerState::build_speech_request(const LoadedModel request_base_, config_.voice_dir, model.voice_presets, model.default_voice_preset, model.accepts_reference_text, model.accepts_speed, model.accepts_speaking_rate, - model.config.default_request_options}); + model.config.default_request_options, + uses_voice_library_speaker_embeddings(model.config.family)}); } struct ServerState::TimedTaskResult { @@ -2175,8 +2176,10 @@ HttpResponse ServerState::handle_voices(const HttpRequest & request) const { } else if (models_.size() == 1) { model_idx = 0; } + bool lists_speaker_embeddings = false; if (model_idx != SIZE_MAX) { const auto & model = *models_.at(model_idx); + lists_speaker_embeddings = uses_voice_library_speaker_embeddings(model.config.family); std::shared_lock metadata_lock(model.metadata_mutex); for (const auto & [name, preset] : model.voice_presets) { (void) preset; @@ -2201,6 +2204,11 @@ HttpResponse ServerState::handle_voices(const HttpRequest & request) const { } } } + if (lists_speaker_embeddings) { + for (auto & name : voice_library_speaker_embedding_names(*config_.voice_dir)) { + voices.push_back(std::move(name)); + } + } } std::sort(voices.begin(), voices.end()); voices.erase(std::unique(voices.begin(), voices.end()), voices.end()); diff --git a/app/server/server_protocol.cpp b/app/server/server_protocol.cpp index 3f1c9cc1f..491d24caa 100644 --- a/app/server/server_protocol.cpp +++ b/app/server/server_protocol.cpp @@ -129,8 +129,20 @@ build_speech_request(const Value &body, const SpeechRequestContext &context) { bool voice_library_resolved = false; if (context.voice_dir.has_value() && !has_explicit_voice_ref) { const std::string voice_name = value->as_string(); - if (const auto wav = - resolve_voice_library_wav(*context.voice_dir, voice_name)) { + // A speaker embedding wins over a wav of the same name. It replaces any + // embedding given in "options", under either option name, since the + // model rejects the two names when they disagree. + std::optional embedding; + if (context.uses_speaker_embeddings) { + embedding = resolve_voice_library_speaker_embedding(*context.voice_dir, + voice_name); + } + if (embedding.has_value()) { + request.options.erase("speaker_embedding_path"); + request.options["speaker_embedding_file"] = embedding->string(); + voice_library_resolved = true; + } else if (const auto wav = resolve_voice_library_wav(*context.voice_dir, + voice_name)) { voice.speaker = engine::runtime::VoiceReference{}; voice.speaker->audio = minitts::cli::read_audio_buffer(*wav); has_voice = true; @@ -361,9 +373,16 @@ load_voice_library_text(const std::filesystem::path &voice_dir, return std::nullopt; } +namespace { + +constexpr std::string_view kSpeakerEmbeddingSuffix = ".speaker.safetensors"; + +// /, when voice_name is a bare file name (never +// a path that could leave voice_dir) and the file exists. std::optional -resolve_voice_library_wav(const std::filesystem::path &voice_dir, - const std::string &voice_name) { +resolve_voice_library_file(const std::filesystem::path &voice_dir, + const std::string &voice_name, + std::string_view suffix) { const std::filesystem::path name_path(voice_name); if (voice_name.empty() || name_path.has_root_name() || name_path.has_root_directory() || name_path.has_parent_path() || @@ -371,12 +390,57 @@ resolve_voice_library_wav(const std::filesystem::path &voice_dir, voice_name == "..") { return std::nullopt; } - const auto wav = voice_dir / (voice_name + ".wav"); + const auto file = voice_dir / (voice_name + std::string(suffix)); std::error_code ec; - if (!std::filesystem::is_regular_file(wav, ec)) { + if (!std::filesystem::is_regular_file(file, ec)) { return std::nullopt; } - return wav; + return file; +} + +} // namespace + +std::optional +resolve_voice_library_wav(const std::filesystem::path &voice_dir, + const std::string &voice_name) { + return resolve_voice_library_file(voice_dir, voice_name, ".wav"); +} + +bool uses_voice_library_speaker_embeddings(std::string_view family) { + // vieneu_v3_turbo declares an option of the same name, but it reads + // comma-separated floats, not a safetensors file. + return family == "irodori_tts"; +} + +std::optional +resolve_voice_library_speaker_embedding(const std::filesystem::path &voice_dir, + const std::string &voice_name) { + return resolve_voice_library_file(voice_dir, voice_name, + kSpeakerEmbeddingSuffix); +} + +std::vector +voice_library_speaker_embedding_names(const std::filesystem::path &voice_dir) { + std::vector names; + std::error_code ec; + if (!std::filesystem::is_directory(voice_dir, ec)) { + return names; + } + for (const auto &entry : + std::filesystem::directory_iterator(voice_dir, ec)) { + if (!entry.is_regular_file()) { + continue; + } + const auto filename = entry.path().filename().string(); + if (filename.size() > kSpeakerEmbeddingSuffix.size() && + filename.compare(filename.size() - kSpeakerEmbeddingSuffix.size(), + kSpeakerEmbeddingSuffix.size(), + kSpeakerEmbeddingSuffix) == 0) { + names.push_back( + filename.substr(0, filename.size() - kSpeakerEmbeddingSuffix.size())); + } + } + return names; } std::unordered_map diff --git a/app/server/server_protocol.h b/app/server/server_protocol.h index eadf35153..48fa42313 100644 --- a/app/server/server_protocol.h +++ b/app/server/server_protocol.h @@ -40,6 +40,15 @@ load_voice_library_text(const std::filesystem::path &voice_dir, std::optional resolve_voice_library_wav(const std::filesystem::path &voice_dir, const std::string &voice_id); +// Irodori-TTS Speaker Inversion embeddings in the voice library, as +// /.speaker.safetensors. Only families that read that file +// through their speaker_embedding_file request option use them. +bool uses_voice_library_speaker_embeddings(std::string_view family); +std::optional +resolve_voice_library_speaker_embedding(const std::filesystem::path &voice_dir, + const std::string &voice_id); +std::vector +voice_library_speaker_embedding_names(const std::filesystem::path &voice_dir); std::unordered_map options_from_object(const engine::io::json::Value *value); @@ -58,6 +67,7 @@ struct SpeechRequestContext { bool accepts_speed = true; bool accepts_speaking_rate = true; const std::unordered_map &default_options; + bool uses_speaker_embeddings = false; }; engine::runtime::TaskRequest build_speech_request( diff --git a/docs/models/irodori_tts.md b/docs/models/irodori_tts.md index 080e12af4..aee67d26b 100644 --- a/docs/models/irodori_tts.md +++ b/docs/models/irodori_tts.md @@ -84,6 +84,7 @@ audiocpp_cli --task tts --family irodori_tts \ - **By file:** request option `speaker_embedding_file` (CLI `--request-option`, or `"options"` in a server request). Legacy `speaker_embedding_path` remains accepted. - **By name:** put the file in an `embeddings` directory next to the model weights as `.safetensors` and pass `` as the voice id (CLI `--voice-id`, server `"voice"`). With the model path set to that directory, the server's `GET /v1/audio/voices` lists the names. A voice id with no such file is ignored, as before. +- **From the server's voice library:** put the file in the server's [`voice_dir`](../../app/server/README.md#voice-library-voice_dir) as `.speaker.safetensors` and send `"voice": ""`. This works with any model path, including a single GGUF file, and the library is shared by all configured Irodori-TTS models. `GET /v1/audio/voices?model=` lists the names, so the WebUI's voice list offers them too. An embedding wins over a `.wav` of the same name. The file must hold a `speaker_embedding` tensor of shape `[tokens, speaker_dim]` (F32, F16 or BF16). An embedding cannot be combined with reference audio or `no_ref=true`. Use it with the base checkpoint it was trained on: an embedding from another checkpoint with the same `speaker_dim` loads, but the speaker match is not guaranteed. diff --git a/tests/unittests/test_server_voice_library.cpp b/tests/unittests/test_server_voice_library.cpp new file mode 100644 index 000000000..a1af5692a --- /dev/null +++ b/tests/unittests/test_server_voice_library.cpp @@ -0,0 +1,163 @@ +// The server voice library resolves "voice" to an Irodori-TTS Speaker Inversion +// embedding (.speaker.safetensors) before a wav of the same name, only for +// families that use such files. +#include "server_protocol.h" + +#include "engine/framework/io/json.h" + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +namespace { + +namespace fs = std::filesystem; +using minitts::server::RuntimeVoicePreset; +using minitts::server::SpeechRequestContext; + +void expect(bool condition, const std::string & message) { + if (!condition) { + throw std::runtime_error(message); + } +} + +void write_file(const fs::path & path, const std::string & bytes) { + std::ofstream out(path, std::ios::binary | std::ios::trunc); + out.write(bytes.data(), static_cast(bytes.size())); + expect(static_cast(out), "could not write " + path.string()); +} + +void put_u16(std::string & out, uint16_t value) { + out.push_back(static_cast(value & 0xff)); + out.push_back(static_cast((value >> 8) & 0xff)); +} + +void put_u32(std::string & out, uint32_t value) { + for (int shift = 0; shift < 32; shift += 8) { + out.push_back(static_cast((value >> shift) & 0xff)); + } +} + +// 16 kHz mono PCM16 silence. +std::string tiny_wav() { + constexpr uint32_t kSamples = 160; + std::string out = "RIFF"; + put_u32(out, 36 + kSamples * 2); + out += "WAVEfmt "; + put_u32(out, 16); + put_u16(out, 1); + put_u16(out, 1); + put_u32(out, 16000); + put_u32(out, 16000 * 2); + put_u16(out, 2); + put_u16(out, 16); + out += "data"; + put_u32(out, kSamples * 2); + out.append(kSamples * 2, '\0'); + return out; +} + +engine::runtime::TaskRequest speech(const fs::path & voice_dir, bool uses_speaker_embeddings, + const std::string & body_text) { + const std::optional dir = voice_dir; + const std::unordered_map presets; + const std::optional default_preset; + const std::unordered_map default_options; + return minitts::server::build_speech_request( + engine::io::json::parse(body_text), + SpeechRequestContext{voice_dir, dir, presets, default_preset, true, true, true, + default_options, uses_speaker_embeddings}); +} + +bool has_audio(const engine::runtime::TaskRequest & request) { + return request.voice.has_value() && request.voice->speaker.has_value() && + request.voice->speaker->audio.has_value(); +} + +} // namespace + +int main() { + const fs::path dir = fs::temp_directory_path() / + ("audiocpp-voice-library-test-" + std::to_string( + std::chrono::steady_clock::now().time_since_epoch().count())); + try { + fs::create_directories(dir); + // The embedding is never opened here: the model reads it. + write_file(dir / "both.speaker.safetensors", "not read by the server"); + write_file(dir / "both.wav", tiny_wav()); + write_file(dir / "embedding_only.speaker.safetensors", "not read by the server"); + write_file(dir / "wav_only.wav", tiny_wav()); + write_file(dir / "plain.safetensors", "not a speaker embedding name"); + + expect(minitts::server::uses_voice_library_speaker_embeddings("irodori_tts"), + "irodori_tts should use voice library speaker embeddings"); + expect(!minitts::server::uses_voice_library_speaker_embeddings("vieneu_v3_turbo"), + "vieneu_v3_turbo reads another embedding format"); + + // An embedding wins over a wav of the same name and replaces an embedding + // given in options under either name. + auto request = speech(dir, true, + R"({"input":"x","voice":"both","options":{"speaker_embedding_path":"other.safetensors"}})"); + expect(request.options.at("speaker_embedding_file") == + (dir / "both.speaker.safetensors").string(), + "voice did not resolve to the library embedding"); + expect(request.options.count("speaker_embedding_path") == 0, + "legacy embedding option was kept next to the resolved one"); + expect(!has_audio(request), "an embedding voice also loaded reference audio"); + + request = speech(dir, true, R"({"input":"x","voice":"embedding_only"})"); + expect(request.options.at("speaker_embedding_file") == + (dir / "embedding_only.speaker.safetensors").string(), + "embedding-only voice did not resolve"); + + request = speech(dir, true, R"({"input":"x","voice":"wav_only"})"); + expect(has_audio(request) && request.options.count("speaker_embedding_file") == 0, + "wav-only voice did not resolve to reference audio"); + + // Other families keep the wav, and never get the embedding. + request = speech(dir, false, R"({"input":"x","voice":"both"})"); + expect(has_audio(request) && request.options.count("speaker_embedding_file") == 0, + "a family without embeddings did not get the wav"); + request = speech(dir, false, R"({"input":"x","voice":"embedding_only"})"); + expect(request.options.count("speaker_embedding_file") == 0 && + request.voice.has_value() && request.voice->speaker.has_value() && + request.voice->speaker->cached_voice_id == "embedding_only", + "a family without embeddings did not fall back to the cached voice id"); + + // An explicit voice_ref bypasses the library, as for wavs. + request = speech(dir, true, + R"({"input":"x","voice":"embedding_only","voice_ref":")" + + (dir / "wav_only.wav").generic_string() + R"("})"); + expect(request.options.count("speaker_embedding_file") == 0 && has_audio(request), + "voice_ref did not bypass the library embedding"); + + // Names that are paths never leave the library. + request = speech(dir, true, R"({"input":"x","voice":"../embedding_only"})"); + expect(request.options.count("speaker_embedding_file") == 0, + "a path-like voice name resolved to an embedding"); + + auto names = minitts::server::voice_library_speaker_embedding_names(dir); + std::sort(names.begin(), names.end()); + expect(names == std::vector{"both", "embedding_only"}, + "embedding names should be the .speaker.safetensors stems only"); + expect(minitts::server::voice_library_speaker_embedding_names(dir / "missing").empty(), + "a missing voice library should list no embeddings"); + } catch (const std::exception & error) { + std::error_code ec; + fs::remove_all(dir, ec); + std::cerr << "voice library test failed: " << error.what() << "\n"; + return 1; + } + std::error_code ec; + fs::remove_all(dir, ec); + std::cout << "server voice library passed\n"; + return 0; +} From 1a797622da3932c8f1f62da1b7d247efb934c733 Mon Sep 17 00:00:00 2001 From: KKTTSPJ <291752079+KKTTSPJ@users.noreply.github.com> Date: Sun, 18 Oct 2026 06:33:15 +0900 Subject: [PATCH 2/3] server: let voice presets carry request options Replace the Irodori-specific voice_dir lookup of this branch with the design suggested in review: a voice_presets (or default_voice_preset) entry may now hold an "options" object. When the preset is selected, its options are added to the request's options; an option the request sets itself keeps the request's value, and default_request_options still come last. The server passes the values through as given and the model validates them like any other request option, so the server no longer knows a family name, an embedding file convention or option aliases. A preset that sets only options is valid; a non-object or an empty options object with nothing else is rejected. Preset names are already listed by /v1/audio/voices. Co-Authored-By: Claude Opus 5.5 --- CMakeLists.txt | 12 +- app/server/README.md | 28 ++- app/server/api.md | 6 +- app/server/config.cpp | 11 +- app/server/config.h | 2 + app/server/parallel_runtime.cpp | 11 +- app/server/runtime.cpp | 11 +- app/server/server_protocol.cpp | 83 ++------- app/server/server_protocol.h | 11 +- docs/models/irodori_tts.md | 1 - tests/unittests/test_server_config.cpp | 44 ++++- tests/unittests/test_server_voice_library.cpp | 163 ------------------ tests/unittests/test_server_voice_preset.cpp | 99 +++++++++++ 13 files changed, 202 insertions(+), 280 deletions(-) delete mode 100644 tests/unittests/test_server_voice_library.cpp create mode 100644 tests/unittests/test_server_voice_preset.cpp diff --git a/CMakeLists.txt b/CMakeLists.txt index 2908f0baf..5b240efbe 100644 --- a/CMakeLists.txt +++ b/CMakeLists.txt @@ -3444,17 +3444,17 @@ if (ENGINE_BUILD_TESTS OR ENGINE_BUILD_EXTENDED_TESTS OR ENGINE_BUILD_MODEL_TEST add_test(NAME server_runtime_selection_test COMMAND server_runtime_selection_test) set_tests_properties(server_runtime_selection_test PROPERTIES TIMEOUT 60) - add_executable(server_voice_library_test - tests/unittests/test_server_voice_library.cpp ${server_selection_sources}) - target_include_directories(server_voice_library_test PRIVATE + add_executable(server_voice_preset_test + tests/unittests/test_server_voice_preset.cpp ${server_selection_sources}) + target_include_directories(server_voice_preset_test PRIVATE ${CMAKE_CURRENT_SOURCE_DIR}/app/server "${AUDIOCPP_SERVER_TEST_GENERATED_DIR}" "${CMAKE_CURRENT_BINARY_DIR}/generated") - target_link_libraries(server_voice_library_test PRIVATE engine_runtime ggml Threads::Threads) + target_link_libraries(server_voice_preset_test PRIVATE engine_runtime ggml Threads::Threads) if(WIN32) - target_link_libraries(server_voice_library_test PRIVATE ws2_32) + target_link_libraries(server_voice_preset_test PRIVATE ws2_32) endif() - add_test(NAME server_voice_library_test COMMAND server_voice_library_test) + add_test(NAME server_voice_preset_test COMMAND server_voice_preset_test) # Exercise real handlers with controlled, tiny model loaders. No native # installer or backend-specific model admission is needed by this test. diff --git a/app/server/README.md b/app/server/README.md index 5eb8d8583..d5e6274fc 100644 --- a/app/server/README.md +++ b/app/server/README.md @@ -520,6 +520,27 @@ reference library before passing it through as a model-native voice ID. For built-in speakers, configure `voice_id` using an ID valid for that model, such as `af_heart` for Kokoro; preset names and native voice IDs are not universal. +A preset can also carry request `options` for the model. The server passes them +through as given, after the request's own options, so a request that sets the +same option keeps its value; the model validates them like any other request +option. For example, an Irodori-TTS speaker embedding selected by name: + +```json +{ + "voice_presets": { + "narrator": { + "options": { + "speaker_embedding_file": "/absolute/path/to/narrator.speaker.safetensors" + } + } + } +} +``` + +A request `{"input": "Hello.", "voice": "narrator"}` then uses that embedding. +Option values are not paths to the server, so a relative path is not resolved +against the config file the way `voice_ref` is; use absolute paths. + ### Voice library (`voice_dir`) Set server-level `"voice_dir"` to a folder of your reference WAV recordings and @@ -556,13 +577,11 @@ Relative `voice_dir` paths resolve against the config file's directory. The comm When a request sends `"voice"` that is not a configured model preset, the server checks `/.wav`; if the file exists it is loaded as the cloning reference, and the `` transcript from `prompt_text` is injected as `reference_text` unless the request already provides one. -For Irodori-TTS, the library can also hold Speaker Inversion embeddings as `.speaker.safetensors` (the files Irodori-TTS training writes). `"voice": ""` then passes the file as the `speaker_embedding_file` request option, in place of reference audio; see [Speaker Inversion Embeddings](../../docs/models/irodori_tts.md#speaker-inversion-embeddings). An embedding wins over a `.wav` of the same name. Other models do not use these files. - Server-side resolution of reference audio and voice names: 1. Explicit `voice_ref` replaces any preset reference audio and bypasses WAV-name lookup. 2. `voice` matching a configured model preset — preset wins. -3. `voice` matching a wav basename in `voice_dir` — voice-library clone. For Irodori-TTS, a `.speaker.safetensors` in `voice_dir` comes first and is used as the speaker embedding. +3. `voice` matching a wav basename in `voice_dir` — voice-library clone. 4. Otherwise, `voice` is passed as the model-native cached voice ID. If neither `voice` nor `voice_ref` is supplied, the configured default preset @@ -571,8 +590,7 @@ applies. Prefer one selection method per request: combining `voice` and is model-specific. Top-level `reference_text` overrides the preset/library text. [`GET /v1/audio/voices`](api.md#get-v1audiovoicesmodelid) lists library WAV -basenames, deduplicated against configured preset and cached voice names, and -for an Irodori-TTS model also the library's speaker embedding names. +basenames, deduplicated against configured preset and cached voice names. With native UI management enabled and no configured `voice_dir`, the server provides embedded demo recordings in a temporary reference library. Those names diff --git a/app/server/api.md b/app/server/api.md index 9acd45de0..72dc521d6 100644 --- a/app/server/api.md +++ b/app/server/api.md @@ -84,6 +84,8 @@ option parsing. If no request voice is provided and the configured model has `default_voice_preset`, the server injects that preset automatically. Request-level `voice`, `voice_ref`, and `reference_text` override the configured default. +A preset's configured `options` are added to the request's `options`; an option the request sets itself keeps the request's value. + `voice_ref` accepts either a plain path string (server-side file) or an object with a `type`: ```json @@ -383,9 +385,7 @@ A browser cannot drive it: `fetch()` request streaming requires HTTP/2 and is ha Lists cached voice IDs, configured model preset names, and shared `voice_dir` WAV basenames. If the configured model path is a directory containing `embeddings/*.safetensors` (used by PocketTTS), their filename stems are also -listed. For an Irodori-TTS model, `voice_dir` speaker embeddings -(`.speaker.safetensors`) are listed by ``. It does not enumerate -speaker tensors packed inside a GGUF; use the +listed. It does not enumerate speaker tensors packed inside a GGUF; use the [model's documented built-in IDs](README.md#built-in-voices-and-discovery) for those. Pass `model` explicitly when several entries are configured. Omitting it selects diff --git a/app/server/config.cpp b/app/server/config.cpp index ae7f169bc..6e1705ace 100644 --- a/app/server/config.cpp +++ b/app/server/config.cpp @@ -63,8 +63,15 @@ ServerModelConfig::VoicePreset parse_voice_preset( if (const auto * reference_text = value.find("reference_text")) { preset.reference_text = reference_text->as_string(); } - if (!preset.voice_id.has_value() && !preset.voice_ref.has_value() && !preset.reference_text.has_value()) { - throw std::runtime_error(context + " must set voice_id, voice_ref, or reference_text"); + if (const auto * options = value.find("options")) { + if (!options->is_object()) { + throw std::runtime_error(context + " options must be an object"); + } + preset.options = options_from_object(options); + } + if (!preset.voice_id.has_value() && !preset.voice_ref.has_value() && !preset.reference_text.has_value() && + preset.options.empty()) { + throw std::runtime_error(context + " must set voice_id, voice_ref, reference_text, or options"); } return preset; } diff --git a/app/server/config.h b/app/server/config.h index dbe444f99..ff24c52d4 100644 --- a/app/server/config.h +++ b/app/server/config.h @@ -37,6 +37,8 @@ struct ServerModelConfig { std::optional voice_id; std::optional voice_ref; std::optional reference_text; + // Request options the preset supplies, passed to the model as given. + std::unordered_map options; }; std::string id; diff --git a/app/server/parallel_runtime.cpp b/app/server/parallel_runtime.cpp index ab4a7298a..5bf23a6e4 100644 --- a/app/server/parallel_runtime.cpp +++ b/app/server/parallel_runtime.cpp @@ -636,6 +636,7 @@ RuntimeVoicePreset ParallelServerState::load_runtime_voice_preset( RuntimeVoicePreset out; out.voice_id = preset.voice_id; out.reference_text = preset.reference_text; + out.options = preset.options; if (preset.voice_ref.has_value()) { out.audio = minitts::cli::read_audio_buffer(*preset.voice_ref); } @@ -829,8 +830,7 @@ engine::runtime::TaskRequest ParallelServerState::build_speech_request(const Loa request_base_, config_.voice_dir, model.voice_presets, model.default_voice_preset, model.accepts_reference_text, model.accepts_speed, model.accepts_speaking_rate, - model.config.default_request_options, - uses_voice_library_speaker_embeddings(model.config.family)}); + model.config.default_request_options}); } struct ParallelServerState::TimedTaskResult { @@ -2338,10 +2338,8 @@ HttpResponse ParallelServerState::handle_voices(const HttpRequest & request) con } if (model_idx != SIZE_MAX) { selected = models_.at(model_idx).get(); } } - bool lists_speaker_embeddings = false; if (selected != nullptr) { const auto & model = *selected; - lists_speaker_embeddings = uses_voice_library_speaker_embeddings(model.config.family); std::shared_lock metadata_lock(model.metadata_mutex); for (const auto & [name, preset] : model.voice_presets) { (void) preset; @@ -2366,11 +2364,6 @@ HttpResponse ParallelServerState::handle_voices(const HttpRequest & request) con } } } - if (lists_speaker_embeddings) { - for (auto & name : voice_library_speaker_embedding_names(*config_.voice_dir)) { - voices.push_back(std::move(name)); - } - } } std::sort(voices.begin(), voices.end()); voices.erase(std::unique(voices.begin(), voices.end()), voices.end()); diff --git a/app/server/runtime.cpp b/app/server/runtime.cpp index 68b755402..41862cb61 100644 --- a/app/server/runtime.cpp +++ b/app/server/runtime.cpp @@ -539,6 +539,7 @@ RuntimeVoicePreset ServerState::load_runtime_voice_preset( RuntimeVoicePreset out; out.voice_id = preset.voice_id; out.reference_text = preset.reference_text; + out.options = preset.options; if (preset.voice_ref.has_value()) { out.audio = minitts::cli::read_audio_buffer(*preset.voice_ref); } @@ -718,8 +719,7 @@ engine::runtime::TaskRequest ServerState::build_speech_request(const LoadedModel request_base_, config_.voice_dir, model.voice_presets, model.default_voice_preset, model.accepts_reference_text, model.accepts_speed, model.accepts_speaking_rate, - model.config.default_request_options, - uses_voice_library_speaker_embeddings(model.config.family)}); + model.config.default_request_options}); } struct ServerState::TimedTaskResult { @@ -2177,10 +2177,8 @@ HttpResponse ServerState::handle_voices(const HttpRequest & request) const { } else if (models_.size() == 1) { model_idx = 0; } - bool lists_speaker_embeddings = false; if (model_idx != SIZE_MAX) { const auto & model = *models_.at(model_idx); - lists_speaker_embeddings = uses_voice_library_speaker_embeddings(model.config.family); std::shared_lock metadata_lock(model.metadata_mutex); for (const auto & [name, preset] : model.voice_presets) { (void) preset; @@ -2205,11 +2203,6 @@ HttpResponse ServerState::handle_voices(const HttpRequest & request) const { } } } - if (lists_speaker_embeddings) { - for (auto & name : voice_library_speaker_embedding_names(*config_.voice_dir)) { - voices.push_back(std::move(name)); - } - } } std::sort(voices.begin(), voices.end()); voices.erase(std::unique(voices.begin(), voices.end()), voices.end()); diff --git a/app/server/server_protocol.cpp b/app/server/server_protocol.cpp index 491d24caa..319bb5f43 100644 --- a/app/server/server_protocol.cpp +++ b/app/server/server_protocol.cpp @@ -121,6 +121,11 @@ build_speech_request(const Value &body, const SpeechRequestContext &context) { request.options.find("reference_text") == request.options.end()) { request.options["reference_text"] = *preset->reference_text; } + // Preset options fill what the request leaves unset; the model validates + // them like any other request option. + for (const auto &[key, value] : preset->options) { + request.options.emplace(key, value); + } } const bool has_explicit_voice_ref = body.find("voice_ref") != nullptr; @@ -129,20 +134,8 @@ build_speech_request(const Value &body, const SpeechRequestContext &context) { bool voice_library_resolved = false; if (context.voice_dir.has_value() && !has_explicit_voice_ref) { const std::string voice_name = value->as_string(); - // A speaker embedding wins over a wav of the same name. It replaces any - // embedding given in "options", under either option name, since the - // model rejects the two names when they disagree. - std::optional embedding; - if (context.uses_speaker_embeddings) { - embedding = resolve_voice_library_speaker_embedding(*context.voice_dir, - voice_name); - } - if (embedding.has_value()) { - request.options.erase("speaker_embedding_path"); - request.options["speaker_embedding_file"] = embedding->string(); - voice_library_resolved = true; - } else if (const auto wav = resolve_voice_library_wav(*context.voice_dir, - voice_name)) { + if (const auto wav = + resolve_voice_library_wav(*context.voice_dir, voice_name)) { voice.speaker = engine::runtime::VoiceReference{}; voice.speaker->audio = minitts::cli::read_audio_buffer(*wav); has_voice = true; @@ -373,16 +366,9 @@ load_voice_library_text(const std::filesystem::path &voice_dir, return std::nullopt; } -namespace { - -constexpr std::string_view kSpeakerEmbeddingSuffix = ".speaker.safetensors"; - -// /, when voice_name is a bare file name (never -// a path that could leave voice_dir) and the file exists. std::optional -resolve_voice_library_file(const std::filesystem::path &voice_dir, - const std::string &voice_name, - std::string_view suffix) { +resolve_voice_library_wav(const std::filesystem::path &voice_dir, + const std::string &voice_name) { const std::filesystem::path name_path(voice_name); if (voice_name.empty() || name_path.has_root_name() || name_path.has_root_directory() || name_path.has_parent_path() || @@ -390,57 +376,12 @@ resolve_voice_library_file(const std::filesystem::path &voice_dir, voice_name == "..") { return std::nullopt; } - const auto file = voice_dir / (voice_name + std::string(suffix)); + const auto wav = voice_dir / (voice_name + ".wav"); std::error_code ec; - if (!std::filesystem::is_regular_file(file, ec)) { + if (!std::filesystem::is_regular_file(wav, ec)) { return std::nullopt; } - return file; -} - -} // namespace - -std::optional -resolve_voice_library_wav(const std::filesystem::path &voice_dir, - const std::string &voice_name) { - return resolve_voice_library_file(voice_dir, voice_name, ".wav"); -} - -bool uses_voice_library_speaker_embeddings(std::string_view family) { - // vieneu_v3_turbo declares an option of the same name, but it reads - // comma-separated floats, not a safetensors file. - return family == "irodori_tts"; -} - -std::optional -resolve_voice_library_speaker_embedding(const std::filesystem::path &voice_dir, - const std::string &voice_name) { - return resolve_voice_library_file(voice_dir, voice_name, - kSpeakerEmbeddingSuffix); -} - -std::vector -voice_library_speaker_embedding_names(const std::filesystem::path &voice_dir) { - std::vector names; - std::error_code ec; - if (!std::filesystem::is_directory(voice_dir, ec)) { - return names; - } - for (const auto &entry : - std::filesystem::directory_iterator(voice_dir, ec)) { - if (!entry.is_regular_file()) { - continue; - } - const auto filename = entry.path().filename().string(); - if (filename.size() > kSpeakerEmbeddingSuffix.size() && - filename.compare(filename.size() - kSpeakerEmbeddingSuffix.size(), - kSpeakerEmbeddingSuffix.size(), - kSpeakerEmbeddingSuffix) == 0) { - names.push_back( - filename.substr(0, filename.size() - kSpeakerEmbeddingSuffix.size())); - } - } - return names; + return wav; } std::unordered_map diff --git a/app/server/server_protocol.h b/app/server/server_protocol.h index 48fa42313..a1944b078 100644 --- a/app/server/server_protocol.h +++ b/app/server/server_protocol.h @@ -40,15 +40,6 @@ load_voice_library_text(const std::filesystem::path &voice_dir, std::optional resolve_voice_library_wav(const std::filesystem::path &voice_dir, const std::string &voice_id); -// Irodori-TTS Speaker Inversion embeddings in the voice library, as -// /.speaker.safetensors. Only families that read that file -// through their speaker_embedding_file request option use them. -bool uses_voice_library_speaker_embeddings(std::string_view family); -std::optional -resolve_voice_library_speaker_embedding(const std::filesystem::path &voice_dir, - const std::string &voice_id); -std::vector -voice_library_speaker_embedding_names(const std::filesystem::path &voice_dir); std::unordered_map options_from_object(const engine::io::json::Value *value); @@ -56,6 +47,7 @@ struct RuntimeVoicePreset { std::optional voice_id; std::optional audio; std::optional reference_text; + std::unordered_map options; }; struct SpeechRequestContext { @@ -67,7 +59,6 @@ struct SpeechRequestContext { bool accepts_speed = true; bool accepts_speaking_rate = true; const std::unordered_map &default_options; - bool uses_speaker_embeddings = false; }; engine::runtime::TaskRequest build_speech_request( diff --git a/docs/models/irodori_tts.md b/docs/models/irodori_tts.md index aee67d26b..080e12af4 100644 --- a/docs/models/irodori_tts.md +++ b/docs/models/irodori_tts.md @@ -84,7 +84,6 @@ audiocpp_cli --task tts --family irodori_tts \ - **By file:** request option `speaker_embedding_file` (CLI `--request-option`, or `"options"` in a server request). Legacy `speaker_embedding_path` remains accepted. - **By name:** put the file in an `embeddings` directory next to the model weights as `.safetensors` and pass `` as the voice id (CLI `--voice-id`, server `"voice"`). With the model path set to that directory, the server's `GET /v1/audio/voices` lists the names. A voice id with no such file is ignored, as before. -- **From the server's voice library:** put the file in the server's [`voice_dir`](../../app/server/README.md#voice-library-voice_dir) as `.speaker.safetensors` and send `"voice": ""`. This works with any model path, including a single GGUF file, and the library is shared by all configured Irodori-TTS models. `GET /v1/audio/voices?model=` lists the names, so the WebUI's voice list offers them too. An embedding wins over a `.wav` of the same name. The file must hold a `speaker_embedding` tensor of shape `[tokens, speaker_dim]` (F32, F16 or BF16). An embedding cannot be combined with reference audio or `no_ref=true`. Use it with the base checkpoint it was trained on: an embedding from another checkpoint with the same `speaker_dim` loads, but the speaker match is not guaranteed. diff --git a/tests/unittests/test_server_config.cpp b/tests/unittests/test_server_config.cpp index d059fefd0..385822e9a 100644 --- a/tests/unittests/test_server_config.cpp +++ b/tests/unittests/test_server_config.cpp @@ -68,6 +68,12 @@ void test_inline_default_and_named_presets() { }, "builtin": { "voice_id": "alba" + }, + "embedding": { + "options": { + "speaker_embedding_file": "voices/embedding.safetensors", + "num_steps": 8 + } } } } @@ -81,10 +87,45 @@ void test_inline_default_and_named_presets() { require(model.default_voice_preset.has_value(), "inline default preset parsed"); require(model.default_voice_preset->voice_ref == root / "voices/default.wav", "default voice_ref path resolved"); require(model.default_voice_preset->reference_text == "default transcript", "default reference_text parsed"); - require(model.voice_presets.size() == 2, "named presets parsed"); + require(model.voice_presets.size() == 3, "named presets parsed"); require(model.voice_presets.at("assistant").voice_ref == root / "voices/assistant.wav", "named voice_ref path resolved"); require(model.voice_presets.at("assistant").reference_text == "assistant transcript", "named reference_text parsed"); require(model.voice_presets.at("builtin").voice_id == "alba", "named voice_id parsed"); + const auto & options = model.voice_presets.at("embedding").options; + require(options.size() == 2, "preset options parsed"); + require(options.at("speaker_embedding_file") == "voices/embedding.safetensors", + "preset option values are kept as given, not resolved as paths"); + require(options.at("num_steps") == "8", "numeric preset option parsed"); + require(model.voice_presets.at("assistant").options.empty(), "presets without options have none"); +} + +void test_bad_preset_options_are_rejected() { + const auto root = make_temp_root(); + const auto not_object = write_config(root, "server_preset_options_type.json", R"JSON({ + "models": [{"id": "tts", "family": "irodori_tts", "path": "models/irodori", + "voice_presets": {"narrator": {"options": "speaker.safetensors"}}}] +})JSON"); + bool rejected = false; + try { + (void) minitts::server::load_server_config(not_object); + } catch (const std::runtime_error & error) { + rejected = std::string(error.what()).find("options must be an object") != std::string::npos; + } + require(rejected, "non-object preset options are rejected"); + + const auto empty = write_config(root, "server_preset_options_empty.json", R"JSON({ + "models": [{"id": "tts", "family": "irodori_tts", "path": "models/irodori", + "voice_presets": {"narrator": {"options": {}}}}] +})JSON"); + rejected = false; + try { + (void) minitts::server::load_server_config(empty); + } catch (const std::runtime_error & error) { + rejected = std::string(error.what()).find("must set voice_id, voice_ref, reference_text, or options") != + std::string::npos; + } + require(rejected, "a preset with only empty options is rejected"); + std::filesystem::remove_all(root); } void test_model_spec_override_selection() { @@ -739,6 +780,7 @@ int main() { test_model_spec_override_selection(); test_builtin_utility_model_paths(); test_default_preset_name(); + test_bad_preset_options_are_rejected(); test_default_request_options(); test_default_session_options(); test_missing_default_preset_name_is_rejected(); diff --git a/tests/unittests/test_server_voice_library.cpp b/tests/unittests/test_server_voice_library.cpp deleted file mode 100644 index a1af5692a..000000000 --- a/tests/unittests/test_server_voice_library.cpp +++ /dev/null @@ -1,163 +0,0 @@ -// The server voice library resolves "voice" to an Irodori-TTS Speaker Inversion -// embedding (.speaker.safetensors) before a wav of the same name, only for -// families that use such files. -#include "server_protocol.h" - -#include "engine/framework/io/json.h" - -#include -#include -#include -#include -#include -#include -#include -#include -#include -#include -#include - -namespace { - -namespace fs = std::filesystem; -using minitts::server::RuntimeVoicePreset; -using minitts::server::SpeechRequestContext; - -void expect(bool condition, const std::string & message) { - if (!condition) { - throw std::runtime_error(message); - } -} - -void write_file(const fs::path & path, const std::string & bytes) { - std::ofstream out(path, std::ios::binary | std::ios::trunc); - out.write(bytes.data(), static_cast(bytes.size())); - expect(static_cast(out), "could not write " + path.string()); -} - -void put_u16(std::string & out, uint16_t value) { - out.push_back(static_cast(value & 0xff)); - out.push_back(static_cast((value >> 8) & 0xff)); -} - -void put_u32(std::string & out, uint32_t value) { - for (int shift = 0; shift < 32; shift += 8) { - out.push_back(static_cast((value >> shift) & 0xff)); - } -} - -// 16 kHz mono PCM16 silence. -std::string tiny_wav() { - constexpr uint32_t kSamples = 160; - std::string out = "RIFF"; - put_u32(out, 36 + kSamples * 2); - out += "WAVEfmt "; - put_u32(out, 16); - put_u16(out, 1); - put_u16(out, 1); - put_u32(out, 16000); - put_u32(out, 16000 * 2); - put_u16(out, 2); - put_u16(out, 16); - out += "data"; - put_u32(out, kSamples * 2); - out.append(kSamples * 2, '\0'); - return out; -} - -engine::runtime::TaskRequest speech(const fs::path & voice_dir, bool uses_speaker_embeddings, - const std::string & body_text) { - const std::optional dir = voice_dir; - const std::unordered_map presets; - const std::optional default_preset; - const std::unordered_map default_options; - return minitts::server::build_speech_request( - engine::io::json::parse(body_text), - SpeechRequestContext{voice_dir, dir, presets, default_preset, true, true, true, - default_options, uses_speaker_embeddings}); -} - -bool has_audio(const engine::runtime::TaskRequest & request) { - return request.voice.has_value() && request.voice->speaker.has_value() && - request.voice->speaker->audio.has_value(); -} - -} // namespace - -int main() { - const fs::path dir = fs::temp_directory_path() / - ("audiocpp-voice-library-test-" + std::to_string( - std::chrono::steady_clock::now().time_since_epoch().count())); - try { - fs::create_directories(dir); - // The embedding is never opened here: the model reads it. - write_file(dir / "both.speaker.safetensors", "not read by the server"); - write_file(dir / "both.wav", tiny_wav()); - write_file(dir / "embedding_only.speaker.safetensors", "not read by the server"); - write_file(dir / "wav_only.wav", tiny_wav()); - write_file(dir / "plain.safetensors", "not a speaker embedding name"); - - expect(minitts::server::uses_voice_library_speaker_embeddings("irodori_tts"), - "irodori_tts should use voice library speaker embeddings"); - expect(!minitts::server::uses_voice_library_speaker_embeddings("vieneu_v3_turbo"), - "vieneu_v3_turbo reads another embedding format"); - - // An embedding wins over a wav of the same name and replaces an embedding - // given in options under either name. - auto request = speech(dir, true, - R"({"input":"x","voice":"both","options":{"speaker_embedding_path":"other.safetensors"}})"); - expect(request.options.at("speaker_embedding_file") == - (dir / "both.speaker.safetensors").string(), - "voice did not resolve to the library embedding"); - expect(request.options.count("speaker_embedding_path") == 0, - "legacy embedding option was kept next to the resolved one"); - expect(!has_audio(request), "an embedding voice also loaded reference audio"); - - request = speech(dir, true, R"({"input":"x","voice":"embedding_only"})"); - expect(request.options.at("speaker_embedding_file") == - (dir / "embedding_only.speaker.safetensors").string(), - "embedding-only voice did not resolve"); - - request = speech(dir, true, R"({"input":"x","voice":"wav_only"})"); - expect(has_audio(request) && request.options.count("speaker_embedding_file") == 0, - "wav-only voice did not resolve to reference audio"); - - // Other families keep the wav, and never get the embedding. - request = speech(dir, false, R"({"input":"x","voice":"both"})"); - expect(has_audio(request) && request.options.count("speaker_embedding_file") == 0, - "a family without embeddings did not get the wav"); - request = speech(dir, false, R"({"input":"x","voice":"embedding_only"})"); - expect(request.options.count("speaker_embedding_file") == 0 && - request.voice.has_value() && request.voice->speaker.has_value() && - request.voice->speaker->cached_voice_id == "embedding_only", - "a family without embeddings did not fall back to the cached voice id"); - - // An explicit voice_ref bypasses the library, as for wavs. - request = speech(dir, true, - R"({"input":"x","voice":"embedding_only","voice_ref":")" + - (dir / "wav_only.wav").generic_string() + R"("})"); - expect(request.options.count("speaker_embedding_file") == 0 && has_audio(request), - "voice_ref did not bypass the library embedding"); - - // Names that are paths never leave the library. - request = speech(dir, true, R"({"input":"x","voice":"../embedding_only"})"); - expect(request.options.count("speaker_embedding_file") == 0, - "a path-like voice name resolved to an embedding"); - - auto names = minitts::server::voice_library_speaker_embedding_names(dir); - std::sort(names.begin(), names.end()); - expect(names == std::vector{"both", "embedding_only"}, - "embedding names should be the .speaker.safetensors stems only"); - expect(minitts::server::voice_library_speaker_embedding_names(dir / "missing").empty(), - "a missing voice library should list no embeddings"); - } catch (const std::exception & error) { - std::error_code ec; - fs::remove_all(dir, ec); - std::cerr << "voice library test failed: " << error.what() << "\n"; - return 1; - } - std::error_code ec; - fs::remove_all(dir, ec); - std::cout << "server voice library passed\n"; - return 0; -} diff --git a/tests/unittests/test_server_voice_preset.cpp b/tests/unittests/test_server_voice_preset.cpp new file mode 100644 index 000000000..f11d59892 --- /dev/null +++ b/tests/unittests/test_server_voice_preset.cpp @@ -0,0 +1,99 @@ +// A voice preset's options reach the model as request options: they fill what the +// request leaves unset, for named and default presets alike, and the server never +// interprets them. +#include "server_protocol.h" + +#include "engine/framework/io/json.h" + +#include +#include +#include +#include +#include +#include + +namespace { + +namespace fs = std::filesystem; +using minitts::server::RuntimeVoicePreset; +using minitts::server::SpeechRequestContext; + +void expect(bool condition, const std::string & message) { + if (!condition) { + throw std::runtime_error(message); + } +} + +engine::runtime::TaskRequest speech( + const std::unordered_map & presets, + const std::optional & default_preset, + const std::unordered_map & default_options, + const std::string & body_text) { + const fs::path base = fs::current_path(); + const std::optional voice_dir; + return minitts::server::build_speech_request( + engine::io::json::parse(body_text), + SpeechRequestContext{base, voice_dir, presets, default_preset, true, true, true, default_options}); +} + +} // namespace + +int main() { + try { + RuntimeVoicePreset narrator; + narrator.options = { + {"speaker_embedding_file", "/voices/narrator.speaker.safetensors"}, + {"num_steps", "8"}, + }; + RuntimeVoicePreset with_text; + with_text.reference_text = "preset transcript"; + with_text.options = {{"reference_text", "option transcript"}}; + const std::unordered_map presets{ + {"narrator", narrator}, + {"with_text", with_text}, + }; + const std::optional no_default; + const std::unordered_map no_defaults; + + auto request = speech(presets, no_default, no_defaults, R"({"input":"x","voice":"narrator"})"); + expect(request.options.at("speaker_embedding_file") == "/voices/narrator.speaker.safetensors", + "preset option did not reach the request"); + expect(request.options.at("num_steps") == "8", "second preset option did not reach the request"); + expect(!request.voice.has_value(), "an options-only preset should not add a voice"); + + // The request's own options win over the preset's. + request = speech(presets, no_default, no_defaults, + R"({"input":"x","voice":"narrator","options":{"num_steps":4}})"); + expect(request.options.at("num_steps") == "4", "request option did not override the preset"); + expect(request.options.at("speaker_embedding_file") == "/voices/narrator.speaker.safetensors", + "unrelated preset option was dropped"); + + // The preset wins over the model's default_request_options. + const std::unordered_map defaults{{"num_steps", "32"}, {"cfg_scale", "2"}}; + request = speech(presets, no_default, defaults, R"({"input":"x","voice":"narrator"})"); + expect(request.options.at("num_steps") == "8", "default_request_options overrode the preset"); + expect(request.options.at("cfg_scale") == "2", "default_request_options were dropped"); + + // The dedicated reference_text field keeps its meaning. + request = speech(presets, no_default, no_defaults, R"({"input":"x","voice":"with_text"})"); + expect(request.options.at("reference_text") == "preset transcript", + "preset reference_text should win over the same key in its options"); + + // A default preset carries its options too, until the request names a voice. + const std::optional default_preset = narrator; + request = speech(presets, default_preset, no_defaults, R"({"input":"x"})"); + expect(request.options.at("speaker_embedding_file") == "/voices/narrator.speaker.safetensors", + "default preset options did not reach the request"); + request = speech(presets, default_preset, no_defaults, R"({"input":"x","voice":"alba"})"); + expect(request.options.count("speaker_embedding_file") == 0, + "default preset options leaked into a request that named another voice"); + expect(request.voice.has_value() && request.voice->speaker.has_value() && + request.voice->speaker->cached_voice_id == "alba", + "a voice that is not a preset should stay a model-native voice id"); + } catch (const std::exception & error) { + std::cerr << "voice preset test failed: " << error.what() << "\n"; + return 1; + } + std::cout << "server voice preset passed\n"; + return 0; +} From 5937a6ab4b7c8500f9ba6d589e000a4b1329c41e Mon Sep 17 00:00:00 2001 From: 0xShug0 <231717474+0xShug0@users.noreply.github.com> Date: Sat, 17 Oct 2026 19:01:28 -0400 Subject: [PATCH 3/3] Reuse preset option merge helper and clarify precedence --- app/server/README.md | 9 +++++---- app/server/api.md | 2 +- app/server/server_protocol.cpp | 6 +----- 3 files changed, 7 insertions(+), 10 deletions(-) diff --git a/app/server/README.md b/app/server/README.md index d5e6274fc..37479fa81 100644 --- a/app/server/README.md +++ b/app/server/README.md @@ -520,10 +520,11 @@ reference library before passing it through as a model-native voice ID. For built-in speakers, configure `voice_id` using an ID valid for that model, such as `af_heart` for Kokoro; preset names and native voice IDs are not universal. -A preset can also carry request `options` for the model. The server passes them -through as given, after the request's own options, so a request that sets the -same option keeps its value; the model validates them like any other request -option. For example, an Irodori-TTS speaker embedding selected by name: +A preset can also carry request `options` for the model. Explicit request options +override preset options with the same key. Different aliases are not merged; +supplying conflicting aliases may produce a model validation error. The model +validates preset options like any other request option. For example, an +Irodori-TTS speaker embedding selected by name: ```json { diff --git a/app/server/api.md b/app/server/api.md index 72dc521d6..1edf318f7 100644 --- a/app/server/api.md +++ b/app/server/api.md @@ -84,7 +84,7 @@ option parsing. If no request voice is provided and the configured model has `default_voice_preset`, the server injects that preset automatically. Request-level `voice`, `voice_ref`, and `reference_text` override the configured default. -A preset's configured `options` are added to the request's `options`; an option the request sets itself keeps the request's value. +A preset's configured `options` fill missing request keys. Explicit request options override preset options with the same key. Different aliases are not merged; supplying conflicting aliases may produce a model validation error. `voice_ref` accepts either a plain path string (server-side file) or an object with a `type`: diff --git a/app/server/server_protocol.cpp b/app/server/server_protocol.cpp index 319bb5f43..0d76e9c91 100644 --- a/app/server/server_protocol.cpp +++ b/app/server/server_protocol.cpp @@ -121,11 +121,7 @@ build_speech_request(const Value &body, const SpeechRequestContext &context) { request.options.find("reference_text") == request.options.end()) { request.options["reference_text"] = *preset->reference_text; } - // Preset options fill what the request leaves unset; the model validates - // them like any other request option. - for (const auto &[key, value] : preset->options) { - request.options.emplace(key, value); - } + request = apply_default_request_options(preset->options, std::move(request)); } const bool has_explicit_voice_ref = body.find("voice_ref") != nullptr;