Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 1 addition & 4 deletions CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -586,16 +586,13 @@ audiocpp_add_model(demucs
audiocpp_add_model(kokoro_tts
SOURCES
src/models/kokoro_tts/assets.cpp
src/models/kokoro_tts/package.cpp
src/models/kokoro_tts/g2p_multilingual.cpp
src/models/kokoro_tts/loader.cpp
src/models/kokoro_tts/frontend.cpp
src/models/kokoro_tts/g2p_en.cpp
src/models/kokoro_tts/predictor.cpp
src/models/kokoro_tts/decoder.cpp
src/models/kokoro_tts/plbert.cpp
src/models/kokoro_tts/session.cpp
INCLUDES engine/models/kokoro_tts/loader.h
INCLUDES engine/models/kokoro_tts/session.h
LOADERS engine::models::kokoro_tts::make_kokoro_tts_loader
)
if(MSVC)
Expand Down
136 changes: 136 additions & 0 deletions docs/models/kokoro_tts.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,136 @@
# Kokoro 82M

Kokoro 82M is a compact multilingual text-to-speech model exposed as
`--family kokoro_tts`. audio.cpp packages the model as standalone GGUF files
with all 54 upstream voice packs.

## Install

```bash
python3 tools/model_manager_v2.py install kokoro_82m_q8_0
```

The default package installs:

```text
models/Kokoro-82M-GGUF/kokoro-82m-q8_0.gguf
```

BF16 is also available:

```bash
python3 tools/model_manager_v2.py install kokoro_82m_bf16
```

## Quick Start

```bash
audiocpp_cli --task tts --family kokoro_tts \
--model models/Kokoro-82M-GGUF/kokoro-82m-q8_0.gguf \
--backend cpu \
--language en-us \
--voice-id af_heart \
--text "Hello from Kokoro." \
--out out.wav
```

Chinese example:

```bash
audiocpp_cli --task tts --family kokoro_tts \
--model models/Kokoro-82M-GGUF/kokoro-82m-q8_0.gguf \
--backend cpu \
--language zh \
--voice-id zf_xiaobei \
--text "你好,这是中文语音测试。" \
--out out_zh.wav
```

## Model

| Field | Value |
|---|---|
| Family | `kokoro_tts` |
| Task | `tts` |
| Modes | `offline` |
| Default package | `kokoro_82m_q8_0` |
| Other package | `kokoro_82m_bf16` |
| Model file | `models/Kokoro-82M-GGUF/kokoro-82m-q8_0.gguf` |

## Voices and Languages

Use `--voice-id <id>` to select one of the packaged voices. The voice prefix
selects the language family:

| Prefix | Language |
|---|---|
| `af`, `am` | American English |
| `bf`, `bm` | British English |
| `ef`, `em` | Spanish |
| `ff` | French |
| `hf`, `hm` | Hindi |
| `if`, `im` | Italian |
| `jf`, `jm` | Japanese |
| `pf`, `pm` | Brazilian Portuguese |
| `zf`, `zm` | Mandarin Chinese |

The request language must match the selected voice. For example, use
`--language zh` with `zf_*` or `zm_*` voices.

## Runtime Resources

The release GGUF includes model weights, config, vocabulary, all voice packs,
and the generated `g2p/ja.json` and `g2p/zh.json` tables.

English, Spanish, French, Hindi, Italian, and Portuguese use the shared eSpeak
runtime. Install or package eSpeak data as described in
[`docs/espeak_phonemizer.md`](../espeak_phonemizer.md).

Chinese works from the release GGUF because `g2p/zh.json` is bundled. Avoid
mixed Latin words inside Chinese text unless they are known to map to Kokoro's
vocabulary.

Japanese also needs MeCab and UniDic. The small release GGUF does not include
UniDic because it is large. Export a local full multilingual GGUF with
`--embed-multilingual-resources` if you need Japanese to work from a bundled
package.

## Options

| Option | Values | Default | Meaning |
|---|---|---:|---|
| `--language <code>` / `--request-option language=<code>` | language code | voice prefix | Text frontend language. |
| `--voice-id <id>` | packaged voice id | `af_heart` | Built-in voice pack. |
| `--seed <n>` / `--request-option seed=<n>` | integer | random | Decoder noise seed. |
| `--text-chunk-size <n>` / `--request-option text_chunk_size=<n>` | integer chars | `240` | Long-form chunk size. |
| `--session-option kokoro_tts.weight_type=<type>` | `native`, `f32`, `f16`, `bf16`, `q8_0` | `native` | Matmul weight storage type. |
| `--session-option kokoro_tts.conv_weight_type=<type>` | `native`, `f32`, `f16` | `native` | Convolution weight storage type. |

For GGUF packages, leave weight options at `native` unless you are testing a
conversion or storage policy.

## Conversion

Convert from the official `hexgrad/Kokoro-82M` source checkout:

```bash
python tools/prepare_kokoro_gguf.py \
--source /path/to/Kokoro-82M \
--output-dir /path/to/Kokoro-82M-GGUF \
--type both \
--overwrite
```

For a fully bundled local multilingual package with eSpeak data and UniDic:

```bash
python tools/prepare_kokoro_gguf.py \
--source /path/to/Kokoro-82M \
--output-dir /path/to/Kokoro-82M-GGUF \
--type both \
--embed-multilingual-resources \
--overwrite
```

The detailed validation notes live in
[`tests/kokoro_tts/MULTILINGUAL_GGUF.md`](../../tests/kokoro_tts/MULTILINGUAL_GGUF.md).
1 change: 1 addition & 0 deletions docs/tts.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,7 @@
| IndexTTS2 | `index_tts2` | `tts` | [IndexTTS](models/index_tts.md) |
| IndexTTS2.5 | `index_tts2` (variant `2.5`) | `tts` | [IndexTTS](models/index_tts.md) |
| Irodori-TTS | `irodori_tts` | `tts`, `vdes` | [Irodori-TTS](#irodori-tts) |
| Kokoro 82M | `kokoro_tts` | `tts` | [Kokoro 82M](models/kokoro_tts.md) |
| GLM-TTS | `glm_tts` | `tts`, `clon` | [GLM-TTS](#glm-tts) |
| Inflect Micro v2 | `inflect_v2` | `tts` | [Inflect v2](#inflect-v2) |
| OuteTTS | `outetts` | `tts`, `clon` | [OuteTTS](#outetts) |
Expand Down
11 changes: 3 additions & 8 deletions include/engine/models/kokoro_tts/assets.h
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
#pragma once

#include "engine/framework/assets/tensor_source.h"
#include "engine/framework/assets/resource_bundle.h"
#include "engine/framework/core/backend_weight_store.h"
#include "engine/framework/core/module.h"
#include "engine/framework/io/json.h"
Expand Down Expand Up @@ -243,9 +244,6 @@ struct KokoroWeights {

std::shared_ptr<engine::core::BackendWeightStore> store;
};
namespace g2p_en {
class EnglishG2P;
}
}

namespace engine::models::kokoro_tts {
Expand All @@ -259,20 +257,17 @@ struct KokoroVoicePack {
};

struct KokoroAssets {
std::shared_ptr<struct KokoroPackage> package;
std::shared_ptr<class MultilingualG2P> multilingual_g2p;
std::filesystem::path model_root;
engine::assets::ResourceBundle resources;
engine::io::json::Value config;
std::shared_ptr<const engine::assets::TensorSource> model_weights;
int64_t context_length = 512;
std::unordered_map<std::string, int32_t> vocab;
std::unordered_map<std::string, KokoroVoicePack> voices;
std::filesystem::path english_lexicon_dir;
std::shared_ptr<const kokoro_ggml::g2p_en::EnglishG2P> english_g2p_us;
std::shared_ptr<const kokoro_ggml::g2p_en::EnglishG2P> english_g2p_gb;
};

std::shared_ptr<const KokoroAssets> load_kokoro_assets(const std::filesystem::path & model_root);
std::shared_ptr<const KokoroAssets> load_kokoro_assets(const std::filesystem::path & model_path);

std::shared_ptr<const kokoro_ggml::KokoroWeights> load_kokoro_backend_weights(
const KokoroAssets & assets,
Expand Down
6 changes: 0 additions & 6 deletions include/engine/models/kokoro_tts/frontend.h
Original file line number Diff line number Diff line change
Expand Up @@ -31,12 +31,6 @@ KokoroFrontendSessionState resolve_kokoro_frontend_session_state(
const std::optional<runtime::VoiceCondition> & voice,
const KokoroAssets & assets);

void validate_kokoro_frontend_session_state(
const runtime::Transcript & text,
const std::optional<runtime::VoiceCondition> & voice,
const KokoroFrontendSessionState & state,
const KokoroAssets & assets);

KokoroSynthesisInput build_kokoro_synthesis_input(
const runtime::Transcript & text,
const KokoroFrontendSessionState & state,
Expand Down
159 changes: 0 additions & 159 deletions include/engine/models/kokoro_tts/g2p_en.h

This file was deleted.

Loading
Loading