Skip to content

fix(rknn): find an exec_plan engine named next to the model - #963

Merged
0xShug0 merged 1 commit into
0xShug0:mainfrom
wOvAN:rknn-artifact-model-root
Oct 18, 2026
Merged

0xShug0 merged 1 commit into
0xShug0:mainfrom
wOvAN:rknn-artifact-model-root

Conversation

@wOvAN

@wOvAN wOvAN commented Oct 17, 2026

Copy link
Copy Markdown
Contributor

What

A .rknn sidecar names its compiled engine through exec_plan, as a path the operator types. Until now that string was used verbatim, so a relative name resolved against the process working directory: to use an engine that sits in the model directory you had to type a board-absolute path. Every other model-adjacent file in the runtime is nameable relative to the model — a ResourceBundle reference, the miotts codec and ASR model defaults, the chatterbox conditional tensors, the yue2 VAE sidecars — so the five RKNN artifact options were the odd ones out.

engine::rknn::resolve_engine_path(artifact, model_root) joins a relative name onto the model root when that file exists, and returns the artifact unchanged in every other case: absolute path, unset artifact, empty root, and a relative name with nothing under the root — which keeps today's working-directory-relative behaviour, the spelling docs/models/cosyvoice3.md already documents. CosyVoice3's engine ladder resolves per element.

Why the root is a new accessor

ResourceBundle::model_root() is where a package's resources live, and for a GGUF source that is assets::prepare_model_directory(model_path).model_root — for a GGUF with embedded sidecars, a directory materialized from the file rather than the directory on disk. Joining an installed file name onto that never hits, which is exactly how this branch first failed on the board (cannot open model flow_estimator_seq200.rknn, engine loads 0).

So load_resource_bundle() now records resolve_model_root(model_path) — the canonical directory --model names, the same root a GGUF sidecar is found under — as ResourceBundle::model_directory(), and the five sidecars resolve against it. model_directory() falls back to model_root() for a bundle that was never given one, so nothing outside load_resource_bundle() changes behaviour.

Validation

Host:

g++ -std=c++20 -Iinclude -Itests/unittests tests/unittests/test_rknn_engine.cpp src/framework/rknn/engine.cpp -o /tmp/rknn_engine_test && /tmp/rknn_engine_test   ->  TEST PASS
g++ -std=c++20 -fsyntax-only -Iinclude -Iexternal/ggml/include <the five session .cpp>    ->  clean
python3 tools/check_loader_catalog_sync.py --self-test && python3 tools/check_loader_catalog_sync.py   ->  ok

rknn_engine_test (existing target, extended) covers: name under root resolves, absolute path unchanged, name with no engine under the root unchanged, unset artifact unchanged, empty root resolves nothing. src/framework/rknn/engine.cpp compiles in the non-RKNN configuration — the resolver lives outside ENGINE_HAVE_RKNN, so a CPU build links it.

Board — Orange Pi 5 Plus (RK3588), native build with -DENGINE_ENABLE_RKNN=ON, whole tree rebuilt, 0 errors. One cosyvoice3 model on models/CosyVoice3-GGUF/cosyvoice3-q8_0.gguf, flow ladder spelled twice:

abs:  "main": "/root/audio.cpp/models/CosyVoice3-RKNN/flow_estimator_seq110.rknn,...seq200.rknn,...seq250.rknn"
bare: "main": "flow_estimator_seq110.rknn,flow_estimator_seq200.rknn,flow_estimator_seq250.rknn"

(the three names are symlinks in the GGUF directory; audiocpp_server is started from an unrelated working directory). Six requests per run, 4/10/14/20/26/34 input chars, seed=1234, same reference audio, one cold server per run:

chars abs bare identical
4 200 200 md5 f2864dfeb0f56734cfe56278be87d49c
10 200 200 a6f082274762d516b4b8b4ce3c34a7a1
14 200 200 e789b8aa7101a7dac84201f1c2cfc43a
20 200 200 def00c2af34c12f92b84609499f7626f
26 200 200 2032a5752b59b4ceec99ec38c79e7fe1
34 200 200 e77335e0bf953223b9fbd13cc94d5b5e

Engine loads 4 in both runs (flow.dit.rknn.loaded_frames 200/250). On a binary without the model_directory() fix the bare run fails all six requests with CosyVoice3 flow estimator RKNN: cannot open model flow_estimator_seq200.rknn and loads 0 — the resolution, not the working directory, is what carries it.

Docs: docs/build/rockchip-npu.md states the resolution rule.

Limits

  • Resolution is existence-gated by design: a typo'd name that exists nowhere keeps the old behaviour (the engine loader reports it), so the change never silently redirects an artifact that the operator meant to be CWD-relative.
  • Only the five RKNN artifact consumers were wired; no other option goes through the resolver.

Every `.rknn` sidecar takes its compiled engine through `exec_plan` as a path the operator
types, and until now that path was used verbatim: a relative name resolved against the process
working directory, so a deployment had to type board-absolute paths (`/root/tts-rknn/...`) for
files that sit in the model directory. Everywhere else in the runtime a model's files are named
relative to the model -- a `ResourceBundle` reference, the `miotts` codec and ASR model defaults,
the `chatterbox` conditional tensors, the `yue2` VAE sidecars -- so the sidecar options were the
odd ones out.

`engine::rknn::resolve_engine_path(artifact, model_root)` joins a relative name onto the model
root when that file exists, and returns the artifact unchanged in every other case: an absolute
path, an unset artifact, an empty root, and a relative name with nothing under the root (which
keeps today's working-directory-relative behaviour, the spelling `docs/models/cosyvoice3.md`
already documents). CosyVoice3's ladder resolves per element.

The root is the directory the weights live in, not the bundle root. `ResourceBundle::model_root()`
is where a package's resources live, and for a GGUF that embeds its sidecars that is a directory
materialized from the file, so a file the operator installed next to the GGUF never resolves
against it. `load_resource_bundle` now also records `resolve_model_root(model_path)` -- the
directory `--model` names -- as `model_directory()`, and all five sidecars (CosyVoice3 flow,
Qwen3-TTS speech decoder, Kokoro generator, Piper decoder, Inflect V2 decoder) resolve against
it, falling back to `model_root()` for a bundle that was never given one.

Validation:
- `rknn_engine_test` (existing target): resolves a name that exists under the root, keeps an
  absolute path, keeps a name with no engine under the root, keeps an unset artifact, and an
  empty root resolves nothing. Built and run on the host: `TEST PASS`.
- `g++ -fsyntax-only` on all five session translation units (`-Iinclude
  -Iexternal/ggml/include`): clean; `src/framework/rknn/engine.cpp` compiles in the
  non-RKNN configuration (the resolver is outside `ENGINE_HAVE_RKNN`, so a CPU build links it).
- `tools/check_loader_catalog_sync.py` -> ok.
- Board (RK3588, native `ENGINE_ENABLE_RKNN=ON` build of this tree, whole tree rebuilt: 0 errors).
  `cosyvoice3` on `models/CosyVoice3-GGUF/cosyvoice3-q8_0.gguf`, the flow ladder spelled once as
  three absolute paths and once as three bare file names (`flow_estimator_seq110.rknn`,
  `flow_estimator_seq200.rknn`, `flow_estimator_seq250.rknn`, symlinks in the GGUF directory),
  server started from an unrelated working directory, six requests (4/10/14/20/26/34 chars,
  `seed=1234`, same reference audio): all HTTP 200 in both runs, engine loads 4
  (`flow.dit.rknn.loaded_frames 200/250`), and all six waveforms bit-identical between the runs.
  On a binary without the `model_directory()` fix the same bare run fails every request with
  `CosyVoice3 flow estimator RKNN: cannot open model flow_estimator_seq200.rknn` and engine
  loads 0, so the resolution is what carries it.
@wOvAN
wOvAN force-pushed the rknn-artifact-model-root branch from e616c87 to 872b80d Compare October 18, 2026 09:20
@0xShug0
0xShug0 merged commit 1d3881c into 0xShug0:main Oct 18, 2026
11 checks passed
@0xShug0

0xShug0 commented Oct 18, 2026

Copy link
Copy Markdown
Owner

@wOvAN Merged 🎉

cunba-ai pushed a commit to cunba-ai/audio.cpp that referenced this pull request Oct 28, 2026
…ke-word/Whistle NEON + Drama3 deepening + per-model server device selection

Upstream commits a8ba5a3..5074956 (16):
- shared object libraries declared in model registrations (0xShug0#966)
- OpenWakeWord Alexa GGUF package (0xShug0#968)
- Whistle host decoder restored + opt-in NEON acceleration (0xShug0#967)
- opt-in NEON network execution for microWakeWord (0xShug0#969)
- fish_drama3: per-layer expert counts, speech-only head (0xShug0#959)
- server: exec_plan engines counted in free-memory guard (0xShug0#964)
- server: config model settings kept on WebUI loads, demo voices with --ui (0xShug0#971)
- fix(rknn): exec_plan engine lookup next to the model (0xShug0#963)
- webui: configured models treated same with/without UI management (0xShug0#972)
- perf(rknn): ladder engine priced by driver report (0xShug0#970)
- feat(server): per-model backend device selection (0xShug0#973)
- perf(fish_dac): chunked reference encoding cuts VRAM peak (0xShug0#960)
- webui: Vietnamese interface translations (0xShug0#839)
- Drama3 base/small release cleanup (0xShug0#976)
- text chunking: closing quotes kept with their sentence, tag-aware (0xShug0#977)
- fish_drama3: default top_k 4097 no cap, matching Fish's Space (0xShug0#978)

# Conflicts:
#	src/community_models/whistle_asr/session.cpp
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants