Repository navigation
fix(rknn): find an exec_plan engine named next to the model - #963
Merged
Merged
Conversation
Every `.rknn` sidecar takes its compiled engine through `exec_plan` as a path the operator types, and until now that path was used verbatim: a relative name resolved against the process working directory, so a deployment had to type board-absolute paths (`/root/tts-rknn/...`) for files that sit in the model directory. Everywhere else in the runtime a model's files are named relative to the model -- a `ResourceBundle` reference, the `miotts` codec and ASR model defaults, the `chatterbox` conditional tensors, the `yue2` VAE sidecars -- so the sidecar options were the odd ones out. `engine::rknn::resolve_engine_path(artifact, model_root)` joins a relative name onto the model root when that file exists, and returns the artifact unchanged in every other case: an absolute path, an unset artifact, an empty root, and a relative name with nothing under the root (which keeps today's working-directory-relative behaviour, the spelling `docs/models/cosyvoice3.md` already documents). CosyVoice3's ladder resolves per element. The root is the directory the weights live in, not the bundle root. `ResourceBundle::model_root()` is where a package's resources live, and for a GGUF that embeds its sidecars that is a directory materialized from the file, so a file the operator installed next to the GGUF never resolves against it. `load_resource_bundle` now also records `resolve_model_root(model_path)` -- the directory `--model` names -- as `model_directory()`, and all five sidecars (CosyVoice3 flow, Qwen3-TTS speech decoder, Kokoro generator, Piper decoder, Inflect V2 decoder) resolve against it, falling back to `model_root()` for a bundle that was never given one. Validation: - `rknn_engine_test` (existing target): resolves a name that exists under the root, keeps an absolute path, keeps a name with no engine under the root, keeps an unset artifact, and an empty root resolves nothing. Built and run on the host: `TEST PASS`. - `g++ -fsyntax-only` on all five session translation units (`-Iinclude -Iexternal/ggml/include`): clean; `src/framework/rknn/engine.cpp` compiles in the non-RKNN configuration (the resolver is outside `ENGINE_HAVE_RKNN`, so a CPU build links it). - `tools/check_loader_catalog_sync.py` -> ok. - Board (RK3588, native `ENGINE_ENABLE_RKNN=ON` build of this tree, whole tree rebuilt: 0 errors). `cosyvoice3` on `models/CosyVoice3-GGUF/cosyvoice3-q8_0.gguf`, the flow ladder spelled once as three absolute paths and once as three bare file names (`flow_estimator_seq110.rknn`, `flow_estimator_seq200.rknn`, `flow_estimator_seq250.rknn`, symlinks in the GGUF directory), server started from an unrelated working directory, six requests (4/10/14/20/26/34 chars, `seed=1234`, same reference audio): all HTTP 200 in both runs, engine loads 4 (`flow.dit.rknn.loaded_frames 200/250`), and all six waveforms bit-identical between the runs. On a binary without the `model_directory()` fix the same bare run fails every request with `CosyVoice3 flow estimator RKNN: cannot open model flow_estimator_seq200.rknn` and engine loads 0, so the resolution is what carries it.
wOvAN
force-pushed
the
rknn-artifact-model-root
branch
from
October 18, 2026 09:20
e616c87 to
872b80d
Compare
Owner
|
@wOvAN Merged 🎉 |
cunba-ai
pushed a commit
to cunba-ai/audio.cpp
that referenced
this pull request
Oct 28, 2026
…ke-word/Whistle NEON + Drama3 deepening + per-model server device selection Upstream commits a8ba5a3..5074956 (16): - shared object libraries declared in model registrations (0xShug0#966) - OpenWakeWord Alexa GGUF package (0xShug0#968) - Whistle host decoder restored + opt-in NEON acceleration (0xShug0#967) - opt-in NEON network execution for microWakeWord (0xShug0#969) - fish_drama3: per-layer expert counts, speech-only head (0xShug0#959) - server: exec_plan engines counted in free-memory guard (0xShug0#964) - server: config model settings kept on WebUI loads, demo voices with --ui (0xShug0#971) - fix(rknn): exec_plan engine lookup next to the model (0xShug0#963) - webui: configured models treated same with/without UI management (0xShug0#972) - perf(rknn): ladder engine priced by driver report (0xShug0#970) - feat(server): per-model backend device selection (0xShug0#973) - perf(fish_dac): chunked reference encoding cuts VRAM peak (0xShug0#960) - webui: Vietnamese interface translations (0xShug0#839) - Drama3 base/small release cleanup (0xShug0#976) - text chunking: closing quotes kept with their sentence, tag-aware (0xShug0#977) - fish_drama3: default top_k 4097 no cap, matching Fish's Space (0xShug0#978) # Conflicts: # src/community_models/whistle_asr/session.cpp
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
A
.rknnsidecar names its compiled engine throughexec_plan, as a path the operator types. Until now that string was used verbatim, so a relative name resolved against the process working directory: to use an engine that sits in the model directory you had to type a board-absolute path. Every other model-adjacent file in the runtime is nameable relative to the model — aResourceBundlereference, themiottscodec and ASR model defaults, thechatterboxconditional tensors, theyue2VAE sidecars — so the five RKNN artifact options were the odd ones out.engine::rknn::resolve_engine_path(artifact, model_root)joins a relative name onto the model root when that file exists, and returns the artifact unchanged in every other case: absolute path, unset artifact, empty root, and a relative name with nothing under the root — which keeps today's working-directory-relative behaviour, the spellingdocs/models/cosyvoice3.mdalready documents. CosyVoice3's engine ladder resolves per element.Why the root is a new accessor
ResourceBundle::model_root()is where a package's resources live, and for a GGUF source that isassets::prepare_model_directory(model_path).model_root— for a GGUF with embedded sidecars, a directory materialized from the file rather than the directory on disk. Joining an installed file name onto that never hits, which is exactly how this branch first failed on the board (cannot open model flow_estimator_seq200.rknn, engine loads 0).So
load_resource_bundle()now recordsresolve_model_root(model_path)— the canonical directory--modelnames, the same root a GGUF sidecar is found under — asResourceBundle::model_directory(), and the five sidecars resolve against it.model_directory()falls back tomodel_root()for a bundle that was never given one, so nothing outsideload_resource_bundle()changes behaviour.Validation
Host:
rknn_engine_test(existing target, extended) covers: name under root resolves, absolute path unchanged, name with no engine under the root unchanged, unset artifact unchanged, empty root resolves nothing.src/framework/rknn/engine.cppcompiles in the non-RKNN configuration — the resolver lives outsideENGINE_HAVE_RKNN, so a CPU build links it.Board — Orange Pi 5 Plus (RK3588), native build with
-DENGINE_ENABLE_RKNN=ON, whole tree rebuilt, 0 errors. Onecosyvoice3model onmodels/CosyVoice3-GGUF/cosyvoice3-q8_0.gguf, flow ladder spelled twice:(the three names are symlinks in the GGUF directory;
audiocpp_serveris started from an unrelated working directory). Six requests per run, 4/10/14/20/26/34 input chars,seed=1234, same reference audio, one cold server per run:f2864dfeb0f56734cfe56278be87d49ca6f082274762d516b4b8b4ce3c34a7a1e789b8aa7101a7dac84201f1c2cfc43adef00c2af34c12f92b84609499f7626f2032a5752b59b4ceec99ec38c79e7fe1e77335e0bf953223b9fbd13cc94d5b5eEngine loads 4 in both runs (
flow.dit.rknn.loaded_frames 200/250). On a binary without themodel_directory()fix the bare run fails all six requests withCosyVoice3 flow estimator RKNN: cannot open model flow_estimator_seq200.rknnand loads 0 — the resolution, not the working directory, is what carries it.Docs:
docs/build/rockchip-npu.mdstates the resolution rule.Limits