Repository navigation
npu plan: a plan that names an op with no kernel here is refused at load, naming the design (rule (c), engine side) - #1420
bong-water-water-bong wants to merge 1 commit into
Conversation
…oad, naming the design (rule (c), engine side) Owner's rule (c): 'plan.json load refuses up front, naming the reason, any op without a kernel for its shape/dtype ... never a silent host fallback.' The plan can be complete as JSON and unservable in fact: designs[].elf is a name, and whether that file is in the kernel directory is a different question from whether the plan is well formed. Until now the answer arrived later -- the target's own probe, at open_prefill, after the setup work. Plan::check_artifacts() asks it at load instead: for each named design, the artifact must be present and non-empty in one of the given kernel directories (the forward's own >= 1 byte rule), and a miss refuses with plan designs[1] (O): the artifact "full_i8_O_K2048_N1024.elf" is in none of 1 kernel dir(s) -- this op has no kernel here which names the design, the role and the artifact. Wired into init_impl before any context is opened, and the target's probe stays as the second line rather than being replaced: this one runs earlier and says more. Deliberately not checked here: whether a weight tensor carries its quantization. That is the container's question (npu/q4nx.h refuses a tensor it cannot decode); this layer only sees the plan, and inventing a second opinion about tensor quantization is how two answers start disagreeing. Tests (tests/npu_plan_test.cpp, no device, temp dirs): both artifacts present is complete; a missing one is refused and the message names designs[1]/O/elf; a second directory is searched first-hit like the forward; a zero-byte file is not a kernel; a plan with no designs section is complete (nothing named, nothing missing); and an empty directory list refuses with 'no kernel directory' rather than crashing.
|
#1420 ready for review at
Two boundary calls worth your eye:
Rows: One environment note rather than a claim: this is a host-only change and I ran it in a host-only build, because my scratch worktree's NPU configure now fails on a stale |
bong-water-water-bong
left a comment
There was a problem hiding this comment.
VERDICT: READY at 6a437e1b6839c2cb862cc33450daf0eb0ef05f45
Plan::check_artifacts() refuses a plan whose designs[] names an ELF that isn't on disk. It runs at load, before any context is opened, and names the design index, role and artifact. That's rule (c) for the part the plan layer can see. The scope it leaves to the container (tensor quantisation, npu/q4nx.h) is the right split, and it's stated where the function is declared.
The wiring checks out:
- it is skipped under
host_gemm_(offline mode opens no ELF, so a missing artifact isn't an error there, consistent withopen_prefill); - it only runs when a plan is loaded;
- it searches
{dir}, the same directoryopen_prefilltakes fromsetup_.kernels_dir, so the two can't disagree on where to look; - the
>= 1byte rule matches the forward's probe; - an empty
designssection passes, so every plan written before the section still loads.
What I ran at this head (base 336fc786, on main; NPU build, thermal-run.sh):
npu_plan_test,npu_plan_op_reference_testandonebit_npu_forward: built clean;ctest -R "npu_plan|designs_parity|design_grid|op_reference": 6/6.npu_plan_op_referenceshows Not Run until its own target is built;tools/copyright.py --check: rc 0.
One latent nit, not reachable today: open_prefill deliberately skips the two pb16i split roles (role_is_pb16i_split; QKVFAQ/QKVFAKV, added for GDN packs at q4nx_forward.cpp:1037), because they "exist only as pb16i kernels … and no int8 ELF on disk". check_artifacts doesn't know that. A plan whose designs lists those two roles under their derived int8 ELF names would be refused for artifacts the forward never opens. Today nothing writes such a plan: tools/npu_plan.py's designs_from_config returns None for GDN configs (linear_num_key_heads). But designs_agree requires the section to match the derived set including those roles, so the first GDN plan with a designs section would hit this. Skipping the QKVFA roles when the pack is pb16i, or checking the ELF the target will actually open, would close it ahead of time.
What — rule (c), engine side
Owner: "
plan.jsonload refuses up front, naming the reason, any op without a kernel for its shape/dtype … never a silent host fallback."A plan can be complete as JSON and unservable in fact:
designs[].elfis a name, and whether that file exists is a different question from whether the plan is well formed. Until now the answer arrived later — the target's own probe insideopen_prefill, after the setup work.Plan::check_artifacts()asks it at load instead:naming the design index, the role and the artifact. It is wired into
init_implbefore any context is opened, and the target's probe stays as the second line rather than being replaced: this one runs earlier and says more.Deliberately not checked here: whether a weight tensor carries its quantisation. That is the container's question (
npu/q4nx.hrefuses a tensor it cannot decode), and this layer only sees the plan — inventing a second opinion about tensor quantisation is how two answers start disagreeing. Stated in the header where the function is declared.Tests (no device, temp dirs)
designs[1],Oand the elf>= 1byte rule the forward's probe uses)designssectionRows
npu_plan(the new cases, plus every earlier one)npu_plan: okctest -R "npu_plan|designs_parity|design_grid|op_reference"Release, host-only tree (build-plan) — this change is device-free by construction; the NPU build in my scratch worktree is currently blocked by a staleONEBIT_NPU_RT_LIBpath, which is an environment issue I am reporting rather than working aroundtools/copyright.py --checkRefs #799.