[bong-water-water-bong]

This opens the Laya router to the NPU (geramyl: "HRX + LOOM + NPU"). The engine gets only a generic hook. The NPU code stays in the private add-on.

Engine side (public):

• laya/encoder.h: a registry of encoders by name. An encoder gets a sequence's embedded rows [rows][1024] with n real tokens, runs the 28 layers in place, and may decline.
• Scorer::set_encoder: score() hands every sequence to the set encoder. If the encoder declines any of them, the whole batch runs on the CPU as before.
• serve --laya with ONEBIT_LAYA_DEVICE=npu uses the C++ scorer with the npu encoder instead of the GGUF daemon. It fails before loading the checkpoint, saying how to build, when no add-on registered one.
• tests/laya_encoder_test.cpp (ctest laya_encoder) covers the registry. tests/laya_gguf_route.sh checks the missing-add-on message.
• Docs: docs/laya.md ("The encoder on the NPU") and docs/npu.md.

Private side, for reference: npu-kernels addons/npu-laya registers npu. It takes the engine's NPU device lock without waiting, and declines while another process holds it, so those decisions run on the CPU. It keeps its hardware context together with the lock for 2 s after a call, then releases both.

Measured on strixhalo (serve --laya with fake backends and tests/route-policy-e2e.json, real checkpoint):

• Classes route correctly: code → hrx, prose → zinc, short → vulkan.
• About 105 ms per warm decision: the NPU encoder is ~50 ms, the CPU head ~41 ms.
• About 500 ms per decision while a peer holds the NPU lock (CPU fallback).
• The lock is free again after 2 s idle.
• Routing accuracy on the 200 cases with the NPU encoder is 194/200 = 97.0%; the CPU scorer gets 95.5%.

Local ctest passes 13/13.

🤖 Generated with Claude Code

[context7]

Docs7 for 1bit-monster/engine

Result	Status	Action
Deployment	➖ Not used	—
Content review	➖ Did not run. This site has no agent runs available this month. Wait for the monthly reset or check your Docs7 plan.	—

Commit fcda1bd

[github-actions]

Preparing review...

[github-actions]

Preparing review...

[github-actions]

Preparing review...

[github-actions]

PR Reviewer Guide 🔍

(Review updated until commit fcda1bd)

Here are some key observations to aid the review process:

⏱️ Estimated effort to review: 3 🔵🔵🔵⚪⚪

🧪 PR contains tests

🔒 No security concerns identified

⚡ Recommended focus areas for review Incorrect NPU encoder check The check for ONEBIT_LAYA_DEVICE=npu occurs before the laya_daemon is initialized, but the laya_daemon is only created when !gguf.empty() && !bin.empty(). If gguf is empty but bin is not, or vice versa, the check for laya_npu will be skipped, and the code will proceed to try to load the encoder without ensuring that the NPU add-on is available. This could lead to a runtime error or an incorrect fallback behavior. const onebit::laya::EncoderFactory* f = laya_npu ? onebit::laya::find_encoder("npu") : nullptr; if (laya_npu && !f) throw std::runtime_error("laya: ONEBIT_LAYA_DEVICE=npu needs a build with the NPU add-on " "(-DONEBIT_NPU_PRIVATE=<npu-kernels checkout>, docs/laya.md)"); Potential performance issue in encoder loop The encoder loop iterates over all sequences in the batch (for (int b = 0; b < N && encoded; ++b)), and if any sequence is declined by the encoder, the entire batch falls back to CPU execution. This could lead to performance degradation if a small number of sequences are declined, as the entire batch must be processed on the CPU instead of just the declined ones. for (int b = 0; b < N && encoded; ++b) encoded = encoder_(out.data() + (size_t)b * L * D, L, (int)seqs[b].size()); if (encoded) { h.swap(out); encoded_ += N; } else { declined_ += N; } }

[github-actions]

Persistent review updated to latest commit fcda1bd