[bong-water-water-bong]

What

tools/q4nx_t2s.py could only be fed from the pb16i image: a model shipped in the plan layer's own
container form (t2, 6.82 GB for Bonsai-2 27B) could not be turned into the stream the XDNA decode kernel
reads without the 14.44 GB sidecar. from_t2(chunk, K, N) closes that gap.

from_t2 unpacks the chunk to its codes and fp16 scale bits, re-derives the image with the packer's own
code (q4nx_pb16i.pack) and re-streams it. Going back through the packer is deliberate: the tile order and
the int16 multipliers are then the packer's, not a second implementation of them (the class of mistake that
cost two bugs in #809 — a hand-written transpose order).

tools/q4nx_t2.unpack is split into unpack_bits (codes int8 [N][K] + the stored fp16 bits [N][K//128])
and unpack (the float32 view), because the conversion needs the stored bits rather than a rounded float.
unpack's signature and arithmetic are unchanged — re-measured, below.

Gate, byte-exact

# the stream contract, synthetic (what CI runs; ctest q4nx_t2s)
python3 tests/q4nx_t2s_test.py
  PASS t2 and t2s are different byte orders  [135476 of 139264 bytes differ]
  PASS from_t2(t2 chunk) == from_pb16i_image()
  ... 14 checks, 0 failures

# every real Bonsai-2 image: t2 chunk -> stream, against the pb16i image -> stream
431 images, 25.66 G weights: 6.82 GB of stream, 431/431 byte-identical via t2 (2.1250 b/w), 648 s
PASS from_t2

# the refactored decoder still passes its own gate over the same 431 images
431 images, 25.66 G weights: pb16i 14.44 GB -> t2 6.82 GB (2.1250 b/w), 431/431 bit-exact
PASS q4nx_t2

The check that matters is two-sided on purpose: from_t2 must equal from_pb16i_image(), and the t2
chunk must differ from the stream (135,476 of 139,264 bytes on the fixture), because t2 is k-major within
each 32-row band and t2s is n-major within each 16x16 block. So a later "simplification" that returns the
chunk unchanged — the one wrong change that would look like a tidy-up — fails the test. Verified by breaking
it in place: a byte passthrough fails, and so does dropping the .T on the scales (both produce a stream that
is not the image's).

Not in this PR

• No device work and no tok/s claim: the kernel that consumes the stream is npu-kernels (private), and
  nothing here touches the array.
• No C++ change, so no ctest/build leg beyond CI's — the t2 chunk's C++ decoder (npu/q4nx.cpp) reads the
  container, not the stream, and is untouched.
• from_t2 inherits pb16i's geometry on top of t2's, i.e. N must be a multiple of 1,024 as well as 32; a t2
  tensor outside that is refused by name (is not a pb16i geometry) rather than silently reshaped. All 431
  Bonsai-2 images are whole images, so this costs nothing today.

Files

file	change
tools/q4nx_t2s.py	from_t2(), one import, one docstring paragraph (additive; the file is the owner's, from #821)
tools/q4nx_t2.py	unpack_bits() extracted, unpack() becomes the float32 wrapper
tests/q4nx_t2s_test.py	check 4b: equality and order-inequality (additive; one import, one docstring line)
docs/npu.md	a paragraph under "The t2s stream": a t2 model needs no sidecar, and the two orders are never interchangeable

python3 tools/copyright.py --check → rc 0.

Context: the m16-ELF item that would let this lane claim a tokens/s number is still blocked (no AIE toolchain
on this box, kernels closed source), so this is the P2-format continuation instead — host-only, no window.

[github-actions]

PR Reviewer Guide 🔍

(Review updated until commit e84e3fe)

Here are some key observations to aid the review process:

🎫 Ticket compliance analysis 🔶 809 - Partially compliant Compliant requirements: Add support for the t2 chunk kind (2-bit ternary + fp16 scale per 128 K, 2.125 b/w) Implement the dequant_tile function for t2 Add support for t2 in the as_q4_1() function Implement the t2_code, t2_scale, and t2_reserved functions Add a tool to pack/unpack t2 data Add tests for the t2 layout and functionality Update documentation to include the t2 chunk kind Non-compliant requirements: None Requires further human verification: None 821 - Partially compliant Compliant requirements: Add support for the t2s stream format (half the weight bytes, expanded on the core) Implement the from_t2 function to derive the t2s stream from a t2 chunk Ensure byte-exact reproduction of the pb16i image stream Add tests for the t2s functionality Update documentation to include the t2s stream format Non-compliant requirements: None Requires further human verification: None

⏱️ Estimated effort to review: 4 🔵🔵🔵🔵⚪

🧪 PR contains tests

🔒 No security concerns identified

⚡ Recommended focus areas for review Code Duplication The unpack function in tools/q4nx_t2.py was refactored to use the new unpack_bits function, but the logic for handling the reserved pattern (3) is duplicated in both functions. This could lead to inconsistencies if changes are made to one but not the other. codes = quad.reshape(rt, kt, ROWS, COLS).transpose(0, 2, 1, 3).reshape(N, K) sb = t[:, CODE_BYTES:].copy().view("<u2").reshape(rt, kt, ROWS, COLS // GROUP) return codes, np.ascontiguousarray(sb.transpose(0, 2, 1, 3).reshape(N, K // GROUP)) def unpack(raw, K, N): """t2 bytes -> float32 [N, K] (the same arithmetic as npu/q4nx.cpp::dequant_tile).""" codes, sb = unpack_bits(raw, K, N) scale = sb.view(np.float16).astype(np.float32) return codes.astype(np.float32) * np.repeat(scale, GROUP, axis=1) Function Clarity The from_t2 function in tools/q4nx_t2s.py re-derives the image using the packer's code, which is good for consistency. However, the comment suggests that this approach is used to avoid a second implementation of the tile order, but it's not immediately clear why this is necessary or how it prevents bugs. def from_t2(chunk, K, N): """A container's t2 chunk -> the t2s stream bytes, without the pb16i image. Re-derives the image with the packer's own code (so the tile order and the int16 multipliers are the packer's, not a second implementation of them) and then streams it; the result is byte- identical to `from_pb16i_image()` on the image the chunk came from. pb16i geometry applies on top of t2's, i.e. N must be a multiple of 1,024 as well as 32. """ codes, bits = q4nx_t2.unpack_bits(chunk, K, N) # [N][K] and [N][K//128], fp16 bits img, _eref, _mode = pack(codes.T.copy(), bits.T.copy(), K, N) return from_pb16i_image(img.reshape(-1), K, N)

[bong-water-water-bong]

VERDICT: READY at 7cc55aa0a — 4 files, +53/−9, and the new from_t2 is pinned equal to from_pb16i_image() by a gate
I ran.

python3 tests/q4nx_t2s_test.py        ->  14 checks, 0 failure(s)
  PASS stream size [139264 B] / ratio 0.47222x / 2.1250 bits per weight
  PASS expand matches the pb16i code area byte for byte [262144 B, 0 differ]
  PASS stream() == from_pb16i_image()
  PASS t2 and t2s are different byte orders [135476 of 139264 bytes differ]
  PASS from_t2(t2 chunk) == from_pb16i_image()
  PASS a non-shared 128-K multiplier is refused
  PASS reserved code refused by expand / by decode

The two-sided check is the right one: equality alone would pass a converter that returned the pb16i image back, and the
135,476-of-139,264 mismatch on the fixture is what proves from_t2 is not that. unpack_bits is the split the body
describes — stored fp16 bits out, float32 view unchanged — and the refactored q4nx_t2 path still passes its own gate.

What I re-derived myself: 23,674,880 = 10,880 tiles x 2,176 for gate/up at K=5120/N=17408, the same stream size #821
measured on the real image, and the 0.472222 ratio. What I did not run is the 431-image leg against the real
Bonsai-2 set (648 s under load); my attempt to spot-check one tensor died because the container on this box carries a
different chunk form for layers.0.mlp.gate_proj (55,705,600 B, not a 23,674,880 B t2 chunk), so that leg stays the
author's number. The synthetic gate pins the same equality against the packer's own code, which is why this is still
READY rather than held.

CI at this head: build pass 5m17s, CodeQL pass, pr_agent pass, hw_gate skipped. Behind main by #824/#825 only
(different files).

[github-actions]

Persistent review updated to latest commit e84e3fe

[bong-water-water-bong]

VERDICT: READY at e84e3fe32 — re-head only: own diff still 4 files / +53/-9, own files byte-identical to 7cc55aa0a,
and I re-ran the gate at this head: python3 tests/q4nx_t2s_test.py → 14 checks, 0 failure(s), including
from_t2(t2 chunk) == from_pb16i_image() and the two-sided "t2 and t2s are different byte orders [135476 of 139264
bytes differ]" that stops a converter passing by returning the input.

The one leg I did not run is unchanged and still the author's: the 431-image comparison (648 s). My attempt at a
one-tensor spot check failed for a reason worth knowing — the container on this box carries a different chunk form for
layers.0.mlp.gate_proj (55,705,600 B, not a 23,674,880 B t2 chunk) — so nothing about that leg changed.

CI at this head: build pass 5m29s, pr_agent pass (then re-run by the label), CodeQL + Analyze pass.
tprs-ready applied.