[bong-water-water-bong]

P3 of the NPU plan (#802), host side only — the device measurement is blocked by the orchestrator's NPU freeze
(two firmware TDRs at 09:48-09:49 in the release gate's NPU row); this lane took no ticket and no launch today.

What this is

A ternary model is 2.125 bits per weight. The pb16i image streams 4.5 (int4 nibbles plus one int16 multiplier
per 32-K group). tools/q4nx_t2s.py derives the stream form the XDNA decode kernel reads, so the codes can travel at
2 bits and be expanded to the mmul's int4 nibbles inside the core, between the DMA and the mmul:

2,176 B per k=256 x n=32 tile, 2.125 b/w      27B decode: pb16i 14.44 GB -> 6.82 GB (0.47222x)
[0, 2048)    2-bit codes, n-major blocks (nb, kb) at (nb*16 + kb)*64; inside a block, byte r*4 + j holds
             the four codes for k = kb*16 + r, n = nb*16 + 4j + {0,1,2,3}, code c at bits 2*c
[2048, 2176) int16 multiplier per 128-K block per column, [nb 2][gb 2][16 c]
expansion:   out[2i] = f(c0) | f(c1)<<4, out[2i+1] = f(c2) | f(c3)<<4, f(v) = v - 1 mod 16 = {15,0,1}

Why it is exact, not a requantisation

• The multiplier is already per-128-K. q4nx_pb16i.pack derives one ms per 128-K block and then repeats it four
  times to reach the image's 32-K granularity, so putting it back at 128-K granularity drops only duplicated bytes. The
  derivation checks that they are duplicates and refuses with a named error otherwise (NotRepresentable) — no
  rounding, ever. (The t2 chunk kind's own gate measured that property at 100 % of blocks for Bonsai-2.)
• The block order is chosen for the expansion, not for the decoder. The mmul's B operand wants element e = r*16 + c
  of a 16x16 block in byte e/2; P2's t2 order (byte r*64 + c/4) would make the expansion a cross-lane transpose.
  With four consecutive-n codes in one wire byte, the expansion is a byte-local 1->2 map — four masks/shifts per output
  vector, no shuffling, no table in DM — and one nb (1,024 B) becomes a single contiguous DMA run.
• Runs do not shrink. The 2,176-B tile is streamed as one contiguous run (the pb16i design uses 18 x 1,024 B): the
  measured 43-50 GB/s needs >= 1 KiB runs, so 2-bit must not buy bytes by shortening runs.

Gate (byte-exact, device-free)

ctest q4nx_t2s (new, 12 checks, numpy only — synthetic weights through the engine's own pb16i packer): size and b/w,
expand() == the image's int4 code bytes block for block, decode() == the image's codes and multipliers,
stream() == from_pb16i_image() byte for byte, the side table identical, and both refusals (a non-shared 128-K
multiplier; a reserved 2-bit pattern).

Run by hand on the real Bonsai-2 images (too big for CI): blk_0_ffn_gate_weight.pb16i, all 10,880 tiles,
0 of 44,564,480 expanded code bytes differ, codes and multipliers identical, 50,135,040 B -> 23,674,880 B; the whole
431-image set 14.44 GB -> 6.82 GB, the same figure the t2 chunk kind reports for its container format.

Also run in the NPU kernel arena against the frozen int32 harness (K=5120, N=17408, seeds 1/2/3): replaying the kernel's
integer arithmetic (int8 x int4 -> int32 per 32-K group, t = P*ms int16 x int16 -> acc32, int32 / acc64) from the t2s
stream reproduces expected with 0 of 278,528 int32 differing, 3/3 seeds.

What is not here

The ELF family that consumes the stream (private npu-kernels; built and DM-checked, unlaunched) and its measured GB/s /
s-token — those arrive as an arena entry and a second PR once the freeze lifts, with numbers in the body. Nothing in this
PR changes an existing path: new tool, new test, one add_test, one docs section.

[github-actions]

PR Reviewer Guide 🔍

(Review updated until commit 9538d31)

Here are some key observations to aid the review process:

🎫 Ticket compliance analysis 🔶 802 - Partially compliant Compliant requirements: Implement a new 2-bit ternary stream (t2s) format that halves the weight bytes and expands them on-core in the XDNA NPU The t2s stream should be derived from the pb16i image and must be byte-exact with the original image The expansion must happen inside the core between DMA and mmul operations The implementation should reduce memory bandwidth usage from 14.44 GB/token to ~6.82 GB/token The solution must handle ternary weights with 2.125 bits per weight The implementation must refuse streams with reserved 2-bit patterns or non-shared 128-K multipliers Non-compliant requirements: The device measurement is blocked by the orchestrator's NPU freeze (this is a host-side only implementation) Requires further human verification: The actual performance improvement on hardware needs to be verified (memory bandwidth reduction) The on-core expansion behavior needs to be validated on actual XDNA hardware

⏱️ Estimated effort to review: 3 🔵🔵🔵⚪⚪

🧪 PR contains tests

🔒 No security concerns identified

⚡ Recommended focus areas for review Reserved code handling The code checks for reserved codes (value 3) in the expand and decode functions, but the check in expand only counts the total number of reserved codes and reports a generic message without indicating which specific codes are problematic. This could make debugging difficult when corrupted streams are encountered. The decode function provides a more specific error message but the expand function should be consistent in providing detailed error information. if (q == RESERVED).any(): raise ValueError(f"{int((q == RESERVED).sum())} reserved codes in the stream; the converter never " f"writes one, so this stream is corrupt") Code duplication in multipliers check The check for non-shared 128-K multipliers in from_pb16i_image function uses np.array_equal with np.repeat to compare 32-K multipliers within a 128-K block. This approach may be inefficient for large tensors and could be simplified by directly checking if all elements in the 4x32-K groups are equal, which would be more readable and potentially faster. ms4 = ms.reshape(-1, NB, 2, 4, 16) if not np.array_equal(ms4, np.repeat(ms4[:, :, :, :1, :], 4, axis=3)): bad = int(np.count_nonzero(ms4 != np.repeat(ms4[:, :, :, :1, :], 4, axis=3))) raise NotRepresentable( f"{bad} of the image's 32-K multipliers are not shared across their 128-K block, so the " f"t2s stream (which carries one multiplier per 128 K) cannot hold this tensor exactly")

[bong-water-water-bong]

VERDICT: READY at fe539f1 (tprs-ready applied)

Re-derived the headline claims on the real images, not only the synthetic fixture.

• python3 tests/q4nx_t2s_test.py → 12/12 PASS, 0 failures (expand matches the pb16i code area byte for byte,
  both refusals, the side table, the 0.472222x ratio, 2.1250 b/w).
• the real tensor in your body, blk_0_ffn_gate_weight.pb16i (K=5120, N=17408), read here:

tiles 10880 (both sides)          stream 23,674,880 B   image 50,135,040 B   ratio 0.47222  (2.1250 b/w)
expand(st) vs the image's int4 code area, block for block:  0 of 44,564,480 bytes differ
decode(st) codes vs _split(image) codes:                    identical
multipliers at 128-K vs the image's 32-K repeats:            identical

My first comparison said "40 million differ" because I compared the expanded output against img[:K*N/2] — wrong
on my side: the image is [tiles][4096 code + 512 ms], so the code bytes are img.reshape(-1,4608)[:, :4096].
With the correct slice it is 0 differ, which is the number your body claims.

• the 431-image figure is arithmetic, not a new measurement: 431 .pb16i files total 14,435,942,400 B = 14.44 GB,
  and since the per-tensor ratio is exactly 2176/4608, the stream total is 6.82 GB — the same 0.47222x.
• tools/copyright.py --check rc 0, and q4nx_t2s is registered as a ctest next to q4nx_t2
  (CMakeLists.txt:458), so CI runs it on every build rather than only by hand.

Not verified by me, stated in the body: the arena int32 replay (0 of 278,528, 3/3 seeds) — private harness, and the
NPU side is frozen; and any device GB/s, which the body already says is blocked. Nothing in the diff changes an
existing path: one new tool, one new test, one add_test, one docs section.

[github-actions]

Persistent review updated to latest commit 8f5c176

[github-actions]

Persistent review updated to latest commit 9538d31

[bong-water-water-bong]

VERDICT: READY at 9538d3155 (head after #822 merged; tprs-ready already on)

9538d31 merges main 3702f78 (#822). Content unchanged, re-checked at it rather than carried over:

• own diff is still the four files (tools/q4nx_t2s.py +177, tests/q4nx_t2s_test.py +125, CMakeLists.txt +4,
  docs/npu.md +24); tools/q4nx_t2s.py and tests/q4nx_t2s_test.py are byte-identical to the head I reviewed
  (fe539f1), and the merge kept both this PR's add_test(NAME q4nx_t2s …) and #820's
  mtp_draft_test … npu/mtp/mtp_head.cpp in CMakeLists.txt;
• python3 tests/q4nx_t2s_test.py → 12/12 checks, 0 failures;
• real-image claim re-derived again at this head, blk_0_ffn_gate_weight.pb16i (K=5120, N=17408): 10,880 tiles,
  stream 23,674,880 B against the image's 50,135,040 B (ratio 0.47222), and expand() → 0 of 44,564,480
  expanded code bytes differ from the pb16i image's int4 code area, block for block;
• CI on this head: build, CodeQL, four Analyze, pr_agent success (331 s), hw_gate skipped;
  mergeable_state=clean.

Unchanged caveats from my first verdict, neither a gate: the arena int32 replay and any device GB/s are the author's
(and the NPU side is frozen), and the PR changes no existing path — new tool, new test, one add_test, one docs
section.