npu: the t2s leaderboard rows say one run, and what the pair did not test

bench/npu-leaderboard/bonsai2-27b-pb16i-20261102.json

@@ -17,8 +17,8 @@
     "stream_per_token_gb": 12.93,
     "tokens": 7
   },
-  "instrument": "NPU_PHASE_PROF=1, the serve's own [ph] tok wall= line, median of the 7 steady-state tokens (first token dropped); request walls are not this number, they pay the per-serve image fill once",
-  "repeats": 7,
+  "instrument": "NPU_PHASE_PROF=1, one run per arm: the serve's own [ph] tok wall= line, median of its 7 steady-state tokens (first token dropped); request walls are not this number, they pay the per-serve image fill once",
+  "repeats": 1,
   "timestamp": "2026-11-02T23:50:00Z",
   "source": "~/briefs/TBONSAI-SPEED-STATE.md, 2026-11-02 20:05 AST entry: three npu-hold tickets (19:47, 19:48, 19:49 AST) on :18099, amdxdna counter 0 in each, spread 409-412 ms/token",
   "notes": "control arm of the P3 (#802) pair: the same container file, the same binary and the same box one minute from bonsai2-27b-t2s2-20261102.json, so the ratio is a weight-format difference and nothing else. This is the 4.5 bpw path the 2-bit stream replaced; it is not a candidate, it is the denominator. What would make this wrong: measuring it in a different window than the t2s arm (the pair is only valid adjacent)."

bench/npu-leaderboard/bonsai2-27b-t2s2-20261102.json

@@ -17,9 +17,9 @@
     "stream_per_token_gb": 6.11,
     "tokens": 7
   },
-  "instrument": "NPU_PHASE_PROF=1, the serve's own [ph] tok wall= line, median of the 7 steady-state tokens (first token dropped); request walls are not this number, they pay the per-serve image fill once",
-  "repeats": 7,
+  "instrument": "NPU_PHASE_PROF=1, one run per arm: the serve's own [ph] tok wall= line, median of its 7 steady-state tokens (first token dropped); request walls are not this number, they pay the per-serve image fill once",
+  "repeats": 1,
   "timestamp": "2026-11-02T23:50:00Z",
   "source": "~/briefs/TBONSAI-SPEED-STATE.md, 2026-11-02 20:05 AST entry: three npu-hold tickets (19:47, 19:48, 19:49 AST) on :18099, amdxdna counter 0 in each, spread 311-317 ms/token",
-  "notes": "P3 (#802) 'after' arm against bonsai2-27b-pb16i-20261102.json: 1.31x per token and 1.42x on the device wait (209 vs 296 ms). Accept: every decode role byte-identical to the pb16i path (6/6 at layer 0, 8/8 at layer 3, seeds 1/2/3) and greedy text identical on the three-prompt window. Stream is 2.12x fewer bytes (431 images, 14.44 -> 6.82 GB). What would make this wrong: quoting a request wall instead of [ph] tok (a fresh serve adds the ~10.6 s weight-image fill), or comparing across boxes - the pb16i arm is the only valid control."
+  "notes": "P3 (#802) 'after' arm against bonsai2-27b-pb16i-20261102.json: 1.31x per token and 1.42x on the device wait (209 vs 296 ms). Accept is byte-exactness, not a text diff: every decode role matched the pb16i path byte for byte at layer 0 (6/6) and layer 3 (8/8) on three seeds, and greedy text is equal to the pb16i arm's on the prompt both were asked (' Paris.\\nThe capital of Germany is'). A three-prompt full-model text window on this pair was NOT run - the 2026-11-06 re-take was blocked by an unrelated serve holding the engine's device lock - so that remains the open acceptance step. Stream is 2.12x fewer bytes (431 images, 14.44 -> 6.82 GB). What would make this wrong: quoting a request wall instead of [ph] tok (a fresh serve adds the ~10.6 s weight-image fill), or comparing across boxes - the pb16i arm is the only valid control."
 }

docs/npu-leaderboard.md

@@ -24,8 +24,8 @@ One row is one measured run of this engine. A record is only ranked if it names
 
 | model | route | quant | metric | value | settings | repeats | instrument |
 | --- | --- | --- | --- | --- | --- | --- | --- |
-| Bonsai-2-27B | npu | PTQ1_0 ternary, t2s 2-bit stream (2.125 bpw) | decode | 313.0 ms/token | device=accel0, port=18099, container=~/models/npu/bonsai2-27b-t2s2 (q4nx_weight_layout=t2s, version 6, sealed SEALED-T2S.elf, M=16), binary=~/wt/bonsai-speed-build2/1bit (2026-11-02, engine 916e90d + the Hadamard SIMD host path), stream_per_token_gb=6.11, tokens=7 | 7 | NPU_PHASE_PROF=1, the serve's own [ph] tok wall= line, median of the 7 steady-state tokens (first token dropped); request walls are not this number, they pay the per-serve image fill once |
-| Bonsai-2-27B | npu | pb16i (int4 codes + int16 magnitudes, 4.5 bpw) | decode | 410.0 ms/token | device=accel0, port=18099, container=~/models/npu/bonsai2-27b-pb16i (q4nx_weight_layout=pb16i, version 4, M=16), binary=~/wt/bonsai-speed-build2/1bit (2026-11-02, engine 916e90d + the Hadamard SIMD host path), stream_per_token_gb=12.93, tokens=7 | 7 | NPU_PHASE_PROF=1, the serve's own [ph] tok wall= line, median of the 7 steady-state tokens (first token dropped); request walls are not this number, they pay the per-serve image fill once |
+| Bonsai-2-27B | npu | PTQ1_0 ternary, t2s 2-bit stream (2.125 bpw) | decode | 313.0 ms/token | device=accel0, port=18099, container=~/models/npu/bonsai2-27b-t2s2 (q4nx_weight_layout=t2s, version 6, sealed SEALED-T2S.elf, M=16), binary=~/wt/bonsai-speed-build2/1bit (2026-11-02, engine 916e90d + the Hadamard SIMD host path), stream_per_token_gb=6.11, tokens=7 | 1 | NPU_PHASE_PROF=1, one run per arm: the serve's own [ph] tok wall= line, median of its 7 steady-state tokens (first token dropped); request walls are not this number, they pay the per-serve image fill once |
+| Bonsai-2-27B | npu | pb16i (int4 codes + int16 magnitudes, 4.5 bpw) | decode | 410.0 ms/token | device=accel0, port=18099, container=~/models/npu/bonsai2-27b-pb16i (q4nx_weight_layout=pb16i, version 4, M=16), binary=~/wt/bonsai-speed-build2/1bit (2026-11-02, engine 916e90d + the Hadamard SIMD host path), stream_per_token_gb=12.93, tokens=7 | 1 | NPU_PHASE_PROF=1, one run per arm: the serve's own [ph] tok wall= line, median of its 7 steady-state tokens (first token dropped); request walls are not this number, they pay the per-serve image fill once |
 | Flash-Next (IQ4_XS 4.25 bpw) | hrx | IQ4_XS | prefill pp512 | 286.63 tok/s | ngl=99, fa=1, ub=512, engine=0d0c846 | 3 | llama-bench -p 512, 3 repeats, s.d. 0.94 tok/s |
 | Flash-Next (IQ4_XS 4.25 bpw) | hrx | IQ4_XS | prefill p1024 ub576 | 269.34 tok/s | ngl=99, fa=1, ub=576, engine=0d0c846 | 1 | llama-bench -p 1024, 1 repeat |
 | Flash-Next (IQ4_XS 4.25 bpw) | hrx | IQ4_XS | prefill pp4096 | 260.59 tok/s | ngl=99, fa=1, ub=512, engine=0d0c846 | 3 | llama-bench -p 4096, 3 repeats, s.d. 4.51 tok/s |
@@ -46,7 +46,7 @@ One row is one measured run of this engine. A record is only ranked if it names
 | Qwen3.8-27B-NPU2 | npu | Q4NX (int8 GEMM) | first token (weight pack included) | 145.7 s | device=accel0, commits=800cf6f, threads=32 | 1 | request wall and driver trace span |
 | Qwen3.8-27B-NPU2 | npu | Q4NX (int8 GEMM) | first token (weight pack included) | 435.9 s | device=accel0, commits=pre-#622, threads=32 | 1 | driver trace span, gated on ~/npu.lock |
 
-- `bonsai2-27b-t2s2-20261102` — 2026-11-02T23:50:00Z; source: ~/briefs/TBONSAI-SPEED-STATE.md, 2026-11-02 20:05 AST entry: three npu-hold tickets (19:47, 19:48, 19:49 AST) on :18099, amdxdna counter 0 in each, spread 311-317 ms/token; P3 (#802) 'after' arm against bonsai2-27b-pb16i-20261102.json: 1.31x per token and 1.42x on the device wait (209 vs 296 ms). Accept: every decode role byte-identical to the pb16i path (6/6 at layer 0, 8/8 at layer 3, seeds 1/2/3) and greedy text identical on the three-prompt window. Stream is 2.12x fewer bytes (431 images, 14.44 -> 6.82 GB). What would make this wrong: quoting a request wall instead of [ph] tok (a fresh serve adds the ~10.6 s weight-image fill), or comparing across boxes - the pb16i arm is the only valid control.
+- `bonsai2-27b-t2s2-20261102` — 2026-11-02T23:50:00Z; source: ~/briefs/TBONSAI-SPEED-STATE.md, 2026-11-02 20:05 AST entry: three npu-hold tickets (19:47, 19:48, 19:49 AST) on :18099, amdxdna counter 0 in each, spread 311-317 ms/token; P3 (#802) 'after' arm against bonsai2-27b-pb16i-20261102.json: 1.31x per token and 1.42x on the device wait (209 vs 296 ms). Accept is byte-exactness, not a text diff: every decode role matched the pb16i path byte for byte at layer 0 (6/6) and layer 3 (8/8) on three seeds, and greedy text is equal to the pb16i arm's on the prompt both were asked (' Paris.\nThe capital of Germany is'). A three-prompt full-model text window on this pair was NOT run - the 2026-11-06 re-take was blocked by an unrelated serve holding the engine's device lock - so that remains the open acceptance step. Stream is 2.12x fewer bytes (431 images, 14.44 -> 6.82 GB). What would make this wrong: quoting a request wall instead of [ph] tok (a fresh serve adds the ~10.6 s weight-image fill), or comparing across boxes - the pb16i arm is the only valid control.
 - `bonsai2-27b-pb16i-20261102` — 2026-11-02T23:50:00Z; source: ~/briefs/TBONSAI-SPEED-STATE.md, 2026-11-02 20:05 AST entry: three npu-hold tickets (19:47, 19:48, 19:49 AST) on :18099, amdxdna counter 0 in each, spread 409-412 ms/token; control arm of the P3 (#802) pair: the same container file, the same binary and the same box one minute from bonsai2-27b-t2s2-20261102.json, so the ratio is a weight-format difference and nothing else. This is the 4.5 bpw path the 2-bit stream replaced; it is not a candidate, it is the denominator. What would make this wrong: measuring it in a different window than the t2s arm (the pair is only valid adjacent).
 - `flashnext-pp512-20261027` — 2026-10-27T02:38:52Z; source: ~/lb/572/run-20261027T015408Z/a1-timing.out (strixhalo); gufo reference on the same box is 1679 tok/s; the analysis is in docs/hrx.md
 - `flashnext-p1024-ub576-20261027` — 2026-10-27T02:50:00Z; source: ~/lb/572/run-20261027T015408Z/a4-ub576.out (strixhalo); shows the ubatch bound above 512 no longer refuses the run; single repeat, do not read as a tuned number