npu: pin the t2 padding convention, and say where the fold lives (P2 follow-up)

docs/npu.md

@@ -124,6 +124,24 @@ t2 tile, 2176 bytes, 32 rows (N) x 256 columns (K) of one tensor
 Tile order is the container's everywhere: row-major over the `[N/32][K/256]` grid, so one
 32-column block's whole K extent is contiguous. Geometry: `N % 32 == 0`, `K % 256 == 0`.
 
+**A tensor that is not whole tiles is padded, not refused-and-forgotten.** The padding is code 0 with
+scale `0.0`, i.e. a weight of exactly 0: a padded K column contributes 0 to the reduction whatever the
+activation is, and a padded N row is never read. `tools/q4nx_t2.py` refuses such a tensor by default and
+pads it with `pad=True` (`pad_to_tile`). The chunk has no field for the original shape, so the padded
+shape is the tensor's shape and the consumer skips the rows the writer did not fill. All 431 ternary
+Bonsai-2 images are already whole tiles (5120 / 6144 / 17408-column rows), so this exists for GGUF
+tensors and for any projection whose width is not a multiple of 32.
+
+**The stored weight is the folded (post-rotation) one, and t2 carries no rotation of its own.** A folded
+container's `W'` already has the block Hadamard and the per-width signs baked into the weights; the
+consumer rotates the *activation* into that basis — `A' = H (A . s)`, signs first, then the normalized
+Sylvester-Walsh butterfly, then the `1/sqrt(block)` — and never touches the weight
+(`npu/forward/src/q4nx_forward.cpp::hadamard_rotate`). So a t2 tensor holds `W'` exactly as the pb16i
+image does, the fold declaration stays where it belongs (the container's `q4nx_hadamard`,
+`q4nx_hadamard_block` and the `HAD1` sign table), and the chunk needs no pre/post-rotation flag: reading
+a t2 tensor in the wrong basis is a consumer bug, not a format one. The inverse direction (the tied
+embedding, `inverse_weight_names` in the pack) is `S (H y)`, which round-trips to the identity.
+
 **The scale is fp16, not bf16, and that is load bearing.** Every source format already stores this
 scale as fp16 — PTQ1_0's, TQ1_0's and TQ2_0's `d` are `ggml_half`, and `pb16i`'s `ms * 2^e_ref` is an
 int16 with a per-column exponent — while bf16 has 8 significand bits against fp16's 11. Measured on

tests/q4nx_t2_test.py

@@ -18,7 +18,9 @@
 usage: tests/q4nx_t2_test.py [--images DIR] [--limit N] [--quiet]
 
 Always-on part (no model needed, numpy only):
-  1. geometry: tiles*2176 bytes, 2.125 bits/weight, and a geometry that is not whole tiles refused;
+  1. geometry: tiles*2176 bytes, 2.125 bits/weight, a geometry that is not whole tiles refused, and
+     pad=True zero-padding it (real block bit-exact, padding exactly 0, so a padded K column
+     contributes nothing);
   2. a synthetic ternary tensor round-trips through pack()/unpack() exactly, for several (K, N);
   3. the reserved pattern is never emitted by pack() and is counted by reserved_count();
   4. from_pb16i() reproduces the pb16i image element for element (checked against
@@ -107,6 +109,31 @@ def always_on():
               f"{K}x{N} round-trips exactly")
         check(T.reserved_count(raw, K, N) == 0, f"{K}x{N} pack() emits no reserved pattern")
     check(T.nbytes(250, 32) == 0 and T.nbytes(256, 33) == 0, "a geometry that is not whole tiles is refused")
+    # 1b. a tensor that is not whole tiles: refused by default, zero-padded with pad=True. The
+    #     convention is what a consumer relies on, so assert the arithmetic and not just the size:
+    #     the real block is bit-exact, the padding is exactly 0 (code 0 x scale 0), so a padded K
+    #     column contributes nothing to a dot product whatever the activation is.
+    Kn, Nn = 384, 30                       # neither % 256 nor % 32 (128-wide scale groups, so 384)
+    c_n = rng.integers(-1, 2, size=(Nn, Kn)).astype(np.int8)
+    s_n = (rng.random((Nn, Kn // 128)) * 0.01).astype(np.float16)
+    try:
+        T.pack(c_n, s_n)
+        check(False, "pack() must refuse a tensor that is not whole tiles")
+    except ValueError:
+        pass
+    raw_n = T.pack(c_n, s_n, pad=True)
+    check(len(raw_n) == T.nbytes(512, 32), "pad=True pads 30x300 to the next whole-tile geometry (32x512)")
+    got_n = T.unpack(raw_n, 512, 32)
+    check(np.array_equal(got_n[:Nn, :Kn], c_n.astype(np.float32) * np.repeat(s_n.astype(np.float32), 128, axis=1)),
+          "pad=True leaves the real block bit-exact")
+    check(not got_n[Nn:, :].any() and not got_n[:, Kn:].any(), "the padding decodes as exactly 0 (code 0, scale 0)")
+    codes_p, scales_p = T.pad_to_tile(c_n, s_n)
+    check(not scales_p[Nn:, :].any() and not scales_p[:, Kn // 128:].any(),
+          "the padding carries a zero scale too (0.0, as the format doc says)")
+    act = rng.standard_normal(512).astype(np.float32)
+    check(float(np.abs(got_n @ act - got_n[:, :Kn] @ act[:Kn]).max()) == 0.0,
+          "a padded K column contributes exactly 0 to the reduction")
+
     try:
         T.pack(np.arange(256 * 32).reshape(32, 256) % 3 - 1, np.zeros((32, 2), np.float16))
         check(True, "pack() accepts codes in {-1,0,1}")

tools/q4nx_t2.py

@@ -31,6 +31,10 @@
 Bonsai-2 images, 87.5 % of the scales are not bf16-representable (worst |rel| 3.9e-3) while 100 % are
 fp16-exact. Same two bytes, so 2176 B and 2.125 b/w do not change.
 
+A tensor that is not whole tiles is refused unless `pad=True`, which zero-pads it to the geometry
+(`pad_to_tile`): code 0 with scale 0.0, so a padded K column contributes exactly 0 and a padded N row
+is never read.
+
 Converters here: pb16i -> t2 (the engine's integer-exact image, lossless), and the GGUF kinds
 PTQ1_0 / TQ1_0 -> t2 (lossless: both are ternary with an fp16 scale) and TQ2_0 -> t2 (lossy, refused
 unless asked: TQ2_0's code 3 means +2, which a ternary chunk cannot hold).
@@ -66,9 +70,38 @@ def scale_offset(r, c):
 # ── pack / unpack ────────────────────────────────────────────────────────────────────────────────
 
 
-def pack(codes, scales):
-    """codes int8 [N, K] in {-1, 0, 1}, scales fp16 [N, K//128] -> t2 bytes."""
+def pad_to_tile(codes, scales):
+    """Zero-pad a tensor to t2 geometry (N -> % 32, K -> % 256). Returns (codes, scales).
+
+    This is the pinned convention for a tensor whose shape is not whole tiles (docs/npu.md): the
+    padding is code 0 with scale 0.0, so a padded K column is a weight of exactly 0 and contributes
+    0 to the reduction for any activation, and a padded N row is never read.  The caller keeps the
+    padded shape -- the chunk has no field to record the original one.
+    """
     codes = np.asarray(codes, dtype=np.int8)
+    scales = np.asarray(scales, dtype=np.float16)
+    N, K = codes.shape
+    if scales.shape != (N, K // GROUP):
+        raise ValueError(f"scales must be [N, K//128] = {(N, K // GROUP)}, got {scales.shape}")
+    Np, Kp = -(-N // ROWS) * ROWS, -(-K // COLS) * COLS
+    if (Np, Kp) == (N, K):
+        return codes, scales
+    c = np.zeros((Np, Kp), np.int8)
+    c[:N, :K] = codes
+    s = np.zeros((Np, Kp // GROUP), np.float16)
+    s[:N, :K // GROUP] = scales
+    return c, s
+
+
+def pack(codes, scales, pad=False):
+    """codes int8 [N, K] in {-1, 0, 1}, scales fp16 [N, K//128] -> t2 bytes.
+
+    pad=True zero-pads a tensor that is not whole tiles (pad_to_tile); default refuses it.
+    """
+    codes = np.asarray(codes, dtype=np.int8)
+    scales = np.asarray(scales, dtype=np.float16)
+    if pad:
+        codes, scales = pad_to_tile(codes, scales)
     N, K = codes.shape
     if N % ROWS or K % COLS:
         raise ValueError(f"t2 needs N % {ROWS} == 0 and K % {COLS} == 0, got {N}x{K}")
@@ -211,7 +244,7 @@ def _trits_base3(b, n_trits):
     return out
 
 
-def from_ptq1_0(rows, K, N):
+def from_ptq1_0(rows, K, N, pad=False):
     """PrismML PTQ1_0 (28 B / 128 values: qs[24], qh[2], fp16 d) -> t2. Lossless.
 
     The trit codec is q4nx_pb16i.read_ptq1_0's, not restated here: PTQ1_0 does not lay its 128
@@ -221,10 +254,10 @@ def from_ptq1_0(rows, K, N):
     import q4nx_pb16i as PB
     codes, dbits = PB.read_ptq1_0(bytes(rows), K, N)          # codes [K][N] int8, d bits [K/128][N]
     d = dbits.T.copy().view(np.float16)                       # [N][K/128]
-    return pack(codes.T.copy().astype(np.int8), d)
+    return pack(codes.T.copy().astype(np.int8), d, pad=pad)
 
 
-def from_tq1_0(rows, K, N):
+def from_tq1_0(rows, K, N, pad=False):
     """ggml TQ1_0 (QK_K=256: qs[48], qh[4], fp16 d = 54 B) -> t2. Lossless (ternary, one d per 256 K)."""
     QK, SZ = 256, 54
     rows = np.asarray(rows, dtype=np.uint8)
@@ -244,10 +277,10 @@ def from_tq1_0(rows, K, N):
     tr[:, :, 240:256] = t.transpose(2, 0, 1).reshape(N, K // QK, 16)
     d = np.frombuffer(b[:, :, 52:54].tobytes(), dtype="<f2").reshape(N, K // QK)
     # two 128-K t2 blocks per 256-K TQ1_0 block share the one d
-    return pack(tr.reshape(N, K), np.repeat(d.astype(np.float16), 2, axis=1))
+    return pack(tr.reshape(N, K), np.repeat(d.astype(np.float16), 2, axis=1), pad=pad)
 
 
-def from_tq2_0(rows, K, N, allow_nonternary=False):
+def from_tq2_0(rows, K, N, allow_nonternary=False, pad=False):
     """ggml TQ2_0 (QK_K=256: qs[64], fp16 d) -> t2.
 
     TQ2_0 is NOT ternary: its code 3 means +2. Converting it is refused unless allow_nonternary,
@@ -269,7 +302,7 @@ def from_tq2_0(rows, K, N, allow_nonternary=False):
                          f"not ternary, so not t2; pass allow_nonternary=True to clamp them to +1")
     tr = np.clip(tr, -1, 1)
     d = np.frombuffer(b[:, :, 64:66].tobytes(), dtype="<f2").reshape(N, K // QK)
-    return pack(tr.reshape(N, K), np.repeat(d.astype(np.float16), 2, axis=1)), n_plus2
+    return pack(tr.reshape(N, K), np.repeat(d.astype(np.float16), 2, axis=1), pad=pad), n_plus2
 
 
 if __name__ == "__main__":