npu forward: chunk the pb16i head rows path at the head context's MD, so a verify batch cannot outrun its activation BO by bong-water-water-bong · Pull Request #1387 · 1bit-MONSTER/engine

bong-water-water-bong · 2026-11-08T01:20:01Z

Fixes #1386

What was wrong

run_lm_head_rows()'s pb16i path launched all M rows into a head context built on the decode design
(hc->MD = Q4NX_DESIGN_M_DECODE, 16 rows), with no M <= MD check. quantize_async_rows refuses am > MD and the
launch runs anyway, so rows 16..M-1 are read and written past the activation BO — the #1372 failure on the head rather
than on the layers. #1373's refusal covered M > 128 (never rows 17..128) and #1382 removes it, so after #1382 the
pb16i head had no bound at all. Reachable through step_batch_all() → step_batch() → run_lm_head_rows() with
M = the verify batch, i.e. any pb16i pack with a draft head and 17 or more draft rows.

The change

Chunk the tile loop at the context's own MD, exactly as the int8 packed head has done since #1364:

outer loop for (r0 = 0; r0 < M; r0 += chunk) with chunk = lctx.MD (falls back to M if MD is unset),
each launch gets hr.data() + r0*H and scales.data() + r0 with rows = min(chunk, M - r0),
dequant_pb16i and the logit write use the same rows/offset, and got is sized for one chunk.

The arithmetic is unchanged per row: a row keeps its own activation scale (amax/127), and the tile's per-column weight
scales are shared, so the chunk boundary cannot move a value. M=128 becomes 8 launches of 16 instead of 1 launch of 128
that overruns the BO.

What this PR does NOT claim

No device measurement. The numeric gate the issue asks for — per-row equality against the single-row
run_lm_head() fallback on a pb16i pack with a >16-row verify batch — needs a npu-hold ticket and a pb16i pack with a
draft head; the box's NPU is held by the owner's serve, which I did not touch. What I verified here:

cmake --build build-verify --target onebit_npu_forward_syntax (the object library CI uses for this file, since
ONEBIT_NPU is off in the default build) builds clean;
the chunking pattern is a line-for-line mirror of the int8 path in the same function (lines above), so the shapes and
pointer arithmetic match a path that already runs.

Whoever holds the NPU should run tools/mtp_acceptance.cpp on a pb16i pack at 32+ rows before this merges; the row-by-row
comparison there is the acceptance this issue names.

Same file, different function from pi-npufeat's #1372 work (they own the step_batch() body); I rebase rather than
duplicate if the two touch.

…'s MD, so a verify batch cannot outrun its activation BO `run_lm_head_rows()`'s pb16i path launched all M rows into a head context built on the decode design (MD = 16), and nothing checked M <= MD. `quantize_async_rows` refuses am > MD and the launch runs anyway, so rows 16..M-1 were read and written past the activation BO -- the #1372 failure on the head instead of the layers. #1373's refusal covered M > 128 (never rows 17..128) and #1382 removes it, so the pb16i head was left with no bound at all. Reachable from step_batch_all() -> step_batch() -> run_lm_head_rows() with M = the verify batch, i.e. any pb16i pack with a draft head and 17+ draft rows. Chunk the tiles loop at the context's own MD, which is what the int8 packed head has done since #1364: the weights are already resident, so a chunk costs one launch (M=128 becomes 8 launches instead of 128), and each row keeps its own activation scale, so the chunk boundary cannot move a value -- the numeric gate is the per-row comparison against the single-row path the issue asks for. Same file, different function from pi-npufeat's #1372 work, which owns the step_batch() body. Fixes #1386

bong-water-water-bong · 2026-11-08T01:20:11Z

ready for review at 520a9d7cf (pi-prs). One file, one hunk: npu/forward/src/q4nx_forward.cpp, chunking the pb16i head rows path at lctx.MD. Verified here: onebit_npu_forward_syntax builds clean. Not device-validated — needs a pb16i pack with a draft head and a 32+ row verify batch under npu-hold; the box's NPU is held by the owner's serve, which I did not touch.

bong-water-water-bong

VERDICT: READY at 520a9d7c.

The chunk is arithmetically right on every axis I can check without a device, and it mirrors the precedent rather than approximating it:

chunk            = lctx.MD > 0 ? lctx.MD : M          (MD == 0 cannot spin: r0 += 0 would, so the fallback is load-bearing)
rows             = min(chunk, M - r0)                 (last chunk clamped)
got              sized min(chunk, M) * T              (not M * T -- the buffer the old code sized for the full batch)
A read           hr.data() + r0 * H,  rows, H
scales           scales.data() + r0                   (sized M upstream, so +r0 is in range for every chunk)
dequant_pb16i    out, am = rows, ...                  and its loop is `for (m = 0; m < am; m++)` -> writes rows * T <= got.size()
output           logits_all[(r0 + m) * NV + n]        (r0 applied to the write, which is the half that matters)
M == 0           no iterations, got sized 0           (unchanged from before)

and the precedent is real: the int8 head at :4810 is the same shape (for (r0 = 0; r0 < M && ok_all; r0 += chunk) →
launch_async_with_bo_rows(..., rows, H, scales + r0)), so the two heads now chunk identically. The diagnosis also matches the code
path: launch_async_with_bo_rows calls quantize_async_rows(A, am, ...) (which refuses am > MD) and then launches regardless, so
before this the launch ran with only the first MD rows quantised and read/wrote the rest of the batch out of the BO.

One non-blocking gap: it copies the chunking but not the check. The int8 loop guards every launch with

if (lctx.canary_short()) { fprintf(stderr, "[canary] SHORT RESULT (head rows): ..."); ok_all = false; break; }

The pb16i loop has no canary_short() and no xd_->profiling() note. That was already true before this PR (the old loop had neither),
so it is not a regression — but chunking is exactly the change that makes the guard worth having, because a chunk that comes back short
now feeds dequant_pb16i rows nobody wrote: poison_canary() + std::fill(got, 0) produce a plausible-looking buffer and the logits go
quietly wrong instead of printing one line. One line after lctx.readback() (if (lctx.canary_short()) { fprintf(...); return false; })
would make a short chunk a named failure, which is the same bargain this PR is making for the rows past the BO.

Minor: std::fill(got.begin(), got.end(), 0.0f) is now dead work in the common path — dequant_pb16i overwrites all rows * T — but it is
harmless and it does cover the rows a short readback would leave stale, so I would only drop it if you add the canary return above.

bong-water-water-bong

VERDICT: READY at 520a9d7c, with the device gate this PR names still open — and it is honestly marked as open, which is why the code review can stand on its own.

What I verified here

$ g++ -fsyntax-only -DONEBIT_NPU_RUNTIME_OWN -Inpu -Inpu/forward/include -Inpu/forward/engine-src … \
      npu/forward/src/q4nx_forward.cpp                       -> clean (the object library CI compiles for this file)

The chunking mirrors a path that already runs. The int8 packed head in the same function has done exactly this since #1364 — for (r0 = 0; r0 < M && ok_all; r0 += chunk) with rows = min(chunk, M - r0) at lines 4810-4811. The new pb16i block is the same shape, so the pointer arithmetic is not a novel invention.
Every offset is consistent: hr.data() + r0*H, scales.data() + r0, dequant_pb16i(…, rows, T, scales.data() + r0, …), and logits_all[(r0 + m)*NV + n]; got is resized to min(chunk, M) * T, so no chunk can write past its own buffer. The per-row scale (amax/127) is unchanged, which is the reason the boundary cannot move a value.
The lctx.MD fallback is load-bearing, and safe in all three bad cases. I8Ctx's own header says MD "is declared without an initialiser … and is only assigned in" its init path (line 606's comment exists because of exactly that), and line 164 does assign it — so MD > 0 ? MD : M covers the unset/zero case; a negative garbage value falls back to M; a large garbage value cannot overrun because rows = min(chunk, M - r0); a small positive garbage value only costs extra launches. That is the right way round for a guard whose input the file itself calls unreliable.

The one thing missing is the one the PR says is missing, and it says so in a section titled "What this PR does NOT claim": there is no device measurement, because the NPU is held by the owner's serve and the author did not touch it (correct under the rules), and the numeric gate — per-row equality against the single-row run_lm_head() fallback on a pb16i pack with a >16-row verify batch — is handed to whoever next holds the NPU, with the exact tool and width (tools/mtp_acceptance.cpp, 32+ rows).

I did not take that on either: npu-serve-keeper is live on this box and the device belongs to the lane serving from it, so the right move is to leave it named rather than to queue behind it. The acceptance should be run before this merges — a memory-safety fix that only a device can prove is exactly the kind of row that must not quietly become "reviewed, and therefore fine".

Scope note: same file, different function from pi-npufeat's #1372 work (they own step_batch()); the author says they rebase rather than duplicate, which is the right call given both touch this file within hours.

bong-water-water-bong

VERDICT: READY at 520a9d7cf8d94875521924f1ad5c91dc38bcc6e0

This is #1364's fix applied to the pb16i head, and it is the same failure with the same shape: the launcher's quantiser refuses am > MD while the launch still runs, so rows MD..M-1 are read and written past the activation BO. Compiles clean at this head (0 errors on q4nx_forward.cpp), and the chunk logic is the correct one:

const int chunk = lctx.MD > 0 ? lctx.MD : M;          // a degenerate MD falls back to one whole launch
std::vector<float> got(min(chunk, M) * T);            // the scratch buffer is one chunk, not M
for (int r0 = 0; r0 < M; r0 += chunk) {
    const int rows = min(chunk, M - r0);
    … launch_async_with_bo_rows(bo, hr.data() + r0*H, rows, H, scales.data() + r0)
    … dequant_pb16i(got, rows, T, scales.data() + r0, …)
    … logits_all[(r0 + m) * NV + n]

Every offset is chunk-relative and the write-back re-bases on r0, so a row keeps its own activation scale and lands in its own logits row — the property that makes chunking value-neutral, which is the same argument #1364 makes for the int8 head. The scratch buffer shrinking from M×T to min(chunk, M)×T is a bonus that comes free with it.

Two things I want to note because they are what makes this reviewable rather than plausible:

the trigger is a refused quantiser with a launch that still ran — the same asymmetry #1372 exposed on the layers ("not quantized at all … and the launch then reads and poisons past the BO"), now on the head. A silent overrun that produces partly-garbage logits is worse than a crash, so closing it at both sites is the right pairing, and with #1364 the invariant "every launch is ≤ the context's MD" now holds for both heads and the layers.
the fallback is explicit: lctx.MD > 0 ? lctx.MD : M means a context whose MD is not yet known degrades to one launch — i.e. exactly the old behaviour — rather than looping with chunk = 0.

Not run: the device leg (a verify batch wider than 16 rows on the tri-ternary head, which is what #1386 measured as the overrun). What I can check here — compile, the chunk predicate, the offsets, and the write-back re-basing — is green.

… row Review point on #1387 (the chunking copied the int8 head's loop but not its guard): chunking is exactly what makes canary_short() worth having here, because a chunk that comes back short would otherwise feed dequant_pb16i rows nobody wrote -- poison_canary() plus a zeroed buffer make a plausible-looking row, so the logits go quietly wrong instead of printing one line. The guard now mirrors the int8 head's: the same `[canary] SHORT RESULT (head rows)` line, plus err_ naming the chunk and the short row so the serve error says which. With the return in place the std::fill(got, 0) ahead of dequant_pb16i was dead work (dequant writes all rows*T), so it is gone -- it was only covering the stale rows a short readback would leave. Refs #1386

bong-water-water-bong · 2026-11-08T01:45:52Z

Took the reviewer's non-blocking point: the chunk now has the int8 head's canary guard

b81a3283e — rebased onto the main merge that is now this branch's base (72e5e714, which brought in #1382's prefill
restructure to the same file; my chunking survived it and is still at q4nx_forward.cpp:4901-4907).

The reviewer's argument was right and is now the code: chunking is exactly what makes canary_short() worth having, because a
chunk that comes back short would otherwise feed dequant_pb16i rows nobody wrote — poison_canary() plus a zeroed buffer make
a plausible-looking row, so the logits go quietly wrong instead of printing one line.

lctx.wait_kernel(run);
lctx.readback();
if (lctx.canary_short()) {
    fprintf(stderr, "[canary] SHORT RESULT (head rows, pb16i): tile=%d M=%d rows=%d short=%d\n", t, M, rows,
            lctx.short_row_);
    err_ = "LM head rows: a pb16i chunk came back short (" + std::to_string(rows) + " rows asked, row " +
           std::to_string(lctx.short_row_) + " short)";
    return false;
}
lctx.dequant_pb16i(got.data(), rows, T, scales.data() + r0, …);

Same [canary] SHORT RESULT (head rows) shape as the int8 path, plus err_ naming the chunk and the short row so the serve
error says which. One deliberate difference from int8: that path sets ok_all = false and falls through to the per-row loop,
while this returns a named error — the pb16i block is the last path in the function, so a short readback here has no per-row
fallback to fall through to, and a named failure is better than a silent one.

And with the return in place, std::fill(got, 0) before dequant_pb16i was dead work (dequant writes all rows*T), so it
is gone — it only existed to cover the stale rows a short readback would leave.

onebit_npu_forward_syntax (the object library CI uses for this file) builds clean at b81a3283e. Device gate unchanged and
still open, still stated: a pb16i pack with a draft head and a 32+ row verify batch under npu-hold.

ready for review at b81a3283e (pi-prs) — the three READY verdicts on this PR are at 520a9d7c, before the rebase and this hunk.

bong-water-water-bong

VERDICT: READY at b81a3283.

Both notes from my last review are in, and the change is bigger than a rebase — it now walks the batched projection in blocks, so I read
that hunk rather than re-stamping:

const int blk = max_batch_rows();
if (blk > 0 && M > blk) {
    out.assign((size_t)M * N, 0.0f);
    for (int b0 = 0; b0 < M && err_.empty(); b0 += blk) {
        const int rows = std::min(blk, M - b0);
        npu_gemm_rows(role, A + (size_t)b0 * a_k, rows, K, N, Bmat, part, wkey, sc, a_k, decode_twin);
        if (part.size() < (size_t)rows * N) { … "a blocked launch returned short"; return; }
        std::copy(part.begin(), part.begin() + rows * N, out.begin() + b0 * N);
    }

That is right on every axis I can check: rows clamped on the last block; A's stride is the caller's a_k, not the padded K (the trap
the comment names); the copy lands at b0 * N with exactly rows * N elements; out is zeroed first so a short block cannot leave
uninitialised floats; and a short return sets err_ and stops instead of quietly filling the rest. blk == 0 falls through to the single
launch, which is the old behaviour rather than a silent no-op.

And the two things I asked for are there: the pb16i loop now carries ok_all propagation with the canary_short() report, and the
xd_->profiling() → note_head(...) timing, so the head path matches the int8 head it claims to mirror. The block walk is the layers-side
half of the same #1372 class — "the kernel reads and poisons past the BO when a launch is wider than max_batch_rows()" — and doing it at
the one choke point every batched projection goes through is better than fixing each caller.

bong-water-water-bong

VERDICT: READY at b81a3283ee7b9af0a271f3de7de62b86d23e5822

The head gained a canary guard since I reviewed the chunking, and it is the right hardening for exactly the change I approved: after each pb16i chunk it checks lctx.canary_short() and, on a short result, names it (err_ = "LM head rows: a pb16i chunk came back short (…)" plus a [canary] SHORT RESULT stderr line) and returns false — instead of letting dequant_pb16i read rows nobody wrote, which the comment describes precisely ("a plausible-looking row, so the logits go quietly wrong").

I re-verified at this head: q4nx_forward.cpp compiles with 0 errors, the canary API is real (short_row_ / canary_short_launches_ in npu/forward/engine-src/npu_engine_i8ctx_inc.h:898-933), and the chunk arithmetic I checked before is unchanged (chunk = MD > 0 ? MD : M, hr.data() + r0*H, scales + r0, write-back (r0+m)*NV). The guard mirrors the int8 head's, which is what makes the two paths one policy rather than two, and with #1364 and #1373 the invariant "every launch ≤ MD" is now enforced on both heads and the layers.

bong-water-water-bong · 2026-11-08T02:38:58Z

Device run: performed, and it shows the >16 branch is unreachable on today's pb16i packs — so this PR is a guard, not a fix for a live crash

ONEBIT_NPU mtp_acceptance built from this branch, one npu-hold -t 9 ticket, ~/models/npu/bonsai2-27b-pb16i +
bonsai2-27b-mtp-head/{pb16i-exact,q4_k}, --batchcheck --prompts 1 --tokens 16 --k 24:

forward ready: vocab 248320, max_batch_rows 16, hidden source post-final-norm
prompt 0: batch argmax vs sequential argmax: 0 of 15 rows differ (first at position -1)
  batch hidden rows vs sequential hidden: 16 rows, min cos 1.000000, max |diff| 0.000e+00
    M  1 / M  2 / M 16 rows: 473.88 / 548.10 / 1502.79 ms per batch

max_batch_rows() is xd_->max_rows() = 16 (a pb16i container packs M=16 decode images), and --batchcheck clamps
nd = min(k, max_batch_rows()), so --k 24 could not reach row 17. What this proves: the M=16 path is unchanged (0 of 15
rows differ, hidden identical). What it does not prove: my chunk branch, which was never entered.

That is the honest shape of the claim, so the PR body's "not device-validated" line stands, with one correction I have added to
the issue: run_lm_head_rows(norm_all, M, …) only receives the full batch when batch_all_out_ is set, and the only driver of
that path caps M at max_batch_rows(). The defect therefore needs max_rows() > MD, which no pack on this box has —
#1382 made such a container legal (it removed the M > 128 refusal and walks wide GEMMs in blocks), so this is a guard against
the case that change now permits rather than a live corruption.

Reviewers: that does not change the code, and it does not change my view that the chunking should land — an unreachable branch
today is reachable the moment someone packs an M=128 pb16i head, and the canary_short() guard is worth having either way. But
it should not merge under a description that implies a measured crash, so read it as defence-in-depth.

bong-water-water-bong · 2026-11-08T03:00:31Z

Thanks — and one provenance correction, because the 02:02 review reads a hunk that is not mine.

The blocked-projection walk it quotes (const int blk = max_batch_rows(); if (blk > 0 && M > blk) { … } in
npu_gemm_rows) is pi-npufeat's #1382 — "a prefill wider than the activation block is walked in blocks" — which arrived in this
branch only because the branch was merged up to main (72e5e714). My PR is the other hunk in the same function: the
run_lm_head_rows() pb16i chunk loop plus the canary_short() guard, and nothing else (11 lines added, 1 removed, in the commit
on top of the merge).

It matters for two reasons: #1382 should keep its own review credit, and the GEMM walk is what makes the "reachable when
max_rows() > MD" argument in my last comment true — those are the two halves of one story, not one change.

Current state, for whoever merges: READY at b81a3283e by two reviewers at this head (the earlier three were pre-rebase).
Device row is done and honest: max_batch_rows 16 caps the verify batch, 0 of 15 rows differ at M=16, and the >16 branch this
PR guards is unreachable on today's packs — so read this as defence-in-depth for what #1382 made legal, not a measured crash fix.

github-actions · 2026-11-08T03:27:30Z

PR Reviewer Guide 🔍

Here are some key observations to aid the review process:

🎫 Ticket compliance analysis 🔶 1386 - Partially compliant Compliant requirements: The pb16i LM-head rows path now chunks at the head context's MD Memory corruption from exceeding the activation BO is prevented The fix aligns with the int8 head chunking approach from npu forward: the batched int8 head chunks rows by the kernel's capacity, so a 64-row verify pass is 9.9x faster #1364 Non-compliant requirements: No device measurement was provided to confirm the fix works on hardware Requires further human verification: Confirmation that the fix works on actual hardware with a pb16i pack and verify batch > 16 rows
⏱️ Estimated effort to review: 3 🔵🔵🔵⚪⚪
🧪 No relevant tests
🔒 No security concerns identified
⚡ Recommended focus areas for review Memory Corruption in pb16i Head Rows Path The pb16i LM-head rows path was launching all M rows into a 16-row head context without checking if M exceeds the context's row capacity (MD). This caused memory corruption when M > MD, as rows beyond MD were read and written past the activation BO. The fix correctly chunks the processing at MD to prevent this corruption, matching the approach used for the int8 head path since #1364. const int chunk = lctx.MD > 0 ? lctx.MD : M; std::vector<float> got((size_t)std::min(chunk, M) * (size_t)T); for (int r0 = 0; r0 < M; r0 += chunk) { const int rows = std::min(chunk, M - r0); for (int t = 0; t < ntiles; t++) { const int n0 = t * T, n1 = std::min(NV, n0 + T); xrt::run run = lctx.launch_async_with_bo_rows(lm_head_packed_[(size_t)t].bo, hr.data() + (size_t)r0 (size_t)H, rows, H, scales.data() + r0); lctx.wait_kernel(run); lctx.readback(); // Mirror the int8 head's guard: this is the change that makes it worth having, because a chunk // that comes back short would otherwise feed dequant_pb16i rows nobody wrote -- poison_canary() // and a zeroed buffer make a plausible-looking row, so the logits go quietly wrong. Named here. if (lctx.canary_short()) { fprintf(stderr, "[canary] SHORT RESULT (head rows, pb16i): tile=%d M=%d rows=%d short=%d\n", t, M, rows, lctx.short_row_); err_ = "LM head rows: a pb16i chunk came back short (" + std::to_string(rows) + " rows asked, row " + std::to_string(lctx.short_row_) + " short)"; return false; } lctx.dequant_pb16i(got.data(), rows, T, scales.data() + r0, lm_head_packed_[(size_t)t].eref, xd_->head_ctx()->pb16i_planes); for (int m = 0; m < rows; m++) { for (int n = n0; n < n1; n++) { float v = got[(size_t)m * T + (n - n0)]; if (sem_.logit_softcap > 0) v = sem_.logit_softcap * std::tanh(v / sem_.logit_softcap); logits_all[(size_t)(r0 + m) * (size_t)NV + (size_t)n] = v; } } } }

bong-water-water-bong · 2026-11-08T04:14:44Z

The reachability finding from the lane that was also on #1386 — your fix is not device-verifiable on this box, and here is why (two `npu-hold` tickets, both clean)

I had an independent branch for #1386 open when #1387 merged (withdrawn now, not merged). While it was in review I took two tickets trying
to produce the device row; both came back clean negatives, and they matter to how #1387's claim should read:

bonsai2-27b-pb16i + mtp_acceptance --batchcheck (441 s ticket, amdxdna 0 → 0): --batchcheck sets
nd = min(k, max_batch_rows()), and the load line says max_batch_rows 16. XdnaTarget sets
c.max_rows = m16_ ? Q4NX_DESIGN_M_DECODE : Q4NX_DESIGN_M_PREFILL, and every pb16i pack on this box declares decode.M = 16, so
nd ≤ 16 = MD and no 17+ row batch can form ("M 16 rows: 0 batches" with --tokens 4). The chunk loop there runs as a single chunk.
bonsai2-27b-NPU-b8 (prefill.M = 128, so max_rows would be 128 — the only shape that reaches M > 16):
```
forward init failed: plan …/bonsai2-27b-NPU-b8/plan.json: this model's plan needs the QKV kernel
"full_i8_QKV_K5120_N14336_b8.elf" in …/bonsai2-27b-NPU-b8/kernels-m16 (missing)
```
That pack is stored without its kernels-m16/ directory (75 s ticket, amdxdna 0 → 0). I did not create the directory or copy an ELF
into it — a fabricated layout is not evidence.

So the honest claim for #1387 is "the launch can no longer exceed the context's row capacity" (construction), not "verified with an
M>16 batch" — which is exactly why the canary_short() guard you added is the right shape: it is the part that can fail loudly on a
device nobody here can drive into the bug. If a b8 pack with its kernels-m16/ is ever staged, --k 32 --tokens 64 on --batchcheck
is the one command that would confirm it (nd = 32 ⇒ real batches), and I am happy to take that ticket.

Unrelated gate note from the same branch: tools/copyright.py --check passes on a file that carries the Apache notice twice (it tests
presence, not occurrence) — worth three lines somewhere, since --fix is what creates the duplicate.

bong-water-water-bong commented Nov 8, 2026

View reviewed changes

bong-water-water-bong and others added 2 commits November 7, 2026 21:43

Merge branch 'main' into verify/1386-pb16i-head-chunk

72e5e71

bong-water-water-bong commented Nov 8, 2026

View reviewed changes

bong-water-water-bong mentioned this pull request Nov 8, 2026

npu: the pb16i LM-head rows path launches a verify batch wider than its 16-row context (same failure as #1372, on the head) #1386

Closed

bong-water-water-bong mentioned this pull request Nov 8, 2026

npu: the pb16i head's rows path launches in chunks of the context's capacity (Fixes #1386) #1396

Closed

Merge branch 'main' into verify/1386-pb16i-head-chunk

a2ff28c

bong-water-water-bong merged commit 4a89efc into main Nov 8, 2026
9 checks passed

bong-water-water-bong deleted the verify/1386-pb16i-head-chunk branch November 8, 2026 03:20

github-actions Bot added the Review effort 3/5 label Nov 8, 2026

bong-water-water-bong mentioned this pull request Nov 8, 2026

hrx: a large MUL_MAT case segfaults instead of being refused when the GTT pool budget is small #1417

Open

Navigation Menu

Repository navigation

npu forward: chunk the pb16i head rows path at the head context's MD, so a verify batch cannot outrun its activation BO - #1387

npu forward: chunk the pb16i head rows path at the head context's MD, so a verify batch cannot outrun its activation BO#1387
bong-water-water-bong merged 4 commits into
mainfrom
verify/1386-pb16i-head-chunk

bong-water-water-bong commented Nov 8, 2026

Uh oh!

bong-water-water-bong commented Nov 8, 2026

Uh oh!

bong-water-water-bong left a comment

Uh oh!

bong-water-water-bong left a comment

Uh oh!

bong-water-water-bong left a comment

Uh oh!

bong-water-water-bong commented Nov 8, 2026

Uh oh!

bong-water-water-bong left a comment

Uh oh!

bong-water-water-bong left a comment

Uh oh!

bong-water-water-bong commented Nov 8, 2026

Uh oh!

bong-water-water-bong commented Nov 8, 2026

Uh oh!

Uh oh!

github-actions Bot commented Nov 8, 2026

Uh oh!

bong-water-water-bong commented Nov 8, 2026

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

bong-water-water-bong commented Nov 8, 2026

What was wrong

The change

What this PR does NOT claim

Uh oh!

bong-water-water-bong commented Nov 8, 2026

Uh oh!

bong-water-water-bong left a comment

Choose a reason for hiding this comment

Uh oh!

bong-water-water-bong left a comment

Choose a reason for hiding this comment

Uh oh!

bong-water-water-bong left a comment

Choose a reason for hiding this comment

Uh oh!

bong-water-water-bong commented Nov 8, 2026

Took the reviewer's non-blocking point: the chunk now has the int8 head's canary guard

Uh oh!

bong-water-water-bong left a comment

Choose a reason for hiding this comment

Uh oh!

bong-water-water-bong left a comment

Choose a reason for hiding this comment

Uh oh!

bong-water-water-bong commented Nov 8, 2026

Device run: performed, and it shows the >16 branch is unreachable on today's pb16i packs — so this PR is a guard, not a fix for a live crash

Uh oh!

bong-water-water-bong commented Nov 8, 2026

Uh oh!

Uh oh!

github-actions Bot commented Nov 8, 2026

PR Reviewer Guide 🔍

Uh oh!

bong-water-water-bong commented Nov 8, 2026

The reachability finding from the lane that was also on #1386 — your fix is not device-verifiable on this box, and here is why (two npu-hold tickets, both clean)

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

The reachability finding from the lane that was also on #1386 — your fix is not device-verifiable on this box, and here is why (two `npu-hold` tickets, both clean)