[bong-water-water-bong]

What

tools/mtp_depth_pool.py + tests/mtp_depth_pool_test.py (+ CMake registration). No engine behaviour change, no
device: this is the instrument behind this lane's per-depth verdict, moved into the tree so the claim is reproducible
by someone who does not have my home directory.

Why it exists

The MTP brief's clause is "live acceptance equals the offline protocol to within 2 percentage points at every depth"
on 8 prompts × 64 tokens pooled. One mtp_acceptance --live run costs ~90 s of array time plus a model + head load
per process, so an 8-prompt window is two or three npu-hold tickets of ≤ 3 min, and the pooled verdict is the sum
of their per-prompt cells. Pooling those logs by eye produced two wrong numbers in this lane (2026-11-05 05:2x: a window
reported as 20 cells that had 12), so the arithmetic lives in a checked tool and each trap is a refusal:

trap	how it bit	the tool's behaviour
the harness echoes the summary after its exit line, so a cell appears twice	a 12-cell window was pooled as 20 cells	everything after == ticket exited is ignored
a cell is keyed by (prompt, depth)	a prompt slice averaged into a claim reads as an 11-26 pp error	a second row for a key with different counts is refused, never averaged
a truncated log looks exactly like a smaller window	a short read would pass silently	a ticket's own run-level row must equal the sum of its per-prompt rows, or the log is refused
a depth no chain reached has 0/0	reported as "delta +0.00 pp PASS" once (the phantom depth, engine#916)	reported as no denominator and excluded from the scored count

Known-answer check (the port is equivalent, measured)

Pointed at the three real tickets that produced the recorded window:

$ tools/mtp_depth_pool.py --selftest ticket-clause55-p0-20261105-115131.log \
    ticket-clause55-p1-4-20261105-115551.log ticket-clause55-p5-7-20261105-120713.log
pooled 32 scored cells = 8 prompts x 4 depths (prompts 0..7, 3 ticket(s))

depth                live  offline (same positions)      delta  verdict
    1    136/181 = 75.14%          136/181 = 75.14%   +0.00 pp  PASS
    2     86/136 = 63.24%           84/134 = 62.69%   +0.55 pp  PASS
    3      63/86 = 73.26%            60/82 = 73.17%   +0.09 pp  PASS
    4      53/63 = 84.13%            49/59 = 83.05%   +1.08 pp  PASS

4 of 4 scored depths within 2 pp at the same positions
selftest PASS: 4 depths reproduce the recorded 8x64 window to the count
selftest PASS differing duplicate / incomplete read / unkeyed depth row / tail repeat

Same integers as the recorded table, to the count — so this port is the script that made the claim, not a re-derivation
of it. (The tool also refuses three log shapes in --selftest with no logs at all, which is what CI can run.)

Test

tests/mtp_depth_pool_test.py — pure python, fixture tickets, no model and no device, registered as
ctest mtp_depth_pool (0.15 s):

PASS  two agreeing tickets pool without a refusal
PASS  3 prompts x 2 depths = 6 cells (the echoed tail is not counted)
PASS  depth 1 pools to 24/30 = 80.00%     PASS  depth 2 pools to 12/27 = 44.44%
PASS  depth 1 is 10.00 pp out, so the depth itself reads FAIL
PASS  the verdict line counts one pass of two scored depths
PASS  the same ticket twice pools to the same 4 cells, not 8
PASS  a disagreement between tickets is refused (rc=1)      [names prompt 1 depth 2]
PASS  a log missing a cell is refused (rc=1)                [names the short read]
PASS  a depth with an empty denominator does not fail the run
PASS  the empty depth is reported as unmeasured and excluded from the scored count
PASS  the tool's refusal self-test passes
all checks passed

ctest -R "mtp_depth_pool|mtp_draft" → 2/2 passed. Apache notice on both files, tools/copyright.py --check clean.

---

Second commit (9ec6cce8): the pooled verdict now names the depths it could not score, and the prompts it did not cover

Self-found after the first commit, in this file. The tool kept only the rows it could score, so the tool's own
not scored line was dropped — and that line is not exotic: mtp_acceptance prints
depth 4: n/a (live 0/0 reached, offline 15/20) not scored whenever one side has no denominator, and
npu/mtp/mtp_head.h:96 says why it must (at tau>0 the chain gate stops before depths 3-4, which is where the
"0/0 is not agreement" rule came from). So a pool over a four-depth run whose chains reached three depths printed

3 of 3 scored depths within 2 pp at the same positions

— a bare full pass over a table that had silently lost a depth. That is the phantom-depth read (#916) one layer up,
in the layer that produces the deliverable's published verdict.

The same hole on the prompt axis: this lane runs one or two prompts per ticket, so prompts 0,1,5 is a normal pool,
and the header printed prompts 0..5 — the range the set spans, not the set that was measured, while the brief's
window is prompts 0..7.

Both are now reported instead of inferred: an unscored depth is carried through as unmeasured and named, and the
verdict counts it (3 of 3 scored depths within 2 pp at the same positions (1 of 4 depths unmeasured: 4)); a
non-contiguous prompt set names what is missing (prompts 0,1,5 -- NOT a contiguous window: 2,3,4 missing of 0..5).

Red on the old tool, green after (two new fixtures in tests/mtp_depth_pool_test.py):

FAIL  the not-scored depth is printed as unmeasured
FAIL  ... and it is depth 4, the one the log named
FAIL  the verdict says 3 of 4 depths, not a bare '3 of 3': 3 of 3 scored depths within 2 pp ...
FAIL  the header names the hole instead of spanning it: ... (prompts 0..5, 3 ticket(s))

The recorded known answer is unchanged to the count — --selftest over the three clause55 tickets still pools
32 cells = 8 prompts x 4 depths, +0.00 / +0.55 / +0.09 / +1.08 pp, 4 of 4 scored depths, selftest PASS.
tests/mtp_depth_pool_test.py → 21 checks, 0 failed; ctest -R mtp_depth_pool 1/1; tools/copyright.py --check
clean; no device, no model, no token.

---

Third commit (0c2afbc3): the pool states the generation budget it covers

Same family, one more axis. The brief's window is 8 prompts × 64 tokens; the pool reported which prompts and
which depths it scored, but never the token budget. The live per-prompt header carries it
(prompt 0: TEXT IDENTICAL, 64 tokens, accepted 44/80, …), so a 64-token ticket pooled with a 32-token one produced a
table whose shape matched neither — a verdict that reads as a window it was not measured over.

The budget is now in the header (…, 64 tokens/prompt, 3 ticket(s)), and a mixed pool says so per prompt rather than
inheriting whichever ticket was read first:

MIXED token budgets: 64 on prompts 0; 128 on prompts 1 -- the pooled verdict is not one window

Mixing lengths across different prompts stays legal (a longer prompt is a legitimate measurement); the same prompt
measured at two lengths is a contradiction about one cell and is refused, like every other disagreement this tool
already refuses. Only the live header shape is read — its N tokens is r.stats.emitted (the --tokens budget),
while the offline header's number in the same position is the stream length, and reading both would have compared two
different quantities.

Red on the previous commit, green after: a uniform budget is stated in the header, the mixed budget is named per prompt and prompt 3 at 64 and at 32 tokens is refused all fail against 9ec6cce8; the suite is 26/26 now, with
--selftest still reproducing the recorded window to the count (32 cells = 8 prompts x 4 depths,
64 tokens/prompt, +0.00 / +0.55 / +0.09 / +1.08 pp). ctest -R mtp_depth_pool 1/1; copyright --check clean.

---

Fourth commit (56487b49): a pre-reached log is refused for the reason it is actually refused

Found by sweeping the lane's whole log corpus (70 tickets, one at a time) through the tool rather than only the three
logs the known answer uses: 2 of the 16 cell-bearing logs are refused — ticket-match-p0-1-20261102-151015.log
and ticket-match-p0-3-diffs-20261102-152047.log — and both refusals were correct but wrongly explained. Their
per-prompt rows are the pre-reached shape

depth 1: live  90.00 % (18/20)  offline  90.00 % (18/20)  delta  +0.00 pp

whose live rate is the marginal one, not the rate conditional on reaching the depth that the current column
reports — pooling them would mix two quantities, which is exactly the accept-len-vs-tokens/pass class of misread this
lane has already published once. But the tool said "the log was read incompletely, do not pool it", i.e. truncation,
which sends a reader to re-run a complete six-minute ticket.

The two cases are now told apart by structure rather than guessed: run-level rows live after the
== live vs offline at the SAME positions header, so a delta-shaped row before it is a per-prompt row in the old
shape, and the refusal names it (with the count: 8 = 2 prompts × 4 depths, 16 = 4 × 4 — both right). A genuinely short
log still gets the "read incompletely" refusal, unchanged.

Corpus re-swept after the change: 14 pool, 54 skipped as offline-only, the same 2 refused — now for the right
reason. Recorded known answer unchanged to the count. 28/28 checks; ctest -R mtp_depth_pool 1/1; copyright clean.

---

Fifth commit (4f53bf61): the chain width (k) is part of the window too

The last axis the window has. Every --live --match row covers the depths that run drafted, so a k=6 ticket pooled
with k=4 ones builds a six-depth table whose depths 5–6 have cells on the k=6 prompts only — and the verdict then
reads 6 of 6 scored depths within 2 pp at the same positions, a pass over depths five of the tickets never attempted.
Against the brief's four-depth table that is not the claim being made.

k is now read from the rows themselves (the deepest depth a prompt has a row for, scored or not-scored, so a depth the
chain never reached still counts) and printed — …, 64 tokens/prompt, k=4, 3 ticket(s) — or named per prompt when it
differs:

MIXED chain width (k): 4 on prompts 0; 6 on prompts 1 -- the depths above 4 come from the wider prompts only,
not from the whole pool

Pooling different widths stays legal (a k sweep is a legitimate measurement); the table just stops implying the whole
pool reached those depths. A prompt measured twice resolves to the wider run, which is the one whose cells the table
keeps.

Red on 56487b49, green after (three checks); suite 34/34; --selftest still reproduces the recorded 8x64 window to
the count, now labelled k=4; ctest -R mtp_depth_pool 1/1; copyright clean. Still no model, no device, no token.

Summary of the tool's window, after these three commits

pooled 32 scored cells = 8 prompts x 4 depths (prompts 0..7, 64 tokens/prompt, k=4, 3 ticket(s)) — i.e. the header
now states what was measured (prompt set, depth count, generation budget, chain width), the table names depths it
could not score, and the verdict counts them. Every one of those five fields exists because this lane misread a pool
once: the tail echo (12 → 20 cells), the cross-ticket disagreement, the phantom depth (#916), the prompt gap, the mixed
budget, and now the chain width.

---

Sixth commit (69bce5ae): the run identity — vocab, batch width, hidden source — is part of the window

The last input that can change what a cell means, and the one with the largest published effect in this lane: the
hidden source alone moved acceptance between 2.1 and 3.19 tokens/pass. mtp_acceptance prints the identity of the
run that produced a log on one line at init —

forward ready: vocab 248320, max_batch_rows 16, hidden source post-final-norm

— so a pool mixing a residual ticket with post-final-norm ones averages two measurements while reporting one table.
Same for a different vocabulary (different model) and a different batch width (different sealed ELF family). None of that
is visible in the cells; all of it is in the log.

It is now in the header, and a mixed pool names each identity with the ticket it came from:

pooled 32 scored cells = 8 prompts x 4 depths (prompts 0..7, 64 tokens/prompt, k=4, vocab 248320,
       max_batch_rows 16, hidden post-final-norm, 3 ticket(s))

MIXED run identity: vocab 248320/M16/post-final-norm from norm.log; vocab 248320/M16/residual from rsd.log --
a different model, batch width or hidden source is a different measurement, not one pool

Only tickets that contributed cells are listed, so an offline log in the same command cannot fake a mix.

The header now states the whole window

prompt set (and any gap) · depth coverage (and any depth no chain reached) · generation budget · chain width · run
identity. That is the complete set of inputs the logs carry that can change a cell, and every field exists because this
lane misread a pool once: the tail echo (12 → 20 cells), the cross-ticket disagreement, the phantom depth (#916), the
prompt gap, the mixed budget, the chain width, and now the identity.

The lane's own corpus is uniform on all of them (86 logs: one model, vocab 248320, M=16, post-final-norm), so no
recorded number changes — verified: --selftest still reproduces the 8x64 window to the count, and the corpus sweep is
unchanged (14 pool / 54 offline-only / 2 refused for the pre-reached shape). Red on 4f53bf61, green after; suite
36/36; ctest -R mtp_depth_pool 1/1; copyright clean; no model, no device, no token.

---

Seventh commit (7d5f34c6 + 673faa2e): the recorded 8×64 window is now checkable in CI, not only on the box that holds the tickets

A gap in this PR's own evidence chain. The tool's known answer — the per-depth live-vs-offline table the brief is judged
on — could only be reproduced with the original npu-hold logs, which live in one home directory on one machine. So the
artefact that produces the deliverable's published verdict could break its own recorded numbers and CI would stay green:
ctest mtp_depth_pool ran only the refusal self-test, with no logs.

The three tickets now ship as fixtures under tests/fixtures/mtp_clause55/ (prompts 0 / 1‑4 / 5‑7, 78 lines together,
extracted verbatim — the extraction command and the tickets it came from are in the directory's README), and the test runs
--selftest over them:

PASS  the fixtures pool to the recorded 8 x 4 cells
PASS  fixture pool reports 136/181 = 75.14%
PASS  fixture pool reports 86/136 = 63.24%
PASS  fixture pool reports 63/86 = 73.26%
PASS  fixture pool reports 53/63 = 84.13%
PASS  all four depths are inside the bar on the fixtures
PASS  the tool's own known-answer check passes on them

Nothing the tool reads was edited — the [ Ns] stamps, the [live joint …] brackets and the match pN: headers are as
the tickets printed them — so the pool still has to reproduce the counts, instead of a fixture written to agree with the code.

Proved not vacuous: adding 1 to live_n in table() reds 8 checks, including
the shipped fixtures reproduce the recorded 8x64 window (rc=1); reverting returns all checks passed.

Branch is up to date with main (14bda3cf, which now carries #1138 and #1143) — the queue updated it to 84844488 and
my commit was rebased onto that. ctest -R mtp_depth_pool 1/1, copyright --check clean, no model, no device, no token.

---

Eighth commit (3880aef8): the fixtures were never committed — .gitignore:19 is *.log

build on 673faa2e failed in one test, and it was this PR's own:

53 - mtp_depth_pool (Failed)
FAIL  tests/fixtures/mtp_clause55 is incomplete: ['/__w/engine/engine/tests/fixtures/mtp_clause55/p0.log', …]

.gitignore:19 is *.log, so git add -A tests/fixtures/mtp_clause55 committed the README and silently left all three
tickets behind. Locally the test passed because the untracked files were sitting in the working tree; in a clean checkout
there was nothing to read. The guard fired — one environment too late, but correctly.

Two changes so it cannot recur quietly:

• the fixtures carry .txt (not matched by any ignore rule) and the directory README quotes this CI failure as the
  reason, so a rename or fresh clone cannot drop them again;
• tests/mtp_depth_pool_test.py now asserts with git ls-files that all three are tracked before pooling them, so an
  unfixtured tree fails locally instead of only on a clean checkout. Measured: files present but unstaged →
  FAIL the three fixtures are tracked by git (0 of 3); staged → PASS … (3 of 3).

tools/copyright.py gains a narrow exemption for tests/fixtures/mtp_clause55/*.txt — the same case as the existing
tests/golden/*/tokens.txt exemption: these are data a parser reads byte-verbatim, so a notice comment would make the
fixture no longer the log it claims to be (the README carries the notice).

Verified from a clean tree, not from my working copy: a fresh git worktree add at this commit runs the suite 43/43
green — which is what CI will do. copyright --check clean.

[bong-water-water-bong]

VERDICT: READY

I re-derived the pooled table from the raw tickets instead of reading its output, and it matches to the count.

The three tickets the selftest names are still on this box (~/wt/mtp-window/ticket-clause55-{p0-20261105-115131,p1-4-20261105-115551,p5-7-20261105-120713}.log), so this could be run for real:

$ tools/mtp_depth_pool.py --selftest ticket-clause55-p0-... ticket-clause55-p1-4-... ticket-clause55-p5-7-...
pooled 32 scored cells = 8 prompts x 4 depths (prompts 0..7, 3 ticket(s))
    1    136/181 = 75.14%          136/181 = 75.14%   +0.00 pp  PASS
    2     86/136 = 63.24%           84/134 = 62.69%   +0.55 pp  PASS
    3      63/86 = 73.26%            60/82 = 73.17%   +0.09 pp  PASS
    4      53/63 = 84.13%            49/59 = 83.05%   +1.08 pp  PASS
selftest PASS: 4 depths reproduce the recorded 8x64 window to the count
selftest PASS differing duplicate / incomplete read / unkeyed depth row / tail repeat   (each with its message)

Independent arithmetic over the same three logs, parsing the per-prompt (... , reached) / offline conditional rows myself and skipping everything after == ticket ... exited: 32 cells, 8 prompts, four depths -> 136/181, 86/136, 63/86, 53/63 live and 136/181, 84/134, 60/82, 49/59 offline. Same integers.

That exercise also reproduced the reason the file exists, at my expense: my first pass read the run-level delta ... PASS lines instead of the per-prompt detail rows and produced depth 4 34/37 = 91.89% for a window whose real depth-4 cell is 53/63. Nothing in the log is wrong there - those are the other shape - which is exactly the confusion the tool refuses to repeat, and it is why I would rather this arithmetic lived in the tree than in an eye.

tests/mtp_depth_pool_test.py: 13 PASS lines, all checks passed, rc 0, no model and no device.

Cross-check against the docs PR in the same queue: engine #1150 quotes 75.14 / 63.24 / 73.26 / 84.13 % and +0.00/+0.55/+0.09/+1.08 pp; those are these pooled numbers, so the two PRs agree with each other and with the tickets.

[bong-water-water-bong]

VERDICT: READY

Re-verdicted at 56487b49 (three commits since a26ef180, all refusals/wording in the tool and its test). The pooled verdict is unchanged by them, and I re-derived it again from the tickets rather than reading the tool's output:

$ tools/mtp_depth_pool.py --selftest ticket-clause55-p0-… ticket-clause55-p1-4-… ticket-clause55-p5-7-…
pooled 32 scored cells = 8 prompts x 4 depths (prompts 0..7, 3 ticket(s))
    1 136/181 = 75.14%  136/181 = 75.14%  +0.00 pp PASS
    2  86/136 = 63.24%   84/134 = 62.69%  +0.55 pp PASS
    3   63/86 = 73.26%    60/82 = 73.17%  +0.09 pp PASS
    4   53/63 = 84.13%    49/59 = 83.05%  +1.08 pp PASS
selftest PASS: 4 depths reproduce the recorded 8x64 window to the count
selftest PASS differing duplicate / incomplete read / unkeyed depth row / tail repeat

tests/mtp_depth_pool_test.py -> all checks passed, including the new "the refusal names the real cause, not a short read" case.

The new coverage line is the useful part and it behaves: with one ticket instead of three the header says 4 scored cells = 1 prompts x 4 depths (prompts 0..0, 64 tokens/prompt, 1 ticket(s)) — it states the window it actually covered instead of echoing the brief's 8x64, which is the shape this lane already got wrong twice. And per-prompt generation budgets are now tracked (budget[prompt]), with a second length for the same prompt refused rather than pooled; that is the case the test names.

Nothing else in the PR changed: still no model, no device, copyright --check clean.

[github-actions]

PR Reviewer Guide 🔍

(Review updated until commit b3bc7dd)

Here are some key observations to aid the review process:

🎫 Ticket compliance analysis 🔶 916 - Partially compliant Compliant requirements: Fix the arithmetic and verdict logic for depth scoring to correctly handle cases where either side has no denominator Ensure that depths with no chain on either side are marked as unmeasured, not a pass Update both print sites in tools/mtp_acceptance.cpp to reflect the new logic Ensure the summary now reads "N of M scored depths within 2 pp" and explicitly states when a depth was not scored Non-compliant requirements: None Requires further human verification: None 1138 - Partially compliant Compliant requirements: Fix the draft loop to advance over either documented hidden-row shape instead of failing the run Ensure that the loop takes the first accepted rows of either shape and fails loudly on any other count Ensure that the fix handles cases where a base keeps its batch buffer Ensure that the fix does not reintroduce silence on wrong row counts Non-compliant requirements: None Requires further human verification: None 1143 - Partially compliant Compliant requirements: Update npu/mtp/forward_base.h to correctly describe the deferral mechanism used Remove dead members and comments that are no longer accurate Ensure last_hidden_rows() is marked with override to prevent silent failures Update comments to accurately reflect the code behavior Non-compliant requirements: None Requires further human verification: None

⏱️ Estimated effort to review: 4 🔵🔵🔵🔵⚪

🧪 PR contains tests

🔒 No security concerns identified

⚡ Recommended focus areas for review Possible Logic Issue in Pooling The tool's logic for pooling per-depth verdicts may have an issue with how it handles depths that are not scored. Specifically, when a depth is marked as not scored in a log, it is carried through as unmeasured, but the tool does not seem to ensure that such depths are properly excluded from the scoring count in the final verdict line. This could lead to a discrepancy between the number of scored depths and the actual depths that were measured. for d in sorted(unmeasured): # Named, not omitted: the depth was asked for and no chain in any ticket reached it, so it is # unmeasured -- and a table that simply lacks the row reads as a window that shrank to fit. print(f"{d:>5} unmeasured -- no chain reached this depth in any ticket; not scored") tail = "" if unmeasured: tail = (f" ({len(unmeasured)} of {len(depths) + len(unmeasured)} depths unmeasured: " f"{','.join(str(d) for d in sorted(unmeasured))})") print(f"\n{inside} of {scored} scored depths within {BAR:g} pp at the same positions{tail}") return inside, scored, totals Potential Incomplete Refusal Handling The tool includes a self-test for refusal cases, but it may not cover all possible edge cases where logs might be malformed or contain unexpected data. Specifically, the refusal logic for logs with pre-reached shape could be more robust to ensure that such logs are properly refused instead of being silently handled. # The tail trap: the same cells repeated after the exit line must not double the window. d1 = "depth 1: live 80.00 % (8/10, reached) offline conditional 80.00 % (8/10)\n" run = "depth 1: live 80.00 % (8/10) offline 80.00 % (8/10) delta +0.00 pp PASS\n" tail = ("== ticket exited rc=0 (hold reached)\n" + "prompt 0: TEXT IDENTICAL\n" + d1 + "depth 1: live 40.00 % (4/10, reached) offline conditional 40.00 % (4/10)\n") with tempfile.NamedTemporaryFile("w", suffix=".log", delete=False) as fh: fh.write("prompt 0: TEXT IDENTICAL\n" + d1 + run + tail) path = fh.name try: cells, _, _, _, _ = parse_ticket(path) if cells != {(0, 1): (8, 10, 8, 10)}: print(f"selftest FAIL tail repeat: parsed {cells}, wanted only the first row", file=sys.stderr) return 1 print("selftest PASS tail repeat: the echoed summary after the exit line is ignored") finally: os.unlink(path) return 0

[bong-water-water-bong]

VERDICT: READY

Re-verdicted at 69bce5ae (two commits since 56487b49, both about stating the run's identity). The pooled table is unchanged to the count, re-derived again from the three real tickets on this box:

    1    136/181 = 75.14%          136/181 = 75.14%   +0.00 pp  PASS
    2     86/136 = 63.24%           84/134 = 62.69%   +0.55 pp  PASS
    3      63/86 = 73.26%            60/82 = 73.17%   +0.09 pp  PASS
    4      53/63 = 84.13%            49/59 = 83.05%   +1.08 pp  PASS

and the header now says what those numbers were measured on, which is the change:

pooled 32 scored cells = 8 prompts x 4 depths (prompts 0..7, 64 tokens/prompt, k=4,
vocab 248320, max_batch_rows 16, hidden post-final-norm, 3 ticket(s))

vocab 248320 matches the model container's own config.json (vocab_size: 248320 for bonsai2-27b-*), and max_batch_rows 16 / hidden post-final-norm are the two knobs that would silently change what a per-depth cell means - a pooled verdict that does not state them can be compared across two different runs and read as a regression or a win. Three separate refusals now guard that identity: a second length for one prompt, a chain width that disagrees, and a mixed vocab/batch/hidden.

tests/mtp_depth_pool_test.py -> all checks passed, including the refusal self-tests; the four named refusals still fire with their messages (differing duplicate, incomplete read, unkeyed depth row, pre-reached log).

[github-actions]

Persistent review updated to latest commit 45f9194

[bong-water-water-bong]

VERDICT: READY

Re-verdicted at e44ba81f. The only commit since 69bce5ae is Merge remote-tracking branch 'ssh-origin/main', and this PR's own files are byte-identical across the two heads (tools/mtp_depth_pool.py, tests/mtp_depth_pool_test.py); the 30 files in the merge are other lanes' landings.

So the verdict from 69bce5ae stands: the pooled table re-derived from the three tickets (136/181, 86/136, 63/86, 53/63 and the offline 136/181, 84/134, 60/82, 49/59), the header stating 64 tokens/prompt, k=4, vocab 248320, max_batch_rows 16, hidden post-final-norm, and tests/mtp_depth_pool_test.py -> all checks passed with its four refusals firing.

[github-actions]

Persistent review updated to latest commit 45f9194

[bong-water-water-bong]

VERDICT: CHANGES

The new test is red in the tree as pushed, and the reason is .gitignore. tests/mtp_depth_pool_test.py case 15 requires three files, and only the README is in the commit:

$ git ls-tree -r --name-only HEAD tests/fixtures/mtp_clause55/
tests/fixtures/mtp_clause55/README.md
$ python3 tests/mtp_depth_pool_test.py
FAIL  tests/fixtures/mtp_clause55 is incomplete: ['…/p0.log', '…/p1-4.log', '…/p5-7.log']
1 CHECK(S) FAILED

$ git check-ignore -v tests/fixtures/mtp_clause55/p0.log
.gitignore:19:*.log	tests/fixtures/mtp_clause55/p0.log

So the three fixture logs exist on the author's box (where the test passed) and were silently skipped by git add; in any clean checkout - and therefore in CI - the case fails. This is the same trap that produced "one file, +1/-1" on #1144 earlier tonight, and it is worth two lines in the repo's own notes rather than only a fix here.

The rest of it is right, and I proved it by putting the files back. Running the README's documented extraction over the real tickets on this box (~/wt/mtp-window/ticket-clause55-*), the three files come out 15 / 35 / 28 lines and the whole case goes green:

PASS  the shipped fixtures reproduce the recorded 8x64 window (rc=0)
PASS  the fixtures pool to the recorded 8 x 4 cells
PASS  fixture pool reports 136/181 = 75.14% / 86/136 = 63.24% / 63/86 = 73.26% / 53/63 = 84.13%
PASS  all four depths are inside the bar on the fixtures

i.e. the recipe in the README reproduces the window to the count, and the known answer is now checkable off this box - which is the point of the commit. (I removed the three files afterwards; the tree is clean.)

Fix, any of the three:

1. git add -f tests/fixtures/mtp_clause55/*.log - the fixtures are inputs, not logs;
2. or add !tests/fixtures/mtp_clause55/*.log after .gitignore:19, which is the version a future fixture author will find;
3. or commit them as .txt and adjust CL - the extension carries no meaning here.

Preferred: (2) plus the -f, because the next person adding a fixture under a *.log-ignored name will hit the same wall, and a negation rule documents why. Also worth making the else branch fail with "missing files, and .gitignore may have eaten them" - the current message names the path but not the likely cause, which cost me a few minutes.

[github-actions]

Persistent review updated to latest commit 3c3e373

[bong-water-water-bong]

VERDICT: READY at 3c3e373c — the CHANGES I filed at 673faa2e is closed, and I re-derived the known answer from a clean checkout rather than from a tree that happens to hold the tickets.

What was wrong: .gitignore:19 is *.log, so git add -A tests/fixtures/mtp_clause55 committed only the README and the three tickets were silently left behind; the suite passed on the authoring box because the files were lying in the working tree.

Re-checked at this head, in git worktree add --detach (empty git status, so nothing untracked can help):

• The fixtures are in the commit: git ls-tree -r 3c3e373c tests/fixtures/mtp_clause55 -> README.md, p0.txt, p1-4.txt, p5-7.txt. Renamed to .txt, which no ignore rule matches.
• Provenance reproduced byte-for-byte: I ran the README's own extraction (grep -E "forward ready:|prompt [0-9]+:|depth [0-9]+: |match p[0-9]+:|== live vs offline at the SAME|== ticket ") over the original ~/wt/mtp-window/ticket-clause55-{p0-20261105-115131,p1-4-20261105-115551,p5-7-20261105-120713}.log on this box and diffed against the committed fixtures: identical for all three, 15 / 35 / 28 lines. So they are the tickets, not a restatement of the answer.
• The suite is green from the clean tree: python3 tests/mtp_depth_pool_test.py -> 43 PASS, all checks passed, including the three fixtures are tracked by git (3 of 3), the shipped fixtures reproduce the recorded 8x64 window (rc=0) and the four counts 136/181 = 75.14%, 86/136 = 63.24%, 63/86 = 73.26%, 53/63 = 84.13%.
• Still not vacuous: live_n -> 1 + live_n in table() reds 11 checks including the shipped fixtures reproduce the recorded 8x64 window (rc=1); reverted, 43 PASS again.
• tools/copyright.py --check rc 0 (the new tests/fixtures/mtp_clause55/*.txt exemption is the same case as tests/golden/*/tokens.txt: data a parser reads verbatim). add_test(NAME mtp_depth_pool ...) is registered in CMakeLists.txt:573.
• CI at this head: build SUCCESS with 53/89 Test #53: mtp_depth_pool … Passed in its own log; CodeQL + five Analyze legs SUCCESS; hw_gate SKIPPED; pr_agent fail (expected while coder-llm is off).

BEHIND is main moving under it (f406769), not a conflict. Merge when the queue reaches it, applying the usual update first.

[github-actions]

Persistent review updated to latest commit b3bc7dd

[bong-water-water-bong]

Re-verdict at b3bc7dd9: READY — my CHANGES at 673faa2e is closed and nothing of the fix moved since the head I verified at 3c3e373c.

git diff 3c3e373c b3bc7dd9 is entirely other PRs' content landing in main (pins.json, npu/moe.cpp from #1153, console-tui from #1155, 10lemonade, vm-check.sh); this PR's eight files — the tool, its test, the three .txt fixtures, the fixture README, CMakeLists.txt, tools/copyright.py — are untouched. Its own diff against its merge-base is still 8 files, 771 insertions(+).

For the record, what I verified at 3c3e373c and re-checked from a clean git worktree (so nothing untracked can help): 43 PASS; the three fixtures are the tickets, reproduced byte-for-byte by the README's own grep -E … over ~/wt/mtp-window/ticket-clause55-*.log (15 / 35 / 28 lines); the recorded 8x4 window reproduces (136/181, 86/136, 63/86, 53/63); and the check is not vacuous — live_n -> 1 + live_n reds 11 checks including the shipped fixtures reproduce the recorded 8x64 window (rc=1).

BEHIND is main moving under it, not a conflict. Merge applying the usual update first.

[bong-water-water-bong]

VERDICT: READY at b3bc7dd9 — my CHANGES is closed, fixtures verified from a clean worktree; detail in the re-verdict above.