[bong-water-water-bong]

What this fixes

The zha q/k dump (src/llama-zha-dump.cpp, #179) marks the tensors it wants with ggml_set_output() and reads them back after graph_compute. On the HRX route its pre-RoPE half came back as zeros — every layer, every row — while the post-RoPE half was fine:

-dev none                                        : qpre L0 rms 1.46 (values)
-dev HRX0 -ngl 99                                : qpre L0 every value 0.0     <- the bug
-dev HRX0 -ngl 99 GGML_HRX_DISABLE_DISPATCH=fused: qpre L0 rms 14.66 (kpre), values back

Cause, in ggml-hrx: graph.cpp:tensor_is_external() treats a GGML_TENSOR_FLAG_OUTPUT value as external, so a fused match writes its root's output for the caller — but nothing does the same for the nodes a match covers. Their only copy stays inside the fused program and their buffer is never written, so a read-back afterwards sees the untouched (zero) buffer.

The change

A match is declined when it would cover a node whose tensor carries the read-back marker, and the plan falls through to a narrower registration that writes the node itself. One GGML_HRX_LOG_DISPATCH=1 line per decline, then a suppression notice after 8.

ggml_hrx dispatch: common.rmsnorm_binary_f32 declined at node 6 (MUL) - it covers tensor zha-readback.Qcur_normed-0, marked as a graph output

The marker is the explicit name prefix llama_zha_dump applies (zha-readback.<name>), not the bare GGML_TENSOR_FLAG_OUTPUT. My first version keyed on the flag alone, and it declined common.rmsnorm_binary_f32 in every ordinary run: llama.cpp marks several tensors as outputs for API reasons (result_norm is t_embd for the embeddings API), and the final-norm fusion was being dropped to serve a debug tool. With the prefix, nothing outside a dump run carries the marker, so an ordinary run declines nothing and is planned exactly as it was — a property stated from the code and checked below, rather than argued from a benchmark. (I did try to price the flag-only version: logits were bit-identical, but the box's llama-bench spread on the same binary was ±147 t/s on a 1119 t/s prefill, which cannot resolve a one-op difference and is exactly why I did not want the change there at all.)

Measured

Qwen3-4B-IQ4_XS, -dev HRX0 -ngl 99 -fa auto, same prompt, one gpu-hold ticket per arm set, f16 dumps:

arm	declines	first token	qpre L0 (zero_frac, rms)	kpre L0
ordinary run (no LLAMA_ZHA_DUMP_DIR)	0	" The" −1.3745098, top-5 identical to 6 dp vs the build without this change	—	—
dump run	8 (+suppressed) naming zha-readback.Qcur_normed-N / Kcur_normed-N	" The" −1.3745098	0.0, 1.5323	0.0, 14.6636
reference: GGML_HRX_DISABLE_DISPATCH=fused	—	—	0.0, 1.5322	0.0, 14.6648

The dumped values match the fusion-disabled reference to 4 decimal places, so this is the real value and not a different approximation; no arm produced unsupported HRX node or a compute error.

One deliberate trade: a dumped intermediate that has no fallback registration now fails loudly instead of writing zeros. That is what a debug tool should do.

Files: ggml/src/ggml-hrx/dispatch_registration/dispatch-registry.cpp (edits in place, their header kept) and src/llama-zha-dump.cpp (ours); 2 files, +92/−2.

Refs engine ggml-org#1078 (where the zero read-back was found), fork #179 (the dump itself). No merge from me.

[bong-water-water-bong]

VERDICT: READY at 154b0de4 — the fix is precisely scoped, and the reasoning that scoped it is in the code.

What I checked, and the part that decided it. The naive fix for "a fused dispatch swallowed a value the caller reads back"
is "decline any match covering a tensor flagged as an output" — and the comment says why that is wrong, from measurement:

// The marker is the explicit name prefix llama_zha_dump applies ("zha-readback."), not GGML_TENSOR_FLAG_OUTPUT on its
// own: llama.cpp marks several tensors as outputs for API reasons (result_norm is t_embd), and honouring the bare flag
// declined the final norm fusion in every ordinary run.

So the predicate needs both the flag and the prefix, the root is exempt (a match always writes its root's output), and a
match that covers a readback node is declined so it falls through to a narrower registration that writes the node. That is
the correct shape: the cost is paid only by a run that actually uses the dump, and no ordinary run is planned differently —
which is the property that makes this safe to land without a device.

Checked in the loop rather than assumed: covered_nodes entries are bounds-checked against nodes.size(), the root index is
skipped, and the value lookup is null-checked before the predicate sees it.

The evidence in the body is the right instrument for this bug: pre-RoPE q/k came back all zeros on the HRX route, the
post-RoPE half was fine, and GGML_HRX_DISABLE_DISPATCH=fused brought the values back — a symptom, a localisation and a
control, which is what a placement bug needs.

Not verified by me: the device run (GPU held). The src/llama-zha-dump.cpp +14 half — the dump marking its own tensors with
the prefix — is what makes the predicate fire; that is a one-line invariant worth keeping in mind if the dump is ever renamed.