[bong-water-water-bong]

A placement guard for 1bit-MONSTER/engine#286. It is not a fix: the root cause stays open in 1bit-MONSTER/engine#286.

Problem

In gpt-oss, when the expert MUL_MAT_IDs run on the CPU and ADD_ID / SWIGLU_OAI (#67) run on HRX between those CPU splits, the output is wrong: KLD vs CPU 1.65 and garbage text. Each op is correct on its own.

Change

• dispatch_registration/common/moe-placement-guard.{h,cpp} (our code, Apache-2.0 notice). HRX claims an ADD_ID or SWIGLU_OAI only when the MUL_MAT_ID it follows (directly or through ADD_ID) will also run on HRX. That needs two things:
  • HRX executes the MUL_MAT_ID;
  • the expert weights sit in a buffer HRX can read (ggml_backend_dev_supports_buft).
• ggml-hrx.cpp: one include, and a one-line hook at the top of device_supports_op.

The buffer check is needed for decode. When HRX declines the expert MUL_MAT_ID at load time, llama.cpp puts the experts in CPU_REPACK. Every MUL_MAT_ID then stays on the CPU, including decode shapes that mul_mat_id.decode_f32_wave64 could run. With only an "HRX supports the op" check, prompt KLD was fine but generated text was still garbage, because decode-shaped MUL_MAT_IDs pass that check while staying on the CPU.

• tests/test-hrx-moe-split.cpp: the gpt-oss MoE block with MXFP4 experts in CPU_REPACK (the real layout) and in a host buffer, with biases and routing in HRX memory. It checks the result against a CPU-only run and checks placement. It also reproduces engine Understanding the basics ggml-org/llama.cpp#286 in isolation:

experts in CPU_REPACK, tokens 1 / 3 / 12 / 40	nmse vs CPU	placement
without the guard	0.054 to 0.060	MUL_MAT_ID on CPU, ADD_ID + SWIGLU_OAI on HRX
with the guard	1.3e-15 to 1.5e-15	all on CPU

Evidence (strixhalo, this exact tree ba0be9a, base 388290b)

• test-hrx-moe-split: all cases OK
• test-backend-ops -b HRX0: 1061/1061

gpt-oss-20b MXFP4, llama-perplexity -c 512 -b 512 --chunks 8 -fa on, KLD vs CPU:

	388290b	this PR
default	0.0294	0.0294 (identical)
GGML_HRX_DISABLE_DISPATCH=mul_mat_id.f32_f32_wmma	1.650 (same top 12.0 %)	0.031 (same top 88.8 %)

Greedy (--temp 0 -n 48, three prompts: sky, primary colors, haiku):

• default: same text before and after, and correct.
• with mul_mat_id.f32_f32_wmma disabled: garbage before ("The first part (the first 2? maybe 2^? ..."); correct after ("The sky appears blue because the Earth's atmosphere scatters sunlight. ...").

llama-bench, default, same box conditions:

	pp512	tg128
388290b	777.0 ± 6.2	26.5 ± 0.8
this PR	777.0 ± 9.1	26.1 ± 0.6

The box was shared and in balanced power mode, so absolute numbers are below earlier runs. Before and after were measured the same way.

No regression: Qwen3.8-27B UD-Q4_K_XL and ZAYA1-8B Q4_K_M produce byte-identical greedy text on 388290b and on this PR (same three prompts).

Known limit, unchanged by this PR

gpt-oss with -ub 4096 fails loudly (graph_compute -1); it does not give wrong output. 1bit serve (engine app/serve.cpp) passes no -ub for chat models, so llama-server's default ubatch of 512 applies and users don't hit this. Only these serve paths set a large ubatch:

• --mmproj sets 4096 (vision models; gpt-oss has none);
• --embedding / --rerank set it to the context size.

The one tuned recipe (-ub 1024) is ROCm-only.

🤖 Generated with Claude Code

[bong-water-water-bong]

Review (PR-Agent duty): approve; it merges first, then #72, then #68.

• Placement: the rule is in our own moe-placement-guard.{h,cpp} with the full notice. AMD's ggml-hrx.cpp gets an include and a one-line hook in device_supports_op.
• The two checks are both needed. The MUL_MAT_ID must be executable on HRX, and the expert weights must sit in a buffer HRX can read (ggml_backend_dev_supports_buft). The second catches CPU_REPACK experts, which keep every MUL_MAT_ID (decode included) on the CPU. f3's first version missed that.
• test-hrx-moe-split reproduces engine Understanding the basics ggml-org/llama.cpp#286 in isolation: CPU_REPACK MXFP4 experts give nmse 0.054-0.060 without the guard and 1.5e-15 with it.
• gpt-oss-20b:
  • with mul_mat_id.f32_f32_wmma disabled, KLD 1.650 -> 0.031 and the text is correct;
  • default unchanged (KLD 0.0294, same text, pp/tg within noise).
• Other models and suite: Qwen3.8-27B and ZAYA1-8B greedy text is byte-identical to 388290b; full suite 1061/1061.
• It's a guard; the root cause stays open in engine Understanding the basics ggml-org/llama.cpp#286.
• The -ub 4096 limit: 1bit serve passes no -ub for chat (llama-server's default 512), so users don't hit it.