[bong-water-water-bong]

Follow-up to #104. The default FA-on path is correct, but an explicit -fa off (the non-FA attention path) still failed with unsupported HRX node SET_ROWS.

Root cause

The non-FA path flattens the K/V rows before the KV-cache write, so llama.cpp emits SET_ROWS element-wise (hidden_size == 1): each value is scattered to its own cache row. The row-wise SET_ROWS dispatcher only supports hidden_size 4..32768 (vector4), so it rejected the element-wise shape and the graph failed.

Fix (in 1bit-MONSTER/llama.cpp@c35833404)

Add a scalar-scatter kernel (ggml_set_rows_scatter) and a hidden_size == 1 matcher that dispatches the element-wise write: for each value i, cache[indices[i]] = f16(rows[i]).

Verification

• -fa off now answers "Paris" on Qwen3-Coder-30B-A3B-Instruct Q4_K_M and GLM-4.7-Flash Q4_K_M (previously 500).
• tests/serve_e2e.sh answers "Paris" on Qwen3-Coder-30B-A3B, GLM-4.7-Flash and Qwen3.6-35B-A3B Q8_0 (FA-on, no regression).

[context7]

Docs7 for 1bit-monster/engine

Result	Status	Action
Deployment	➖ Not used	—
Content review	➖ Did not run. This site has no agent runs available this month. Wait for the monthly reset or check your Docs7 plan.	—

Commit b66e8b5

[github-actions]

PR Reviewer Guide 🔍

(Review updated until commit b66e8b5)

Here are some key observations to aid the review process:

🎫 Ticket compliance analysis 🔶 104 - Partially compliant Compliant requirements: Pin third_party/llama.cpp to commit 96f6b89 Fix GLM-4.7-Flash and Qwen3-Coder at full context Remove 32768 context default for --device hrx Update docs (registry.md, PORTING and serve.md) and registry/checked.json Verify on Strix Halo with fresh build (-DONEBIT_HRX=ON) Ensure 7 of 7 HRX tests pass at full context Ensure serve_e2e passes with vulkan --prefill-device hrx Ensure test-backend-ops -b HRX0 passes with 790/790 Non-compliant requirements: None Requires further human verification: None

⏱️ Estimated effort to review: 3 🔵🔵🔵⚪⚪

🧪 No relevant tests

🔒 No security concerns identified

⚡ Recommended focus areas for review Flash Attention Flag Handling The new -fa flag is added to control flash attention behavior, but the handling logic in launch_for function may not correctly propagate this flag when --prefill-device is used. Specifically, when split is true, the -fa on is hardcoded, which may override the user's explicit -fa off setting. This could lead to inconsistent behavior between the prefill and generation devices. if (!o.flash_attn.empty() && !split) { argv.insert(argv.end(), {"-fa", o.flash_attn}); } if (split) { argv.insert(argv.end(), {"-fa", "on"});

[github-actions]

Persistent review updated to latest commit b66e8b5