From f2cea2d5fb5b7d9255f60e7389566e03e77032d6 Mon Sep 17 00:00:00 2001 From: bong-water-water-bong Date: Tue, 29 Sep 2026 14:57:55 -0300 Subject: [PATCH 1/2] Pin llama.cpp c9283cf: HRX ZAYA decode fusions (62 -> ~90 tok/s) Co-Authored-By: Claude Opus 5.5 --- third_party/llama.cpp | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/third_party/llama.cpp b/third_party/llama.cpp index 45c177083..c9283cff7 160000 --- a/third_party/llama.cpp +++ b/third_party/llama.cpp @@ -1 +1 @@ -Subproject commit 45c17708344627b0731ff28d4075da65a3bfa6ff +Subproject commit c9283cff724ec3d558e46da725f36d01f771c8b0 From 901446d50cd737fd73a2fa7dc10a03566353a5c7 Mon Sep 17 00:00:00 2001 From: bong-water-water-bong Date: Tue, 29 Sep 2026 14:58:11 -0300 Subject: [PATCH 2/2] registry + docs/hrx: llama.cpp (hrx) at c9283cf, ZAYA decode fusions Co-Authored-By: Claude Opus 5.5 --- docs/hrx.md | 14 ++++++++++++++ registry/architectures.json | 2 +- 2 files changed, 15 insertions(+), 1 deletion(-) diff --git a/docs/hrx.md b/docs/hrx.md index b877cb368..9ae341f2d 100644 --- a/docs/hrx.md +++ b/docs/hrx.md @@ -170,6 +170,20 @@ were observed. a ZAYA token is small dispatches and the ~1.8 us gaps between them, not matmuls; the profiler (`HRX_PROFILE_MODE=dispatch`) shows which ones. Profile through `llama-server`: `llama-bench` under the profiler fails. +- **ZAYA decode fusions (llama.cpp #36-#40).** ZAYA1-8B HRX0 decode 67 -> ~90 tok/s (Vulkan 93), + each step measured in interleaved A/B runs: + - #36 skips CONTs of fresh contiguous results in `src/models/zaya.cpp` (views keep theirs: on HRX + their consumers take slower strided paths), bitwise the same output: +2.4%. + - #39 runs the CCA convolution (Q/K concat, conv-state update, depthwise conv, grouped 2-tap conv + and biases, ~35 graph nodes) as one dispatch (`zaya.cca_conv.decode_f32`): 72 -> 81 tok/s. + - #40 runs the CCA query/key mixing and per-head norms (~30 nodes) as one dispatch + (`zaya.cca_qk_norm.decode_f32`): 81 -> 90 tok/s. + #39 and #40 accumulate in their own order, so output is not bitwise the same; teacher-forced top-1 + against the FP32 model stays 69/96 and batch-1 perplexity moves 33.31 -> 33.49 -> 33.45 (CPU backend + 34.30). Both match decode only (one token, one sequence). #37 and #38 remove runtime integer + division from #35's and #32's kernels, which newer Loom rejects. Each new `.loom` file carries a + reference kernel and `check.case` differentials that pass under `iree-test-loom --sanitizer=access`. + `GGML_HRX_DISABLE_DISPATCH=zaya.cca` turns the two ZAYA kernels off for comparison. ### Known issues on `HRX0` diff --git a/registry/architectures.json b/registry/architectures.json index 1c404230a..1cccbdbfe 100644 --- a/registry/architectures.json +++ b/registry/architectures.json @@ -2,7 +2,7 @@ "about": "HF architecture -> GGUF architecture and the backends whose code accepts it. Generated by tools/registry_build.py from the pinned sources; do not edit.", "sources": { "llama.cpp (vulkan)": "cecf3ee01d9d99378e98bfea95f51cac714b04b8", - "llama.cpp (hrx)": "45c17708344627b0731ff28d4075da65a3bfa6ff", + "llama.cpp (hrx)": "c9283cff724ec3d558e46da725f36d01f771c8b0", "zinc": "29bc350ac4cbf9f110ec628b9e177ea04ac816fb" }, "counts": {