Skip to content

Pin llama.cpp 6617e78: Q3_K/IQ3_S decode on HRX, no spinning wait (27B 97% of Vulkan) - #236

Merged
bong-water-water-bong merged 1 commit into
mainfrom
pin-hrx-q3k-wait
Sep 29, 2026
Merged

bong-water-water-bong merged 1 commit into
mainfrom
pin-hrx-q3k-wait

Conversation

@bong-water-water-bong

Copy link
Copy Markdown
Collaborator

Pins third_party/llama.cpp to 6617e78, which brings in two fork PRs.

llama.cpp #43: Q3_K and IQ3_S in the K-quant decode kernels.

  • IQ3_S keeps its 512-entry grid in workgroup memory.
  • With this, every weight type in the UD-Q4_K_XL 27B goes through these kernels.
  • Qwen3.8-27B UD-Q4_K_XL decode: 11.60 → 12.15 tok/s, against 12.48 on Vulkan (97.3%).
  • Decode-shaped wikitext PPL: 7.1595 (CPU 7.1475).
  • test-backend-ops -o MUL_MAT -b HRX0 passes.

llama.cpp #44: HRX decode no longer spins a CPU core.

  • The per-token stream wait sat in ROCr's signal wait, which returned from the KFD event ioctl on every completed command without sleeping.
  • Long expected waits (over 20 ms) now sleep through 80% of the wait; shorter waits are unchanged.
  • 27B decode now runs at 71–76 °C and about 125 W, where it was 83–93 °C and 133–137 W, and is 0.4% slower.
  • Qwen3-0.6B and ZAYA1-8B decode are unchanged.

Also in this PR:

  • registry/architectures.json is regenerated; --check-pins passes.
  • docs/hrx.md is updated.

🤖 Generated with Claude Code

…B 97% of Vulkan, 74 C)

llama.cpp #43: Q3_K and IQ3_S in the K-quant decode kernels (Qwen3.8-27B
UD-Q4_K_XL 11.60 -> 12.15 tok/s, Vulkan 12.48). llama.cpp #44: long stream
waits sleep instead of spinning a CPU core in ROCr's signal wait (27B decode
71-76 C / ~125 W instead of 83-93 C / 133-137 W, -0.4%). docs/hrx.md updated.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@context7

context7 Bot commented Sep 29, 2026

Copy link
Copy Markdown

Docs7 for 1bit-monster/engine

Result Status Action
Deployment ➖ Not used —
Content review ➖ Did not run. This site has no agent runs available this month. Wait for the monthly reset or check your Docs7 plan. —

Commit 67a9015

@github-actions

Copy link
Copy Markdown

PR Reviewer Guide 🔍

Here are some key observations to aid the review process:

🎫 Ticket compliance analysis 🔶

43 - Partially compliant

Compliant requirements:

  • Performance improvements for Qwen3.8-27B UD-Q4_K_XL on HRX (12.15 tok/s vs 12.48 on Vulkan)
  • Documentation updates in docs/hrx.md reflecting new kernel support and performance data
  • Updated registry/architectures.json to reflect new llama.cpp commit

Non-compliant requirements:

Requires further human verification:

  • Verification that the performance claims (e.g., 2.3-2.9x speedup) are accurate and measured on hardware
  • Confirmation that the blog post content matches the project's voice and tone

44 - Partially compliant

Compliant requirements:

  • No changes to tools/site.py, README, or pages.yml found in the diff

Non-compliant requirements:

  • Site address change not implemented in this PR

Requires further human verification:

  • Need to verify that the DNS and GitHub Pages configuration changes are handled separately
⏱️ Estimated effort to review: 3 🔵🔵🔵⚪⚪
🧪 No relevant tests
🔒 No security concerns identified
⚡ Recommended focus areas for review

Performance Claim Verification

The documentation claims a 12.15 tok/s decode speed for Qwen3.8-27B UD-Q4_K_XL on HRX, which is 97.3% of Vulkan performance. This specific performance number should be verified to ensure it matches the actual hardware measurements mentioned in the ticket.

| Qwen3.8-27B UD-Q4_K_XL | 12.15 | 12.48 | 97.3% (was 9.0) |
Thermal and Power Claim Verification

The documentation states that 27B decode now runs at 71–76 °C and about 125 W, compared to 83–93 °C and 133–137 W previously. These thermal and power claims need to be confirmed as they represent significant hardware behavior changes.

the wait; shorter ones are unchanged. 27B decode: 71-76 C and ~125 W instead of 83-93 C and
133-137 W, 0.4% slower; Qwen3-0.6B and ZAYA1-8B unchanged. `ONEBIT_HRX_BLOCKING_WAIT=1` turns it

@bong-water-water-bong
bong-water-water-bong merged commit f04d933 into main Sep 29, 2026
11 checks passed
@bong-water-water-bong
bong-water-water-bong deleted the pin-hrx-q3k-wait branch September 29, 2026 22:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant