Skip to content

llama-hadamard: fold qwen3next ssm_ba; refuse stamped files with unfoldable Q4_0 weights - #71

Merged
bong-water-water-bong merged 1 commit into
1bit/hrx-vulkan-patchedfrom
1bit/hadamard-onebit-guard
Oct 2, 2026
Merged

bong-water-water-bong merged 1 commit into
1bit/hrx-vulkan-patchedfrom
1bit/hadamard-onebit-guard

Conversation

@bong-water-water-bong

Copy link
Copy Markdown

llama-hadamard: fold qwen3next's ssm_ba; refuse stamped files with unfoldable Q4_0 weights

onebit.hadamard_q4_0 files (#58) are written by the engine's
tools/hadamard_q4_0.py, which checks that every Q4_0 tensor it writes,
token_embd aside, is rotated. The loader folded only the Q4_0 tensors on
its whitelist and left the rest alone, so a rotated Q4_0 weight outside the
whitelist ran without its activation transform. That would answer wrongly
with no error.

  • ssm_ba (qwen3next's fused beta/alpha projection) joins the whitelist:
    qwen3next builds it through build_lora_mm (models/qwen3next.cpp), like
    ssm_alpha/ssm_beta in qwen35.
  • A stamped file with any other non-embedding Q4_0 weight off the whitelist
    is refused with the weight's name, not run.

No existing file changes behaviour. The H32 files on hand, by Q4_0 weights
the loader can't fold: Qwen3.8-27B (qwen35) 0 of 456, Qwen3-Coder-30B-A3B
(qwen3moe) 0 of 337, Qwen3.6-35B-A3B (qwen35moe) 0 of 401. Qwen3.8-27B-H32 on
HRX0: PPL 5.8714 before and after (wikitext-2, 4 x 512). GLM-4.7-Flash-H32
(deepseek2, 235 MLA weights not foldable) is still refused by the earlier
arch check. No qwen3next H32 file exists to run. tools/hadamard_q4_0.py does
not rotate ssm_ba yet, and its invariant check rejects any Q4_0 tensor it
hasn't rotated.

🤖 Generated with Claude Code

…foldable Q4_0 weights

onebit.hadamard_q4_0 files (#58) are written by the engine's
tools/hadamard_q4_0.py, which checks that every Q4_0 tensor it writes,
token_embd aside, is rotated. The loader folded only the Q4_0 tensors on
its whitelist and left the rest alone, so a rotated Q4_0 weight outside the
whitelist ran without its activation transform. That would answer wrongly
with no error.

- ssm_ba (qwen3next's fused beta/alpha projection) joins the whitelist:
  qwen3next builds it through build_lora_mm (models/qwen3next.cpp), like
  ssm_alpha/ssm_beta in qwen35.
- A stamped file with any other non-embedding Q4_0 weight off the whitelist
  is refused with the weight's name, not run.

No existing file changes behaviour. The H32 files on hand, by Q4_0 weights
the loader can't fold: Qwen3.8-27B (qwen35) 0 of 456, Qwen3-Coder-30B-A3B
(qwen3moe) 0 of 337, Qwen3.6-35B-A3B (qwen35moe) 0 of 401. Qwen3.8-27B-H32 on
HRX0: PPL 5.8714 before and after (wikitext-2, 4 x 512). GLM-4.7-Flash-H32
(deepseek2, 235 MLA weights not foldable) is still refused by the earlier
arch check. No qwen3next H32 file exists to run. tools/hadamard_q4_0.py does
not rotate ssm_ba yet, and its invariant check rejects any Q4_0 tensor it
hasn't rotated.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@bong-water-water-bong

Copy link
Copy Markdown
Author

Review (PR-Agent duty): approve. The change is in our own src/llama-hadamard.cpp: qwen3next's ssm_ba joins the fold whitelist, and a stamped file with any other non-embedding Q4_0 weight the folder can't handle is refused by name instead of answering wrongly. Existing H32 files (27B, Coder-30B, 3.6-35B) have 0 unfoldable Q4_0 weights, so they're unaffected.

@bong-water-water-bong
bong-water-water-bong merged commit 388290b into 1bit/hrx-vulkan-patched Oct 2, 2026
10 of 24 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant