Repository navigation
llama-hadamard: fold qwen3next ssm_ba; refuse stamped files with unfoldable Q4_0 weights - #71
Merged
bong-water-water-bong merged 1 commit intoOct 2, 2026
Conversation
…foldable Q4_0 weights onebit.hadamard_q4_0 files (#58) are written by the engine's tools/hadamard_q4_0.py, which checks that every Q4_0 tensor it writes, token_embd aside, is rotated. The loader folded only the Q4_0 tensors on its whitelist and left the rest alone, so a rotated Q4_0 weight outside the whitelist ran without its activation transform. That would answer wrongly with no error. - ssm_ba (qwen3next's fused beta/alpha projection) joins the whitelist: qwen3next builds it through build_lora_mm (models/qwen3next.cpp), like ssm_alpha/ssm_beta in qwen35. - A stamped file with any other non-embedding Q4_0 weight off the whitelist is refused with the weight's name, not run. No existing file changes behaviour. The H32 files on hand, by Q4_0 weights the loader can't fold: Qwen3.8-27B (qwen35) 0 of 456, Qwen3-Coder-30B-A3B (qwen3moe) 0 of 337, Qwen3.6-35B-A3B (qwen35moe) 0 of 401. Qwen3.8-27B-H32 on HRX0: PPL 5.8714 before and after (wikitext-2, 4 x 512). GLM-4.7-Flash-H32 (deepseek2, 235 MLA weights not foldable) is still refused by the earlier arch check. No qwen3next H32 file exists to run. tools/hadamard_q4_0.py does not rotate ssm_ba yet, and its invariant check rejects any Q4_0 tensor it hasn't rotated. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Author
|
Review (PR-Agent duty): approve. The change is in our own |
bong-water-water-bong
merged commit Oct 2, 2026
388290b
into
1bit/hrx-vulkan-patched
10 of 24 checks passed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
llama-hadamard: fold qwen3next's ssm_ba; refuse stamped files with unfoldable Q4_0 weights
onebit.hadamard_q4_0 files (#58) are written by the engine's
tools/hadamard_q4_0.py, which checks that every Q4_0 tensor it writes,
token_embd aside, is rotated. The loader folded only the Q4_0 tensors on
its whitelist and left the rest alone, so a rotated Q4_0 weight outside the
whitelist ran without its activation transform. That would answer wrongly
with no error.
qwen3next builds it through build_lora_mm (models/qwen3next.cpp), like
ssm_alpha/ssm_beta in qwen35.
is refused with the weight's name, not run.
No existing file changes behaviour. The H32 files on hand, by Q4_0 weights
the loader can't fold: Qwen3.8-27B (qwen35) 0 of 456, Qwen3-Coder-30B-A3B
(qwen3moe) 0 of 337, Qwen3.6-35B-A3B (qwen35moe) 0 of 401. Qwen3.8-27B-H32 on
HRX0: PPL 5.8714 before and after (wikitext-2, 4 x 512). GLM-4.7-Flash-H32
(deepseek2, 235 MLA weights not foldable) is still refused by the earlier
arch check. No qwen3next H32 file exists to run. tools/hadamard_q4_0.py does
not rotate ssm_ba yet, and its invariant check rejects any Q4_0 tensor it
hasn't rotated.
🤖 Generated with Claude Code