Repository navigation
Docs: the onebit recipe lives in our Lemonade fork - #24
Merged
Merged
Conversation
…nade Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
bong-water-water-bong
enabled auto-merge (squash)
September 23, 2026 22:57
This was referenced Oct 7, 2026
bong-water-water-bong
added a commit
that referenced
this pull request
Oct 22, 2026
…-256 layers (Qwen3.5-4B 32K decode 23.7 -> 47.1 tok/s, Qwen3.8-27B 7.5 -> 11.0) (#500) gpu-kernels 4f586cac adds, since cf222e7d: - #24: hip.fa_decode_d256.gate, one dispatch for the gated full attention of Qwen3.5 / Qwen3.6 / Qwen3.8 (FLASH_ATTN_EXT -> RESHAPE -> MUL by SIGMOID(gate)) at 1..8 query tokens (decode and MTP verify), head 256, f16 KV, n_kv >= 1024. The Loom gated kernel ran it on one workgroup per query head (16-24 workgroups); profile at 26,644 tokens: 55% of a Qwen3.5-4B decode step and 34% of a Qwen3.8-27B one, about 41 GB/s of KV. The HIP kernel splits the cache over T x Hkv x S workgroups, reads each K/V row once per KV head and merges with agent-scope fences and runtime-zeroed counters (about 217 GB/s at n_kv 32768 in its bench). Measured on strixhalo gfx1151, performance mode, private build (llama.cpp af04c9f2 + hrx-system 82804197 + the add-on), llama-bench -fa 1 -p 0 -n 64 -d D, interleaved x3, medians, tok/s (ours): d=0 4096 16384 32768 Qwen3.5-4B before 61.13 50.93 34.17 23.71 Qwen3.5-4B after 61.20 59.19 53.31 47.07 Qwen3.8-27B before 12.34 11.36 9.27 7.49 Qwen3.8-27B after 12.32 12.11 11.62 10.97 (Lemonade's Vulkan llama.cpp b11393, same command and files: 4B 64.03 / 60.99 / 54.80 / 48.27, 27B 12.47 / 9.06 / 11.78 / 11.14; 27B d<32768 rows on the 08bb2241 base.) Qwen3.8-27B --mtp, llama-server, 26,644-token prompt, 64 greedy tokens: 16.40 -> 23.78 tok/s (medians of 4). Correctness: kernel bench vs a double-precision reference on 17 shapes (G 1..8, T 1..8, gate on/off) PASS, 5 repeats bit-identical; greedy text vs the Loom kernel identical on Qwen3.5-4B at 180 / 6,662 / 26,644 tokens (mean top-10 KL <= 1.7e-4), Qwen3.8-27B 26,644 tokens mean KL 3.7e-5 then a near-tie flip at token 10 (logprob gap 0.008); identical requests repeat exactly; no GPU fault in dmesg. Needs the hw-gate run (label) before merge: goldens with Qwen3.8-27B decode past 1024 KV cells may drift on near-ties. Co-authored-by: bong-water-water-bong <bong-water-water-bong@1bit.gg> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
bong-water-water-bong
pushed a commit
that referenced
this pull request
Nov 6, 2026
…wn serve, not to /tmp's glob
`tests/hybrid_route.sh` asserted "no handoff file left in the slot directory" with
[ -z "$(ls /tmp/1bit-hybrid-*/ 2>/dev/null)" ]
which is every hybrid serve's slot directory on the box, not that test's. Another test under `ctest -j` (or
another lane's serve) keeps its own /tmp/1bit-hybrid-* directory alive with a handoff file in it for exactly
the window this check looks, so the test fails while its own directory is clean. Measured, on today's main:
ctest -j2 89 passed, 1 failed out of 90 -- #24 hybrid_route, "FAIL no handoff file left in the slot
directory", with two slot dirs alive and this serve's own one empty
serial the same test passes (twice, 28 s each), which is how the flake reads as a route failure
Deterministic reproducer, no racing: `mktemp -d /tmp/1bit-hybrid-XXXXXX` plus a `handoff.bin` in it, then run
the test -- it fails on that line before the change and passes after it. The same shape was in
`hybrid_kvshare_route.sh`'s "the socket is gone from the slot directory" (a foreign `kvshare-*.sock`).
Both checks now name the directory the serve under test actually owns, read off its child's argv
(`--slot-save-path`, which is `/tmp/1bit-hybrid-*`), so a foreign file cannot satisfy or fail them:
* "no handoff file left in the slot directory" -> `[ -n "$slotdir" ] && [ -z "$(ls "$slotdir")" ]`
* "the slot directory is removed at exit" -> `[ -n "$slotdir" ] && [ ! -d "$slotdir" ]`
(this one used to count the glob before/after, so it passed whenever *some* directory vanished)
The `[ -n "$slotdir" ]` half is deliberate: if the capture ever stops working the checks fail loudly instead
of passing on an empty string. Teeth kept, checked directly: with a file in the serve's own directory the
assertion is false, with it empty it is true.
Gates: `tests/hybrid_route.sh` PASS, `tests/hybrid_kvshare_route.sh` PASS, both with a planted foreign slot
directory present (the failing condition), `ctest -R hybrid -j2` 2/2 twice, `bash -n` clean on both.
Same class as engine#1070 (a CI-only concurrency failure that reads as a route failure); different mechanism
(the slot-directory glob, not the port).
bong-water-water-bong
added a commit
that referenced
this pull request
Nov 6, 2026
…wn serve, not to /tmp's glob (#1207) `tests/hybrid_route.sh` asserted "no handoff file left in the slot directory" with [ -z "$(ls /tmp/1bit-hybrid-*/ 2>/dev/null)" ] which is every hybrid serve's slot directory on the box, not that test's. Another test under `ctest -j` (or another lane's serve) keeps its own /tmp/1bit-hybrid-* directory alive with a handoff file in it for exactly the window this check looks, so the test fails while its own directory is clean. Measured, on today's main: ctest -j2 89 passed, 1 failed out of 90 -- #24 hybrid_route, "FAIL no handoff file left in the slot directory", with two slot dirs alive and this serve's own one empty serial the same test passes (twice, 28 s each), which is how the flake reads as a route failure Deterministic reproducer, no racing: `mktemp -d /tmp/1bit-hybrid-XXXXXX` plus a `handoff.bin` in it, then run the test -- it fails on that line before the change and passes after it. The same shape was in `hybrid_kvshare_route.sh`'s "the socket is gone from the slot directory" (a foreign `kvshare-*.sock`). Both checks now name the directory the serve under test actually owns, read off its child's argv (`--slot-save-path`, which is `/tmp/1bit-hybrid-*`), so a foreign file cannot satisfy or fail them: * "no handoff file left in the slot directory" -> `[ -n "$slotdir" ] && [ -z "$(ls "$slotdir")" ]` * "the slot directory is removed at exit" -> `[ -n "$slotdir" ] && [ ! -d "$slotdir" ]` (this one used to count the glob before/after, so it passed whenever *some* directory vanished) The `[ -n "$slotdir" ]` half is deliberate: if the capture ever stops working the checks fail loudly instead of passing on an empty string. Teeth kept, checked directly: with a file in the serve's own directory the assertion is false, with it empty it is true. Gates: `tests/hybrid_route.sh` PASS, `tests/hybrid_kvshare_route.sh` PASS, both with a planted foreign slot directory present (the failing condition), `ctest -R hybrid -j2` 2/2 twice, `bash -n` clean on both. Same class as engine#1070 (a CI-only concurrency failure that reads as a route failure); different mechanism (the slot-directory glob, not the port). Co-authored-by: bong-water-water-bong <bong-water-water-bong@1bit.gg>
bong-water-water-bong
pushed a commit
that referenced
this pull request
Nov 8, 2026
…ached the model The catalog paragraph counted "23 HRX entries, each with a proven row on the box ... fork #23's 22 plus flint from #24". flint is added but not proven, and the fork's own page says so; what it no longer says is the old reason (a missing architecture -- that landed 2026-10-23, c3eef46c0). The measured reason, from the proof run on 2026-11-08 under one gpu-hold claim with engine rc-2026-11-05-c25: the engine's planner refused the load in 223 ms -- 1bit serve: flint-UD-Q4_K_XL does not fit: need 24.07 GiB (weights 23.33 + KV/state 0.24 + compute 0.50) at the model's own ctx 262144, free 6.03 GiB (GTT 29.1 of 120.0 GiB free, MemAvailable 6.0 GiB) -- four foreign HRX clients resident, box load 46.6. So the row needs an idle box (>= 24 GiB free), not a different build, and the chat-short span now says it is over the 22 proven rows. The 27-row total (23 + 4) is unchanged: the arithmetic was never wrong, only the attribution.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The Lemonade recipe that runs
1bit serve(onebit) is merged into our fork 1bit-MONSTER/lemonade (fork PR #1), not upstream. This updatesdocs/lemonade.md(how to buildlemondfrom the fork, plus the test evidence), the README status and PORTING step 1.Docs only.
🤖 Generated with Claude Code