Repository navigation
Docs: the 1bit engine runs inside Lemonade - #23
Merged
Merged
Conversation
Lead with it everywhere: README intro and status, the thank-you, lemonade.md (Lemonade is the host; its onebit recipe runs 1bit serve like llama-server; the recipe passes Lemonade's LLM test suite on Vulkan and HRX), serve.md, PORTING step 1 and diagram, npu.md, and the serve/main code comments. 1bit serve on its own, for any OpenAI client, is the secondary note. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
bong-water-water-bong
enabled auto-merge (squash)
September 23, 2026 22:55
bong-water-water-bong
added a commit
that referenced
this pull request
Sep 26, 2026
…a.cpp-vulkan cecf3ee) (#167) * MoE streaming: expert reads in 256 KiB chunks by 16 threads (pin llama.cpp-vulkan cecf3ee) Fork PR #23: a layer's misses are a few ~1 MiB expert parts, so whole-part reads kept 6-7 reads in flight and each took 4.5-4.8 ms on Qwen3.8-Flash-Next. 256 KiB chunks with 16 threads (the streamer's new default) are 14-18% faster there; Coder-30B is within the noise. moe/ExpertCache gains read_chunk and per-read timing, as in the fork's copy; 1bit moe-cache gets --chunk-kb. docs/moe-streaming.md also records that the gate-ahead prefetch doubles Flash-Next's decode time. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * registry: regenerate for llama.cpp-vulkan cecf3ee Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: bong-water-water-bong <bong-water-water-bong@1bit.gg> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
bong-water-water-bong
pushed a commit
that referenced
this pull request
Oct 21, 2026
…) with the llama.cpp bump Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
bong-water-water-bong
added a commit
that referenced
this pull request
Oct 22, 2026
* Pin llama.cpp af04c9f2 and gpu-kernels cf222e7d llama.cpp moves from 96f81977 to af04c9f2 (1bit/hrx-vulkan-patched), which carries: - #123 ggml-hrx: claim views of q8_0 KV caches, so test-backend-ops actually runs the q8_0 FLASH_ATTN_EXT cases (29 were silently reported unsupported; now 28/28 pass on type_K=q8_0 and on type_V=q8_0) - #124 common: opt-in cumulative-confidence draft budget for MTP (--spec-draft-cum-p-min, default 0) with the ONEBIT_MTP_TRACE measurement hook in our own common/onebit-mtp-budget.h gpu-kernels moves from b373c3ec's predecessor to cf222e7d, which carries #22 (the q8_0 KV decode kernel takes short caches too: depth 512 -27% -> -7.4% on Qwen3-4B) and #23 (L2_NORM over views). It is not a submodule; docs/hrx.md now names the commit a release builds against. Hardware gate on this pair (engine main + these two pins, kernels present and off by default): ops 695/695, 7 kat rows, 5 kld rows, 4 needle rows - PASS, 0 ring events. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Record the gpu-kernels commit and the new llama.cpp pin in the registry docs/hrx.md's pin table gains the private-kernels row (gpu-kernels cf222e7d, not a submodule), and registry/architectures.json - which records the pins it was generated from - is regenerated for llama.cpp af04c9f2 (scripts/registry-regen.sh). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Pin gpu-kernels cf222e7d (q8_0 KV decode routing #22, L2_NORM views #23) with the llama.cpp bump Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: bong-water-water-bong <bong-water-water-bong@1bit.gg>
This was referenced Oct 23, 2026
This was referenced Nov 1, 2026
Merged
Merged
Merged
bong-water-water-bong
pushed a commit
that referenced
this pull request
Nov 8, 2026
…ached the model The catalog paragraph counted "23 HRX entries, each with a proven row on the box ... fork #23's 22 plus flint from #24". flint is added but not proven, and the fork's own page says so; what it no longer says is the old reason (a missing architecture -- that landed 2026-10-23, c3eef46c0). The measured reason, from the proof run on 2026-11-08 under one gpu-hold claim with engine rc-2026-11-05-c25: the engine's planner refused the load in 223 ms -- 1bit serve: flint-UD-Q4_K_XL does not fit: need 24.07 GiB (weights 23.33 + KV/state 0.24 + compute 0.50) at the model's own ctx 262144, free 6.03 GiB (GTT 29.1 of 120.0 GiB free, MemAvailable 6.0 GiB) -- four foreign HRX clients resident, box load 46.6. So the row needs an idle box (>= 24 GiB free), not a different build, and the chat-short span now says it is over the 22 proven rows. The 27-row total (23 + 4) is unchanged: the arithmetic was never wrong, only the attribution.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The engine's docs now lead with how it's used: the 1bit engine runs inside Lemonade. Lemonade is the host users talk to (catalog, downloads, router, UI), and its
onebitrecipe runs1bit serveas a backend, the way it runsllama-server.Updated: the README intro and status, the thank-you,
docs/lemonade.md,docs/serve.md, PORTING step 1 and its diagram,docs/npu.md, and theserve/maincode comments. Using1bit servedirectly with any OpenAI client is kept as a secondary note.Docs and comments only.
🤖 Generated with Claude Code