Skip to content

Bump Lemonade: 1bit-MONSTER/lemonade 64eb505cc62a - #1175

Merged
bong-water-water-bong merged 2 commits into
mainfrom
bump-lemonade/64eb505cc62a
Nov 5, 2026
Merged

bong-water-water-bong merged 2 commits into
mainfrom
bump-lemonade/64eb505cc62a

Conversation

@bong-water-water-bong

@bong-water-water-bong bong-water-water-bong commented Nov 5, 2026 •

Copy link
Copy Markdown
Collaborator

Moves third_party/lemonade from f44ff953bcd8 to the fork's main 64eb505cc62a.

Fork commits (last 30):

What this pin changes for the appliance: nothing, by construction

The one commit makes the fixed sleep_for(seconds(2)) that follows every backend child's termination (platform/process_linux.cpp, upstream #514) settable through LEMONADE_GPU_CLEANUP_WAIT_MS. The default is still 2000 ms and nothing in this repository sets the variable — scripts/lemond-onebit.sh is unchanged in this PR — so the board behaves exactly as it does at f44ff953, and the knob is opt-in for a deployment that has measured its own backends.

Why it exists (measured on Strix Halo, onebit/HRX path, max_loaded_models: 1 so every load evicts): the exit-and-respawn the wait covers is 0.115 s median / 0.197 s max over 19 switches, while the wait itself is 2.055 s; 100 evict+load cycles at a shortened wait were clean (no failed load, no child exiting non-zero, no router retry, no leftover child). The NPU path pays the same constant (2.003 s of a 5.657 s median NPU switch) and is not covered by that measurement, which is why nothing sets it yet.

Evidence (Strix Halo, this pin)

check result
lemond built from this sha lemond version 2026.47.0~1760.64eb505c (cmake + Ninja, BUILD_WEB_APP=OFF)
ctest -L cpp-ci on this content 111/111 passed, 0 failed, 85.81 s
catalogue 293 entries, 27 onebit, 4 of them onebit NPU (Qwen3-0.6B/1.7B/4B-NPU-1bit, Qwen3.6-35B-A3B-NPU-1bit) — tools/model_sizes.py --all → Errors: 0
generated docs docs/tools/gen_api_boilerplate.py --check → "API reference docs are up to date."
Lemonade's LLM suite through the onebit recipe on HRX (docs/releases.md, "Bumps") Ran 31 tests in 67.098s — OK (skipped=2), the pin's own lemond; one chat 1BIT SMOKE OK at 293.82 tok/s, load 200 in 2.974 s, 13 switches with 0 child failures and 0 retries (comment below)

CI here has no GPU. The suite row is the one the bump table requires; the branch is bump-lemonade/<sha> so scripts/weekly.sh's bumps stage picks it up.

@github-actions

github-actions Bot commented Nov 5, 2026 •

Copy link
Copy Markdown

PR Reviewer Guide 🔍

(Review updated until commit a2597ff)

Here are some key observations to aid the review process:

🎫 Ticket compliance analysis 🔶

37 - Partially compliant

Compliant requirements:

  • Support for ROCmFP4 (Vulkan) and ROCmI4 (ROCm, W4A4) formats is added
  • Builds and serving via 1bit serve --lean are implemented
  • ROCmFPX is pinned at the correct commit with NOTICE credits
  • CMake files updated to support ONEBIT_LEAN and ONEBIT_LEAN_ROCM
  • Tests added for serve_e2e_lean
  • Documentation updated in relevant files

Non-compliant requirements:

  • None

Requires further human verification:

  • Verification of actual hardware behavior with the new formats on Strix Halo
  • Confirmation that the documented performance metrics are met

514 - Partially compliant

Compliant requirements:

  • USB4 v2 / Thunderbolt stack is loaded with required modules
  • Module list includes thunderbolt, typec, ucsi_acpi, typec_displayport, and thunderbolt_net
  • Image verification against enrolled MOK is confirmed

Non-compliant requirements:

  • None

Requires further human verification:

  • Actual USB4 enumeration and tunnelling on hardware needs to be verified
⏱️ Estimated effort to review: 2 🔵🔵⚪⚪⚪
🧪 PR contains tests
🔒 No security concerns identified
⚡ Recommended focus areas for review

Subproject Update

The PR updates the third_party/lemonade subproject to commit 64eb505cc62a4a34d263792b1ad630e7c380e2db. While this is a straightforward update, it's important to ensure that the changes in this commit do not introduce any regressions or unexpected behavior in the Lemonade platform, especially with regard to GPU cleanup waits and process management.

Subproject commit 64eb505cc62a4a34d263792b1ad630e7c380e2db

@bong-water-water-bong bong-water-water-bong left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

VERDICT: READY — the pin delta is #37 and only #37, the shipped default is still 2 s (measured), and the knob does what the body says when it is set.

Re-derived on this box at 7e0e9ba (merge-base 14bda3c):

  • Own diff is the submodule only: git diff 14bda3c..7e0e9ba = third_party/lemonade | 2 +-, f44ff953 -> 64eb505cc. scripts/lemond-onebit.sh is untouched, as the body claims.
  • The pin's content delta is exactly #37, nothing else: git diff f44ff953 64eb505cc in the fork = 2 files, +52/-2 — src/cpp/server/utils/platform/process_linux.cpp (the env-var reader and the two sleep_for call sites) and test/cpp/test_process_manager.cpp (the two new arms).
  • Nothing in this repository sets LEMONADE_GPU_CLEANUP_WAIT_MS: git grep LEMONADE_GPU_CLEANUP_WAIT_MS pr-1175 -- ':!third_party' -> 0 lines. So the appliance at this pin behaves as it does at f44ff953.
  • The default is unchanged, measured rather than read: a built tree at 64eb505cc, ./build/test_process_manager -> 14/14 PASS. Its two terminate log lines are 18:03:37.952 -> 18:03:40.052, i.e. 2.100 s for the default arm, while the whole binary including the LEMONADE_GPU_CLEANUP_WAIT_MS=0 arm ran in 2.284 s.
  • ctest -L cpp-ci on that tree: 111/111 passed, cpp-ci = 85.44 sec*proc (the body says 85.81 s — same suite, ordinary run-to-run spread).
  • CI at this head: build, CodeQL and all five Analyze legs SUCCESS, hw_gate SKIPPED, pr_agent pass.

Two notes, neither blocking:

  1. The pin is one commit behind fork main: refs/heads/main is now 31f7814b (#38, docs, merged in the lemonade repo). Docs-only, so no build difference, but "the fork's main" in the body is no longer literally true — repin if the intent is main.
  2. The bump table's HRX suite row is not in the PR yet ("running — this PR's comment will carry"). The cpp-ci line above is the Lemonade-side half; post the onebit-recipe suite line before merge, or say it runs post-merge.

@bong-water-water-bong

Copy link
Copy Markdown
Collaborator Author

Release gate: Lemonade's LLM suite through the onebit recipe on HRX — green on this pin

docs/releases.md ("Bumps") asks for the suite through the onebit recipe plus one chat, run against the lemond the pin
ships — not against the release package. That run is this comment. gpu-hold -l "pi lemonade" -t 30, ticket after 34 s,
83 s window, no other GPU work of mine in it.

The binary under test is the pin's own: ~/wt/lemonade-evwait checked out at 64eb505cc, build/lemond built from it →
lemond version 2026.47.0~1760.64eb505c, and /api/v1/health answered the same version.

Ran 31 tests in 67.098s
OK (skipped=2)

The two skips are the ones every onebit run has had, both declared by the recipe itself, not failures:
echo_parameter not supported by onebit and generation_parameters not supported by onebit.

check result
load Qwen3-0.6B-1bit @ ctx 8192 HTTP 200 in 2.974 s
one chat reply 1BIT SMOKE OK, finish_reason: stop, 293.82 tok/s predicted / 902.38 tok/s prompt, 11 tokens
suite 31 tests, OK (skipped=2), 67.1 s, SUITE_RC=0
model switches during the suite 13, 0 children exiting non-zero, 0 router retries, no backend left in lemond's group after its SIGTERM
eviction-wait default on the device cleanup→Process started n=7, min 2092, med 2156, max 14436 ms — i.e. the shipped 2000 ms is still what runs (the max is a cold load of a larger model, not the wait)

Box conditions, because they move these numbers: load 17.88 at start (two foreign HRX0 llama-server clients resident, not
mine), Tctl 56.9 °C, die 56 °C at claim. Nothing else of mine ran on the GPU during the window; :18097 and :8088 untouched.

The single [Error] (Server) 400 … POST /api/v1/embeddings line in the log is a test's own negative case (embeddings against a
chat model), and the suite that contains it is the one that passed.

One scope note, so the pin does not look stale by accident

Fork main is now 31f7814b5 — one commit past this pin, docs only (#38, the eviction-wait page). Not retargeting: the
branch name encodes the sha (bump-lemonade/<sha>, which is what scripts/weekly.sh's bumps stage scans for) and the daily
bump-lemonade cron at 03:23 AST picks it up on its own. Nothing in #38 touches code, so the appliance delta of this pin
remains exactly the one commit described above.

Moves third_party/lemonade from f44ff953bcd8 to the fork's main 64eb505cc62a.
@github-actions

github-actions Bot commented Nov 5, 2026

Copy link
Copy Markdown

Persistent review updated to latest commit fbdba1d

@bong-water-water-bong bong-water-water-bong left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-verdict at fbdba1d: READY, unchanged from my review at 7e0e9ba. The head moved, the content did not — third_party/lemonade is 64eb505cc62a at both heads (git ls-tree fbdba1d third_party/lemonade), and the PR's own diff against its merge-base is still third_party/lemonade | 2 +-, nothing else.

Everything in my earlier comment still stands (pin delta = #37 only; LEMONADE_GPU_CLEANUP_WAIT_MS appears nowhere in this repo; default 2 s measured at 2.100 s with 14/14 test_process_manager PASS; ctest -L cpp-ci 111/111).

The one nit is still open and now slightly older: fork main is 31f7814b (#38, docs — merged in the lemonade repo), so this pin is one commit behind it. Docs only, so no build difference; repin if "the fork's main" is what you want to say.

@bong-water-water-bong bong-water-water-bong left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

VERDICT: READY at fbdba1d1 — content-identical to my review at 7e0e9ba (third_party/lemonade is 64eb505cc62a at both heads, own diff still third_party/lemonade | 2 +-). Detail and evidence in the re-verdict above.

@github-actions

github-actions Bot commented Nov 6, 2026

Copy link
Copy Markdown

Persistent review updated to latest commit a2597ff

bong-water-water-bong added a commit that referenced this pull request Nov 6, 2026
…e server this pin ships" (#1212)

docs/lemonade.md's HRX row said the ported `lemond` `f44ff953` is "the server this pin ships".
True when written, false now: #1175 bumped third_party/lemonade to `64eb505cc62a`, so the row
names a build the pin no longer carries — and this is the second time that sentence expired, which
is why #1141 was closed unmerged (it said the pin was `ef0f3b49`, then #1126 landed and made the
original text true again).

The durable wording states the sha that was measured and points at
`git ls-tree third_party/lemonade` for which one ships today, so the next bump cannot invalidate it.
docs/releases.md's v2026.43 candidate table gets the same treatment in one word: "the sha
`bump-lemonade`'s PR #1126 pins" -> "pinned", because that table is a dated record and the tip has
moved since.

Verified: site renders 57 pages rc 0 against this tree, the new sentence is in lemonade.html, one
<h1> per page, no bare closing tag added (diff grep: 0), copyright rc 0.

Co-authored-by: agent <agent@local>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant