Skip to content

weekly: run Lemonade's suite against a running lemond; clearer held reasons - #135

Merged
bong-water-water-bong merged 1 commit into
mainfrom
weekly-suite-deps
Sep 26, 2026
Merged

bong-water-water-bong merged 1 commit into
mainfrom
weekly-suite-deps

Conversation

@bong-water-water-bong

Copy link
Copy Markdown
Collaborator

A dry-run rehearsal of the Sunday run (WEEKLY_DRY_RUN=1) held the Lemonade bump for the wrong reason:

  • Lemonade's test/server_llm.py expects a lemond already listening. The bump check now starts the build's lemond on its own port (13399, LEMONADE_TEST_PORT) with this build's 1bit, and stops it afterwards.
  • The suite needs numpy. The venv now installs Lemonade's own test/requirements.txt.
  • The held reason quoted the passing ctest log. It now names the step that failed and the last lines of that step's log.

Also, for the first release: the notes say "commit" for a newly added pin and drop "None →" from the Linux line.

Rehearsal on Strix Halo: #132 (ZINC) and #133 (Lemonade) both pass. Lemonade's LLM suite through onebit on Vulkan ran 31 tests, OK (skipped=2).

🤖 Generated with Claude Code

…; name the failing step

Found by a dry-run rehearsal: the suite expects a lemond already listening (port 13305) and
needs numpy from test/requirements.txt, so the Lemonade bump was held for the wrong reason, and
the held reason quoted the (passing) ctest log. Now lemond from the build runs on its own port
for the suite; a held bump names the step and the log that failed. First-release notes say
"commit" for a new pin and drop "None →".

Rehearsal on Strix Halo (WEEKLY_DRY_RUN=1): #132 (ZINC) and #133 (Lemonade) pass; Lemonade's
LLM suite through onebit on Vulkan: Ran 31 tests, OK (skipped=2).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@context7

context7 Bot commented Sep 26, 2026

Copy link
Copy Markdown

Docs7 for 1bit-monster/engine

Result Status Action
Deployment ➖ Not used —
Content review ➖ Did not run. This site has no agent runs available this month. Wait for the monthly reset or check your Docs7 plan. —

Commit 9195301

@github-actions

Copy link
Copy Markdown

PR Reviewer Guide 🔍

Here are some key observations to aid the review process:

🎫 Ticket compliance analysis 🔶

132 - Partially compliant

Compliant requirements:

  • Move third_party/zinc from 3a35e76d64eb to upstream main 29bc350ac4cb

Non-compliant requirements:

  • Run CI test on Strix Halo with specific command

Requires further human verification:

  • Verification of the specific command execution and performance metrics on Strix Halo

133 - Partially compliant

Compliant requirements:

  • Move third_party/lemonade from fb65f488457f to the fork's main 2b491610b828

Non-compliant requirements:

  • Run CI test on Strix Halo with Lemonade's LLM suite through the onebit recipe

Requires further human verification:

  • Verification of Lemonade's LLM suite execution on Strix Halo
⏱️ Estimated effort to review: 3 🔵🔵🔵⚪⚪
🧪 No relevant tests
🔒 No security concerns identified
⚡ Recommended focus areas for review

Lemonade suite dependency installation

The script now installs dependencies for Lemonade's test suite using test/requirements.txt instead of a hardcoded list. This change is good for maintainability, but it's important to ensure that the dependencies specified in test/requirements.txt are compatible with the current environment and that the installation process doesn't introduce any conflicts or regressions.

[ -x "$venv/bin/python" ] || python3 -m venv "$venv"
"$venv/bin/pip" -q install -r "$SRC/third_party/lemonade/test/requirements.txt"  # the suite's own list
# the suite talks to a running lemond: ours, on a port of its own, with this build's 1bit
Lemonade suite test execution

The script now starts a lemond process on a dedicated port (13399) for the Lemonade suite tests. This approach isolates the test environment, which is good for preventing interference between tests. However, it's important to ensure that the lemond process is properly cleaned up even if the test fails, which the script seems to handle with kill "$pid" and wait "$pid".

local port=13399 pid rc=0
LEMONADE_ONEBIT_BIN="$BUILD/1bit" "$W/lemonade/bin/lemond" --port "$port" > "$LOGS/lemond.log" 2>&1 &
pid=$!
( cd "$SRC/third_party/lemonade" &&
  PATH="$W/lemonade/bin:$BUILD:$PATH" LEMONADE_ONEBIT_BIN="$BUILD/1bit" LEMONADE_TEST_PORT="$port" \
  "$venv/bin/python" test/server_llm.py --wrapped-server onebit --backend vulkan ) > "$LOGS/lemonade-suite.log" 2>&1 || rc=$?
kill "$pid" 2>/dev/null; wait "$pid" 2>/dev/null || true
return $rc
Improved error reporting in bump checks

The error reporting in stage_bumps has been improved to include the specific step that failed and the last lines of the relevant log file. This makes it easier to diagnose failures in the bump checks. However, the implementation relies on ls -t and head -1 to find the most recent log file, which could be fragile if there are unexpected log files or if the file system behavior changes.

# the step that failed, and the last lines of the log it wrote (the newest besides ours)
local last; last=$(ls -t "$LOGS"/*.log | grep -v "/bump-" | head -1)
result=held why="failed at $(cat "$W/.step") on Strix Halo ($(basename "$last")): $(tail -n 2 "$last" | tr '\n' ' ' | cut -c1-240)"

@bong-water-water-bong
bong-water-water-bong merged commit be7d412 into main Sep 26, 2026
3 checks passed
@bong-water-water-bong
bong-water-water-bong deleted the weekly-suite-deps branch September 26, 2026 13:27
bong-water-water-bong pushed a commit that referenced this pull request Oct 23, 2026
Fork #135 lets the vendored jinja lexer accept \xNN hex escapes, so a
chat template that opens with replace("\x00", "") (Flint's does) no
longer aborts serve and the onebit backend before the model loads.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
bong-water-water-bong added a commit that referenced this pull request Oct 25, 2026
llama.cpp along 1bit/hrx-vulkan-patched 851fa6c8d8f2 -> e7e5cd08f9bb: DFlash2 drafters on the pinned branch (#139) + FILL cache key / TOP_K tests (#140), Qwen3.8-Flash-Next on HRX with the HIP-defer guard (#142) and IQ2_S routing (#144), DFlash2 Hadamard folds for Ternary-Bonsai (#143), HRX test-suite / Flint / jinja changes (#126-#135). hrx-system held at 82804197bcbe. hw-gate green after the KLD base logits were restored on the gate box. The private kernel pin follows in its own PR.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant