Skip to content

ZAYA1 legacy checkpoints and remote chat templates: pin llama.cpp 567d2dc; ZAYA1-74B docs - #129

Merged
bong-water-water-bong merged 2 commits into
mainfrom
zaya1-74b
Sep 26, 2026
Merged

bong-water-water-bong merged 2 commits into
mainfrom
zaya1-74b

Conversation

@bong-water-water-bong

Copy link
Copy Markdown
Collaborator

Moves third_party/llama.cpp from 4380daf (#127) to 567d2dc, the merge of 1bit-MONSTER/llama.cpp#14. It changes the converter only, not the runtime:

  • Legacy ZAYA checkpoints. ZAYA1-base, ZAYA1-reasoning-base and Zyphra's *-legacy repos use the original Megatron-style layout, which transformers cannot load. The converter maps them to the transformers layout.
  • --remote fetches chat_template.jinja. Remote conversions (such as ZAYA1-74B-preview's) had no chat template before. Upstream has the same gap; a draft patch is in ~/upstream-drafts for the user.

Checked on Strix Halo:

  • --remote Zyphra/ZAYA1-8B-legacy converts to a GGUF whose 1,283 tensors are byte-identical to the one from Zyphra/ZAYA1-8B, with identical model metadata.
  • The normalized legacy configs equal the new-format ones for the 8B and 74B pairs, including the 74B's sliding window and per-layer rope bases.
  • ZAYA1-74B-preview Q4_K_M, llama-bench on Vulkan0: pp512 505.8 tok/s, tg128 35.4 tok/s. Chat with the model's own template gives coherent answers.
  • ZAYA1-8B F16 on the new code matches transformers FP32 at 95 of 96 teacher-forced positions, unchanged.

Docs:

registry/architectures.json is regenerated for the new pin; only the recorded sha changes.

ZAYA1-base and ZAYA1-reasoning-base GGUFs are being converted now; their checks will follow in a separate PR.

🤖 Generated with Claude Code

…d2dc (#14); ZAYA1-74B docs

llama.cpp #14 converts Zyphra's legacy (Megatron-style) ZAYA checkpoints, such as
ZAYA1-base and ZAYA1-reasoning-base, and makes --remote fetch chat_template.jinja.
docs/vulkan.md gets a ZAYA1-74B-preview section (convert, requantize, serve,
bench) and a legacy-checkpoint section; the README status names the Zyphra family.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@context7

context7 Bot commented Sep 26, 2026 •

Copy link
Copy Markdown

Docs7 for 1bit-monster/engine

Result Status Action
Deployment ➖ Not used —
Content review ➖ Did not run. This site has no agent runs available this month. Wait for the monthly reset or check your Docs7 plan. —

Commit d32bd46

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown

Preparing review...

@github-actions

Copy link
Copy Markdown

PR Reviewer Guide 🔍

Here are some key observations to aid the review process:

🎫 Ticket compliance analysis 🔶

127 - Partially compliant

Compliant requirements:

  • Move third_party/llama.cpp from 1e775cd to 4380daf
  • Add ZAYA1-74B-preview's sliding-window layers (4,097-token window, rope theta 1e4)
  • Add second rope base (1e7)
  • Regenerate registry/architectures.json to record 4380daf

Non-compliant requirements:

  • None

Requires further human verification:

  • Verification of Q4_K_M perplexity on Vulkan for ZAYA1-8B and ZAYA1-74B-preview
  • Confirmation that test-llama-archs -a zaya passes on Vulkan and CPU
  • Confirmation that tests/serve_e2e.sh passes for ZAYA1-74B-preview on Vulkan

14 - Partially compliant

Compliant requirements:

  • Pin ZINC at upstream main c50b4add
  • Build ZINC with Zig 0.15.2
  • Add documentation for ZINC build and verification
  • Verify Qwen3.5-9B on RTX 5090 with CUDA backend
  • Verify Qwen3.5-0.6B on Strix Halo with Vulkan backend
  • Verify ROCm build and link

Non-compliant requirements:

  • None

Requires further human verification:

  • Confirmation that the user's fork bong-water-water-bong/zinc commits are not included
  • Confirmation that no Lemonade backend spawns zinc yet
⏱️ Estimated effort to review: 3 🔵🔵🔵⚪⚪
🧪 No relevant tests
🔒 No security concerns identified
⚡ Recommended focus areas for review

Documentation Inconsistency

The documentation states that ZAYA1-74B-preview's Q4_K_M perplexity matches to four decimals with ZAYA1-8B, but the PR description mentions that ZAYA1-8B F16 on the new code matches transformers FP32 at 95 of 96 teacher-forced positions. This discrepancy should be clarified to ensure consistency in the documentation.

| Check | Result |
|---|---|
| `tests/serve_e2e.sh`, `--device vulkan --ctx-size 8192` | PASS |
| Chat with the model's own template (code, prose, a one-line fact) | coherent answers |
| `llama-bench`, Vulkan0 | prefill (pp512) 505.8 tok/s, decode (tg128) 35.4 tok/s |

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant