zamba2-vl: the conversion lived only in the Vulkan-era checkout — ported to the pinned fork, with the config rule it needs (PR 206)

Status: the port is in review as 1bit-MONSTER/llama.cpp#206 and reproduces both published artifacts. The base_model half of this issue was my mistake — see the bottom.

What is published

1bit-MONSTER/Zamba2-VL-1.2B-GGUF, -2.7B-GGUF, -7B-GGUF: Zamba2-VL-<size>-Q8_0.gguf plus
mmproj-Zamba2-VL-<size>-F16.gguf. Each model file's tokenizer.chat_template:

9df322391abffe14587e6f3cfaea42420eaffec364835debfcb13c0e4c6eeab6   (1045 B, all three files)

That is _CHAT_TEMPLATE from conversion/zamba2vl.py — a port of Zyphra's own multimodal format
(Zyphra/Zamba2-VL-* ship chat_template.jinja = 74475f26d1eeaf2e, <s>-delimited turns,
<|vision_start|><image><|vision_end|> bridge) with the media markers as llama.cpp's
<__media_<id>__> bridge.

The real gap, and the fix

The module lived only in the Vulkan-era checkout (llama.cpp-vulkan/conversion/zamba2vl.py), which
left with that build (RFC #213): the pinned fork at 6e42864a6 has conversion/zaya.py and no
zamba2*, so a pin bump could not re-derive these files.

Fork PR 1bit-MONSTER/llama.cpp#206 brings it to the pinned tree as two commits:

conversion/zamba2vl.py (unchanged) + "Zamba2_VLForConditionalGeneration": "zamba2vl" in both
TEXT_MODEL_MAP and MMPROJ_MODEL_MAP.
conversion/base.py: Zamba2_VLForConditionalGeneration added to the TEXT exception list that
already exists for StepVLForConditionalGeneration, Sarashina2VisionForCausalLM, Exaone4_5_…,
Step3p7_…. Without it the registration is dead code: get_model_architecture() prefers
text_config.architectures[0] for the text pass and Zamba2-VL's text_config says
Zamba2ForCausalLM, so a conversion built the plain Zamba2Model and silently wrote Zyphra's
chat_template.jinja with an unpadded 32004-token vocab instead of our 1045-byte template and 32064
tokens.

Verified end to end by converting Zyphra/Zamba2-VL-1.2B through the branch:

	published	reconverted
text `general.architecture`	zamba2	zamba2
text `tokenizer.chat_template`	`9df322391a…` (1045 B)	`9df322391a…` (1045 B)
text tokens	32064	32064
text tensors	398	378
mmproj tensors	519	519, name/shape set identical

The 20-tensor difference is this repo's own #203 ("store Zamba2's shared transformer block once per
block"), which post-dates the published files: blk.11.* and blk.17.* are no longer duplicated. Worth
knowing before anyone re-quantises from today's tree — the result will not be byte-identical to what is on
the Hub.

The engine side needs no change beyond the next llama.cpp pin bump.

Withdrawn: the `base_model` metadata claim

I wrote earlier that the published files' general.base_model.0 naming the text Zyphra/Zamba2-1.2B
was a defect against the hub card. It is not: Zyphra/Zamba2-VL-1.2B's own README declares

base_model:
- Zyphra/Zamba2-1.2B

so the GGUF metadata is a faithful copy of the vendor's card, and a fresh conversion with the card present
reproduces it (Metadata.load() reads the README; my card-less test conversion was the only thing missing
those keys). Our hub card's base_model: Zyphra/Zamba2-VL-1.2B is a summarisation choice, not a fix. No
re-quantise is needed for it.

Authoring site restored as a fork PR: 1bit-MONSTER/llama.cpp#206 (1bit/zamba2vl-conversion, a6cf0fc4, base 1bit/hrx-vulkan-patched) — conversion/zamba2vl.py brought over unchanged from the Vulkan-era checkout and registered in both TEXT_MODEL_MAP and MMPROJ_MODEL_MAP.

Verified against the pinned tree 6e42864a6: both classes register for Zamba2_VLForConditionalGeneration (TEXT → Zamba2VLTextModel, arch ZAMBA2 = 67; MMPROJ → Zamba2VLVisionModel, arch 1), and sha256(_CHAT_TEMPLATE) = 9df322391abffe14587e6f3cfaea42420eaffec364835debfcb13c0e4c6eeab6, 1045 B — byte-identical to the template in all three published GGUFs. A full conversion of Zyphra/Zamba2-VL-1.2B is running to confirm end-to-end and will be reported on the fork PR.

Still open here, and it needs a re-quantise rather than a converter: the published files' general.base_model.0 names the text Zyphra/Zamba2-1.2B. Nothing in this issue requires a template change.

Engine-side integration checked, so the pin bump has no surprises: rebuilding the registry against the fork branch adds exactly one row and nothing else.

$ tools/registry_build.py --out /tmp/arch.json      # third_party/llama.cpp -> the branch at 95b1b579
wrote /tmp/arch.json: hrx 298, npu 7, architectures 298, mapped 298      # main: 297 / 297
added:   ['Zamba2_VLForConditionalGeneration']
removed: []
gguf_architectures.hrx: unchanged (ZAMBA2 is already a runtime arch)

So the bump PR is the usual one: pin llama.cpp to the fork commit and regenerate registry/architectures.json (CI's registry_pins test fails a pin that moved without it, and scripts/registry-regen.sh is what the bump workflows run). Zamba2_VLForConditionalGeneration then appears as mapped on hrx — the converter names it and Zamba2Model's GGUF architecture (zamba2) is already in the fork's runtime — with no new GGUF architecture and no significant.json gap.

The registration alone would not have shown up here: without the get_model_architecture() exception in the same PR, the converter resolves this config family to Zamba2ForCausalLM and the class never runs, which is how a conversion could have shipped the vendor's chat_template.jinja instead of ours.

third_party: bump llama.cpp to 391bad862 (3 fork commits: #204 MUL_MA…

Resolved on main — the fork port is merged, the engine pin includes it, and the registry maps the class. Verified at a083e31:

Fork PR convert: Zamba2-VL (Zamba2 language model + Qwen2.5-VL tower), so the published GGUFs have an authoring site (engine#1098) llama.cpp#206 ("convert: Zamba2-VL …") merged 2026-11-05T12:28Z as 391bad86 (it carries the get_model_architecture() exception this issue says the registration needs).
Engine pin bumped to it by third_party: bump llama.cpp to 391bad862 (3 fork commits: MUL_MAT_ID packet, attention-device pin, Zamba2-VL) #1104 ("third_party: bump llama.cpp to 391bad862 (3 fork commits: README: status for W4A4 MoE and serve --long-model #204 MUL_MAT_ID packet, serve: Hadamard-rotated files run 1024-token micro-batches #205 attention pin, Laya GGUF scorer: ggmlc's laya on Vulkan, 24 ms a routing decision #206 Zamba2-VL)", merged 13:49Z), later moved forward to 4a375bf2 by Bump HRX: llama.cpp 4a375bf2d36f + hrx-system 82804197bcbe #1120.
registry/architectures.json on main records llama.cpp (hrx): 4a375bf2d36f…, Zamba2_VLForConditionalGeneration -> {"gguf": "zamba2", "backends": ["hrx"]}, counts 298/298 mapped — exactly the one row the last comment measured would be added, with no new GGUF architecture and no significant.json gap.
The registry is regenerated at the pin, so CI's registry_pins test is the standing guard that the row and the pin cannot drift apart.

Closing: the conversion no longer lives only in the Vulkan-era checkout, and the "usual bump PR" this issue was waiting on is #1104.

Navigation Menu

zamba2-vl: the conversion lived only in the Vulkan-era checkout — ported to the pinned fork, with the config rule it needs (PR 206) #1098

What is published

The real gap, and the fix

Withdrawn: the `base_model` metadata claim

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

zamba2-vl: the conversion lived only in the Vulkan-era checkout — ported to the pinned fork, with the config rule it needs (PR 206) #1098

Description

What is published

The real gap, and the fix

Withdrawn: the base_model metadata claim

Activity

bong-water-water-bong commented on Nov 5, 2026

bong-water-water-bong commented on Nov 5, 2026

bong-water-water-bong commented on Nov 6, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions

Withdrawn: the `base_model` metadata claim