Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -57,8 +57,8 @@ Windows and 1bit OS ([docs/releases.md](docs/releases.md)).
> 16.3-16.5 tok/s decode ([docs/npu.md](docs/npu.md#private-routes)). Step 4, the Laya router,
> has landed as an opt-in: `1bit serve --device auto --laya-model <dir>` picks the device per
> request with the C++ scorer, which matches the Python reference; a decision takes 0.38 s on
> Strix Halo ([docs/laya.md](docs/laya.md)). Step 5, the model registry, has landed: of 332,565
> HF text-generation models with an architecture, 93.28% are mapped to a backend and 64.18%
> Strix Halo ([docs/laya.md](docs/laya.md)). Step 5, the model registry, has landed: of 332,726
> HF text-generation models with an architecture, 94.88% are mapped to a backend and 64.33%
> have an architecture checked end to end on Strix Halo; a daily census keeps the counts
> current ([docs/registry.md](docs/registry.md)). The working engine is being ported from 1bit-MONSTER,
> our private development repository; see [docs/PORTING.md](docs/PORTING.md).
Expand Down
2 changes: 1 addition & 1 deletion docs/PORTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ Each step below is one PR (or a short series) that builds and runs on Strix Halo
| 3x | **35B MoE on the NPU (experimental, closed source).** Qwen3.6-35B-A3B on the NPU | the private `1bit-MONSTER/npu-kernels` repository | **landed as a private add-on** ([docs/npu.md](npu.md#private-routes)): built into `1bit` with `-DONEBIT_NPU_PRIVATE`; parity against the fp64 reference passes (3 positions, argmax 846 / 198 / 3710), 16.3-16.5 tok/s decode, and `1bit serve --device npu` answers "The capital of France is Paris." Without the add-on, `serve` says the route is not part of the build |
| + | **Linux kernel.** The kernel that provides `amdxdna` and `amdgpu`, pinned to upstream | `torvalds/linux` release tags; config from the Strix Halo kernel of 2026-09-23 | **pinned** ([docs/kernel.md](kernel.md)): v7.3-rc4 builds into Debian packages with `amdxdna` in-tree; kept current by `bump-linux.yml`. Installing it on Strix Halo is a separate, deliberate step |
| 4 | **Laya router.** A non-autoregressive scorer that picks where each request runs | `src/laya_scorer.cpp`, `include/laya_scorer.h` on `backup/laya-and-results-2026-09-22`; model at `~/models/laya` | **landed, opt-in** ([docs/laya.md](laya.md)): source `NandhaKishorM/laya` + the three HF checkpoints pinned, hash-verified fetch, `bump-laya.yml` (#18); the C++ scorer matches the Python reference on the root and `typed-decisions/` checkpoints (max logit diff 8.6e-6, same argmax) and `1bit serve --device auto --laya-model <dir>` routes each request (#90). Load 2.8 s once, then 0.38 s per decision (was 8.75 s, #91). Next: `multilingual/` (mmBERT-base) |
| 5 | **Every HF model, kept current.** The architecture registry and the daily HF census that finds new architectures and ranks the unmapped ones by model count (1bit-MONSTER's registry mapped 2,030 HF arch strings to its own kernels; here the backends' pinned code decides) | `src/model_registry*.cpp`, `Testing/census_*.py` and `.json`, `.github/workflows/census-{watch,sweep,autopr}.yml` | **landed** ([docs/registry.md](registry.md)): `registry/architectures.json` generated from the pinned llama.cpp, HRX fork and ZINC (265 HF architectures); `census.yml` sweeps HF daily. First sweep: 415,414 text-generation models, mapped 93.28%, checked 64.18% of those with an architecture (reported separately). All seven checked GGUF architectures pass on HRX since the llama.cpp pin `96f6b89` (#95) |
| 5 | **Every HF model, kept current.** The architecture registry and the daily HF census that finds new architectures and ranks the unmapped ones by model count (1bit-MONSTER's registry mapped 2,030 HF arch strings to its own kernels; here the backends' pinned code decides) | `src/model_registry*.cpp`, `Testing/census_*.py` and `.json`, `.github/workflows/census-{watch,sweep,autopr}.yml` | **landed** ([docs/registry.md](registry.md)): `registry/architectures.json` generated from the pinned llama.cpp, HRX fork and ZINC (265 HF architectures); `census.yml` sweeps HF daily. First sweep: 415,414 text-generation models, mapped 94.88%, checked 64.33% of those with an architecture (reported separately). All seven checked GGUF architectures pass on HRX since the llama.cpp pin `96f6b89` (#95) |
| 5z | **ZAYA1 (Zyphra) on the GPU.** An architecture upstream llama.cpp lacks, in our llama.cpp; `1bit serve --device vulkan` routes it there | `1bit-MONSTER/llama.cpp` `1bit/hrx2-archive` (`src/models/zaya.cpp`) and 1bit-MONSTER's `tools/convert_zaya_safetensors_to_gguf.py` | **landed** ([docs/vulkan.md](vulkan.md#zaya1-zyphra-from-our-llamacpp)): ported to the pinned fork with a converter; matches transformers in FP32 at 95/96 teacher-forced positions; Q4_K_M decodes at 93 tok/s on Vulkan (74.8 on the old ggml-hrx2 build), 61.5 on ROCm and 25.5 on HRX, and passes `serve_e2e` on Vulkan and HRX |
| 6 | **ZINC (NVIDIA and more).** Upstream `zolotukhin/zinc`, a Zig GGUF engine with Vulkan, ROCm, CUDA and Metal backends; its CUDA backend reaches NVIDIA GPUs (Ada `sm_89`, Blackwell `sm_120`) | not in 1bit-MONSTER; pinned from upstream `main` | **pinned** ([docs/zinc.md](zinc.md)): `scripts/build-zinc.sh` builds it privately; the Vulkan build gives 12095 (" Paris") at 295 tok/s on Strix Halo; the CUDA build answers " Paris." at 167–173 tok/s on an RTX 5090 (Qwen3.5-9B); kept current by `bump-zinc.yml`. `1bit serve --device zinc` runs it (`-DONEBIT_ZINC=ON`, e2e passes) |

Expand Down
16 changes: 8 additions & 8 deletions docs/registry.md
Original file line number Diff line number Diff line change
Expand Up @@ -90,20 +90,20 @@ counts and the coverage read against the registry and the checked results. The
`census.yml` workflow runs daily at 05:17 UTC. It rebuilds the registry from the pins,
sweeps HF, and opens a PR when anything changed.

The first full sweep, 2026-09-25:
The census of 2026-09-27 (the first full sweep ran 2026-09-25), with `registry/census.json` holding the latest:

| | models | share of those with an architecture |
|---|---|---|
| Text-generation models on HF | 415,414 | |
| With an architecture in their config | 332,565 (2,610 architectures) | |
| Mapped | 310,221 | 93.28% |
| Checked | 213,456 | 64.18% |
| Text-generation models on HF | 415,695 | |
| With an architecture in their config | 332,726 (2,616 architectures) | |
| Mapped | 315,701 | 94.88% |
| Checked | 214,033 | 64.33% |

| Backend | Mapped | Checked |
|---|---|---|
| vulkan | 93.28% | 64.18% |
| hrx | 93.19% | 64.18% |
| zinc | 69.69% | 10.28% |
| vulkan | 94.88% | 64.33% |
| hrx | 94.78% | 64.32% |
| zinc | 69.83% | 10.28% |
| npu | 9.29% | 9.29% |

"Checked" counts every model whose architecture has a passing model. It does not mean each
Expand Down
Loading