From d4bf65c9ab79910e789d618067c33008d9bfaa81 Mon Sep 17 00:00:00 2001 From: bong-water-water-bong Date: Sat, 7 Nov 2026 22:32:40 -0400 Subject: [PATCH 1/2] docs/lora: a row for main's current pin, and the rule that the drift is transient MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Trap 1's table had rows for the c20 pin and for 4a375bf2d, and main now pins ed43da0b4 — on a page whose whole point is that the zero-adapter offset is a property of a pin. Measured on a clean build of that pin (source tree at ed43da0b4 with an empty `git status`, frozen before use, device named in the log): pin base zero-adapter offset a5271d9 2.265140 2.272791 +0.34 % 4a375bf2d 2.267098 2.265357 -0.08 % ed43da0b4 2.265140 2.272791 +0.34 % <- same as a5271d9 So the 4a375bf2d drift is gone rather than compounded: all four metrics (base, zero-adapter, after-drop and the +/-eps probe, plus the x10 response) are back to a5271d9's exact values, and the row was taken twice per pair in one ticket (reference pair and this pin alternating) so it is a measured identity, not a lookalike. Consequence stated on the page: a loss from one pin ago is neither an upper nor a lower bound on today's — re-measure the row, do not interpolate. Two smaller corrections from the same round: - the cost table named engine#973 but not the pin its numbers came from; it now says llama.cpp 32559003a (the pin main carried then), so it cannot be read as a claim about the current pin; - a new paragraph, "attribute the build before you record a row — the commit is not enough": two source trees at the same sha are not the same build (one is the pin, one is that sha plus a lane's four modified files), so a row records pin sha + working-tree cleanliness + the frozen libraries' sha256sum + the device line. Refs #973, #1357 (the page's own merge). --- docs/lora.md | 24 ++++++++++++++++++++++-- 1 file changed, 22 insertions(+), 2 deletions(-) diff --git a/docs/lora.md b/docs/lora.md index b21a9f70..33f86f9a 100644 --- a/docs/lora.md +++ b/docs/lora.md @@ -55,7 +55,8 @@ There is now an in-place path in `include/llama.h` (engine#973): The factors are copied into the tensors the graph already points at, so there is no parse, no allocation, no adapter-set change and no graph rebuild. -**Measured** (engine#973, Qwen3-0.6B Q4_K_M, rank 2 over 196 projections, batch 16, per update, HRX0, +**Measured** (engine#973, on a build of llama.cpp `32559003a` — the pin main carried then, so these are that +pin's numbers and not the current pin's; Qwen3-0.6B Q4_K_M, rank 2 over 196 projections, batch 16, per update, HRX0, min of two passes, `write` = the factor update, `apply` = reload + adapter-set change): | path | bank RT 2 | bank RT 26 | @@ -85,13 +86,25 @@ does not, deterministically: adding the factor pair to the graph changes the mat |---|---|---|---| | `a5271d9` (c20) | 2.265140 | 2.272791 | **+0.34 %** above base | | `4a375bf2d` (engine #1120) | 2.267098 | 2.265357 | **−0.08 %** below base | -| Vulkan, either pin | 2.219879 | 2.219879 | bit-identical | +| `ed43da0b4` (current) | 2.265140 | 2.272791 | **+0.34 %** above base — same as `a5271d9` | +| Vulkan, any of the three | 2.219879 | 2.219879 | bit-identical | Same four fixed prompts, same model, same seeds, each row reproduced bit-identically across runs. So: an adapter-on vs adapter-off loss comparison on HRX carries an offset of about 0.3 %, **it changes sign between pins**, and it must be read against the zero-factor row of the *same build*, never against a number remembered from another one. A difference-based method (two adapter-on evaluations) is unaffected. +The third row also shows the drift is **transient rather than monotonic**: `4a375bf2d` moved the HRX base loss to +2.267098 and flattened the probe's ±ε ordering, and `ed43da0b4` puts all four metrics back to `a5271d9`'s exact +values. So a loss measured one pin ago is not a lower or upper bound on today's — it is simply another pin's number. +The practical rule is the same one: re-measure the row, do not interpolate between rows. + +That third row is the most measured of the three: reference pair (`a5271d9`) and this pin alternating, **two runs +each inside one GPU ticket**, four identical fingerprints — base 2.265140, zero 2.272791, drop 2.265140, ±ε +2.086712/2.256433, ×10 13.670712/13.830040, and Vulkan 2.219879 on both pairs. Two runs per pair is what separates +"the numbers are deterministic and this pin equals that one" from "the numbers looked like this once". A single run +that differs is worth nothing until it repeats. + ## Trap 2: the adapter's tensors are the model's buffer type, so "repack to CPU" is not what happens `llama_adapter_lora_init_impl` picks the buffer type per tensor from the model tensor it shadows. On a repacking @@ -113,6 +126,13 @@ moving the HRX base loss by +0.086 %: of GPU, and the whole mechanism is ~30 lines against `libllama` (engine#973's feasibility study kept the harness and the reference rows). +**Attribute the build before you record a row — the commit is not enough.** Two trees at the same llama.cpp sha are +not the same build: one can be the pin with a clean working tree, the other that same sha plus someone's four +modified sources. Both produce a `libggml-hrx.so` for the same commit, and only one of them is the pin. So a row +records four things: the pin sha, whether the source tree was **clean** (`git status --porcelain` empty), the frozen +copy of the libraries actually loaded (with `sha256sum`), and the log line that names the device. Skipping the +cleanliness check is how a table gets a row labelled with a pin it was never built from. + ## What is deliberately not here - **Per-sequence factor slots / co-serving updates inside `serve --parallel`.** Nothing in the engine needs it yet, From 16e54b5d83e2a0cc41d6dbf21ef803d37387f3f1 Mon Sep 17 00:00:00 2001 From: bong-water-water-bong Date: Sat, 7 Nov 2026 22:40:37 -0400 Subject: [PATCH 2/2] docs/lora: show the six numbers the reversion claims, instead of counting them MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Both count/precision notes from the review of #1399, taken in the section that asks readers not to trust a remembered number: 1. "all four metrics back to a5271d9's exact values" was asserted, not shown — a5271d9's ±ε and ×10 appeared nowhere on the page while this pin's did. The evidence is now a two-row table: the reference pair and this pin side by side on base, zero-adapter, after-drop, ±ε, ×10 and Vulkan, every cell equal and every cell taken twice. That turns the reversion claim into something a reader can check in the table rather than trust from the prose. 2. "four identical fingerprints" was followed by five items (six with Vulkan). The sentence now says six and the table lists exactly six, so the count and the list agree. Refs #973, #1357, #1399. --- docs/lora.md | 13 +++++++++---- 1 file changed, 9 insertions(+), 4 deletions(-) diff --git a/docs/lora.md b/docs/lora.md index 33f86f9a..cbaefdd0 100644 --- a/docs/lora.md +++ b/docs/lora.md @@ -100,10 +100,15 @@ values. So a loss measured one pin ago is not a lower or upper bound on today's The practical rule is the same one: re-measure the row, do not interpolate between rows. That third row is the most measured of the three: reference pair (`a5271d9`) and this pin alternating, **two runs -each inside one GPU ticket**, four identical fingerprints — base 2.265140, zero 2.272791, drop 2.265140, ±ε -2.086712/2.256433, ×10 13.670712/13.830040, and Vulkan 2.219879 on both pairs. Two runs per pair is what separates -"the numbers are deterministic and this pin equals that one" from "the numbers looked like this once". A single run -that differs is worth nothing until it repeats. +each inside one GPU ticket**, and all six numbers equal in all four runs — so the identity is measured, not inferred: + +| | base | zero-adapter | after drop | ±ε probe | ×10 probe | Vulkan base | +|---|---|---|---|---|---|---| +| `a5271d9`, reference pair (runs 1, 2) | 2.265140 | 2.272791 | 2.265140 | 2.086712 / 2.256433 | 13.670712 / 13.830040 | 2.219879 | +| `ed43da0b4`, this row (runs 1, 2) | 2.265140 | 2.272791 | 2.265140 | 2.086712 / 2.256433 | 13.670712 / 13.830040 | 2.219879 | + +Two runs per pair is what separates "the numbers are deterministic and this pin equals that one" from "the numbers +looked like this once". A single run that differs is worth nothing until it repeats. ## Trap 2: the adapter's tensors are the model's buffer type, so "repack to CPU" is not what happens