hrx: at -ngl 99 the weights are resident twice in GTT (one whole-model BO plus per-tensor BOs) - 5.9 GiB for a 2.38 GiB model

What was measured

amdgpu_bo_create tracepoint (one event per KFD Buffer Object with its exact pages), per allocating process, on
strixhalo (gfx1151, ttm.pages_limit = 120 GiB GTT, no-SVM trio HSA_USE_SVM=0 HSA_USERPTR_FOR_PAGED_MEM=0 HRX_ALLOW_NO_SVM=1), pinned fork d8bd0ff6052a (llama-cli from ~/wt/i674-repro/stage-pin788, build b0-d8bd0ff6052),
-fit off -c 256 --temp 0 -n 8 -no-cnv, model Qwen3-4B-Q4_K_M (398 tensors, 2375.9 MiB of tensor bytes, 2.31 GiB file).

The trace accounts for essentially the whole footprint: BO creation sums vs the same run's own peak drm-total-gtt:
-ngl 0 3299.3 / 3301 MiB, -dev none 296.9 / 298 MiB, -ngl 99 5901.3 / 5903 MiB.

arm	BOs	BO bytes	largest BO	per-tensor weight BOs
`-dev none -ngl 0` (control)	54	296.9 MiB	140 MiB	none
`-dev HRX0 -ngl 0`	416	3299.3 MiB	306 MiB	yes: 20 MiB ×18, 14 MiB ×90, 6 MiB ×72, 4 MiB ×18 …
`-dev HRX0 -ngl 99`	363	5901.3 MiB	2376 MiB	the same set again: 306 MiB ×3, 20 MiB ×18, 14 MiB ×90, 6 MiB ×72, 4 MiB ×19, 2 MiB ×54

The 2376 MiB BO is exactly ceil(2491323904, 2 MiB) — the GGUF's whole tensor region rounded up to the 2 MiB allocation
granularity this driver uses — and the per-tensor BOs next to it are byte-for-byte the same size classes the -ngl 0 arm
materialises, at counts that match the GGUF's own tensor families (18 blk.N.ffn_down Q6_K, 90 ffn_gate+ffn_up+
ffn_down Q4_K, 72 attn_q+attn_output, 18 attn_v).

So with all layers offloaded the same weights are resident in GTT twice: once as the whole-model buffer (~2.32 GiB) and
once as per-tensor device buffers (~2.07 GiB). That is where this arm's 5.9 GiB for a 2.38 GiB model goes; it is not KV,
arenas or transients (the control run fixes those at 296.9 MiB, and a first request adds only a few MiB).

Same numbers for the other models at -ngl 0, where the attribution is exact (observed BO counts vs GGUF classes rounded
to 2 MiB, fixed cost subtracted): 0.6B 122 MiB ×2, 4 MiB ×14/14, 2 MiB class 182 of 295 (the 113 missing are exactly the
113 F32 norm tensors, so -ngl 0 materialises the MUL_MAT weights and leaves norms host-side); 8B 422 MiB ×1, 72 MiB
×40/40, 54 MiB ×20/20, 36 MiB ×20/20.

Interpretation (flagged, not claimed)

Either the residency cache re-materialises weights that are already device-resident under -ngl 99 — the -ngl 0 defect of
#669 in the other mode — or the whole-model BO is a different backing (import/loader buffer) that the compute path does not
read, and the per-tensor BOs are the only live copy. What would settle it: knowing which of the two the graph binds. From
outside, both copies are charged to GTT for the process's lifetime.

Reproduction

# under gpu-hold, trio on, root only for tracefs (bo_trace restores it and drops the workload back to $SUDO_USER)
gpu-hold -l <lane> -t 5 -- bash tools/scripts/mem-botrace.sh      # 3 arms in ~5 s
gpu-hold -l <lane> -t 3 -- bash tools/scripts/mem-sizes.sh        # allocation-size sweep, <1 s

Instrument: tools/bo_trace.py (selftest included) — enables only events/amdgpu/amdgpu_bo_create, raises
buffer_size_kb for the run, restores both, attributes events to the child by tid/comm, and reports BO sizes plus the
tracepoint's type/prefer/allow/visible fields. Raw evidence for this report: evidence/memory/bo-trace-20261102-1647.log
(three arms) and bo-trace-multi-20261102-1659.log (0.6B + 8B).

Scope / caveats

One model at -ngl 99 (the 4B). -ngl 0 was verified at three models.
-fit off, single short prompt, one ticket, one box state; the -ngl 0 unbounded total moves 3301→4113 MiB between
sessions (recorded separately), but the composition above reproduced exactly within this ticket.
Small (sub-2 MiB) allocations are suballocated inside slabs, so their BO counts are not a tensor census; the ≥ 4 MiB
classes are exact one-for-one.

Triaged: bug added. Owner hrx-study (the lane that measured it), bucket A. Three things from the tree at the pin
d8bd0ff6052a, all device-free — one of them settles the question you flagged rather than deferring it.

1. The 2376 MiB BO is the loader's model buffer, not an import or a registry mapping

The size is the giveaway and the source confirms it: llama.cpp allocates one buffer per buffer type, covering every
tensor assigned to that buft, and prints its size.

src/llama-model.cpp:1649   buf = ggml_backend_alloc_ctx_tensors_from_buft(ctx, buft);   // one buffer per buft
src/llama-model.cpp:1696   "%s: %12s model buffer size = %8.2f MiB\n"

So at -ngl 99 that BO is the model buffer (2375.91 MiB of tensor bytes → ceil(…, 2 MiB) = 2376 MiB, exactly the
event you traced), and it is neither an import nor a shared/registry mapping. That kills one of your two readings: the
whole-model BO is not "a different backing the compute path does not read" because it is an import — it is the buffer
the loader placed the weights in, which is what -ngl 99 means.

2. The per-tensor set most plausibly matches HRX's own weight materialisation, which would explain the identical size classes

ggml/src/ggml-hrx/runtime/host-memory.cpp:306 materialize_symmetric_i4_row64() and
runtime/host-memory-ternary.cpp:111 materialize_ternary_q4_0_k128() exist to repack a weight into the layout a kernel
wants (the i4 one is row64 at 144 B/row), keyed on HostWeightSource.materialized_length. That is a per-tensor
allocation whose sizes track the GGUF's tensor classes exactly — which is what both arms show (the -ngl 0 arm's
2.07 GiB of per-tensor copies, and the same classes again at -ngl 99).

If that is the second set, the finding is not "#669 in the other mode" and not "the same weights imported twice": it is
"the loader's model buffer is 2.32 GiB of GTT the compute path never reads once the weights are repacked" — a
different fix (do not place, or free, a buffer whose tensors are all consumed through prepared copies) with a different
risk. Worth knowing before anyone writes the fix, because the two candidate fixes point in opposite directions.

3. The discriminator costs one `-v` run and no trace at all

-v prints one model buffer size line per buft. In your -ngl 0 arm it is the CPU_Mapped line. Run the -ngl 99
arm with -v and read which buffer names 2375.91 MiB:

an HRX buffer of that size → §1, and the open question becomes §2's: is that buffer ever read after the repack
(hit=757 says the per-tensor copies are the live ones)? If nothing reads it, the fix is to not create it.
a CPU_Mapped buffer of that size plus a 2376 MiB device BO → the model buffer is host-side and something copies
the whole region to the device, which is a third mechanism neither hypothesis in the report names.

Then GGML_HRX_* materialisation lines (the same log) say whether the per-tensor set is the repack cache, which is the
one fact that decides §2.

Where the instrument needs to land before this is verifiable by anyone else

tools/bo_trace.py, tools/scripts/mem-botrace.sh and tools/scripts/mem-sizes.sh exist in no branch on the remote
(git ls-remote --heads swept; git log --all --diff-filter=A -- tools/bo_trace.py is empty) — the only copy is
~/amd-oss/hrx-study/tools/bo_trace.py, and the evidence/memory/*.log paths in the report are local too. Same class as
#790 and #713: a measurement nobody else can re-run is a claim, not evidence. The tracker is genuinely useful (one
tracepoint, per-tid attribution, exact pages), so it is worth a small PR of its own — with its selftest — independent of
the fix.

Bucket A stands: the change is ours (ggml-hrx placement/materialisation) and the way to prove it is a GTT delta on
the same three arms, no new ELF, no near-ceiling load.

Follow-up measurements on this report (strixhalo, same pin d8bd0ff6052a, same binary/model/trio, -c 256 -n 8 -no-cnv).
Three findings, one correction, one method note.

1. Correction to the size of the second copy

The body says the per-tensor set is "~2.07 GiB (2124 MiB)". That was my arithmetic error. The observed -ngl 99 classes are
306 MiB x3 + 20 MiB x18 + 14 MiB x90 + 6 MiB x72 = 2970 MiB (2.90 GiB), against 306 x2 + 20 x18 + 14 x90

6 x72 = 2664 MiB at -ngl 0. The conclusion (the model is resident twice) is unchanged; the duplicate is ~840 MiB
larger than reported.

2. What each copy is — they come from different allocators, and only one is charged to the pool budget

component (`-ngl 99`, 4B)	size	allocator	charged to `pool_reserve`?
runtime bring-up	296.9 MiB	libhrx via llama-cli, first 30 ms	no
model buffer (GGUF layout, whole tensor region)	2376.0 MiB = `ceil(2491323904, 2 MiB)`, one BO	`ggml_backend_alloc_ctx_tensors_from_buft` -> `buffer_alloc` (`ggml-hrx.cpp:378-420`)	yes
output-layer buffer	36.0 MiB (`-ngl 99` only)	same path, second ctx	yes
per-weight materialised copies, unbounded	2970 MiB	`allocate_device_buffer` per weight (`host-memory.cpp:1205-1228`)	no
residency arena, bounded (`GGML_HRX_HOST_WEIGHT_LIMIT_MB=N`)	one BO of exactly N (1024.0, 2720.0 observed)	`arena_reserve` -> `allocate_device_buffer(device, target)`, `target = limit_` (`host-memory.cpp:1690-1755`)	no

Time-ordered trace of a 5.1 s -ngl 99 run: bring-up at +0.00-0.03 s, the 2376 MiB model buffer at +1.369 s, the
per-weight copies from +1.375 s to +4.23 s, all from the hal-memory thread. So the duplicate is not the loader copying
twice — the loader allocates the whole-model buffer, and the residency cache fills alongside it as weights are first used.

GGML_HRX_POOL_LIMIT_MB is a hard ceiling inside buffer_alloc (ggml-hrx.cpp:406-412) and its refusal prints the
request size plus the cumulative charge (hrx-1bit-safety.cpp:99-111), so a budget ladder separates charged from
uncharged allocations:

budget	outcome at `-ngl 99`
200	refuses: "a device buffer of 304.3 MiB does not fit (0.0 MiB of 200.0 MiB already in use)"
2200 / 2300 / 2400	refuse at "177.2 MiB ... 2198.8 in use" / "93.8 ... 2282.1 in use" / "12.5 ... 2400.0 in use"
2500 / 2700 / 3300 / 3600 / 4200 / 5200	run, 5903 MiB of GTT each

2198.8 + 177.2 = 2376.0 and 2282.1 + 93.8 = 2375.9, i.e. the refusals happen at the model buffer, and ggml splits that
buffer at buffer_max_size = pool_remaining(budget) (ggml-hrx.cpp:444-448 in
ggml_backend_alloc_ctx_tensors_from_buft_impl, ggml-alloc.c:1186-1196) instead of failing cleanly. Total charged is
therefore ~2412 MiB of a 5903 MiB footprint: GGML_HRX_POOL_LIMIT_MB prices ggml buffers, not the process. The 2970 MiB
of materialised copies and the 296.9 MiB bring-up are invisible to it.

3. Which copy does the graph bind? The materialised one (behavioural proof; the tracepoint does not exist)

GGML_HRX_HOST_WEIGHT_LIMIT_MB=1024 at -ngl 99 refuses exactly like it does at -ngl 0:

E hrx: refusing to run: GGML_HRX_HOST_WEIGHT_LIMIT_MB=1024 MiB cannot hold this model's per-execution working set -
a 304 MiB weight does not fit with 0 MiB resident and 838 MiB parked for the running execution.
Raise it to at least 1144 MiB or unset it (engine #788).

and the run stops at 2.7 s (GTT 4085 MiB, 111 BOs). A residency budget only matters if the residency path is on the critical
path — so under -ngl 99 the command programs read the materialised copies, and the 2376 MiB model buffer holds the
GGUF-layout originals. Source agrees: materialize_host_bindings (command-program-executor.cpp:570-630) replaces a
read-only weight binding with the resident copy when requires_materialization() (host data) or
access.transformed_read (layout repack), and it sources the copy from host data or the bound device buffer
(source.device_buffer = binding.host_data == nullptr ? binding.buffer : nullptr, :602) — so a device-resident weight is
repacked into the arena exactly like a host-resident one.

4. It is not a `-fit off` artefact

Default -fit and -fit off both give 5903 MiB and 364/360 BOs.

5. Method note: BO use is not observable on this kernel

To answer (3) by measurement rather than inference I tried joining amdgpu_bo_create with amdgpu_bo_list_set on the BO
pointer. Across 20 arms and 6,412 BO creations, amdgpu_bo_list_set and amdgpu_cs fire zero times: those events
belong to the GEM amdgpu_cs_ioctl path and HSA/KFD compute submissions never build an amdgpu_bo_list. Anything that wants
to attribute GPU reads per BO on this platform has to come from the runtime side.

The question this raises

If every MUL_MAT weight that needs a repacked layout gets a materialised copy that the programs bind, the loader's whole-model
buffer is a staging input that stays resident for the process lifetime — 2376 MiB for this 2.38 GiB model, and proportional
for larger ones, i.e. -ngl 99 costs ~2.25x the weights. Is that intended (the model buffer as the repack source of record),
or should the materialised copy be able to replace it once every read is served from the arena?

Instruments and raw evidence: tools/bo_use.py (selftest included; enables only amdgpu_bo_create +
amdgpu_bo_list_set + amdgpu_cs, restores tracefs on exit, drops the workload back to $SUDO_USER), driven by
tools/scripts/mem-bouse{,2,3}.sh; logs bo-use-20261102-1738.log, bo-use2-20261102-1744.log,
bo-use3-20261102-1750.log, with per-BO raw dumps and the two trace dumps.

Reproduced, and the mechanism is now settled: the 2376 MiB model buffer is a repack source that is written once, downloaded once, and never read by a kernel

Ran your reproducer at the pin with today's binary (stage-pin788, tools/scripts/mem-botrace.sh, one gpu-hold
ticket, trio on). Same numbers as the report, so this is a confirmation, not a re-derivation:

# bo_trace: llama-cli in 2.6 s, rc 0, own peak drm-total-gtt 5903 MiB
# 364 BOs, 5901.3 MiB of BO bytes
  2491416576 B x   1  =  2376.0 MiB   <- ceil(2491323904, 2 MiB): the whole tensor region
   320864256 B x   3  =   918.0 MiB \
   20971520 B x  18  =   360.0 MiB  |
   14680064 B x  90  =  1260.0 MiB  |  the repacked per-weight copies
    6291456 B x  72  =   432.0 MiB  |
    4194304 B x  24  =    96.0 MiB  |
    2097152 B x  57  =   114.0 MiB /

1. Every GEMM weight is repacked, so no kernel ever reads the GGUF-layout copy

The kernels do not bind weights in GGUF layout: dispatch-mul-mat.cpp:861 binds a q4_K weight as
kQ4KPackedK256Row64Layout, :543 a q6_K weight as kQ6KPackedK256Row64ScaleRowLayout, and so on. The only native
(ggml-native) weight bindings are the F32 norms (dispatch-rms-norm etc.) and the MoE expert gate/up bindings
(dispatch-routed-ffn.cpp:944-945, dispatch-moe-router.cpp:512).

So at -ngl 99 the model's GEMM weights arrive in a device buffer in GGUF layout and are then repacked: acquire()
takes source.device_buffer (command-program-executor.cpp:602), and because the layout is not native it takes the
download_synchronous branch (host-memory.cpp:1893-1901) - device -> host -> CPU repack -> arena upload. The device
copy is only ever a repack source.

For this model the dead share is essentially all of it: 398 tensors, 2375.9 MiB, of which the natively-read F32 norms are
145 tensors totalling 766 KiB (largest 10 KiB). Against a 2376 MiB model BO that is 2375.3 MiB, 99.97 %, written
and then only downloaded from.

2. This is not fixable inside `ggml/src/ggml-hrx` without telling llama.cpp a lie

Where a weight lives is decided at load by select_weight_buft -> weight_buft_supported
(src/llama-model-loader.cpp:1058, :1236), and its only question to the backend is
ggml_backend_dev_supports_op(dev, op) with a zero-size dummy buffer of the candidate buft (:1236-1240). Replying
"no" for a device-resident weight would move the weight to the mmap - and would also move every MUL_MAT of that graph to
the CPU, because the scheduler asks the same predicate about the real buffers at runtime. Distinguishing the two by the
0-byte dummy buffer is an implicit contract of exactly the class this fleet keeps filing (#868); I am not shipping that.

There is a road in llama.cpp that already has the right policy - use_mmap && buft == host_buffer_type(dev) is
redirected to the CPU buffer "avoid using a host buffer when using mmap" (:1277-1284) - it simply does not apply to a
device-local buft, which is what -ngl 99 selects.

3. The three roads, priced

road	change	effect on this model	risk
A weights stay in the mmap for HRX (no device model buffer)	loader-level: prefer the host/CPU buft for weight tensors on HRX, i.e. extend the mmap rule to a device buft the backend will repack	-2376 MiB GTT (-44 %: 5903 -> ~3300), and the repack loses its device->host download (strictly less work)	Ops stay on HRX, but natively-read weights (norms; MoE expert weights) then need staging per execution - so this is only free for dense models, and MoE needs a per-tensor split
B repack in place on first use (no second copy)	device-side repack kernels; the model buffer becomes the resident copy	-2970 MiB instead	new kernels, new correctness surface, weeks
C free the model buffer once every tensor in it is superseded	ggml buffer lifetime surgery (one BO per buft; partial release impossible)	-2376 MiB	a later execution that binds the raw buffer (norms) or a different layout reads freed memory; not a "smallest correct fix"

This needs an owner word, and it is the same question the issue asked: is the loader's model buffer the repack source
of record (then 2.25x weights is by design and the fix is documentation + a guardrail), or should HRX keep weights in the
mmap and pay a repack on first use (then A, and the MoE/dense split decides how much of it is free)?

Recommended on the evidence: A, gated to tensors whose readers are all transformed, with the MoE native bindings left
alone - that is the version whose dead share is 99.97 % and whose extra cost is zero (the repack already round-trips
through the host today). It still needs the loader-side hook, which is why it is not a PR tonight.

I did not open a PR for #856. Nothing merged, no device config touched; the trace log is
~/amd-oss/hrx-study/evidence/memory/bo-trace-20261102-2341.log.

A related but distinct duplication, measured in the merged tree (428937ae) on 2026-11-04 17:34 AST, 4B Q4_K_M,
-dev HRX0 -ngl 0 -c 256 -fit off -n 8, one 4 s gpu-hold ticket:

hrx alloc trace: weight-materialised size=26654720 (25.420 MiB) layout=q6k-i8-k32-row64 type=14 in=9728 out=2560 src=20428800   x 17
hrx alloc trace: weight-materialised size=20428800 (19.482 MiB) layout=ggml-native      type=144 in=0    out=0    src=20428800  x 18
hrx alloc trace: weight-materialised size=319065600 (304.285 MiB) layout=ggml-native      ...  x 1
hrx alloc trace: weight-materialised size=319065600 (304.285 MiB) layout=q6k-packed-k256-row64-scalerow ... x 1

(GGML_HRX_ALLOC_TRACE, a ~30-line diagnostic in my tree only — it prints one line per device allocation the fork makes,
tagged by call site; the arm without the variable is the control and reproduces the same totals, so it is inert.)

What this says, and how it differs from this issue:

The residency cache holds one entry per (weight, layout). ffn_down.weight (Q6_K, 20 428 800 B) is materialised
once in ggml-native (18 tensors) and, for 17 of them, a second time in q6k-i8-k32-row64
(dispatch-mul-mat.cpp:596) at 26 654 720 B = 1.3047x the raw size. Net cost: 432 MiB per 4B run, on top of
the native copies that stay resident. It is created by the first decode, not by the loader: the load-only arm has
none of it, so the decode shape selects a MUL_MAT variant the prefill shape did not.
The "duplicated token_embd" that was long unexplained is the same mechanism: its two BOs are exactly
ggml-native and q6k-packed-k256-row64-scalerow, both 319 065 600 B. Not a duplicate allocation — one tensor
resident in two layouts.
This is not hrx: at -ngl 99 the weights are resident twice in GTT (one whole-model BO plus per-tensor BOs) - 5.9 GiB for a 2.38 GiB model #856's duplication: hrx: at -ngl 99 the weights are resident twice in GTT (one whole-model BO plus per-tensor BOs) - 5.9 GiB for a 2.38 GiB model #856 is the loader's model buffer plus the repacked per-weight copies at
-ngl 99. This one appears at -ngl 0 as well, is caused by layout specialisation across shapes, and would survive
fixing hrx: at -ngl 99 the weights are resident twice in GTT (one whole-model BO plus per-tensor BOs) - 5.9 GiB for a 2.38 GiB model #856.

Operator-relevant consequence: --mem-check's estimate is computed from the GGUF header, so it does not model a second
layout — the real peak can exceed the estimate by ~1.3x the affected weight bytes (432 MiB at 4B, more at larger models
with more Q6_K ffn_down). Worth knowing when sizing a context, since this lane already measured context at
0.915 MiB/token on the same stack.

Whether evicting an unused layout is desirable is a design question (the packed layout presumably buys decode speed);
I am reporting the cost, not asking for a change. Evidence: ~/amd-oss/hrx-study/evidence/memory/alloctrace-detail-20261104-1734.log,
written up as §7.21(c) of ~/amd-oss/hrx-study/FACTS-memory.md.

Follow-up on the "second resident layout" mechanism I posted above (17 x 26 654 720 B, q6k-i8-k32-row64, the packed
ffn_down Q6_K copy): the count is 17 because the eighteenth is the last layer, and the reason is in the model graph,
not in the cache. Measured on this box (Qwen3-4B -dev HRX0 -ngl 0 -c 256 -fit off -n 8, merged tree 428937ae), with a
per-node dispatch trace that names the winning matcher and the weight:

the executed 13-token prompt-prefill plan has 17 ffn_down Q6_K MUL_MATs at token_count 13, all matched by
common.mul_mat.q6_k_i8_prepacked — the only matcher that requests q6k-i8-k32-row64;
blk.35.ffn_down — the last layer — runs in that same plan at token_count 1
(src/models/qwen3.cpp:114-117: after the last layer's attention, cur/inpSA are reduced with
ggml_get_rows(cur, inp_out_ids), so the last layer's FFN is computed for n_outputs tokens only), and
match_q6_k_i8_prepacked_dispatch rejects token_count <= 5 (dispatch-mul-mat.cpp:576-604), so the registry picks
q6_k_token1_final_projection_q8 (priority 305 vs 285) and binds the weight in its native layout;
every 1-token decode plan uses that native route for all 18, so the packed copies are created by the prompt prefill and
by nothing else — a load-only run creates none, a decode creates none, and a prefill of ≤5 or ≥17 tokens cannot reach
that matcher either.

Two consequences for anyone sizing --mem-check from the GGUF header: the extra cost is one packed copy per Q6_K
ffn_down except the last layer (432.1 MiB at 4B; a full 18 would be 457.6 MiB, so the asymmetry is not the lever), and
it is a prefill artifact — an eviction policy trades prefill latency, not decode latency.

Full evidence, including the two instruments that cannot answer this (matcher-name dedup, shape-deduped program dumps):
FACTS-memory §7.21(e) in ~/amd-oss/hrx-study/.

Correction to my own comment above (and a smaller number to plan with): the "one packed copy per Q6_K ffn_down except the
last layer, 432.1 MiB at 4B" is only true for a short prompt. Measured on the same binary/flags, only the prompt moved:

the q6k-i8-k32-row64 request comes from common.mul_mat.q6_k_i8_prepacked, which requires token_count in (5,16]
(dispatch-mul-mat.cpp:576-604); at 1-5 tokens a different matcher wins, and at 17+ tokens every Q6_K ffn_down and
attn_v goes to the generic common.mul_mat.f32_f32_wmma, which requests no layout at all (pack_rows is
token_count <= 5, :1058-1063) — measured at 18, 28, 58 and 268 tokens, zero packed copies;
peak GTT for the same 4B -ngl 0 -c 256 -n 8 run: 3 845.44 MiB at a 13-token prompt vs 3 361.43 MiB at a 21-token
one — 484.0 MiB, of which 432.1 MiB is the repack;
so a server that prefills 512 or 2 000 tokens (any scoreboard pp512/ttft2k arm, and most real traffic) carries no
ffn_down repack, while a 6-16-token prompt creates and keeps it for the life of the process.

Two consequences for this issue: the repack is not a fixed per-model cost to budget, it is a per-prompt one, and an
eviction policy would free it for nearly every real server while costing one repack on the next short prompt. The other
q6k-packed-k256-row64-scalerow copy I mentioned (the 304.3 MiB token_embd one) does appear at 1 token, so the two
second-layout terms do not share the same trigger.

Measurement, tables and the architecture census (the last-layer reduction that gives "except the last layer" is
unconditional in 100 of 150 src/models/*.cpp): FACTS-memory §7.21(e)+(f) in ~/amd-oss/hrx-study/.

Measured negative result, and it names the real duplicate. The loader-placement change was implemented, built and measured — and it does not reduce GTT:

arm (Qwen3-4B-Q4_K_M, `-dev HRX0 -ngl 99 -c 256 -n 16 -fit off`, A B B A, one gpu-hold)	CPU_Mapped model buffer	HRX0 model buffer	peak drm-total-gtt	footprint
A: `GGML_HRX_WEIGHT_HOST_PLACEMENT=0`	304.28 MiB	2375.91 MiB	41 258 MiB	3 697
B: `=1` (weights kept host-side)	2375.90 MiB	305.03 MiB	41 418 MiB	3 857

The 2071 MiB drop in the device model buffer is cancelled by the HRX arena (requires_materialization() is true for any host data, so each reader materialises a device copy) — net +160 MiB. Generated text is byte-identical across all four arms (same normalised stdout sha), so correctness is not the problem.

Why the issue's premise does not hold at this revision: at decode token_count=1 the matchers read the weights in the native layout too, so the device model buffer is the live native copy, not a repack source. The real duplicate is the arena's packed copies, not the model buffer. A correct fix needs in-place device repack (road B) or fewer native-layout readers — both kernel-side; loader placement cannot win here.

Evidence: fork branch 1bit/856-resident-once @ d8c1898d3 (pushed, no PR — a PR would regress the metric), strixhalo ~/wt/tile856-20261106T035932 (A/B logs) and ~/wt/tile856-diag-20261106T035757 (the tied-token_embd conflict bisect with -ot). Unproven: the served/long-prompt pp4096 workload (not measured) and the engine's pinned llama.cpp 4a375bf2d (the test ran on tip 59d7f8284).

Disposition: parked on a roadmap/owner decision — this is now a kernel-work question (in-place device repack), not a loader bug. Unpark: choose road B (in-place repack) or reduce the native-layout readers, with the footprint above as the acceptance metric and the same A B B A + text-identity control.

Sweep (pi-verify) at `918f731a` — the repro is not reproducible from `origin/main`: its instrument is not in the tree

The issue's Reproduction block points at tools/scripts/mem-botrace.sh, tools/scripts/mem-sizes.sh, tools/bo_trace.py and
evidence/memory/bo-trace-20261102-1647.log. None of them is in the repository, and none ever was:

$ ls tools/bo_trace.py tools/scripts/mem-botrace.sh tools/scripts/mem-sizes.sh
   (all absent)
$ git log --all --oneline --name-only | grep -i "bo_trace\|mem-botrace"
   (no output — never committed on any ref)
$ grep -rn "mem-botrace" --include=*.md --include=*.sh .
   (no output)

So the double-residency claim cannot be re-measured from a clean checkout of main: -ngl 99 still allocates a whole-model BO
plus per-tensor BOs on the code path this issue describes, but the census that showed it lives only in a lane worktree that is
gone. Landing tools/bo_trace.py (or pointing the issue at a runner in the tree) is a prerequisite for anyone else to verify a
fix here. Still open.

third_party: llama.cpp d097c4cd2 (engine #856)

third_party: llama.cpp 92ced18f3 (engine #856)

third_party: llama.cpp 360944b0e, rebased on 1bit/hrx-vulkan-patched …

third_party: llama.cpp a3e8d42c9 — HRX imports the weight mapping, so…

Fixed and merged — the before/after, and exactly what is still open

The change: fork 1bit-MONSTER/llama.cpp#226, merged as a3e8d42c9 on 1bit/hrx-vulkan-patched; carried to main by
engine #1303, merged as a729e0fa6. This backend now imports the loader's weight mapping
(device_buffer_from_host_ptr + hrx_allocator_import_buffer) into its own buffer type, so the loader's GGUF-layout
copy of a repacked dense weight is never made. Compute placement is unchanged — the tensor is still this backend's buffer, so
the scheduler keeps the op on the device.

Control for every row below: GGML_HRX_WEIGHT_HOST_PLACEMENT=0, which restores the previous placement.

Before/after, peak of the process's own `drm-total-gtt` (never the box counter)

model / flags	before	after	delta
Qwen3-4B-Q4_K_M, `-ngl 99 -fit off -c 256 -n 32`	3927 MiB	1245 MiB	−2682 MiB
zaya1-8b-Q4_K_M, `-ngl 99`	queued — behind a 60-min `fnpre` hold and an `i679` ticket; will be appended as a comment
Qwen3-4B, `-ngl 99 -ot 'ffn_.*=CPU'` (CPU-placed tensors)	2135 MiB	1005 MiB	−1130 MiB
Qwen3-Coder-30B-A3B, `-ngl 99 -ot exps=CPU` (MoE)	1509 MiB	1509 MiB	0, by design

The original report's arm was this one: a 4B whose weights are 2375.9 MiB of tensors held 5903 MiB at -ngl 99 (2376 MiB
model buffer + ~2970 MiB of repacked copies). On the current tree the same arm is 3927 MiB before and 1245 MiB after, and
the 2376 MiB model buffer is gone from GTT entirely.

RssFile/RssAnon are unchanged across the arms (~2410 / ~670 MiB on the 4B): the pages stay file-backed, shared and
reclaimable, because the mapping is MAP_SHARED|PROT_READ|PROT_WRITE over an O_RDWR file — a private writable mapping
was rejected during review precisely because MAP_POPULATE CoW-copies the whole file into anonymous memory. The reviewer
independently verified the model file is not written (sha256 and mtime unchanged after a run).

Correctness: generated text is token-identical to control at --temp 0 on every dense row.

What is not measured, stated rather than implied

Perplexity before/after does not exist. The lane that offered it (cl-856) never ran it: its build was of an earlier,
retired variant and its tickets were blocked by low box memory. My correctness evidence is token equality, not PPL. If the
maintainers want a PPL row before this is considered closed on evidence, it is a single llama-perplexity pair with
GGML_HRX_WEIGHT_HOST_PLACEMENT 0 vs unset, and I will run it.
Large-model coverage is still missing, and not for lack of trying. cl-856 offered a 27B GTT row and a
perplexity pair; neither ran (its build was of the retired variant, then its tickets were blocked by low box memory). My
own 8B row is queued behind a legitimate 60-minute fnpre hold and an i679 ticket and has not run either, so I am not
going to hold this issue open on a queue position. It will be appended below when it lands, and a 27B-class
reproduction is still worth having.
MoE does not win yet: -ot exps=CPU falls back by design, because the from-ptr region is [first,last] over
interleaved experts and pinning the whole file made the load not finish (>14 min, versus 4 s with the gate). Making MoE
win needs the import restricted to the ranges the device actually uses — a loader-side split, tracked as the follow-up.
The census instruments behind the original numbers live outside this repo (~/amd-oss/hrx-study/tools/, and
bo_trace.py + mem-botrace.sh there); pi-verify's sweep is right that the Reproduction block cannot be run from a clean
checkout. The measurement path used for every row above is tools/scripts/mem-856-arms.sh in that tree, and landing it
here as a tool PR is a reasonable follow-up if wanted.

Closing as fixed: the duplicate is gone for dense models on main, with the A/B switch and the fallback gated so nothing else
changes.

Appended after the close: the 8B large-model row (the one I said was queued)

~/wt/i856-8b-0101, on the merged code (build from a3e8d42c9), zaya1-8b-Q4_K_M (5.6 GB GGUF), -ngl 99 -fit off -c 256 -n 32 --temp 0, two interleaved repeats, peak of the process's own drm-total-gtt:

arm	peak own GTT	`RssFile`	`RssAnon`	gen t/s	generated text
control (`GGML_HRX_WEIGHT_HOST_PLACEMENT=0`)	6215 MiB	5335 MiB	436 MiB	91.8 / 96.2	—
this fix	497 MiB	5338 MiB	242-243 MiB	95.3 / 25.4	identical

−5718 MiB of GTT on an 8B — the whole-model buffer for a 5.6 GB model is gone, and it is gone from GTT only: RssFile
is unchanged (5335 → 5338 MiB) and RssAnon does not grow, i.e. the weights are still the file's own pages, shared and
reclaimable. Generated text is token-identical to control.

On the rate column: the second fix repeat is a contention artefact, not a policy cost — its prompt rate was 13.0 t/s against
45-49 for the other three runs, on a box where another lane was holding the GPU token. The clean pair has the fix slightly
ahead (95.3 vs 91.8). As with the 4B, I would not quote a rate effect from this either way; the memory number is the one
that is bit-reproducible across repeats.

That makes the dense before/after complete for two sizes:

model	before	after	delta
Qwen3-4B-Q4_K_M	3927 MiB	1245 MiB	−2682 MiB
zaya1-8b-Q4_K_M	6215 MiB	497 MiB	−5718 MiB

Still not done, unchanged from the close comment: a perplexity before/after, and the loader-side range split that would let
MoE win as well.

The pin that carries #226 will not load Qwen3.8-Flash-Next (76.3 GiB) — 4 attempts, 2 builds, 0 successes

Cross-posted from the #679 lane: this blocks every Flash-Next row on strixhalo (decode, prefill, and #1251's acceptance).

Symptom (--no-repack, -c 19456 -ngl 99 --device HRX0, shipped configuration, no dispatch disables):

I load_tensors: loading model tensors, this can take a while... (load_mode = mmap)
I add: tensor per_layer_token_embd.weight (size = 27465 MiB) lazy read enabled
<72 minutes of nothing>

state S, wchan futex_do_wait, 40 threads, GTT flat at 5-6 GiB, RSS ~32 GiB flat, no error line, no gfxhub fault, no
reset, /health never answers.

Two builds at a3e8d42c9 (version 10774 in both): my worktree with the preinstalled libhrx, and pi-hrx-compat's
~/wt/i1251-build with from-source hrx-system. Same stall.

Before/after, from grep -c "model loaded" over every server.log in this lane's artifacts:

pin	loads that reached `model loaded`
`469c69f5d` (9 arms, identical flags)	9 / 9
`a3e8d42c9` (4 arms, 2 builds)	0 / 4

765fa1c4c..a3e8d42c9 contains #226 (weight-mapping import), #228, #229, #214 and #227; #226 is the only load-path change.
Artifacts: ~/lb/aval/i679-shipped-20261107T052856Z and …T083554Z (server.log + 1 Hz GTT/RSS sample.txt).

Question that decides bug vs missing refusal: does the import path need the repack that --no-repack skips, and if the
mapping is not there, does it wait instead of erroring? A loud refusal would have cost 15 seconds instead of two 90-minute
claims.

Sweep (pi-verify) at `0f45221e` — fixed; verifying the close (with the follow-on it caused)

Fix in main: engine a729e0fa (PR #1303) moves third_party/llama.cpp to the fork commit whose own subject is the fix — the
pin is now a3e8d42c "hrx: import the weight mapping instead of copying it to the device (engine #856)" (fork #226) — so the
loader's second, never-read GGUF-layout copy is no longer made.

Measured in the PR (Qwen3-4B-Q4_K_M, -ngl 99 -fit off -c 256 -n 32 --temp 0, peak of the process's own drm-total-gtt):

arm	peak own GTT	generated text
control `GGML_HRX_WEIGHT_HOST_PLACEMENT=0`	3927 MiB	—
this bump	1245 MiB (−2682)	token-identical at `--temp 0`
`-ot 'ffn_.*=CPU'`, control → bump	2135 → 1005 MiB	identical

with RssFile held at ~2410 MiB and RssAnon not growing with the model (the mapping is MAP_SHARED over an O_RDWR file, so the
pages stay file-backed and reclaimable), and the MoE arm (-ot exps=CPU, 18.6 GB) falling back to the control placement rather
than regressing: 1509 MiB, 4 s load, vs the import path pinning 18.6 GB before the gate.

Verified at the current pin (0f45221e → third_party/llama.cpp = a3e8d42c): device_buffer_from_host_ptr and the
GGML_HRX_WEIGHT_HOST_PLACEMENT A/B switch are both in the pinned source (ggml/src/ggml-hrx/ggml-hrx.cpp:160-163, :908-910,
backend registration at :1488).

Two things this close is not claiming:

It cost something, and that is filed: hrx: token-embedding GET_ROWS leaves the device at -ngl 99 after the host-pointer weight import (#226/#228), 1 node + 1 split per graph #1316 reports that after Bump XDNA: amd/xdna-driver 342187206b1f #226/laya scorer: vectorizable linear and threaded head attention (CPU scorer 352 -> 208 ms a decision) #228 the token-embedding GET_ROWS is no longer placed
on HRX at -ngl 99 (1 node + 1 graph split per graph, every model; GGML_HRX_STRICT=1 aborts with "embd" is GET_ROWS f32 ne=[2560,2,1,1] … 1 of 3024 nodes off HRX), and names this PR's host-pointer region as a suspect. That is a placement
regression introduced by the same move — it belongs to hrx: token-embedding GET_ROWS leaves the device at -ngl 99 after the host-pointer weight import (#226/#228), 1 node + 1 split per graph #1316, not here, but nobody should read this close as "the bump had no
downside".
The reproducer this issue names still is not in the tree (tools/bo_trace.py, tools/scripts/mem-botrace.sh,
evidence/memory/ — never committed on any ref), so re-measuring the original defect needs a new census; the numbers above are
the PR's harness, not an in-tree tool.

Confirmed with your own control: #226's weight-mapping import is what stops this model loading — 72-minute stall with it, 75-second load without it

Follow-up to my report above. You documented the A/B control in the commit message: "GGML_HRX_WEIGHT_HOST_PLACEMENT=0 turns
the capability off and reproduces the previous placement". I ran exactly that, nothing else changed:

arm	result
pin `a3e8d42c9`, shipped config	`per_layer_token_embd.weight` then 72 min in `futex_do_wait`, GTT flat 5-6 GiB, RSS 32 GiB flat, never `/health` — on two builds (mine and `~/wt/i1251-build`)
same pin, same flags (`-c 19456 --no-repack -ngl 99 --device HRX0`), `+GGML_HRX_WEIGHT_HOST_PLACEMENT=0`	loads in 75 s (08:28:31 headroom ok → 08:29:46 `up`), then completes the full 18.8k row: decode 11.28 tok/s, prefill 226.3, 0 NaN, 0 asserts

That is the capability, not the model, not --no-repack, not the box (the no-import arm ran at MemAvailable 86 GiB,
peak_gtt 70.4 GiB). Artifacts: stall ~/lb/aval/i679-shipped-20261107T{052856Z,083554Z}, control
~/lb/aval/i679-noimport-20261107T122759Z, flags in ~/lb/i679-window13.sh.

Why I think the import is the culprit in this model's case specifically (76.3 GiB, -ngl 99, three GGUF shards): your three
requirements are page-aligned address, writable region, MAP_SHARED — so the loader now opens and maps the weight files
read-write, and HSA locks them. On a 122 GiB box that means locking ~76 GiB of page cache while the process also holds a
~28-32 GiB RSS; the stall sits exactly where the first large tensor would be locked, and it is a wait, not an error, so it
surfaced as a 72-minute futex with no diagnostic. Suggestions, in the order that costs least: (1) if the lock cannot be
satisfied, refuse loudly rather than wait — a 15-second error would have saved two 90-minute claims; (2) log per-tensor import
progress at -lv 4 so the next stall names its tensor; (3) gate the import by file size or available memory so a 76 GiB model
falls back to the copy path by default.

I am not asking you to revert: with the control the model loads and the row is taken. But until this is fixed, the shipped
default cannot load Qwen3.8-Flash-Next at all, and every row quoted on #679 by any lane should state whether it ran with the
import on.

Sweep (pi-verify) at `4480a6df` — reopening: the fix is in the pin but off by default, so the default configuration still double-resides

I closed this one this morning on the strength of the pin carrying fork #226 and its 3927 → 1245 MiB measurement. That was right
about the code and wrong about the default. The pin then moved twice, and the second move is decisive:

67d8c42ec  hrx: do not pack a device-resident weight that another node gathers natively (engine #1316)
092eb9dbc  hrx: make the weight-mapping import opt-in until it stops hanging large loads (engine #1339)
a3e8d42c9  hrx: import the weight mapping instead of copying it to the device (engine #856)

092eb9dbc says it plainly: "GGML_HRX_WEIGHT_HOST_PLACEMENT=1 now opts in; unset or 0 keeps the device copy and the read-only
mapping, the placement every control run loaded with." With no environment variable — the shipped configuration — the loader's copy
is made again, so the double residency this issue measured is back.

Why it was turned off, and that was the right call: #1339 measured that on a dense 16.4 GiB file and on the 76.3 GiB Flash-Next
the import never finishes loading (41 threads in hrtimer_nanosleep, own GTT flat at 298 MiB, killed at 14 min; the 76 GiB arm sat
72 min in futex_do_wait, and loads in 75 s with the import off).

There is now a newer answer than "flip the default", and it does not close this issue. Fork #234 / PR #1351 (open) reverts
#226/#231 outright: the loader read()s the weights into their buffers and drops each weight's page cache with
posix_fadvise(DONTNEED), and GGML_HRX_WEIGHT_HOST_PLACEMENT stops doing anything. It fixes #1339 (16.4 GiB: 14.5 s load / 16.5 GiB
page cache → 18.8 s / 0 page cache, peak RSS 16352 → 2859 MiB, text and PPL identical) — and it says explicitly:

Not in scope: HRX's repacked weight copy, the second GTT copy, is unchanged.

So that PR removes the mapping, not the second copy: the repacked resident set HRX materialises on first use is still there, and its
own GTT row is 20497 MiB on the 27B in every arm. This issue is about that copy, so it stays open until something removes it — the
HRX-side resident set, not the loader's mapping.

My earlier comment's evidence for the opt-in path still stands (the …=0 A/B control, and RssFile/RssAnon showing the pages stay
file-backed when the import is on). What was wrong was the prediction that fixing the stall would let the default be inverted: #1339's
fix deletes the switch instead.

docs: the Zyphra page named the wrong cause for the CPU-side input GE…

Where the second copy is now (cl-856, at the #1351 pin ed43da0b4)

Since #1351 (Fixes #1339), the loader reads the weights into one HRX model buffer and keeps no page-cache copy. I split the process's GTT using -v and GGML_HRX_HOST_WEIGHT_TRACE=1, with a 5 Hz GTT trace (one gpu-hold ticket, -ngl 99 -c 256 -n 16):

	Qwen3-4B Q4_K_M	Qwen3.8-27B UD-Q4_K_XL
process GTT, peak	3925 MiB	20497 MiB
HRX0 model buffer (loader)	2376 MiB	15718 MiB
HRX's second copy: weights re-laid-out for decode	796 MiB (38 weights)	2631 MiB (66 weights)
KV + recurrent state + compute	79 MiB	237 MiB
not attributed by these logs	~675 MiB	~1910 MiB

Where the double residency is: only the re-laid-out weights are held twice, which is 34% of the weight bytes on the 4B and 17% on the 27B. The whole model is not.
Both copies are used: HRX builds the second copy when a weight is first used. Decode reads it, while prefill above a token threshold still reads the weight as stored in the model buffer. When GGML_HRX_HOST_WEIGHT_LIMIT_MB is set, the stored bytes are also what an evicted weight is rebuilt from. So the duplicate is not dead memory that the loader can drop.
Getting to one copy is kernel work: either prefill reads the decode layout, or decode reads the stored layout.
The ~0.7 / ~1.9 GiB not attributed is not weights by these numbers. I have not traced it yet.

docs: the Zyphra page named the wrong cause for the CPU-side input GE…

docs: stop citing the deleted import knob as the Zyphra page's eviden…

Data point from an unrelated lane (kernel/driver), because it lands on this issue's blast radius rather than on mine:

While measuring a dispatch-gate change with test-backend-ops -o MUL_MAT -b HRX0 on 2026-11-08 23:58 AST, the box had
90.6 GiB of 120 GiB GTT in use (other lanes' models). The HRX0 guardrail printed, correctly:

ggml_hrx: HRX0 guardrail: GTT total 120.0 GiB, in use 90.6 GiB, ceiling 104.0 GiB (reserve 16.0 GiB),
          pool budget 13.4 GiB - a launch that needs more is refused here rather than faulting the device (engine #680)

and then, at case ~253 of the suite, the process died with SIGSEGV instead of that refusal:

  MUL_MAT(type_a=q4_0,type_b=f32,m=2880,n=32,k=2880,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1):   <- no verdict
rc=139

So on a box at ~90 GiB the "refuse, do not fault" path is not what a large case actually gets — it segfaults. That matters for
this issue because duplicated GTT residency is exactly what pushes the box into that regime: at 40-60 GiB of resident
weights, a suite or a second model can cross into it and take the process down rather than getting a clean error.

Not my repo and not a claim about the cause: it is one observation, from a stock main binary (no local kernel changes), with
the exact case and the guardrail line above. If it is useful, the reproduction is just that suite on a loaded box; if it is
already known, ignore it.

Navigation Menu

Repository navigation

hrx: at -ngl 99 the weights are resident twice in GTT (one whole-model BO plus per-tensor BOs) - 5.9 GiB for a 2.38 GiB model #856

What was measured

Interpretation (flagged, not claimed)

Reproduction

Scope / caveats

1. The 2376 MiB BO is the loader's model buffer, not an import or a registry mapping

2. The per-tensor set most plausibly matches HRX's own weight materialisation, which would explain the identical size classes

3. The discriminator costs one `-v` run and no trace at all

Where the instrument needs to land before this is verifiable by anyone else

1. Correction to the size of the second copy

2. What each copy is — they come from different allocators, and only one is charged to the pool budget

3. Which copy does the graph bind? The materialised one (behavioural proof; the tracepoint does not exist)

4. It is not a `-fit off` artefact

5. Method note: BO use is not observable on this kernel

The question this raises

Reproduced, and the mechanism is now settled: the 2376 MiB model buffer is a repack source that is written once, downloaded once, and never read by a kernel

1. Every GEMM weight is repacked, so no kernel ever reads the GGUF-layout copy

2. This is not fixable inside `ggml/src/ggml-hrx` without telling llama.cpp a lie

3. The three roads, priced

Sweep (pi-verify) at `918f731a` — the repro is not reproducible from `origin/main`: its instrument is not in the tree

Fixed and merged — the before/after, and exactly what is still open

Before/after, peak of the process's own `drm-total-gtt` (never the box counter)

What is not measured, stated rather than implied

Appended after the close: the 8B large-model row (the one I said was queued)

The pin that carries #226 will not load Qwen3.8-Flash-Next (76.3 GiB) — 4 attempts, 2 builds, 0 successes

Sweep (pi-verify) at `0f45221e` — fixed; verifying the close (with the follow-on it caused)

Confirmed with your own control: #226's weight-mapping import is what stops this model loading — 72-minute stall with it, 75-second load without it

Sweep (pi-verify) at `4480a6df` — reopening: the fix is in the pin but off by default, so the default configuration still double-resides

Where the second copy is now (cl-856, at the #1351 pin ed43da0b4)

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

hrx: at -ngl 99 the weights are resident twice in GTT (one whole-model BO plus per-tensor BOs) - 5.9 GiB for a 2.38 GiB model #856

Description

What was measured

Interpretation (flagged, not claimed)

Reproduction

Scope / caveats

Activity

bong-water-water-bong commented on Nov 2, 2026

1. The 2376 MiB BO is the loader's model buffer, not an import or a registry mapping

2. The per-tensor set most plausibly matches HRX's own weight materialisation, which would explain the identical size classes

3. The discriminator costs one -v run and no trace at all

Where the instrument needs to land before this is verifiable by anyone else

bong-water-water-bong commented on Nov 2, 2026

1. Correction to the size of the second copy

2. What each copy is — they come from different allocators, and only one is charged to the pool budget

3. Which copy does the graph bind? The materialised one (behavioural proof; the tracepoint does not exist)

4. It is not a -fit off artefact

5. Method note: BO use is not observable on this kernel

The question this raises

bong-water-water-bong commented on Nov 3, 2026

Reproduced, and the mechanism is now settled: the 2376 MiB model buffer is a repack source that is written once, downloaded once, and never read by a kernel

1. Every GEMM weight is repacked, so no kernel ever reads the GGUF-layout copy

2. This is not fixable inside ggml/src/ggml-hrx without telling llama.cpp a lie

3. The three roads, priced

bong-water-water-bong commented on Nov 4, 2026

bong-water-water-bong commented on Nov 5, 2026

bong-water-water-bong commented on Nov 5, 2026

bong-water-water-bong commented on Nov 6, 2026

bong-water-water-bong commented on Nov 6, 2026

Sweep (pi-verify) at 918f731a — the repro is not reproducible from origin/main: its instrument is not in the tree

bong-water-water-bong commented on Nov 7, 2026

Fixed and merged — the before/after, and exactly what is still open

Before/after, peak of the process's own drm-total-gtt (never the box counter)

What is not measured, stated rather than implied

bong-water-water-bong commented on Nov 7, 2026

Appended after the close: the 8B large-model row (the one I said was queued)

bong-water-water-bong commented on Nov 7, 2026

The pin that carries #226 will not load Qwen3.8-Flash-Next (76.3 GiB) — 4 attempts, 2 builds, 0 successes

bong-water-water-bong commented on Nov 7, 2026

Sweep (pi-verify) at 0f45221e — fixed; verifying the close (with the follow-on it caused)

bong-water-water-bong commented on Nov 7, 2026

Confirmed with your own control: #226's weight-mapping import is what stops this model loading — 72-minute stall with it, 75-second load without it

bong-water-water-bong commented on Nov 7, 2026

Sweep (pi-verify) at 4480a6df — reopening: the fix is in the pin but off by default, so the default configuration still double-resides

bong-water-water-bong commented on Nov 7, 2026

Where the second copy is now (cl-856, at the #1351 pin ed43da0b4)

bong-water-water-bong commented on Nov 8, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions

3. The discriminator costs one `-v` run and no trace at all

4. It is not a `-fit off` artefact

2. This is not fixable inside `ggml/src/ggml-hrx` without telling llama.cpp a lie

Sweep (pi-verify) at `918f731a` — the repro is not reproducible from `origin/main`: its instrument is not in the tree

Before/after, peak of the process's own `drm-total-gtt` (never the box counter)

Sweep (pi-verify) at `0f45221e` — fixed; verifying the close (with the follow-on it caused)

Sweep (pi-verify) at `4480a6df` — reopening: the fix is in the pin but off by default, so the default configuration still double-resides