fix(ple): strict shard schema/geometry, nofollow + bounded discovery, mutation evidence, and pinned-bank admission/rollback

[Avicennasis]

Stacked on nothing: four commits directly on main @ 0d652e7 (git merge-tree --write-tree origin/main <head> is clean). 7 files, +979/-105; no C++ touched.

What this is

The Qwen3.8-Flash-Next PLE n-gram table (128 shard_<i> F8_E4M3 tensors over 33 checkpoint files, 320,001,536 rows / 51,200,245,760 bytes) is read by two backends that share nothing: source_from_safetensors (--ple-backend disk, #311/#548) and load_ple_table (--ple-backend pinned). Each parsed the headers on its own and accepted whatever covered the hash range; neither tied a payload read to the file it had parsed; the pinned backend allocated 47.7 GiB without looking at the host and leaked the bank on any failure after that. This PR gives both backends one preflight and closes those gaps, one concern per commit, each with a failing test first.

1. Aggregate schema + exact geometry (723a5ae). ple_table_layout() parses every PLE-bearing shard header and refuses, naming the tensor and file with expected vs actual: a part that is not F8_E4M3 or not a positive 2-D shape, a shape different from the first part's, data_offsets that are not rows x cols x 1 byte or run past the payload, a weight_scale that is not exactly one finite positive BF16 scalar, tensors from two layers, duplicate or non-contiguous indices. check_ple_geometry() then requires the table to be exactly what the config addresses: split_ngram_parts parts, ngram_head_dim wide, on a configured PLE layer, and a total row count equal (not merely covering) to expected_ple_rows() — the per-head n-gram primes from the same derive_ngram_hash_constants the dummy-weight path uses, summed and padded to make_ngram_vocab_size_divisible_by. The model passes its args to resolve_row_source, so the disk backend is gated the same way as pinned. The scale is read as its two raw bytes instead of through safe_open, so a malformed header no longer surfaces as a SafetensorError.
2. Discovery hardening (6324c51). Shard paths are canonicalised once (realpath(strict=True), because the HF cache is symlinks into blobs/; a dangling link is refused by name) and every later open is O_RDONLY|O_NOFOLLOW|O_CLOEXEC|O_NONBLOCK on that target, accepting only a regular file — a symlink that has since appeared at the canonical path is refused as "is a symlink; refusing to follow it". The header length is checked against a 64 MiB budget and the file size before a byte of it is read, with both numbers in the refusal; the header must decode as a UTF-8 JSON object (a length one past the JSON used to escape as an uncaught UnicodeDecodeError, u64-max as OverflowError, a six-byte file as struct.error).
3. Mutation evidence (45058cf). Preflight records, per shard file, what it saw through the nofollow open (PleShardIdentity: device, inode, size, mtime, ctime, header base, SHA-256 of the header bytes). revalidate_ple_shard() re-opens the canonical path nofollow and refuses with the field that moved ("changed after preflight: st_ctime_ns A -> B") or, for a same-inode edit whose timestamps were put back, with the digest ("header changed after preflight: sha256 A -> B"). The pinned loader runs it right before each file's first read and re-stats every file after the fill ("changed while reading"); PleRowSource carries the identities and DiskRowTable runs it before PleStore opens its descriptors. Best-effort concurrent-change evidence, not authentication.
4. Pinned admission + rollback (1e18e28). Before the HostBank exists, the table must fit under 88% of effective_memory_available() (the tighter of MemAvailable and cgroup v2 memory.max - memory.current; None admits best-effort like the rest of the loader), and the refusal carries the numbers. HostBank.discard() unmaps an unpublished bank and forgets it; the loader calls it from a BaseException handler around fill and pin, releasing its memoryview first and never letting the cleanup mask the failure. disk allocates ~21 MiB of fixed staging and needs neither.

Not in this PR — already on main via #311 / #548

Discovery through the index and the consolidated-shard filter, bounded fd-pinned range reads (the C++ row store validates every extent against fstat at construction, refuses out-of-range row ids, keeps one O_DIRECT fd per file, io_uring), admission-by-construction and no rollback need for the default disk backend. The C++ store is untouched.

Evidence on the real table

Machine: worklappy — Intel Core Ultra 9 185H, 62 GiB RAM, NVIDIA RTX 3000 Ada Laptop GPU (8188 MiB, driver 580.178.04, CUDA 13.0), torch 2.11.0+cu130, Qwen/Qwen3.8-Flash-Next-FP8 @ 236dfdf2 (33 PLE shards, sha256 verified against the Hub earlier the same day). An independent oracle (os.pread on the shards + numpy re-derivation of the hash from the checkpoint's own constants) is the reference throughout.

• Layout: 33 files, 128 x 2,500,012 x 160, layer 1, 320,001,536 rows == expected_ple_rows(config) (0.096 s); check_ple_geometry passes; forcing split_ngram_parts=129 -> "PLE table needs shards 0..128, found 128"; ple_layer_ids=(5,) -> "PLE table tensors are for layer 1, config ple_layer_ids is [5]".
• Disk backend through the new checks: fills for T=1/7/64/256/1024 staged bytes bitwise == oracle, GPU max|diff| 0.0, 0.33 ms/token decode; peak max_memory_allocated 32.0 MiB, nvidia-smi <= 208 MiB, RSS 0.872 GiB while the 47.7 GiB table is served.
• Corruption (hardlink scratch, one copied shard, download untouched): dtype E4M3->E5M2 -> "PLE shard ...shard_0.weight in model-00005-of-00131.safetensors has dtype F8_E5M2, expected F8_E4M3"; shape 2500012->2500013 -> "spans 400001920 bytes; shape [2500013, 160] x 1 byte needs 400002080"; header length +1 -> "safetensors header is not UTF-8 JSON: 'utf-8' codec can't decode byte 0x89 in position 143816"; header length 64 MiB+1 / u64-max / file-size -> "declares a 67108865-byte safetensors header; the budget is 67108864 bytes" (and 18446744073709551615, 1717267690), unread.
• Symlink policy: model-00005 as a link into blobs/ resolves at discovery (HF-cache layout loads, total_rows 320,001,536); opening the link directly, or a link swapped in at the canonical blob path after preflight, -> "is a symlink; refusing to follow it".
• Mutation after preflight on model-00035 (same inode, same size, mtime restored): stat gate -> "changed after preflight: st_ctime_ns 1790637455308811250 -> 1790637455393811633"; with ctime forged back, the digest gate -> "header changed after preflight: sha256 ab0db134... -> 9805e6f0..."; the same edit against a built PleRowSource makes DiskRowTable(...) refuse before the store opens a descriptor; a one-byte append -> "st_size 1600008336 -> 1600008337".
• Pinned admission: real effective_memory_available() 61,184,512,000 B (56.98 GiB), ceiling 53,842,370,560 -> admitted; forced 40 GiB -> "PLE table needs 51200245760 bytes of host memory for the pinned bank; effective memory available is 42949672960 bytes, 37795712204 after the 12% headroom"; 54 GiB -> refused (ceiling 51,024,211,476); 55 GiB -> admitted and stopped at HostBank((320001536, 160)). No bank existed before any of the refusals (RSS delta <= 1 MiB).
• Induced rollback (copied model-00021, holds part 64, utimed once 6.17 GiB had been read): refused after 14.4 s with "changed after preflight: st_mtime_ns 1790637463669848903 -> 1790637467496866138"; RSS 0.655 -> 23.168 -> 0.66 GiB, live mmaps 0 -> 0, torch.cuda.memory_allocated 0 -> 0, Mlocked 32768 -> 32768, no live CPU tensor > 256 MiB. Nothing published.
• Full pinned load: 30.1 s (1.585 GiB/s), bank 51,200,245,760 B PINNED; UVA gathers for T=1/7/64/256/1024 bitwise == oracle, GPU max|diff| 0.0, 0.066 ms/token; peak max_memory_allocated 6.4 MiB, nvidia-smi <= 286 MiB (273 samples at 500 ms, desktop included), RSS 48.503 GiB. tests/models/qwen4_exp/test_ple.py::test_pinned_uva_real_table with FREETOKEN_QWEN4EXP_MODEL set: 1 passed in 33.11 s.
• Suites on the box (test_ple_integrity, test_ple, test_ple_disk, test_weight, tests/kernels/test_row_store): 102 passed, 4 skipped, 1 failed, normal and -O — the failure is test_ple.py::test_track_snapshot_equals_a_prefill_stopped_at_the_boundary (torch.equal on tensors identical to four decimals), which fails identically on unmodified main on this host and is unrelated to this change.

Notes for review

• load_ple_table now needs the geometry fields on its args (ngram_size, heads_per_ngram, ngram_vocab_size_base, make_ngram_vocab_size_divisible_by, ple_layer_ids); the model already passes Qwen4ExpArgs, the two in-repo tests that passed a bare SimpleNamespace were updated.
• HostBank.discard() cannot unmap a registered mmap (no cudaHostUnregister binding); the loader only discards before/at a failed pin(), where the mapping is unregistered.
• The 64 MiB per-file header budget is a choice (real shard headers are ~150 KB; safetensors' own cap is 100 MB).