You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
hyperloom custom workload: a DFlash arm names no drafter, so the 50.87 best of record cannot say which of this box's two drafter quants produced it #977
Class: the artifact does not describe the measurement (same as #892, #930, #932, #947, #954 — all fixed).
HL_DFLASH is not a configuration on its own: serve_lever_args() hands --dflash the file HL_DFLASH_GGUF names,
defaulting to the relative path dflash.gguf (so the run's working directory decides it too), and HL_MTP_HEAD
substitutes an external MTP head for the model's own. levers() — the only place a result echoes its configuration —
lists HL_DFLASH and HL_MTP_MAX but neither HL_DFLASH_GGUF nor HL_MTP_HEAD.
This model has two drafter quants on this box (~/models/qwen38/DFlash2/{q8_0,q4k}.gguf, 2,056,414,560 B and
1,207,231,840 B), so the gap is not academic — and the tuner itself treats the pair as one configuration: it drops a
drafter-quant variant because overwriting HL_DFLASH_GGUF "would make its number incomparable to the baseline"
(hl-workspace-26030/... orchestrator log, 2026-10-30 18:14:39).
The arm of record, 50.87 (HL_DFLASH=1 HL_MTP_MAX=6), is exactly this shape. Its artifact:
So which drafter produced 50.87 is recoverable only from serve.log line 29 of that session
(loading draft model '/home/bcloud/models/qwen38/DFlash2/Qwen3.8-27B-DFlash2-q8_0.gguf') — or from the session's config.yaml, which is Hyperloom's file, not the measurement's. Until #892 a second arm in the same RESULT_DIR
overwrote that serve.log; 50 HL_DFLASH-carrying artifacts (25 distinct runs x benchmark_report.json + inferencex_result.json) exist in that workspace and none of them names its drafter.
Same class, smaller: the pp512/tg128 medians are over -r HL_REPS repetitions (default 3) and the result never says how
many, so two numbers a couple of percent apart cannot be told apart from a rep-count change.
Wanted: every result names the drafting files it really used (resolved path + hash, and whether the path came from the
environment or the default) and the repetition count behind its medians.
Activity