Repository navigation
hrx: attach a separate MTP draft head, and let the sampler support probe skip metadata views (engine #681) - #219
Merged
bong-water-water-bong merged 2 commits intoNov 6, 2026
Conversation
…ph skip metadata views (engine ggml-org#681) A qwen4exp MTP draft head has no tok_embd/output and needs its target context; the memory-fitting probe has none, so the context constructor threw 'requires ctx_other to be set' before the real draft context (which sets ctx_other in common_speculative_init) was ever built. Warn instead of throw: fitting measures, the real draft context still gets ctx_other. The sampler backend support probe rejected the top-k candidate graph on HRX0 because it contains RESHAPE, a metadata-only view the scheduler resolves; skip view ops in the probe. Assisted-by: DeepSeek Harness
…e fitting probe (engine ggml-org#681) Assisted-by: DeepSeek Harness
This was referenced Nov 6, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
engine ggml-org#681: a separate MTP draft head is rejected for qwen4exp on the HRX route. Two minimal fixes, plus what I could measure on the box today.
Fixes
src/llama-context.cpp: a qwen4exp/eagle3/dflash draft head has notok_embd/outputand needs its target context. The memory-fitting probe has no target, so the constructor threw "requires ctx_other to be set" before the real draft context (which setsctx_otherincommon_speculative_init) was ever built. Warn instead of throw; real draft contexts still getctx_other.src/llama-sampler.cpp: the backend support probe rejected the top-k sampler candidate graph on HRX0 because it contains RESHAPE. Reshape/view/permute/transpose are metadata-only views resolved by the scheduler, so skip them in the probe instead of failing the sampler graph.What this measured (strixhalo, fork tip + these commits)
Target:
Qwen3.8-Flash-Next-UD-IQ3_XXS(76.3 GiB),-dev HRX0 -ngl 99 -c 32768 --parallel 1 -fa on --cache-reuse 256, house draft headmtp-Qwen3.8-Flash-Next-shared-Q4_K_M.gguf,--spec-type draft-mtp --spec-draft-n-max 2, greedy 512 tokens, 1 rep per cell (not the issue's median of 3).Head-run clocks sampled per request: short 655-2857 MHz, medium 1064-2803 MHz (pp_dpm_sclk). The no-head arm's clock samples were not persisted before its ticket expired, so the rows are not clock-matched; the +48-53 % is far above the run-to-run spread seen in these cells.
Why this is not the issue's exact row
-ctv q4_0(the issue's protocol) is rejected by HRX during prefill:unsupported HRX node ... SET_ROWS ... q4_0cache. The table above uses-ctk f16 -ctv f16.ggml-cpu/ops.cpp:5136: GGML_ASSERT(i1 >= 0 && i1 < ne1) failedinsideggml_compute_forward_set_rows_impl(GGML_OP_SET_ROWS). It reproduces without-mdand on the unpatched fork tip 59d7f82 at the same prompt, so it is pre-existing and unrelated to this change (and unrelated to-md). It needs the SET_ROWS/CPU-fallback gap fixed before the long-context row can be measured.TOP_Kfor the backend sampler (warning only, falls back).Unproven
-ctv q4_0row: all blocked as above.Assisted-by: DeepSeek Harness