Speculative decoding (dflash draft) gives zero speedup at 262k context, +76-133% at 32k #367

Setup

• Build: PrismML llama.cpp b10777 (commit e3de71522), CUDA 13.3, Windows 11
• GPU: RTX 5080 16 GB, driver 616.92
• Target: Ternary-Bonsai-2-27B-PQ2_0.gguf (27B ternary, ~2.13 bpw)
• Draft: Ternary-Bonsai-2-27B-dspark-dflash-Q4_0.gguf (DSpark v1 draft, converted with --drop-shared-tensors, quantized Q4_0, ~592 MiB)
• Server flags: -ngl 99 -fa on --spec-type draft-dspark --spec-draft-n-max 4 -ngld 999 --cache-type-k q4_0 --cache-type-v q4_0 -np 1 --jinja
• Workload: code generation prompts (~1.2–1.4k tokens in, 128 out), temperature 0, single slot

Results

(Math/chat workloads at 32k showed up to +133%; at 262k the gain is gone across the board.)

Observation

Draft acceptance stays high at 262k, so the draft itself works fine — but the per-step cost of the draft appears to scale with context size, wiping out the speculative gain entirely at large ctx. The target model alone is unaffected (91.7 vs 91.8 t/s).

Is this expected behavior for the dflash draft path (per-step draft cost scaling with ctx), or a performance bug worth investigating? Happy to provide more measurements (e.g. intermediate ctx sizes) if useful.