This repository was archived by the owner on Oct 8, 2026. It is now read-only.
Repository navigation
Merge upstream llama.cpp (2026-10-07) - #34
Merged
Merged
Conversation
graph_inputs was populated while splitting the graph, so it only contained the inputs that are used as srcs of some node. With pipeline parallelism (n_copies > 1), each graph input contributes n_copies leafs to graph_copy, so switching between batches that consume different inputs (e.g. token batches that do not use the embeddings input vs image batches that do) changed the graph composition. This shifted the input copies in graph_copy, making the backend ids comparison report spurious changes and forcing the scheduler to re-reserve. The re-reserve could then record smaller input sizes (e.g. out_ids with n_outputs = 0) and abort later on a graph with an unchanged size via GGML_SCHED_DEBUG_REALLOC. Collect the inputs after the split instead, from all input leafs of the graph, so that the graph composition depends only on which inputs exist, not on which inputs are used. Assisted-by: pi:llama.cpp/MiMo-V2.6-Flash-RL
- Check for buffered write errors when closing downloaded files. - Use UTF-8 paths when writing ETag files on Windows. - Write in binary mode on Windows. Signed-off-by: Adrien Gallouët <angt@huggingface.co>
Signed-off-by: Adrien Gallouët <angt@huggingface.co>
…erved (ggml-org#29565) With --path or --no-ui, /sw.js returned 404, and a 404 does not remove a service worker, so browsers kept showing the cached built-in UI. Serve a worker that unregisters itself, clears its caches and reloads open tabs. A sw.js in the --path folder is still served first. Assisted-by: Claude Opus 5.5
* common : stop accepting draft tokens at EOG * cont : remove the test
* hexagon: add FP32 GELU_ERF and GEGLU_ERF support * hex-erf: reduce register pressure in kernels
) Supersedes ggml-org#29530 Signed-off-by: Adrien Gallouët <angt@huggingface.co>
* vocab : keep </s> NORMAL in PLaMo-2 and PLaMo-3 The PLaMo-2 and PLaMo-3 vocabularies mark </s> as NORMAL. Current EOG token heuristic matched it by text and added its attribute to CONTROL. Skip this heuristic for the PLAMO2 vocab type so </s> stays NORMAL and is not treated as EOG. * use <|plamo:eos|> for detection
…9504) * fix c++ odr by properly using GGML_COMMON_DECL_CPP * using actual field rather than macro Co-authored-by: Georgi Gerganov <ggerganov@gmail.com> --------- Co-authored-by: XZiar <xziar@xziar.xziar> Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
mut_mul_id selected its matmul tile with total token count. For MoE dispatch grid the true N per workgroup is per-expert rows. At pp128 on Sarvam 30B that is 6, not 128, so the picker took the l-tile for ~6 live rows. Most workers in each group had nothing to do. This wasted time. The slow part was 55% of the whole job.
…9254) It turns out Intel doesn't particularly like loading F32s one at a time and we already have the _2aliagned load logic in mul_mat_vec, so here we use it. While we do already check all the requirements to load elements 4 at a time across [B]F16 and F32, it turns out [B]F16 loading 4 at a time is sometimes slower on very specific shapes on Intel BMG. Loading 4 at a time is a bit faster on F32, but its not material and I assume might be slower on other platforms. Note that we also need to validate `a_offset` is 2-aligned in `mul_mat_vec.comp`, which was missing in the original 2-way-load patch. Some selected speedups from `test-backend-ops perf` on a B60. MUL_MAT(type_a=f32,type_b=f32,m=4096,n=1,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1,src_overlap=0): 1704 runs - 767.17 us/run - 117.44 MFLOP/run - 153.08 GFLOPS MUL_MAT(type_a=f32,type_b=f32,m=4096,n=1,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1,src_overlap=0): 2556 runs - 529.81 us/run - 117.44 MFLOP/run - 221.66 GFLOPS MUL_MAT(type_a=f32,type_b=f32,m=4096,n=2,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1,src_overlap=0): 1704 runs - 727.13 us/run - 234.88 MFLOP/run - 323.03 GFLOPS MUL_MAT(type_a=f32,type_b=f32,m=4096,n=2,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1,src_overlap=0): 2130 runs - 528.84 us/run - 234.88 MFLOP/run - 444.15 GFLOPS MUL_MAT(type_a=f32,type_b=f32,m=4096,n=3,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1,src_overlap=0): 1704 runs - 702.19 us/run - 352.32 MFLOP/run - 501.74 GFLOPS MUL_MAT(type_a=f32,type_b=f32,m=4096,n=3,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1,src_overlap=0): 1988 runs - 532.14 us/run - 352.32 MFLOP/run - 662.08 GFLOPS MUL_MAT(type_a=f32,type_b=f32,m=4096,n=4,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1,src_overlap=0): 1278 runs - 919.50 us/run - 469.76 MFLOP/run - 510.89 GFLOPS MUL_MAT(type_a=f32,type_b=f32,m=4096,n=4,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1,src_overlap=0): 1917 runs - 543.69 us/run - 469.76 MFLOP/run - 864.03 GFLOPS MUL_MAT(type_a=f32,type_b=f32,m=4096,n=5,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1,src_overlap=0): 1197 runs - 892.12 us/run - 587.20 MFLOP/run - 658.21 GFLOPS MUL_MAT(type_a=f32,type_b=f32,m=4096,n=5,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1,src_overlap=0): 1881 runs - 575.17 us/run - 587.20 MFLOP/run - 1.02 TFLOPS MUL_MAT(type_a=f32,type_b=f32,m=4096,n=8,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1,src_overlap=0): 1498 runs - 716.40 us/run - 939.52 MFLOP/run - 1.31 TFLOPS MUL_MAT(type_a=f32,type_b=f32,m=4096,n=8,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1,src_overlap=0): 1819 runs - 576.36 us/run - 939.52 MFLOP/run - 1.63 TFLOPS MUL_MAT(type_a=f32,type_b=f32,m=4096,n=512,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1,src_overlap=0): 134 runs - 7467.09 us/run - 60.13 GFLOP/run - 8.05 TFLOPS MUL_MAT(type_a=f32,type_b=f32,m=4096,n=512,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1,src_overlap=0): 134 runs - 7478.12 us/run - 60.13 GFLOP/run - 8.04 TFLOPS
* vulkan: tune GDN shader * tune for Intel
* ci: add zdnn backend build but not test Signed-off-by: Aaron Teo <aaron.teo1@ibm.com> * ci: attempt to run a ubuntu 26.04 container Signed-off-by: Aaron Teo <aaron.teo1@ibm.com> * ci: set shell to bash Signed-off-by: Aaron Teo <aaron.teo1@ibm.com> * ci: clean up comments Signed-off-by: Aaron Teo <aaron.teo1@ibm.com> * ggml-zdnn: fix compiler errors Signed-off-by: Aaron Teo <aaron.teo1@ibm.com> * vendor: attempt to ignore warnings from vendored files Signed-off-by: Aaron Teo <aaron.teo1@ibm.com> --------- Signed-off-by: Aaron Teo <aaron.teo1@ibm.com>
…sub4`) (ggml-org#29478) * ggml-cuda: HIP: optimize non-saturating packed byte subtraction (`__vsubss4`) * CI: ignore 1 spilled vgpr in fattn_vec --------- Co-authored-by: Carl Philipp Klemm <carl@uvos.xyz>
Without CUB (HIP, MUSA) argsort ran the bitonic kernel with one thread per padded column, so any row above 1024 entries launched an invalid block configuration. Each thread now owns several columns, every stage of the network runs all owned columns before the barrier, and the block is capped at 1024 threads. Shared memory becomes the only bound, which supports_op checks against the device instead of a fixed 1024. Rows up to 1024 run the same work as before. Bit-exact with the CUB path on rows of 2048.
* hex-concat: reduce pkts in gather/transpose hot loop gather directly into dst buffer, use special instruction for gather sync * hex-concat: use fastdiv replace calls to sw divide with fastpath * hex-concat: optimize DMA-HVX pipeline and add transpose helpers
* model : support classifier_pooling for ModernBERT rerankers Assisted-by: Claude Opus 5.5 * model : read classifier pooling type in load_hparams Write classifier.pooling_type from _try_set_pooling_type whenever the config has classifier_pooling, and read it in llama_model_base::load_hparams. ModernBERT falls back to mean when it is unspecified. Assisted-by: Claude Opus 5.5 * conversion : only accept cls and mean for classifier_pooling Assisted-by: Claude Opus 5.5 * model : rename classifier_pooling_type to pooling_type_cls Assisted-by: Claude Opus 5.5
Signed-off-by: Adrien Gallouët <angt@huggingface.co>
GGML_PAD(nbytes, alignment) wraps to 0 when nbytes is within (alignment - 1) of SIZE_MAX, which silently bypassed the size overflow guard in gguf_init_from_reader. Reject the tensor before padding when nbytes + (alignment - 1) would overflow. Adds a test-gguf handcrafted case (F32, ne = [4, 2^30-1, 2^30+1, 1]) whose ggml_nbytes = 2^64 - 16 lands in the wrap window. Fails on master, passes with the guard.
* hexagon: add F16 support for activation ops (SILU/GELU/GELU_QUICK/GEGLU/SWIGLU) Widens ggml_hexagon_supported_activations() to accept F16 (src0/dst/src1 must agree on type), and adds F16 per-thread worker functions in act-ops.c mirroring the existing F32 workers, backed by new HVX f16 kernels (hvx_sigmoid_f16_aa, hvx_tanh_f16_aa, hvx_mul_mul_f16_aa, hvx_min_scalar_f16 family). SILU, GELU, GELU_QUICK, GEGLU, and SWIGLU are verified correct on-device (QRD8850) via test-backend-ops CPU-diffed correctness tests. SWIGLU_OAI's F16 path is code-complete and builds clean on host + all 4 DSP arch variants (v73/v75/v79/v81), but has no F16 test-case coverage in test-backend-ops and is therefore unverified on-device in this change. * hex-ops: align macros * hex-ops: minor formatting --------- Co-authored-by: Max Krasnyansky <maxk@qti.qualcomm.com>
cont: fix code style Signed-off-by: Aaron Teo <aaron.teo1@ibm.com>
* Rebase GLM-Next support onto master, and migrate to llama-memory-hybrid-idx * Add initial MTP support * Merge branch optimizations. Reduce allocated compute buffer size, speed up long context decode, fla, and slight MTP improvements. * Review driven changes, remove env vars, protect tensors * Strip MTP for initial PR * Clean up after mtp strip * Clean up after mtp strip * Update speculative.cpp * Update llama-context.h * Clean up after mtp strip * Fix tokenizer ignore merges * Improve quantization protection selection * Refactor mhc helpers, graph base * Lint Fixes * Apply suggestions from code review Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co> * Skip glm5-next in model saver, fix CRLF * Skip glm5-next in sweep * Remove T4 fallback * Review cleanup * Review suggestions * Defer separate MTP gguf handling to MTP PR, drop filter * Repad n_head_kv * kpool init apply * Order by descending score * Drop guard * read kpool from hparams, clarify kpool cache flags, remove kpool_build_state(nullptr) * Add glm5-next support to model saver and add arch test fixture * Review cleanup * Kpool pooled caching clarify * Add multi stream support * Finish Rebase * Sparse FA fir DSA prefill * Const * Update llama-model.cpp to fix rebase error * gguf-py : merge tensor map entries for HC tensors * model : use build_gdn_l2_norm in GLM5_NEXT implementation * chore : remove trailing whitespace * model : use new OP precision setting API in GLM5_NEXT implementation * mtmd : use ggml_swiglu_clamp in GLM5V and apply the image token limit The two clamps around swiglu_split are what ggml_swiglu_clamp already does, so the clamp bounds collapse back to one value. GLM5V also never called set_limit_image_tokens(), so --image-max-tokens had no effect. Assisted-by: Claude Opus 5 (cherry picked from commit 46d18e1) * llama : keep the GLM5-Next k-pool layout across ubatches The layout was rebuilt from a full cell scan on every ubatch. Pools are fixed by the positions relative to the sequence's first one, so the layout now lives on the memory and a ubatch only appends to it. A sequence edit no longer stales every pooled key either, only the ones at or after the edited position, which makes a tail seq_rm free. The pooling subgraph is built unconditionally so the graph shape no longer changes every kpool tokens, and the pool axis is folded into rows before soft_max, which otherwise exceeds the CUDA gridDim.y limit past n_kv 262144. Assisted-by: Claude Opus 5 (cherry picked from commit 5d1c40b) * model : write the GLM5-Next recurrent rollback checkpoints The conv state and the delta net state were only written to the live row, so a rollback restored whatever the checkpoint rows happened to hold. Take the same route as kimi-k3: build_recurrent_attn for the state, and write all K_rs conv groups. That also drops a state view that assumed contiguous rows. Enroll the arch in test-recurrent-state-rollback, which catches this under its garbage-filled cache pass. Assisted-by: Claude Opus 5 (cherry picked from commit 5ace37e) * llama: fix PR ggml-org#27773 test-save-load-state restore failure Clear the attention and indexer cache data after a failed hybrid state restore so restored NaNs cannot affect a later sequence. Assisted-by: Codex * llama: fix PR ggml-org#27773 gpu-rocm graph reallocation Reserve the full GLM5-Next pool capacity and dirty pool count. The gpu-rocm Test step aborts when n_new grows while the graph node count stays fixed; CUDA, Vulkan, Metal, and WebGPU checks report the same error. Assisted-by: Codex * llama : fix GLM5-Next k-pool layout staleness after edits and shared teardown Two defects in the cross-ubatch k-pool layout added by the k-pool commit: 1. Wrong results. An edited sequence only rebuilt its pool layout when its cell count changed, so if the first ubatch after an edit added back exactly as many cells as were removed, the stale position-to-cell list survived. With a unified cache and more than one sequence, where another sequence takes the freed cells, the reused layout points at the wrong cells (CPU: large logit drift, CUDA: NaN). Rebuild whenever the sequence is stale, not only on a size mismatch. 2. Slowdown. "shared" mode was assumed to end only with an edit that forces a rebuild, but sharing also ends when the other sequence is removed. The survivor kept shared = true, pinning cache_safe off and re-pooling every pool on every ubatch (server trigger: n>1 completions with -kvu, via the seq_cp in copy_state_to). In seq_rm, if the layout has shared cells, stale every sequence so one rebuild re-derives sharing and cache_safe returns to 1. Assisted-by: Claude Opus 5 * llama : fix build_attn_mha stream stride for non-contiguous q build_attn_mha split the batch into streams with a stream stride of q->nb[3]/n_stream. That only equals one stream's span, (ne[2]/n_stream)*nb[2], when q is contiguous. GLM5-Next is nope-only, so it does not concat a rope part and passes the permuted q_absorbed straight in, where nb[3] != ne[2]*nb[2]; the stride was then n_head times too large and every stream s >= 1 read another head's queries. Split-KV (-np N without --kv-unified) multi-stream prefill was wrong for every stream past the first. Unified KV and decode were unaffected (n_stream == 1, and decode takes the gather path). Other MLA models concat rope so q is contiguous and the computed value is unchanged for them. Compute the stride from the token dimension, which is identical for a contiguous q. Assisted-by: Claude Opus 5 * llama : re-derive GLM5-Next k-pool sharing on state_read/state_drop The shared-cell teardown added to seq_rm (stale every sequence when the layout has shared cells, so a survivor does not keep shared = true and pin cache_safe off) was missing from the other paths that can free shared cells: state_read and state_drop staled only the one sequence. Apply the same re-derivation there and correct the comment that claimed sharing ends only via an edit or seq_rm. Assisted-by: Claude Opus 5 * quant : drop duplicate GLM5-Next hc_ filter The hc_ name filter was listed twice in the GLM5_NEXT protection block. Assisted-by: Claude Opus 5 * glm5-next: scope K-pool cache access to indexed operations * glm5-next: keep K-pool access in hybrid index memory * glm5-next: keep mHC graph builders model-local * glm5-next: mark only touched pools per ubatch --------- Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co> Co-authored-by: Stanisław Szymczyk <sszymczy@gmail.com> Co-authored-by: Piotr Wilkin <ilintar@gmail.com>
* add models backend check * t4-medium for faster build
The MUSA vendor header never defined __CUDA_ARCH__, so every architecture test in the shared ggml-cuda sources evaluated to 0. Kernel bodies gated on the architecture therefore compiled to nothing, for example the q8_0 -> f16 dequantization kernel in convert.cu, whose NO_DEVICE_CODE fallback expands to an empty body in host code. Report the newest architecture like the HIP backend does and exclude the NVIDIA-only features explicitly, as they are not usable on MUSA. Define it for device passes only: CUB uses defined(__CUDA_ARCH__) to detect device compilation, which is also how nvcc behaves. Drop the now-redundant defined(__CUDA_ARCH__) checks in the architecture comparisons: __CUDA_ARCH__ is undefined in host passes for CUDA and MUSA, and HIP defines it for every pass, so both forms select the same branch.
ggml-org#27773 adds the glm5-next arch without its rows in the Metal fusion baseline, so test-fusion --check fails on it. The rows come from test-fusion --record on an M5 Max, and --check passes 270/270.
…gml-org#29797) * sampling : use greedy selection for eligible temperature-zero chains Assisted-by: OpenAI Codex * Apply suggestion from @ggerganov Co-authored-by: Georgi Gerganov <ggerganov@gmail.com> * sampling: simplify zero-temperature greedy eligibility Allow the same greedy selection on CPU and grammar/reasoning-budget paths. Keep distribution sampling for dynamic temperature and requested probabilities. Cover the common sampler selection and probability behavior in the existing sampler tests. Assisted-by: OpenAI Codex * sampling : use greedy selection after final top-k with k=1 Assisted-by: OpenAI Codex * cont : clean-up --------- Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
…ggml-org#29179) * Fix KeyError converting Qwen3-TTS CustomVoice variant without speaker_encoder_config (fixes ggml-org#29088) Signed-off-by: SIDDARTHA REDDY <75976672+SIDDARTHAREDDY8@users.noreply.github.com> * Address review feedback: drop tests/test-convert-qwen3tts.py Per reviewer feedback on PR ggml-org#29179, remove the regression test file. The fix itself is unchanged. Signed-off-by: SIDDARTHA REDDY <75976672+SIDDARTHAREDDY8@users.noreply.github.com> --------- Signed-off-by: SIDDARTHA REDDY <75976672+SIDDARTHAREDDY8@users.noreply.github.com>
* llama : add GLM5-Next NextN (MTP) graph Build the GLM5-Next multi-token-prediction head as graph_mtp: the NextN block embeds enorm(tok)+hnorm(h) through eh_proj, runs one plain DSA layer and the shared lm_head, reusing the trunk's builders through the no_build tag ctor. llama_memory_recurrent also tolerates a partial seq_rm when the context holds no recurrent layers, which is what the MTP draft context needs. Assisted-by: Claude * llama : glm5-next: skip dead compute in headless NextN forwards A NextN forward with no output rows (the MTP catch-up and the draft-context prefill) persists only through its cache writes, so the headless graph keeps the MLA latent, indexer key|gate and pooled-key writes and drops the query path, the indexer selection, the attention body, the FFN and the LM head. The 4-token catch-up falls from 6.9 ms to 0.33 ms of kernels; the greedy output hashes and the draft acceptance are unchanged. Assisted-by: Claude * llama : glm5-next: fix NextN extraction contracts and shared-tail rollback Three fixes from the architectural review. The headless graph prune now also requires that no unmasked nextn extraction is live, because that mode reads n_tokens hidden rows regardless of the logits flags. Masked extraction publishes the hidden rows gathered by the output ids, so a batch whose output flags are not a prefix exports the right rows. A partial recurrent rollback whose tail cell is shared with another sequence is now rejected instead of silently moving that sequence's tail. Assisted-by: Claude * llama : glm5-next: tidy comments in the MTP changes Assisted-by: Claude * llama : glm5-next: crop the MTP graph to the output rows instead of pruning it Replace the headless NextN prune with the crop pattern the other MTP graphs use: gather the attention output and the block input at the output ids before the position-wise FFN and the shared head. A NextN forward with no output rows (the MTP catch-up and the draft-context prefill) then runs the FFN and the head over zero rows. The 4-token catch-up falls from 6.9 ms to 2.9 ms of kernels; greedy output hashes are unchanged. Assisted-by: Claude * glm5-next: use the nextn crop helpers in the MTP graph Replace the local crop condition and the masked select of t_h_nextn with crop_before_nextn and crop_after_nextn, so the MTP graph narrows its rows the same way as the main graph and the other models. Describe the shared cell and empty filter branches of the recurrent partial rollback. * glm5-next: load MTP-only and trunk-only GGUF files Make the trunk tensors optional when the file only holds the NextN layer, and the NextN tensors optional when the file only holds the trunk, so the split MTP GGUF loads as a draft model. Co-authored-by: Georgi Gerganov <ggerganov@gmail.com> --------- Co-authored-by: Pascal <admin@serveurperso.com> Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
Brings in 221 upstream commits, including ggml-org#14891 (activation-based imatrix statistics for GGUF imatrices). Resolutions: - qwen4exp: take upstream's MTP (ggml-org#29761) over this fork's #9 head. The two write different GGUFs (upstream keeps the MTP block's own hyper-connection mixer as nextn.hc_head_* and runs the draft with QSA), and current MTP files use upstream's layout. Also drop #9's attention-only MTP memory for qwen4exp in llama-model.cpp and its extra NEXTN entries in constants.py. - speculative: keep --spec-draft-adaptive's n_draft_eff bound on top of upstream's probabilistic draft sampling (result_q); begin() resets both the acceptance EMA and the sampler. - ggml-backend: take upstream's pass-6 input collection (ggml-org#29634) and ggml_backend_sched_copy_input() (ggml-org#29943), keeping #23's copy-0 original with input rotation, async-read pinning, and the sanitizer hook. - vulkan: keep #12's UMA read-back gate (host-cached or <= 64 KiB direct); upstream's AMD-only fix is a subset of it. - server: keep spec-prefill's prompt_src() next to upstream's use_spec_rejection(); use add_embd() with M-RoPE positions. - speculative-prefill: migrate to common_batch / llama_process, as upstream ggml-org#29601 removed common_batch_add/clear. - README: keep this fork's quick start; common.h, test-alloc: keep both. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
github-actions
Bot
added
documentation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
20 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Brings in the 221 upstream commits since the 09-29 sync, among them ggml-org#14891 (activation-based statistics for GGUF imatrices). This is a real merge per SYNC.md, so merge with "Create a merge commit" and do not squash. After merging,
git rev-list --count HEAD..upstream/masteris 0.Resolutions worth reviewing
blk.N.nextn.hc_head_*and runs the draft with QSA, where qwen4exp: MTP draft head for speculative decoding #9 attended dense and reused the trunk's mixer. Current MTP GGUFs use upstream's layout:mtp-Qwen3.8-Flash-Next-Q8_0.ggufcarriesblk.48.nextn.hc_head_{norm,down,up}, which qwen4exp: MTP draft head for speculative decoding #9's loader doesn't know. Soqwen4exp.cpp,models.hand the converter take upstream's side. qwen4exp: MTP draft head for speculative decoding #9's attention-only MTP memory inllama-model.cppand its extra NEXTN list entries inconstants.pyare reverted too. MTP GGUFs converted with qwen4exp: MTP draft head for speculative decoding #9's converter will need re-converting.common/speculative.cpp:--spec-draft-adaptive(speculative: size drafts from measured acceptance (--spec-draft-adaptive) #1) still bounds the draft length (n_draft_eff), now on top of upstream's probabilistic draft sampling (Make the drafter probabilistic and the target verify by rejection sampling for simple draft and MTP ggml-org/llama.cpp#27694,result_q).begin()resets both the acceptance EMA and the sampler.ggml-backend.cpp(scheduler, Halo/scheduler uma ring 27311 #23 / upstream 27311, still open there): upstream moved input-copy creation into a new pass 6 (ggml : collect all input tensors into graph_inputs ggml-org/llama.cpp#29634) and the per-split copy loop intoggml_backend_sched_copy_input()(ggml: refactor selective expert copying to user code ggml-org/llama.cpp#29943). The resolution takes both refactors and keeps 27311's behaviour: copy 0 is the original tensor and inputs rotate throughgraph_input_slots, async-read pinning stays, and the sanitizer hook moves into the new copy function.common_batch/llama_process, because upstream batch: migrate the rest of examples to llama_batch_ext ggml-org/llama.cpp#29601 removedcommon_batch_add/common_batch_clear.prompt_src()alongside upstream'suse_spec_rejection(), and usesadd_embd()with M-RoPE positions. The README keeps our quick start.common.handtest-alloc.cppkeep both sides.Verified
Vulkan build (RADV, Radeon 8060S) with no warnings. ctest: 57/59 pass. The two failures are the same pair as the 09-29 sync:
test-recurrent-state-rollbackfails only on deepseek4 (pristine upstream fails the same way), andtest-jinja-pyfails only because the image has nojinja2. Not yet run on ROCm and not measured on the qwen4exp MTP path; worth a speed run on Qwen3.8-Flash with--spec-type draft-mtpbefore tagging.