You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
npu: a rope_scaling type the route does not implement is served plain RoPE (yarn/linear/longrope), silently #1087
Two front doors accept a model that declares a RoPE scaling block they do not apply, and neither says so:
tools/gguf_to_q4nx.py (the GGUF → packed-container route, 1bit npu-run <gguf>) reads *.rope.freq_base and
nothing else, so rope.scaling.type = yarn | linear | longrope | dynamic is dropped. The container is written with rope_theta alone and the lane computes theta^(-i/half) (npu/pack.cpp fill_rope), which is right only inside the
model's original window.
npu/forward implements two rules — llama3 (engine#1048) and LongRoPE short_factor — and reads nothing else.
A container whose config.json declares yarn (or any other type) loads and serves as if the block were absent.
Why it matters: the failure is silent and looks like a model that is merely a bit dumb past the original window. The
size of it is measured for the case that is implemented — npu/forward/include/q4nx_rope.h: ignoring the llama3 rule
on Llama-3.2-1B gives KL 0.0058 at ctx 104, 0.0107 at 754, 0.0252 at 1497 against the same checkpoint with it applied.
A YaRN model served plain has the same shape of error and no line in any log to say so.
Refusing is the honest behaviour, not implementing every rule: YaRN's attention-factor schedule, dynamic NTK and
look-up scaling are each a feature of their own, and guessing one would be worse than a named refusal.
Found while closing the #844 comment item; not the same defect as #1048 (that one had an implementation and no wiring,
these have no implementation at all).
Activity
Merge origin/main (05da4d8) into npu/plan-p4-target — 12 commits, one…