You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
feat: pass through all backend-supported sampling/serving parameters #151
Raven forwards only a fixed subset of sampling parameters to the serving backend. Anything outside that subset is silently dropped, even when the backend supports it. Two incidents where this bit us:
repetition_penalty — sglang supports it, but there is no way to get it through Raven's model config. We had to interpose a local proxy shim on the request path just to set one sampling knob.
enable_thinking: false (Qwen3-style chat-template switch, normally passed via extra_body / chat_template_kwargs) — unreachable through Raven config; required patching the package.
Both are legitimate, backend-documented parameters; needing a fork for each new knob doesn't scale.
Proposal
A passthrough dict on the model/backend config, forwarded verbatim into the request payload (merged into the OpenAI-compatible body / extra_body):
model:
sampling_extra: # merged into sampling paramsrepetition_penalty: 1.05request_extra: # merged into extra_bodychat_template_kwargs: { enable_thinking: false }
Exact shape is bikesheddable — the ask is: anything the backend accepts should be reachable from config without a fork.
Activity
feat(providers): endpoint failover and provider path close-out (EverM…
feat(providers): endpoint failover and provider path close-out (EverM…