Repository navigation
fix(openai): preserve reasoning for local OpenAI-compatible models via R1 toggle #1119
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
beef72e
f4a69cc
60b1b45
de3725b
229555e
1a47887
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -1118,6 +1118,46 @@ describe("OpenAiHandler", () => { | |
| expect(model.id).toBe("") | ||
| expect(model.info).toBeDefined() | ||
| }) | ||
|
|
||
| it("should set preserveReasoning when openAiR1FormatEnabled is on", () => { | ||
| const r1Handler = new OpenAiHandler({ | ||
| ...mockOptions, | ||
| openAiR1FormatEnabled: true, | ||
| }) | ||
| const model = r1Handler.getModel() | ||
| expect(model.info).toEqual({ ...openAiModelInfoSaneDefaults, preserveReasoning: true }) | ||
| }) | ||
|
|
||
| it("should set preserveReasoning for deepseek-reasoner model ids without the toggle", () => { | ||
| const deepseekHandler = new OpenAiHandler({ | ||
| ...mockOptions, | ||
| openAiModelId: "deepseek-reasoner", | ||
| }) | ||
| const model = deepseekHandler.getModel() | ||
| expect(model.info.preserveReasoning).toBe(true) | ||
| }) | ||
|
|
||
| it("should merge preserveReasoning into custom model info when openAiR1FormatEnabled is on", () => { | ||
| const customInfo: ModelInfo = { | ||
| ...openAiModelInfoSaneDefaults, | ||
| contextWindow: 32_768, | ||
| supportsImages: false, | ||
| } | ||
| const r1Handler = new OpenAiHandler({ | ||
| ...mockOptions, | ||
| openAiCustomModelInfo: customInfo, | ||
| openAiR1FormatEnabled: true, | ||
| }) | ||
| const model = r1Handler.getModel() | ||
| expect(model.info).toEqual({ ...customInfo, preserveReasoning: true }) | ||
| expect(model.info.contextWindow).toBe(32_768) | ||
| expect(model.info.supportsImages).toBe(false) | ||
| }) | ||
|
|
||
| it("should not set preserveReasoning by default", () => { | ||
| const model = handler.getModel() | ||
| expect(model.info.preserveReasoning).toBeUndefined() | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This pins the flag, but the consumer it feeds —
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Added in 1a47887: two tests in |
||
| }) | ||
| }) | ||
|
|
||
| describe("Azure AI Inference Service", () => { | ||
|
|
||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Both new tests leave
openAiCustomModelInfounset, soinfois value-identical to the defaults here — the{ ...info }merge only gets exercised on the defaults path. A case combining custom model info (e.g. a distinctcontextWindow) with the toggle would pin the merge for the llama.cpp / LM Studio / Ollama setups this PR targets. Worth adding?There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Added in 1a47887:
should merge preserveReasoning into custom model info when openAiR1FormatEnabled is onbuildsopenAiCustomModelInfowith a distinctcontextWindow(32_768) andsupportsImages: false, and asserts the full merged object{ ...customInfo, preserveReasoning: true }, so the{ ...info }merge path is actually exercised.