Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .changeset/ollama-stream-length.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
'@tanstack/ai-ollama': patch
---

Preserve Ollama's `done_reason: "length"` in streamed chat completions, including turns with tool calls, so truncated output does not execute tools.
2 changes: 2 additions & 0 deletions docs/adapters/ollama.md
Original file line number Diff line number Diff line change
Expand Up @@ -212,6 +212,8 @@ const stream = chat({

Sampling options do not go at the root of `chat()`. For Ollama, they go to `modelOptions.options.temperature`, `modelOptions.options.top_p`, and `modelOptions.options.num_predict`. See [Moving Sampling Options into modelOptions](../migration/sampling-options-to-model-options). The full option list is in [All runner options](#all-runner-options).

When Ollama reports `done_reason: "length"`, the final `RUN_FINISHED` event carries `metadata.tanstack.finishReason: "length"`. This also applies when the stream contains tool calls. `chat()` keeps the partial reply and skips tool execution for that truncated turn.
Comment thread
TPSJASPER marked this conversation as resolved.
Resolved

### Reasoning

Running a thinking model like Qwen 3 or gpt-oss? Set `reasoning` on `chat()`:
Expand Down
2 changes: 1 addition & 1 deletion docs/config.json
Original file line number Diff line number Diff line change
Expand Up @@ -1715,7 +1715,7 @@
"label": "Ollama",
"to": "adapters/ollama",
"addedAt": "2026-04-15",
"updatedAt": "2026-10-22"
"updatedAt": "2026-10-25"
},
{
"label": "Grok (xAI)",
Expand Down
7 changes: 6 additions & 1 deletion packages/ai-ollama/src/adapters/text.ts
Original file line number Diff line number Diff line change
Expand Up @@ -432,7 +432,12 @@ export class OllamaTextAdapter<TModel extends string> extends BaseTextAdapter<
threadId,
model: chunk.model,
timestamp: Date.now(),
finishReason: toolCallsEmitted.size > 0 ? 'tool_calls' : 'stop',
finishReason:
chunk.done_reason === 'length'
? 'length'
: toolCallsEmitted.size > 0
? 'tool_calls'
: 'stop',
// usage is optional under exactOptionalPropertyTypes; omit the key
// entirely when Ollama reported no token counts.
...(finishUsage && { usage: finishUsage }),
Expand Down
85 changes: 85 additions & 0 deletions packages/ai-ollama/tests/stream-finish-reason.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,85 @@
import { afterEach, describe, expect, it, vi } from 'vitest'
import { chat } from '@tanstack/ai'
import { resolveDebugOption } from '@tanstack/ai/adapter-internals'
import { createOllamaChat } from '../src/adapters/text'

afterEach(() => vi.unstubAllGlobals())

describe('Ollama stream finish reason', () => {
it.each([
{ doneReason: 'length', toolCall: false, expected: 'length' },
{ doneReason: 'length', toolCall: true, expected: 'length' },
{ doneReason: 'stop', toolCall: false, expected: 'stop' },
{ doneReason: 'stop', toolCall: true, expected: 'tool_calls' },
{ doneReason: undefined, toolCall: false, expected: 'stop' },
{ doneReason: undefined, toolCall: true, expected: 'tool_calls' },
])(
'maps $doneReason with toolCall=$toolCall to $expected',
async ({ doneReason, toolCall, expected }) => {
const lines = [
{
message: {
role: 'assistant',
content: 'Partial answer',
...(toolCall && {
tool_calls: [{ function: { name: 'lookup', arguments: {} } }],
}),
},
done: false,
},
{
message: { role: 'assistant', content: '' },
done: true,
done_reason: doneReason,
prompt_eval_count: 14,
eval_count: 5,
},
]
const fetch = vi.fn(
async () =>
new Response(
lines.map((line) => JSON.stringify(line) + '\n').join(''),
{ headers: { 'content-type': 'application/x-ndjson' } },
),
)
vi.stubGlobal('fetch', fetch)
const adapter = createOllamaChat('llama3.2', 'http://ollama.test')
const messages = [{ role: 'user' as const, content: 'Reply' }]
const chunks = []
for await (const chunk of adapter.chatStream({
threadId: 'ollama-finish-reason',
logger: resolveDebugOption(false),
model: adapter.model,
messages,
}))
chunks.push(chunk)

expect(chunks.at(-1)).toMatchObject({
type: 'RUN_FINISHED',
finishReason: expected,
})

if (doneReason === 'length') {
const execute = vi.fn(() => 'result')
const events = []
for await (const event of chat({
threadId: 'ollama-finish-reason',
adapter,
messages,
tools: [{ name: 'lookup', description: 'Look up an item', execute }],
modelOptions: { model: adapter.model, options: { num_predict: 5 } },
debug: false,
}))
events.push(event)

expect(events.at(-1)).toMatchObject({
type: 'RUN_FINISHED',
metadata: { tanstack: { finishReason: 'length' } },
})
expect(events.some((event) => event.type === 'RUN_ERROR')).toBe(false)
expect(execute).not.toHaveBeenCalled()
expect(fetch).toHaveBeenCalledTimes(2)
}
},
)
})
Loading
Loading