Skip to content

[Bug] Ollama endpoint /api/chat fails with empty chunks when stream: true (OpenAI endpoint works) #436

Closed

Description

@TRS-WOB

Environment

  • OS: Windows 11
  • Hardware: AMD Ryzen AI 300 / Strix Point (Radeon 860M NPU)
  • FastFlowLM Version: v0.9.36
  • Model: qwen3.5:4b (also tested with qwen3:0.6b)
  • Command: flm serve qwen3.5:4b --port 8080 --ctx-len 32768 --pmode turbo -r 1 --q-len 10

Description

When using the Ollama-compatible endpoint (/api/chat) with "stream": true, the server instantly returns empty SSE chunks containing only metadata (no content) and immediately closes the stream with "done": true.

This completely breaks tools like Claude Code (via proxies like claudish or LiteLLM) that rely on the Ollama streaming API.

Steps to Reproduce:
Run a simple streaming request against the Ollama endpoint:

curl -s -N -X POST http://localhost:8080/api/chat -H "Content-Type: application/json" -d "{\"model\": \"qwen3.5:4b\", \"messages\": [{\"role\": \"user\", \"content\": \"Hello, reply with 'hi'.\"}], \"stream\": true}"

Observed Behavior (Empty Chunks):

{"model":"qwen3.5:4b","message":{"role":"assistant","content":""},"done":true,"done_reason":"stop","prompt_eval_count":24,"eval_count":0,"total_duration":1966916608,"load_duration":4300,"prompt_eval_duration":964379584,"eval_duration":0}

Expected Behavior:
The stream should return the generated text chunk by chunk.
Note: The OpenAI-compatible endpoint (/v1/chat/completions) works perfectly fine with stream: true on the exact same model. Also, setting "stream": false on the Ollama endpoint works correctly.

Activity

  1. ZaneNi commented on Mar 23, 2026

    @ZaneNi
    Contributor

    @TRS-WOB We no longer maintain the Ollama API as of v0.9.17. Thank you.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions