Why is opencode not working with local llms via Ollama?

Question

Why is opencode not working with local llms via Ollama?
Hello. I have tried numerous local llms with opencode and I can not seem to get any to work. I have a decent PC that can run up to a 30b model smoothly. I have tried them. I can not get anything to work. Below is an example of what keeps happening. This is with llama3.2:3b.

Any help is appreciated.

EDIT: Added my config.

Hm I don't think the llama models will treat you very well, also what is your context size for ollama

Hm I don't think the llama models will treat you very well

What would you recommmend?

also what is your context size for ollama

Right now it is set at 32k. Is this too small?

I think that it's viewing OpenCode as the context instead of the actual project directory.

I think that it's viewing OpenCode as the context instead of the actual project directory.

Exactly what I was thinking as well. Seems weird.

What you are experiencing is actually common with local models. This is classic tool calling capability gap. I use Anthropic most of the time, which has native, robust tool calling support.

But I also do a lot of local model stuff. Models like qwen3-coder:30b don't have it. I can see you're using Llama3.2:3b which is an incredibly small model which is likely also the problem.

Some of the guys at unsloth have been working on this issue: https://huggingface.co/unsloth/Qwen3-Coder-30B-A3B-Instruct-GGUF/discussions/10

To provide you guys with context here:

Models must be specifically fine-tuned to understand structured tool calling

They gotta learn to output valid JSON in a specific format when tools are available
The HuggingFace discussion on Qwen3-Coder states it struggles because:
- It outputs XML-style tags (<tool_call>, <function=...>) instead of JSON
- It hallucinates tool names (like todonext) that don't exist
- It doesn't consistently follow the expected format

I've been hinking about this - and there might be a PR here in theory. I believe LM Studio has a transformation layer that intercepts the models raw output, spots xml tool calls and changes them to json format.

hmm....

Looking at /packages/opencode/src/provider/transform.ts, in theory one could add a new transformation for Ollama/local models and Parse XML-style outputs and convert to JSON then hook into the wrapLanguageModel from AI SDK

maybe... I pulled this out my rear

Yeah I think its possible technically

I started fiddling with this because I have ADHD and am now hyper fixated. This is def a challenge. I think however the model might still be the challenge. TBD

But I also do a lot of local model stuff. Models like qwen3-coder:30b don't have it. I can see you're using Llama3.2:3b which is an incredibly small model which is likely also the problem.

The issue isn't solely due to tiny models like llama3.2
This is an example with gpt-oss:20b

But I also do a lot of local model stuff. Models like qwen3-coder:30b don't have it. I can see you're using Llama3.2:3b which is an incredibly small model which is likely also the problem.

The issue isn't solely due to tiny models like llama3.2 This is an example with gpt-oss:20b

Yeah it's the models ability to call the correct tools and use them. I've been testing with Qwen3-coder:30B for an hour now running with an idea. It hallucinates tool names nonstop. So an even smaller model will do even worse.

I built an adapter that catches when Qwen+Ollama mess up tool names, uses fuzzy matching to figure out what they actually meant (like mapping exec to bash), and fixes the call before it fails.

That was the easy part .... but it's still 💩

The problem is I get inconsistent behavior out the wazoo.

Sometimes outputs XML as plain text: <function=files></function></tool_call>
Sometimes makes actual tool calls (but with wrong arguments)
Sometimes just describes what it would do instead of doing it :|

TLDR:
✅ Fuzzy matching works
✅ Alias mapping works
❌ Model still can't edit files
❌ Model still can't run commands
❌ User experience is broken

I gotta think on this one. Maybe test more models. It might just be that the model itself is the limitation. Claude works great though. I love this TUI. I just discovered this whole project randomly earlier today when browsing Reddit about Ollama.

I have heard most people have better experiences with lmstudio over ollama

There isn't a ton we can do on our end besides maybe extra prompting for local models?

Just tested both Ollama and LM Studio on Opencode with multiple models the past couple days.

After many issues with Ollama (mostly that all models default to a very small context window and you have to modify them or find versions with bigger context window settings, and tool call formatting issues), after installing LM Studio I was able to consistently use qwen/qwen3-30b-a3b-2507 with tools, and had varied success with openai/gpt-oss-20b, but it was still usable for shorter tasks.

TL;DR: ditch Ollama, try LM Studio

I was able to verify today that the MODEL (qwen3-coder:30b-a3b-q4_K_M) actually is making tool calls. The disconnect was happening between Ollama and the time it hit the AI SDK. Pretty sure Ollama's parser layer is the problem.

By the time it got to the SDK it was plain text.

// What Ollama returns:
  {
    "content": "<function=read>\n<parameter=file_path>/etc/hostname</parameter>"
  }

So need to intercept that at the middleware layer BEFORE it hits the AI SDK. But there might be another issue... TBD

I'll come back to this if I figure something out

Hello people:

1.- I was banned from anthropic 4 days ago..... XD (fun, just 1 week of vibe coding got me banned) cause I created a "clever" super wrapper for claude code (That's what I understood).
2.- I was working at the same time on a very very detailed architecture for other really big project that, when considered by AI models, allows them to ez build a lot of stuff just cause the architecture is pre-defining "nearly everything".
3.- After being banned from anthropic, I started creating my own "model/agent/claude-code-like". It's been like 2 days of coding and it's kinda "intelligent". it can do commits and basic stuff, but still hallucinating.
4.- How? I created a bunch of deteministic rules for behaviour, for hallucination, differente agentic personalities, different roles, orchestrators, etc...
5.- For those wondering: yes, the basic Idea is to input always rules to the agents or skills (whatever you want to do, does not matter) in order to format with Json. This way you can define like an own "MCP" or bunch of skills and funcionts that can talk between them
4.- When you have all of this, then you need to create a binary that controls and starts the model with ur preferences, etc. and routes commands and responses to / from the console-

it's hard work, not ez task, but not impossible. Maybe I'll update if I continue pushing for it.

hope it lhelps

facing the same issue with:
GLM4.7/UD-IQ2_M

below is the ollama model file used:

FROM /custom_models/GLM4.7/UD-IQ2_M/GLM-4.7-MERGED.gguf

PARAMETER num_ctx 65536
# Use a low temperature for better JSON/Tool accuracy
PARAMETER temperature 0.1
PARAMETER num_gpu 999
PARAMETER num_predict 4096

# --- Tool-Enabled Agentic Template ---
TEMPLATE """[gMASK]<sop>{{ if .System }}<|system|>
{{ .System }}{{ end }}{{ if .Tools }}<|system|>
In addition to plain text responses, you can choose to call one or more of the provided functions.
Use the following rule to decide when to call a function:
* if you need external information or need to edit files, generate a function call.
If you decide to call functions:
* all function calls should be generated in a single JSON list formatted as functools[{"name": [function name], "arguments": [function arguments as JSON]}, ...]
Available functions as JSON spec:
{{ .Tools }}<|eot_id|>{{ end }}{{ if .Prompt }}<|user|>
{{ .Prompt }}{{ end }}<|assistant|>
{{- if .Response }}
{{ .Response }}
{{- end }}
{{- if .ToolCalls }}
<|thought|>
I will call the following tools to fulfill the requirement:
functools[
{{- range .ToolCalls }}
{"name": "{{ .Function.Name }}", "arguments": {{ .Function.Arguments }}}
{{- end }}
]
{{- end }}"""

# --- Senior Engineer System Prompt ---
SYSTEM """You are a Senior Software Engineer with deep access to the system.
You MUST use the provided file-editing tools (edit, write, read) for all building tasks.
Always provide a brief reasoning block (<|thought|>) before emitting a tool call.
Output production-grade, secure code following Clean Code principles."""

Update: I played a few days with opencode, ollama, litellm and other LLM engines and what i found out is: just don't use ollama. It's simple to deploy but more for testing purposes. It does not do proper tool calling and in general lacks a lot of features.

@Mr-Philipp

@Mr-Philipp , what other alternate do you suggest if not ollama, my main goal is to make it work with opencode with local models.
I have around 250-300VRAM for it.

@Mr-Philipp , what other alternate do you suggest if not ollama, my main goal is to make it work with opencode with local models. I have around 250-300VRAM for it.

i moved to vllm and all my problems basically fixed themselves.

nice setup! can you elaborate what GPU's you use and how they are connected?

@Mr-Philipp , what other alternate do you suggest if not ollama, my main goal is to make it work with opencode with local models. I have around 250-300VRAM for it.

i moved to vllm and all my problems basically fixed themselves.

nice setup! can you elaborate what GPU's you use and how they are connected?

if possible can you guide me with the setup you did?

which model you used
parameter you used etc
how did you connect opencode and vllm ?

Also my setup:
I have 4 NVIDIA H200, its setup in a store room with AC's

@Mr-Philipp , what other alternate do you suggest if not ollama, my main goal is to make it work with opencode with local models. I have around 250-300VRAM for it.

i moved to vllm and all my problems basically fixed themselves.
nice setup! can you elaborate what GPU's you use and how they are connected?

if possible can you guide me with the setup you did?

which model you used

parameter you used etc

how did you connect opencode and vllm ?

Also my setup: I have 4 NVIDIA H200, its setup in a store room with AC's

have a look at https://github.com/BoltzmannEntropy/vLLM-5090 to get some inspiration.

I made Qwen 30B work by using appropriate context length: https://docs.ollama.com/context-length

@nesterow

Setting the context length as mentioned by @nesterow worked for me!

OLLAMA_CONTEXT_LENGTH=64000 ollama serve

Unbelievable difference. Worked for me with with gpt-oss:20b, qwen3-coder:30b.

Bumped OLLAMA_CONTEXT_LENGTH=128000 for nemotron-3-nano:30b. It thinks too much!

Ollama was working fine for me with the 1.1.20ish versions of Opencode. After upgrading to 1.1.30 and 1.1.40 I am not getting any feedback in OpenCode. The Ollama setup works fine, e.g. in our OpenWebUI setup. So it seems to be an Opencode issue.

@rasoliveira

To summarize this ticket since I last posted:

@rasoliveira suggests that OLLAMA_CONTEXT_LENGTH really does solve this issue, although keep in mind that that sets the context length for every API request, not per-request, which would be ideal.

@pawanjenu Could you kindly describe where you built your "# --- Tool-Enabled Agentic Template ---" from? That looks very technical for an LLM to process, and I am curious about the reasoning behind that?

For me, I leveraged the 32GB RAM to load 20GB models onto an 8GB GPU and got typical robust tool usages with Ollama. I have not had a chance to use the new 32GB GPU with 96GB RAM to load large-LLMs which may be sufficient to do the multi-sequence OpenCode style calls, switching off between MCPs, tool uses, file reads, and generations with large contexts. I will return when I get a chance to do that.

@shochdoerfer what did you experience before & after the update?

@Mr-Philipp is vLLM (from Berkeley) a model service leveraging most HuggingFace models? In that regard, you are suggesting that is a one-to-one replacement for Ollama? (Except perhaps the CLI interface of Ollama?) Is it validated, tested?

@davidbernat

@davidbernat after upgrading to Ollama 0.15, I got opencode working again.

Before that the opencode UI would just do nothing. It did communicate with Ollama - as in the LLM instance got properly started - but even waiting for 30min showed no reponse in the opencode UI. Also no errors in the logs which made it really hard to debug.

@Mr-Philipp is vLLM (from Berkeley) a model service leveraging most HuggingFace models? In that regard, you are suggesting that is a one-to-one replacement for Ollama? (Except perhaps the CLI interface of Ollama?) Is it validated, tested?

Yes, vllm can load HuggingFace models natively but you need to select exactly what model type you want to load. vllm is openai-API compatible but in my case i am using litellm in front of it. It is not a one-to-one replacement because of how it works and how it needs to be configured is different and more complex.

Some differences:

vllm does not support switching models on demand. you start it with one model and that's it
vllm is able to handle multiple requests at a time

It's like MacOS vs Arch Linux. There is no answer which one is better - it depends on what you want it to do.

To stay organized issues are automatically closed after 90 days of no activity. If the issue is still relevant please open a new one.

Navigation Menu

Repository navigation

Why is opencode not working with local llms via Ollama? #4428

Question

10 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Why is opencode not working with local llms via Ollama? #4428

Description

Question

Activity

rekram1-node commented on Nov 17, 2025

TheLevic commented on Nov 17, 2025

Datal0re commented on Nov 18, 2025

TheLevic commented on Nov 18, 2025

1337hero commented on Nov 18, 2025

1337hero commented on Nov 18, 2025

rekram1-node commented on Nov 18, 2025

1337hero commented on Nov 18, 2025

Datal0re commented on Nov 18, 2025

1337hero commented on Nov 18, 2025

rekram1-node commented on Nov 18, 2025

p0ns commented on Nov 18, 2025

1337hero commented on Nov 18, 2025

10 remaining items

ateixeiramunoz commented on Jan 9, 2026

pawanjenu commented on Jan 17, 2026

Mr-Philipp commented on Jan 17, 2026

pawanjenu commented on Jan 17, 2026

Mr-Philipp commented on Jan 17, 2026

pawanjenu commented on Jan 17, 2026

Mr-Philipp commented on Jan 19, 2026

nesterow commented on Jan 25, 2026

rasoliveira commented on Feb 5, 2026

shochdoerfer commented on Feb 5, 2026

davidbernat commented on Feb 13, 2026

shochdoerfer commented on Feb 13, 2026

Mr-Philipp commented on Feb 13, 2026

github-actions commented on Apr 15, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions