fix: cache write tokens always 0 for Anthropic models behind an OpenAI-compatible provider #36749

Activity

1. github-actions

   assigned

   kitlangton

   on Jul 13, 2026
2. padraig-myers-sh commented on Aug 10, 2026

   

   padraig-myers-sh

   on Aug 10, 2026

   More actions

   Is it possible to get an update on this?
   Currently having to monkey patch opencode with the fix in #36752 to correct cost when routing via llmlite.
3. Zaczero commented on Aug 18, 2026

   

   Zaczero

   on Aug 18, 2026

   More actions

   This reproduces in V2 as well, in a completely different file, so the fix will need porting rather
   than merging.

   V1's analysis pointed at packages/opencode/src/session/llm/ai-sdk.ts. V2 rewrote the provider layer
   into packages/ai/src/protocols/, and the same gap is now here:

   opencode/packages/ai/src/protocols/openai-chat.ts

   Lines 582 to 600 in 39be159

   	const mapUsage = (usage: OpenAIChatEvent["usage"]): Usage | undefined => {
   	if (!usage) return undefined
   	const input = usage.prompt_tokens ?? undefined
   	const output = usage.completion_tokens ?? undefined
   	const cached = usage.prompt_tokens_details?.cached_tokens ?? undefined
   	const cacheWrite = usage.prompt_tokens_details?.cache_write_tokens ?? undefined
   	const reasoning = usage.completion_tokens_details?.reasoning_tokens ?? undefined
   	const nonCached = ProviderShared.subtractTokens(input, ProviderShared.sumTokens(cached, cacheWrite))
   	return new Usage({
   	inputTokens: input,
   	outputTokens: output,
   	nonCachedInputTokens: nonCached,
   	cacheReadInputTokens: cached,
   	cacheWriteInputTokens: cacheWrite,
   	reasoningTokens: reasoning,
   	totalTokens: ProviderShared.totalTokens(input, output, usage.total_tokens ?? undefined),
   	providerMetadata: { openai: usage },
   	})
   	}

   This is the path every OpenAI-compatible provider takes. openai-compatible-chat.ts doesn't define
   its own protocol — it reuses OpenAIChat.protocol end to end and overrides only the route id:

   opencode/packages/ai/src/protocols/openai-compatible-chat.ts

   Lines 17 to 23 in 39be159

   	export const route = Route.make({
   	id: ADAPTER,
   	providerMetadataKey: "openai",
   	protocol: OpenAIChat.protocol,
   	endpoint: Endpoint.path("/chat/completions"),
   	framing: Framing.sse,
   	})

   So litellm, vLLM, LM Studio, Ollama-compat and every gateway fronting Anthropic all land on that one
   mapUsage, which reads cache counts only from prompt_tokens_details.cached_tokens and
   prompt_tokens_details.cache_write_tokens. A gateway reporting cache_creation_input_tokens at the
   top level of usage — exactly the payload in your report — matches neither key.

   Why this fails silently, which I think is the more important half

   The two protocols use opposite conventions, and only one of them can hide a missing field.

   Native Anthropic adds up:

   opencode/packages/ai/src/protocols/anthropic-messages.ts

   Lines 686 to 700 in 39be159

   	const mapUsage = (usage: AnthropicUsage | undefined): Usage | undefined => {
   	if (!usage) return undefined
   	const nonCached = usage.input_tokens
   	const cacheRead = usage.cache_read_input_tokens ?? undefined
   	const cacheWrite = usage.cache_creation_input_tokens ?? undefined
   	const inputTokens = ProviderShared.sumTokens(nonCached, cacheRead, cacheWrite)
   	return new Usage({
   	inputTokens,
   	outputTokens: usage.output_tokens,
   	nonCachedInputTokens: nonCached,
   	cacheReadInputTokens: cacheRead,
   	cacheWriteInputTokens: cacheWrite,
   	reasoningTokens: usage.output_tokens_details?.thinking_tokens,
   	totalTokens: ProviderShared.totalTokens(inputTokens, usage.output_tokens, undefined),
   	providerMetadata: { anthropic: usage },

   input_tokens there is exclusive of cache, so inputTokens = nonCached + cacheRead + cacheWrite. Drop
   a field on that path and the total comes out too small — visible, and something downstream can catch.

   OpenAI Chat subtracts: prompt_tokens is inclusive, so nonCached = prompt_tokens - (cached + cacheWrite). Drop cacheWrite here and the total stays perfectly correct — the tokens simply move
   into the nonCached column. Nothing reconciles wrong, no invariant breaks, no total looks off. The
   bill is the only place the difference is observable, which is why this keeps getting refiled per
   provider (#35974, #37745, this one) instead of being caught once.

   That also means your cost estimate is understated by more than the 1.25x you mentioned whenever the
   provider prices cache writes above input, and it is silently understated in every case.

   The fix needs no schema change

   The alternate keys already survive decoding. Both the usage struct and prompt_tokens_details are
   declared with a rest record:

   opencode/packages/ai/src/protocols/openai-chat.ts

   Lines 155 to 181 in 39be159

   	const OpenAIChatUsage = Schema.StructWithRest(
   	Schema.Struct({
   	prompt_tokens: optionalNull(Schema.Number),
   	completion_tokens: optionalNull(Schema.Number),
   	total_tokens: optionalNull(Schema.Number),
   	prompt_tokens_details: optionalNull(
   	Schema.StructWithRest(
   	Schema.Struct({
   	cached_tokens: optionalNull(Schema.Number),
   	cache_write_tokens: optionalNull(Schema.Number),
   	}),
   	[Schema.Record(Schema.String, Schema.Unknown)],
   	),
   	),
   	completion_tokens_details: optionalNull(
   	Schema.StructWithRest(
   	Schema.Struct({
   	reasoning_tokens: optionalNull(Schema.Number),
   	accepted_prediction_tokens: optionalNull(Schema.Number),
   	rejected_prediction_tokens: optionalNull(Schema.Number),
   	}),
   	[Schema.Record(Schema.String, Schema.Unknown)],
   	),
   	),
   	}),
   	[Schema.Record(Schema.String, Schema.Unknown)],
   	)

   Schema.StructWithRest(..., [Schema.Record(Schema.String, Schema.Unknown)]) keeps unknown keys on the
   decoded value, so cache_creation_input_tokens is present on the object mapUsage already receives.
   Reading it there is additive — no schema edit, no new plumbing.

   One caveat worth stating, because it constrains where the fix can go: mapUsage sets
   providerMetadata: { openai: usage }, but that does not survive persistence. I checked 105,379
   assistant rows in my database and none carries a providerMetadata key — the persisted message keeps
   only agent, model, content, finish, cost, tokens, time, snapshot, error. So a downstream fallback
   that reads provider metadata off the stored message cannot work; it has to happen inside mapUsage
   while the raw payload is still in hand.

   Scope

   #35974 is the same function and the same gap, with DeepSeek's key names instead. Both are one patch.
4. neriousy commented on Oct 1, 2026

   

   neriousy

   on Oct 1, 2026

   Member

   More actions

   Closing as a duplicate of #35974, which tracks cache-write tokens not being recorded for OpenAI-compatible providers. As noted there, both cases are the same gap in the usage mapping and need one fix. Please add your gateway/model details there.
5. neriousy

   closed this as a duplicateof DeepSeek provider: prompt_cache_miss_tokens not mapped to cache.write #35974 on Oct 1, 2026