Skip to content

Azure AI Claude rejects requests after tool abort: assistant message prefill not supported #19517

Closed as not planned

Description

@stefanc-makersite

Bug Description

When using OpenCode with Azure AI Claude (via LiteLLM proxy), aborting a tool call (Ctrl+C or interrupt) causes the next LLM request to fail with:

litellm.BadRequestError: Azure_aiException - "This model does not support assistant message prefill. The conversation must end with a user message."

The raw XML of the attempted tool call leaks into the conversation as visible text:

<invoke name="task">
<parameter name="description">...</parameter>
<parameter name="prompt">...</parameter>
</invoke>
litellm.BadRequestError: Azure_aiException - ...

The user must send a new message to reset the conversation turn. The question tool also consistently triggers this error on Azure AI Claude.

Environment

  • OpenCode 1.3.0 (Homebrew)
  • Provider: Azure AI Claude (Opus 4.6) via LiteLLM proxy (Orion)
  • Model group: azure_ai/claude-opus-4-6

Root Cause

Azure AI's Claude inference layer does not support "assistant message prefill" (trailing role: "assistant" in the messages array). The standard Anthropic API does support this, so OpenCode works fine with direct Anthropic, but fails through Azure AI / LiteLLM.

There are two code paths that produce a trailing assistant message:

1. Aborted assistant message with text but no tool calls

When a user aborts mid-stream after text has started streaming but before any tool call is registered, the aborted assistant message (with partial text) gets included in history.

The error filtering logic in toModelMessages() intentionally keeps aborted messages that have "substantive" parts (text qualifies):

packages/opencode/src/session/message-v2.ts lines 690-698:

if (
  msg.info.error &&
  !(
    MessageV2.AbortedError.isInstance(msg.info.error) &&
    msg.parts.some((part) => part.type !== "step-start" && part.type !== "reasoning")
  )
) {
  continue  // skip this assistant message
}

When an aborted message has tool calls, the tool results get appended as role: "user" (Anthropic convention), so the sequence is fine. But when it only has text, the sequence ends with role: "assistant".

2. isLastStep prefill injection

When the agentic loop hits the step limit, a hardcoded assistant message is appended unconditionally:

packages/opencode/src/session/prompt.ts lines 700-710:

messages: [
  ...(await MessageV2.toModelMessages(msgs, model)),
  ...(isLastStep
    ? [{ role: "assistant" as const, content: MAX_STEPS }]
    : []),
],

This is valid prefill for the direct Anthropic API but rejected by Azure AI Claude.

Suggested Fix

Option A (recommended): Strip trailing assistant messages in normalizeMessages()

packages/opencode/src/provider/transform.ts function normalizeMessages() (line 49) already performs provider-specific message normalization. Add logic to detect when the message array ends with role: "assistant" and either remove it or append a synthetic user message, for providers that don't support prefill.

The codebase already has precedents for this pattern:

  • Synthetic user messages for Gemini reasoning models (prompt.ts lines 520-543)
  • Message sequence fixes for Mistral (transform.ts lines 138-151)
  • LiteLLM dummy tool injection (llm.ts lines 160-178)

Detection criteria for "no prefill support":

  • Provider uses @ai-sdk/openai-compatible (LiteLLM setups)
  • Provider ID contains azure_ai
  • Or a provider-level capability flag

Option B: Guard isLastStep by provider capability

Only inject the MAX_STEPS assistant message for providers known to support prefill (direct Anthropic).

Option C: Proxy-side workaround

Configure LiteLLM to strip trailing assistant messages before forwarding to Azure AI. Less ideal since it papers over the issue.

Related Code Paths

File Lines Role
packages/opencode/src/session/message-v2.ts 690-698 Error filtering includes aborted messages
packages/opencode/src/session/message-v2.ts 756-766 Pending/running tool call -> error result conversion
packages/opencode/src/session/prompt.ts 700-710 isLastStep prefill injection
packages/opencode/src/session/processor.ts 354-424 Abort handling
packages/opencode/src/provider/transform.ts 49-190 normalizeMessages() - provider-specific cleanup
packages/opencode/src/session/llm.ts 160-178 Existing LiteLLM workarounds

Workaround

After the error occurs, send any new user message (even just "continue") to reset the conversation turn. This works because the new user message ensures the history no longer ends with an assistant message.

Activity

  1. added
    coreAnything pertaining to core functionality of the application (opencode server stuff)
    on Mar 28, 2026
  2. github-actions commented on Mar 28, 2026

    @github-actions
    Contributor

    This issue might be a duplicate of existing issues. Please check:

  3. stefanc-makersite commented on Mar 28, 2026

    @stefanc-makersite
    Author

    Relationship to flagged duplicates

    Checked the linked issues and the two open PRs that fix the loop exit condition:

    Both PRs are solid and would fix the infinite loop trigger (especially in opencode web with clock skew). However, neither PR addresses the two code paths described in this issue:

    1. Aborted text-only assistant messages — when abort happens after text streaming starts but before tool calls, the aborted message stays in history ending the sequence with role:assistant. This is in message-v2.ts:690-698 and is unrelated to the loop exit condition.

    2. isLastStep unconditional prefill injection — prompt.ts:700-710 unconditionally appends {role:"assistant", content: MAX_STEPS} at the step limit. This is valid for direct Anthropic API (which supports prefill) but rejected by Azure AI Claude / Bedrock / any provider behind LiteLLM that doesn't support assistant message prefill.

    So even with #17010 and #17149 merged, providers that don't support prefill will still hit "This model does not support assistant message prefill" through these two paths. The fix for both would be stripping/replacing trailing assistant messages in normalizeMessages() (provider/transform.ts:49) for non-prefill providers — the codebase already has similar provider-specific message fixups for Gemini and Mistral.

  4. bussnet commented on Apr 1, 2026

    @bussnet

    PR #20414 partially addresses this. It fixes root cause #1 (empty/aborted assistant messages surviving) by unifying the Anthropic detection logic — normalizeMessages() now strips empty content for all Claude models regardless of SDK package, not just @ai-sdk/anthropic.

    Root cause #2 (isLastStep prefill injection in prompt.ts L700-710) is a separate code path that unconditionally appends a trailing assistant message and is not addressed by this PR.

  5. removed
    coreAnything pertaining to core functionality of the application (opencode server stuff)
    on May 3, 2026
  6. github-actions commented on Jul 3, 2026

    @github-actions
    Contributor

    To stay organized issues are automatically closed after 60 days of no activity. If the issue is still relevant please open a new one.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions