Skip to content

fix(llm): preserve Gemini thought signatures on tool calls - #1031

Open
xh123637 wants to merge 1 commit into
dataelement:mainfrom
xh123637:fix/1028-gemini-thought-signature
Open

xh123637 wants to merge 1 commit into
dataelement:mainfrom
xh123637:fix/1028-gemini-thought-signature

Conversation

@xh123637

@xh123637 xh123637 commented Nov 3, 2026

Copy link
Copy Markdown

Summary

Fixes #1028. With a Gemini 3 model on the native Gemini provider, the first tool call succeeds and the follow-up request fails with HTTP 400: Google returns a thoughtSignature as a sibling of each functionCall part and requires it back, unchanged, in the same part on the next request — but Clawith dropped it at every step:

  • GeminiClient._parse_response_data copied only keys from inside functionCall into _gemini_extra; the part-level thoughtSignature never reached the normalized tool call.
  • GeminiClient.stream() collapsed all parts' signatures into one response-level reasoning_signature that is not tied to any tool call.
  • _sanitize_tool_calls_for_context (caller.py) and the durable runtime (model_step_service.py) rebuild tool calls from id/name/arguments only, so the signature also vanished between steps.
  • GeminiClient._build_payload never emitted a part-level thoughtSignature (it merged _gemini_extra into the functionCall object instead).

Changes

The signature now travels on the normalized tool call as _gemini_thought_signature, next to the existing _gemini_extra:

  • backend/app/services/llm/client.py
    • _parse_response_data stores the part-level thoughtSignature on the tool call.
    • stream() binds the signature of the part that carries a functionCall to that tool call (the response-level reasoning_signature behaviour is unchanged).
    • _build_payload emits {"functionCall": {...}, "thoughtSignature": ...} as a part-level sibling when the tool call carries a signature.
  • backend/app/services/llm/caller.py — _sanitize_tool_calls_for_context carries _gemini_thought_signature exactly like _gemini_extra.
  • backend/app/services/agent_runtime/model_step_service.py — _with_call_instances stores the signatures keyed by Call Instance ID in additional_kwargs["provider_thought_signatures"] (mirroring provider_call_ids, and only when at least one signature exists, so stored messages for other providers keep their exact previous shape); _prompt_messages re-attaches the signature to the rebuilt tool call by Call Instance ID.

Tool calls without a signature serialize byte-identically to before.

Scope note: this covers the native Gemini provider from the issue's reproduction steps. The OpenAI-compatible Gemini endpoint (.../v1beta/openai/), where the signature travels as extra_content.google.thought_signature on OpenAI-shaped tool calls, loses it in a different place (stream aggregation in OpenAICompatibleClient) and is not changed here.

Tests

New backend/tests/test_llm_gemini_thought_signature.py (9 tests): non-streaming parse keeps the signature on the tool call and the next payload sends it back as a part-level sibling; streaming (httpx MockTransport SSE) binds the signature to its own tool call; the context sanitizer keeps it; the runtime stores it under the Call Instance ID and restores it onto the rebuilt tool call, ending in the final Gemini payload. With the three source files reverted, the 6 behaviour tests fail (the signature is lost end to end, matching the issue) and the 3 guards still pass (a response without a signature gains no key, a payload without a signature is byte-unchanged, runtime storage without signatures keeps its exact previous shape); with the fix, all 9 pass.

Full backend suite (root pytest config): 9 failed / 2564 passed with the fix vs 9 failed / 2555 passed on unmodified main in the same sandbox — the failure sets are identical (pre-existing test_deploy_tools.py environment failures) and the +9 are the new tests. ruff check on the changed files reports the same 38 pre-existing findings before and after; this change adds none.

Gemini returns a thoughtSignature as a sibling of each functionCall
part, and Gemini 3 function calling rejects the follow-up request
with HTTP 400 when the signature is not sent back with the same tool
call. The client dropped it end to end:

- _parse_response_data only copied keys from inside functionCall
  into _gemini_extra, so the part-level thoughtSignature was lost.
- stream() collapsed every part's signature into one response-level
  value that is not tied to any tool call.
- _build_payload never emitted a part-level thoughtSignature.
- The context sanitizer and the durable runtime rebuilt tool calls
  without the signature, so it also vanished across steps.

Keep the signature on the normalized tool call as
_gemini_thought_signature, carry it through the sanitizer, store it
in the runtime keyed by Call Instance ID (provider_thought_signatures,
mirroring provider_call_ids), restore it when the prompt is rebuilt,
and emit it as a part-level sibling of functionCall in the Gemini
payload. Tool calls without a signature serialize exactly as before.

Fixes dataelement#1028

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug] Gemini 3 tool calls fail with HTTP 400: thought signature not sent back

1 participant