Repository navigation
fix(anthropic): stop repeating streamed thinking text in the signature chunk - #44127
Conversation
…e chunk The Anthropic stream handler emitted, on the signature_delta, a thinking block carrying every thinking delta seen so far plus the signature, after it had already streamed that text as per-delta blocks. Every additive consumer (the Agents SDK, stream_chunk_builder, the Responses bridge) stored the text twice under one signature and replayed the doubled block on the next turn The signature chunk now carries a signature-only block, matching the Bedrock emitter, so accumulators rebuild the text once and the saved history replays to Anthropic exactly as it was streamed
|
I'll fix CI failures and address comments from users with write access. I'll skip comments containing "(aside)".
|
|
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
|
bugbot run |
|
bugbot run |
| assert second.status_code == 200, second.text | ||
| second_deltas: Final = deltas_of(chunks_of(second.text)) | ||
| assert content_text(second_deltas) == answer(marker), second.text | ||
| assert all(block == thinking_block(THINKING, SIGNATURE) for block in accumulate(second_deltas)), second.text |
There was a problem hiding this comment.
Cache test can pass without thinking
If the cached stream has no thinking_blocks, accumulate(second_deltas) returns an empty tuple and all(...) still passes. The test would report success even though the cache replay lost the signed thinking it is meant to check. Compare the accumulated blocks with the expected block directly.
| assert all(block == thinking_block(THINKING, SIGNATURE) for block in accumulate(second_deltas)), second.text | |
| assert accumulate(second_deltas) == (thinking_block(THINKING, SIGNATURE),), second.text |
There was a problem hiding this comment.
Direct equality fails because the cached replay drops thinking blocks before this PR, tracked separately. The check now admits only empty or one signed block
There was a problem hiding this comment.
That context makes sense. Since cached stream replays dropping thinking_blocks is a separately tracked pre-existing behavior, requiring the expected block here would make this PR’s test fail for an unrelated issue. The current assertion still rejects the regression this PR targets (a duplicated or otherwise unexpected non-empty block) while allowing the known empty replay, so I’m withdrawing the finding.
|
bugbot run |
There was a problem hiding this comment.
✅ Bugbot reviewed your changes and found no new issues!
Comment @cursor review or bugbot run to trigger another review on this PR
Reviewed by Cursor Bugbot for commit 775fe9f. Configure here.
…106.0) (#443) This PR contains the following updates: | Package | Update | Change | |---|---|---| | [ghcr.io/berriai/litellm](https://images.chainguard.dev/directory/image/wolfi-base/overview) ([source](https://github.com/BerriAI/litellm)) | minor | `v1.105.1` → `v1.106.0` | --- ### Release Notes <details> <summary>BerriAI/litellm (ghcr.io/berriai/litellm)</summary> ### [`v1.106.0`](https://github.com/BerriAI/litellm/releases/tag/v1.106.0) [Compare Source](https://github.com/BerriAI/litellm/compare/v1.105.1...v1.106.0) #### Verify Docker Image Signature All LiteLLM Docker images are signed with [cosign](https://docs.sigstore.dev/cosign/overview/). Every release is signed with the same key introduced in [commit `0112e53`](https://github.com/BerriAI/litellm/commit/0112e53046018d726492c814b3644b7d376029d0). **Verify using the pinned commit hash (recommended):** A commit hash is cryptographically immutable, so this is the strongest way to ensure you are using the original signing key: ```bash cosign verify \ --key https://raw.githubusercontent.com/BerriAI/litellm/0112e53046018d726492c814b3644b7d376029d0/cosign.pub \ ghcr.io/berriai/litellm:v1.106.0 ``` **Verify using the release tag (convenience):** Tags are protected in this repository and resolve to the same key. This option is easier to read but relies on tag protection rules: ```bash cosign verify \ --key https://raw.githubusercontent.com/BerriAI/litellm/v1.106.0/cosign.pub \ ghcr.io/berriai/litellm:v1.106.0 ``` Expected output: ``` The following checks were performed on each of these signatures: - The cosign claims were validated - The signatures were verified against the specified public key ``` *** #### What's Changed - test(mcp): fire MCP client test deadlines on conditions instead of wall-clock time by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44166](https://github.com/BerriAI/litellm/pull/44166) - fix(cost-map): add OpenAI TTS and GPT-5.x deprecation dates by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44175](https://github.com/BerriAI/litellm/pull/44175) - fix(tracing): unify ClickHouse storage configuration by [@​yujonglee-berri](https://github.com/yujonglee-berri) in [#​43941](https://github.com/BerriAI/litellm/pull/43941) - feat(docker): one-command quickstart that starts the gateway, Postgres, and the admin UI by [@​misbahsy](https://github.com/misbahsy) in [#​43673](https://github.com/BerriAI/litellm/pull/43673) - fix(lens): simplify the example investigation preview by [@​moe-berri](https://github.com/moe-berri) in [#​44123](https://github.com/BerriAI/litellm/pull/44123) - chore(helm): drop migrationJob values the chart never reads by [@​Bernedotcom2312](https://github.com/Bernedotcom2312) in [#​42141](https://github.com/BerriAI/litellm/pull/42141) - fix(ui): register tencent in the Add Model provider dropdown by [@​jimaldon](https://github.com/jimaldon) in [#​40924](https://github.com/BerriAI/litellm/pull/40924) - test: remove 130 legacy tests owned by stronger unit proofs by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44157](https://github.com/BerriAI/litellm/pull/44157) - feat(mcp)!: disable stdio MCP servers by default by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​44066](https://github.com/BerriAI/litellm/pull/44066) - fix(proxy): exit when DATABASE_URL is set but the Prisma toolchain is missing by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44207](https://github.com/BerriAI/litellm/pull/44207) - fix(proxy): preserve lifespan state across all entrypoints by [@​tin-berri](https://github.com/tin-berri) in [#​44214](https://github.com/BerriAI/litellm/pull/44214) - fix(oci): resolve the GenAI endpoint realm from the compartment OCID instead of hardcoding oraclecloud.com by [@​fede-kamel](https://github.com/fede-kamel) in [#​43180](https://github.com/BerriAI/litellm/pull/43180) - fix(chatgpt): preserve requested service tier in Responses calls by [@​SteffanWolter](https://github.com/SteffanWolter) in [#​40108](https://github.com/BerriAI/litellm/pull/40108) - fix(ui): leave unset callback select params out of the save payload by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44213](https://github.com/BerriAI/litellm/pull/44213) - fix(proxy-extras): hand libpq a root cert, not Prisma's sslcert, when the migration job builds indexes by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44203](https://github.com/BerriAI/litellm/pull/44203) - feat: add Laya gateway and OSS classifier providers by [@​tin-berri](https://github.com/tin-berri) in [#​43626](https://github.com/BerriAI/litellm/pull/43626) - fix(auto-router): count usage savings by selected UTC request day by [@​tin-berri](https://github.com/tin-berri) in [#​44115](https://github.com/BerriAI/litellm/pull/44115) - fix(lens): preserve framework agent names and GenAI message content by [@​moe-berri](https://github.com/moe-berri) in [#​44218](https://github.com/BerriAI/litellm/pull/44218) - feat(jwt): auto_register_map_existing_key maps JWT to the user's existing virtual key by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​42375](https://github.com/BerriAI/litellm/pull/42375) - feat(ui): select Laya for OSS classification by [@​tin-berri](https://github.com/tin-berri) in [#​43768](https://github.com/BerriAI/litellm/pull/43768) - fix(otel): tolerate non-dict callback_settings.otel and ignore bare EXCLUDED_SERVICES env by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44086](https://github.com/BerriAI/litellm/pull/44086) - fix(proxy): carry key, team and project tags into pass-through spend logs by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​42662](https://github.com/BerriAI/litellm/pull/42662) - feat(ui): add System One (Jev) tab to the playground by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44043](https://github.com/BerriAI/litellm/pull/44043) - chore: bump litellm-proxy-extras 0.4.104 -> 0.4.105 by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44234](https://github.com/BerriAI/litellm/pull/44234) - chore(prices): add xAI grok-voice-transcribe-1.0 deprecation date by [@​berriai-litellm-provider-info-sync](https://github.com/berriai-litellm-provider-info-sync)\[bot] in [#​44237](https://github.com/BerriAI/litellm/pull/44237) - fix(exa): fall back to highlights/summary when text is missing by [@​wakqasahmed](https://github.com/wakqasahmed) in [#​42213](https://github.com/BerriAI/litellm/pull/42213) - fix(scim): apply path-less group PATCH ops instead of storing them under an empty metadata key by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​43978](https://github.com/BerriAI/litellm/pull/43978) - fix(lens): run investigations with configured wildcard models by [@​moe-berri](https://github.com/moe-berri) in [#​44233](https://github.com/BerriAI/litellm/pull/44233) - fix(ui): shrink the sidebar logo so it stops outweighing page titles by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​44247](https://github.com/BerriAI/litellm/pull/44247) - feat(tracing): support claude agent sdk traces with agent name, logo and chat content by [@​ishaan-berri](https://github.com/ishaan-berri) in [#​44248](https://github.com/BerriAI/litellm/pull/44248) - fix(ui): make model leaderboard chart bars wide and readable by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44249](https://github.com/BerriAI/litellm/pull/44249) - test: repair stale and polluting tests red on scheduled main CI by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​44229](https://github.com/BerriAI/litellm/pull/44229) - feat(traces): type queries and align read access with log visibility by [@​yujonglee-berri](https://github.com/yujonglee-berri) in [#​44228](https://github.com/BerriAI/litellm/pull/44228) - chore(ui): untrack tsconfig.tsbuildinfo and gitignore \*.tsbuildinfo by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44261](https://github.com/BerriAI/litellm/pull/44261) - refactor(mcp): extract upstream preparation and support modern clients by [@​joshua-berri](https://github.com/joshua-berri) in [#​44232](https://github.com/BerriAI/litellm/pull/44232) - fix(lens): default to traces and reopen span details by [@​moe-berri](https://github.com/moe-berri) in [#​44260](https://github.com/BerriAI/litellm/pull/44260) - fix(bedrock): surface unrecognized converse-stream event frames instead of an empty turn by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44112](https://github.com/BerriAI/litellm/pull/44112) - perf(proxy): batch daily model usage writes instead of upserting per request by [@​ishaan-berri](https://github.com/ishaan-berri) in [#​44243](https://github.com/BerriAI/litellm/pull/44243) - test(integration): opt the config pass-through spend-log case into auth by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​44265](https://github.com/BerriAI/litellm/pull/44265) - test: fix three order-dependent and timing-flaky tests by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​44271](https://github.com/BerriAI/litellm/pull/44271) - fix(ui): make the Lens traces refresh button always clickable by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44252](https://github.com/BerriAI/litellm/pull/44252) - feat(lens): add sample previews and improve setup and worker feedback by [@​moe-berri](https://github.com/moe-berri) in [#​44268](https://github.com/BerriAI/litellm/pull/44268) - chore(openrouter): sync prices, limits and deprecation dates from the models API by [@​berriai-litellm-provider-info-sync](https://github.com/berriai-litellm-provider-info-sync)\[bot] in [#​44287](https://github.com/BerriAI/litellm/pull/44287) - refactor(traces): type the ClickHouse query help response by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44285](https://github.com/BerriAI/litellm/pull/44285) - fix(proxy): stop queued registry read-throughs spending the resync budget by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​44277](https://github.com/BerriAI/litellm/pull/44277) - feat(lens): show investigation progress as one staged bar with time left by [@​ishaan-berri](https://github.com/ishaan-berri) in [#​44301](https://github.com/BerriAI/litellm/pull/44301) - refactor(ui): compose dashboard pages with shared layouts by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44306](https://github.com/BerriAI/litellm/pull/44306) - fix(proxy): share ownership permissions for spend logs and traces by [@​yujonglee-berri](https://github.com/yujonglee-berri) in [#​44239](https://github.com/BerriAI/litellm/pull/44239) - fix(proxy-extras): retry P3009 when a peer already recovered the named migration row by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44283](https://github.com/BerriAI/litellm/pull/44283) - fix(bedrock): price amazon nova 2 pro preview at the standard tier by [@​berriai-litellm-provider-info-sync](https://github.com/berriai-litellm-provider-info-sync)\[bot] in [#​44302](https://github.com/BerriAI/litellm/pull/44302) - feat: add Bespoke Nimble gateway and OSS classifier support by [@​tin-berri](https://github.com/tin-berri) in [#​44246](https://github.com/BerriAI/litellm/pull/44246) - fix(anthropic): stop repeating streamed thinking text in the signature chunk by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44127](https://github.com/BerriAI/litellm/pull/44127) - fix(responses): drop bridge-minted reasoning items from OpenAI replays by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44132](https://github.com/BerriAI/litellm/pull/44132) - feat(bedrock): drop lookaround regex patterns from tool schemas for Converse models that reject them by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44138](https://github.com/BerriAI/litellm/pull/44138) - feat(bedrock): serve gpt-5.6+ chat completions natively by default, with chat_completions/ opt-in for gpt-oss and grok by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44307](https://github.com/BerriAI/litellm/pull/44307) - feat(lens): issue briefs with problem, user goal, outcome and test cases by [@​ishaan-berri](https://github.com/ishaan-berri) in [#​44311](https://github.com/BerriAI/litellm/pull/44311) - fix(responses): keep gpt-5.4/5.5 tool calls on chat and merge bridged tool calls into one choice by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44295](https://github.com/BerriAI/litellm/pull/44295) - fix(cost): price rule-only model names at the deployment's rate by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44144](https://github.com/BerriAI/litellm/pull/44144) - fix(auto-router): attribute router-day savings without spend logs or a session by [@​tin-berri](https://github.com/tin-berri) in [#​44226](https://github.com/BerriAI/litellm/pull/44226) - test(integration): let run.py select cells by pytest node id by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44319](https://github.com/BerriAI/litellm/pull/44319) - fix(bedrock): honor unsupported reasoning effort levels for OpenAI GPT models by [@​Aasif-Multani](https://github.com/Aasif-Multani) in [#​44183](https://github.com/BerriAI/litellm/pull/44183) - feat(lens): add preset watch-for checks to investigation setup by [@​ishaan-berri](https://github.com/ishaan-berri) in [#​44313](https://github.com/BerriAI/litellm/pull/44313) - fix(proxy): use OpenAI workload identity federation tokens on /openai_passthrough by [@​6matt](https://github.com/6matt) in [#​44140](https://github.com/BerriAI/litellm/pull/44140) - feat(interactions): durable cross-pod settlement for background interaction billing by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​41955](https://github.com/BerriAI/litellm/pull/41955) - fix(streaming): keep the finish_reason of the provider's last content chunk in the logged response by [@​joeym82956](https://github.com/joeym82956) in [#​44318](https://github.com/BerriAI/litellm/pull/44318) - fix(bedrock): send the anthropic-workspace-id header to Bedrock Mantle by [@​kohjunhao](https://github.com/kohjunhao) in [#​44173](https://github.com/BerriAI/litellm/pull/44173) - fix(guardrails): run unified guardrails on /v1/images/edits by [@​michelligabriele](https://github.com/michelligabriele) in [#​44195](https://github.com/BerriAI/litellm/pull/44195) - feat(scaleway): add rerank support by [@​ankit373](https://github.com/ankit373) in [#​44160](https://github.com/BerriAI/litellm/pull/44160) - fix(vector_stores): return managed file ids from vector store file list by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​43800](https://github.com/BerriAI/litellm/pull/43800) - fix(proxy): return 4xx instead of 500 for missing required params, invalid pagination and unknown ids by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​43787](https://github.com/BerriAI/litellm/pull/43787) - fix(guardrails): encrypt guardrail litellm_params secrets at rest by [@​yucheng-berri](https://github.com/yucheng-berri) in [#​43627](https://github.com/BerriAI/litellm/pull/43627) - fix(search_tools): encrypt search tool litellm_params at rest by [@​yucheng-berri](https://github.com/yucheng-berri) in [#​43631](https://github.com/BerriAI/litellm/pull/43631) - feat(roi): add GitLab sources and branch cost attribution by [@​moe-berri](https://github.com/moe-berri) in [#​44324](https://github.com/BerriAI/litellm/pull/44324) - fix(model-prices): mark azure us/eu responses-only models as mode responses by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44323](https://github.com/BerriAI/litellm/pull/44323) - revert(cost): revert "fix(cost): price rule-only model names at the deployment's rate" ([#​44144](https://github.com/BerriAI/litellm/issues/44144)) by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44335](https://github.com/BerriAI/litellm/pull/44335) - fix(proxy): stamp the client alias on a copy of each streamed chunk so pricing sees the deployment model by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44341](https://github.com/BerriAI/litellm/pull/44341) - revert(responses): revert "fix(responses): keep gpt-5.4/5.5 tool calls on chat and merge bridged tool calls into one choice" ([#​44295](https://github.com/BerriAI/litellm/issues/44295)) by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44344](https://github.com/BerriAI/litellm/pull/44344) - refactor(dashboard): migrate to zod 4 and openai 6 by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44345](https://github.com/BerriAI/litellm/pull/44345) - feat(ui): move LiteAdmin into the header with a docked side panel by [@​tin-berri](https://github.com/tin-berri) in [#​44293](https://github.com/BerriAI/litellm/pull/44293) - fix(responses): merge bridged tool calls into the same choice as the text by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44346](https://github.com/BerriAI/litellm/pull/44346) - feat(ui): improve trace inspection and ROI estimation by [@​moe-berri](https://github.com/moe-berri) in [#​44351](https://github.com/BerriAI/litellm/pull/44351) - refactor(ui): reorganize Lens dashboard components by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44354](https://github.com/BerriAI/litellm/pull/44354) - refactor(ui): inject Lens backends as services instead of a fake HTTP client by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44369](https://github.com/BerriAI/litellm/pull/44369) - refactor: clean up fresh tech debt from 2026-10-02 by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44362](https://github.com/BerriAI/litellm/pull/44362) - feat(traces): tracing development seed by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44363](https://github.com/BerriAI/litellm/pull/44363) - refactor(types): replace Any with proven types in 4 files by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44370](https://github.com/BerriAI/litellm/pull/44370) - build(deps): suppress unfixed braces GHSA-vfj7-8cjw-p6xm to clear osv-scan by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44347](https://github.com/BerriAI/litellm/pull/44347) - fix(azure): add MAI-Image max_input_tokens from models sold directly page by [@​berriai-litellm-provider-info-sync](https://github.com/berriai-litellm-provider-info-sync)\[bot] in [#​44375](https://github.com/BerriAI/litellm/pull/44375) - fix(proxy): emit postgres service spans only on real DB reads in auth cache helpers by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44148](https://github.com/BerriAI/litellm/pull/44148) - fix(otel): nest cache spans under their operation and name service spans by purpose by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44150](https://github.com/BerriAI/litellm/pull/44150) - fix(otel): name postgres service spans by operation and table by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44240](https://github.com/BerriAI/litellm/pull/44240) - perf(proxy): stop prompt-cache eligibility from tokenizing the whole conversation by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44221](https://github.com/BerriAI/litellm/pull/44221) - fix(proxy): keep the database error when the log_db_metrics failure hook raises by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44383](https://github.com/BerriAI/litellm/pull/44383) - feat(azure): add azure_ai/kimi-k2-thinking from Azure Kimi pricing page by [@​berriai-litellm-provider-info-sync](https://github.com/berriai-litellm-provider-info-sync)\[bot] in [#​44382](https://github.com/BerriAI/litellm/pull/44382) - feat(harness): add Harness.TOOL_LOOP, a minimal in-process tool-calling loop by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44391](https://github.com/BerriAI/litellm/pull/44391) - feat(decisions): add unified /v1/decisions endpoint for Jev-compatible providers by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44236](https://github.com/BerriAI/litellm/pull/44236) - fix(lens): paginate trace reads within ClickHouse limits by [@​moe-berri](https://github.com/moe-berri) in [#​44384](https://github.com/BerriAI/litellm/pull/44384) - feat(lens): live dot field timeline and full-screen traces view by [@​ishaan-berri](https://github.com/ishaan-berri) in [#​44390](https://github.com/BerriAI/litellm/pull/44390) - feat(ui): make LiteAdmin enterprise-only by [@​tin-berri](https://github.com/tin-berri) in [#​44399](https://github.com/BerriAI/litellm/pull/44399) - fix(bedrock): stop emitting Converse cachePoint blocks for Kimi K3 by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44292](https://github.com/BerriAI/litellm/pull/44292) - test(cost): pin OpenAI reported web search count with mixed actions by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44414](https://github.com/BerriAI/litellm/pull/44414) - fix(ci): give the pass-through auth regression test a real FastAPI app by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44403](https://github.com/BerriAI/litellm/pull/44403) - fix(azure): add azure_ai/flux.2-pro input token limit from the models sold directly page by [@​berriai-litellm-provider-info-sync](https://github.com/berriai-litellm-provider-info-sync)\[bot] in [#​44405](https://github.com/BerriAI/litellm/pull/44405) - fix(lens): batch run reads and reset trace pagination by [@​moe-berri](https://github.com/moe-berri) in [#​44398](https://github.com/BerriAI/litellm/pull/44398) - fix(lens): preserve full trace access and expose investigation failures by [@​moe-berri](https://github.com/moe-berri) in [#​44406](https://github.com/BerriAI/litellm/pull/44406) - feat: add make lens-dev for one-command lens local dev by [@​ishaan-berri](https://github.com/ishaan-berri) in [#​44413](https://github.com/BerriAI/litellm/pull/44413) - fix(bedrock): honor the per-request timeout on Converse and Invoke streaming (internal copy of [#​38210](https://github.com/BerriAI/litellm/issues/38210)) by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44134](https://github.com/BerriAI/litellm/pull/44134) - feat(traces): render curated SQL examples from shared files by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44388](https://github.com/BerriAI/litellm/pull/44388) - fix(model_prices): registry audit 2026-10-03, add cohere embed v5, grok-imagine-video-1.5-lite and openrouter gpt-image rows by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44376](https://github.com/BerriAI/litellm/pull/44376) - refactor(traces): extract snapshot cache by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44424](https://github.com/BerriAI/litellm/pull/44424) - feat(lens): always-on investigations with findings and investigations tables by [@​ishaan-berri](https://github.com/ishaan-berri) in [#​44418](https://github.com/BerriAI/litellm/pull/44418) - feat(docker): add a Windows quickstart in PowerShell, and a styled terminal for both quickstarts by [@​misbahsy](https://github.com/misbahsy) in [#​44310](https://github.com/BerriAI/litellm/pull/44310) - feat(proxy): serve a Codex-native model catalog with per-model service_tiers from /v1/models by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44136](https://github.com/BerriAI/litellm/pull/44136) - refactor(types): replace Any with proven types in 9 files by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44389](https://github.com/BerriAI/litellm/pull/44389) - fix(proxy): treat Postgres connection exhaustion as DB unavailable, not poison spend-log rows by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44270](https://github.com/BerriAI/litellm/pull/44270) - fix(proxy): treat Postgres connection exhaustion as backpressure, not poison rows by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44266](https://github.com/BerriAI/litellm/pull/44266) - fix(lens): release budget reservations when the analysis model call fails by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44431](https://github.com/BerriAI/litellm/pull/44431) - fix(mcp): preserve upstream tool schemas and parameter headers by [@​joshua-berri](https://github.com/joshua-berri) in [#​44425](https://github.com/BerriAI/litellm/pull/44425) - fix(bedrock_mantle): route Claude chat completions to the native Messages endpoint by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​43646](https://github.com/BerriAI/litellm/pull/43646) - fix(tracing): preserve spend identity and gateway correlation by [@​yujonglee-berri](https://github.com/yujonglee-berri) in [#​44421](https://github.com/BerriAI/litellm/pull/44421) - feat(roi): measure shipping velocity, quality, and recorded spend by [@​moe-berri](https://github.com/moe-berri) in [#​44426](https://github.com/BerriAI/litellm/pull/44426) - fix(azure): set text-embedding max input to 8192 from the models sold directly page by [@​berriai-litellm-provider-info-sync](https://github.com/berriai-litellm-provider-info-sync)\[bot] in [#​44449](https://github.com/BerriAI/litellm/pull/44449) - feat(ui): show invitation and reset password links in a copyable field by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​44454](https://github.com/BerriAI/litellm/pull/44454) - feat(lens): coordinate worker releases and bundled installs by [@​moe-berri](https://github.com/moe-berri) in [#​44428](https://github.com/BerriAI/litellm/pull/44428) - fix(auth): resolve hidden model_group_alias entries in the zero-cost budget check by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​43741](https://github.com/BerriAI/litellm/pull/43741) - fix(vertex_ai): forward system and tools to partner model count_tokens by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​43900](https://github.com/BerriAI/litellm/pull/43900) - fix(traces): reject conflicting spend aliases and unrelated HTTP siblings by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44456](https://github.com/BerriAI/litellm/pull/44456) - chore(cost-map): add azure_ai/kimi-k2-thinking retirement date from the Azure retired models page by [@​berriai-litellm-provider-info-sync](https://github.com/berriai-litellm-provider-info-sync)\[bot] in [#​44455](https://github.com/BerriAI/litellm/pull/44455) - fix(health): probe Bedrock Mantle Claude deployments over the Anthropic Messages API by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44419](https://github.com/BerriAI/litellm/pull/44419) - fix(tests): match the OS bind error in the owned-proxy port-race retry by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44462](https://github.com/BerriAI/litellm/pull/44462) - feat(anthropic): workload identity federation and pluggable identity sources by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44448](https://github.com/BerriAI/litellm/pull/44448) - test(integration): exact four-part translation cases on a shared fake provider and shared YAML deployment by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44451](https://github.com/BerriAI/litellm/pull/44451) - feat(roi): default people and branch lists to matched accounts by [@​moe-berri](https://github.com/moe-berri) in [#​44465](https://github.com/BerriAI/litellm/pull/44465) - feat(enterprise): bundle LiteAdmin Slack with native gateway login by [@​tin-berri](https://github.com/tin-berri) in [#​44444](https://github.com/BerriAI/litellm/pull/44444) - fix(caching): count tool_call cache_control marks in the injection census by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​43556](https://github.com/BerriAI/litellm/pull/43556) - chore(cost-map): sync openrouter prices from the models API by [@​berriai-litellm-provider-info-sync](https://github.com/berriai-litellm-provider-info-sync)\[bot] in [#​44466](https://github.com/BerriAI/litellm/pull/44466) - fix(lens): preserve approved worker digests and harden its image by [@​moe-berri](https://github.com/moe-berri) in [#​44467](https://github.com/BerriAI/litellm/pull/44467) - feat: improve Lens dev seeding and live UI by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44468](https://github.com/BerriAI/litellm/pull/44468) - test(ui): wait for step search value to settle in TraceDrawer test by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44470](https://github.com/BerriAI/litellm/pull/44470) - feat(ui): share trace drawer as a closable SidePanel and polish Lens by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44473](https://github.com/BerriAI/litellm/pull/44473) - fix(lens): align source setup with available worker images by [@​moe-berri](https://github.com/moe-berri) in [#​44476](https://github.com/BerriAI/litellm/pull/44476) - feat(sdk): add run_tool_loop and arun_tool_loop helpers by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44381](https://github.com/BerriAI/litellm/pull/44381) - feat(lens): guide setup through the first investigation by [@​moe-berri](https://github.com/moe-berri) in [#​44475](https://github.com/BerriAI/litellm/pull/44475) - test(ci): fix six CircleCI test regressions on main by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44429](https://github.com/BerriAI/litellm/pull/44429) - fix(proxy-extras): log v1 migration failures at ERROR so LITELLM_LOG=ERROR shows them by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44202](https://github.com/BerriAI/litellm/pull/44202) - feat(ui): inline Lens settings tab and investigation editor by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44479](https://github.com/BerriAI/litellm/pull/44479) - feat(mcp): hand listed-tool description and input schema to pre-call hooks per caller by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​41162](https://github.com/BerriAI/litellm/pull/41162) - refactor: clean up fresh tech debt from 2026-10-03 by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44484](https://github.com/BerriAI/litellm/pull/44484) - fix(otel): read the registered v2 logger without importing the proxy by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44485](https://github.com/BerriAI/litellm/pull/44485) - test(mcp): keep the SSO assertion round trip from matching its refresh token inside random ciphertext by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44494](https://github.com/BerriAI/litellm/pull/44494) - test(ci): pin the ROI estimator flag and the prompt-cache counter in two drifted tests by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44499](https://github.com/BerriAI/litellm/pull/44499) - refactor(ui): move TraceView into components/lens/traces by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44501](https://github.com/BerriAI/litellm/pull/44501) - refactor(ui): rename view_logs to logs and split request, audit and detail by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44505](https://github.com/BerriAI/litellm/pull/44505) - refactor(ui): share CopyButton between Lens traces and logs by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44513](https://github.com/BerriAI/litellm/pull/44513) - refactor(lens): storage-independent trace reads, shared keyset pager, typed read failures by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44422](https://github.com/BerriAI/litellm/pull/44422) - fix(ci): send the Lens preview body as selection in the tracing endpoint tests by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44525](https://github.com/BerriAI/litellm/pull/44525) - fix(azure): set gpt-realtime-2.1 retirement dates from the retirement schedule by [@​berriai-litellm-provider-info-sync](https://github.com/berriai-litellm-provider-info-sync)\[bot] in [#​44526](https://github.com/BerriAI/litellm/pull/44526) - fix(ci): repair the security sweep and Lens billing integration tests for ROI, JWKS, and release identity changes by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44529](https://github.com/BerriAI/litellm/pull/44529) - fix(dashboard): route dev API calls on Accept and fail fast on non-JSON 2xx by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44528](https://github.com/BerriAI/litellm/pull/44528) - chore(cost-map): sync openrouter prices from the models API by [@​berriai-litellm-provider-info-sync](https://github.com/berriai-litellm-provider-info-sync)\[bot] in [#​44533](https://github.com/BerriAI/litellm/pull/44533) - refactor(ui): route dashboard URL state through nuqs parsers by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44537](https://github.com/BerriAI/litellm/pull/44537) - fix(logging): deduplicate streaming failure callbacks by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44442](https://github.com/BerriAI/litellm/pull/44442) - feat(proxy): add LITELLM_FIPS_MODE startup gate with provider assertion and loud password migration failure by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​42700](https://github.com/BerriAI/litellm/pull/42700) - fix(guardrails): run the end-of-stream post_call scan when the client disconnects mid-stream by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​43839](https://github.com/BerriAI/litellm/pull/43839) - refactor(mcp): fix Sequence/list return mismatch and collapse record_listed_tools wrapper by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44556](https://github.com/BerriAI/litellm/pull/44556) - chore(cost-map): add azure retirement dates for gpt-4o-realtime-preview-2024-10-01 and jamba-instruct by [@​berriai-litellm-provider-info-sync](https://github.com/berriai-litellm-provider-info-sync)\[bot] in [#​44567](https://github.com/BerriAI/litellm/pull/44567) - feat(ui): add persistent columns and loading skeletons to Lens runs by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44579](https://github.com/BerriAI/litellm/pull/44579) - fix(ui): remove Top models by task card from Model Leaderboard by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44502](https://github.com/BerriAI/litellm/pull/44502) - refactor(rust): track ClickHouse migrations in a checksummed ledger by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44580](https://github.com/BerriAI/litellm/pull/44580) - refactor(ui): extract shared timeline and time-range controls by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44584](https://github.com/BerriAI/litellm/pull/44584) - fix(azure): set gpt-4o-transcribe retirement date from the retirement schedule by [@​berriai-litellm-provider-info-sync](https://github.com/berriai-litellm-provider-info-sync)\[bot] in [#​44585](https://github.com/BerriAI/litellm/pull/44585) - ci: move Postgres, MCP and Redis suites to CircleCI integration by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44453](https://github.com/BerriAI/litellm/pull/44453) - chore!: retire the integrated ROI calculator by [@​moe-berri](https://github.com/moe-berri) in [#​44477](https://github.com/BerriAI/litellm/pull/44477) - fix(caching): key response cache by router model group in litellm_metadata by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44542](https://github.com/BerriAI/litellm/pull/44542) - fix(auto-router): align preview and serving default resolution by [@​moyai-berriai](https://github.com/moyai-berriai)\[bot] in [#​44436](https://github.com/BerriAI/litellm/pull/44436) - fix(mcp): bind OAuth clients to their upstream issuer by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​37777](https://github.com/BerriAI/litellm/pull/37777) - feat(guardrails): add llm shield pii redaction and rehydration guardrail by [@​ninadphalak](https://github.com/ninadphalak) in [#​42645](https://github.com/BerriAI/litellm/pull/42645) - refactor(tracing): generate existing HTTP request models from Rust schemas by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44591](https://github.com/BerriAI/litellm/pull/44591) - ci: run unit selections from GHA test-path and drop the CircleCI unit jobs by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44461](https://github.com/BerriAI/litellm/pull/44461) - fix(ui): make mobile sidebar and top bar responsive by [@​moe-berri](https://github.com/moe-berri) in [#​44603](https://github.com/BerriAI/litellm/pull/44603) - refactor(types): replace Any with proven types in 137 files by [@​mateo-berri](https://github.com/mateo-berri) in [#​44478](https://github.com/BerriAI/litellm/pull/44478) - fix(ui): restore inline Lens onboarding and responsive layout by [@​moe-berri](https://github.com/moe-berri) in [#​44604](https://github.com/BerriAI/litellm/pull/44604) - test(integration): anthropic-route basic translation cases for six Claude models on messages, chat completions and responses by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44607](https://github.com/BerriAI/litellm/pull/44607) - fix(responses): report truncated bridged output as incomplete and echo request params by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44460](https://github.com/BerriAI/litellm/pull/44460) - fix(proxy): enforce internal-user model creation prohibition by [@​moyai-berriai](https://github.com/moyai-berriai)\[bot] in [#​44438](https://github.com/BerriAI/litellm/pull/44438) - chore(harness): drop the unused Any import left in harness options by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44625](https://github.com/BerriAI/litellm/pull/44625) - fix(harness): drop the unused Any import that fails ruff on main by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44627](https://github.com/BerriAI/litellm/pull/44627) - refactor(proxy): rename management/teams/access.py to authz.py by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​44624](https://github.com/BerriAI/litellm/pull/44624) - feat(tracing)!: return only data from SQL queries by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44609](https://github.com/BerriAI/litellm/pull/44609) - fix(openai/realtime): drop model from upstream URL for intent=transcription by [@​michelligabriele](https://github.com/michelligabriele) in [#​43854](https://github.com/BerriAI/litellm/pull/43854) - fix(gemini): add priority tier audio input price to three Gemini rows by [@​berriai-litellm-provider-info-sync](https://github.com/berriai-litellm-provider-info-sync)\[bot] in [#​44632](https://github.com/BerriAI/litellm/pull/44632) - fix(cost): bill per-second transcription models outside chat modes by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44458](https://github.com/BerriAI/litellm/pull/44458) - fix(ui): explain why team member reset spend is unavailable instead of hiding it by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44629](https://github.com/BerriAI/litellm/pull/44629) - test(integration): bedrock_converse-route basic translation cases for six Claude models on messages, chat completions and responses by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44646](https://github.com/BerriAI/litellm/pull/44646) - test(integration): run the anthropic /v1/responses basic translation cases now that the bridge echoes request params by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44615](https://github.com/BerriAI/litellm/pull/44615) - feat(lens): restore compact navigation with live investigation review by [@​ishaan-berri](https://github.com/ishaan-berri) in [#​44472](https://github.com/BerriAI/litellm/pull/44472) - feat(proxy): limit which models an end user can call by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​43904](https://github.com/BerriAI/litellm/pull/43904) - fix(sso): let CLI and Claude Code gateway sign-in through on DISABLE_ADMIN_UI nodes by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44620](https://github.com/BerriAI/litellm/pull/44620) - feat(ui): declare shared search operators and flush queries on blur by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44665](https://github.com/BerriAI/litellm/pull/44665) - fix(lens): make tool steps and conversations readable by [@​moe-berri](https://github.com/moe-berri) in [#​44645](https://github.com/BerriAI/litellm/pull/44645) - fix(ui): send null instead of $0 when a team's member default budget is cleared by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​44644](https://github.com/BerriAI/litellm/pull/44644) - test(integration): azure_ai-route basic translation cases for six Claude models on messages, chat completions and responses by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44658](https://github.com/BerriAI/litellm/pull/44658) - test(integration): basic translation cases for the openai_responses route by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44663](https://github.com/BerriAI/litellm/pull/44663) - feat(lens): analyze trace workspaces with confined Python and compaction by [@​moe-berri](https://github.com/moe-berri) in [#​44640](https://github.com/BerriAI/litellm/pull/44640) - fix(vertex-ai): add deprecation_date to gemini-3.1-flash-lite-image by [@​berriai-litellm-provider-info-sync](https://github.com/berriai-litellm-provider-info-sync)\[bot] in [#​44639](https://github.com/BerriAI/litellm/pull/44639) - fix(bedrock): add priority and flex prices for Grok 4.3, 4.6, 4.7 and Kimi K3 by [@​berriai-litellm-provider-info-sync](https://github.com/berriai-litellm-provider-info-sync)\[bot] in [#​44670](https://github.com/BerriAI/litellm/pull/44670) - test(integration): pin the Claude-tokenizer recount for streamed claude-sonnet-5 no-usage cases by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44643](https://github.com/BerriAI/litellm/pull/44643) - feat(proxy): embed enterprise LiteAdmin MCP in LiteLLM images by [@​tin-berri](https://github.com/tin-berri) in [#​44610](https://github.com/BerriAI/litellm/pull/44610) - test(integration): basic translation cases for the azure route by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44672](https://github.com/BerriAI/litellm/pull/44672) - test(integration): assert the Messages API health probe on Mantle Claude by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44612](https://github.com/BerriAI/litellm/pull/44612) - refactor(rust): rename litellm-framing crate to litellm-framer by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44683](https://github.com/BerriAI/litellm/pull/44683) - test(integration): basic translation cases for the gemini route by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44662](https://github.com/BerriAI/litellm/pull/44662) - test(integration): basic translation cases for the openai route by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44667](https://github.com/BerriAI/litellm/pull/44667) - refactor(rust): derive string enum serde through strum and serde_with by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44675](https://github.com/BerriAI/litellm/pull/44675) - fix(spend-tracking): stop caching failed spend-log metadata lookups as confirmed misses by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​43560](https://github.com/BerriAI/litellm/pull/43560) - fix(proxy): let a listed team alias win over a same-named key alias in the customer model check by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44677](https://github.com/BerriAI/litellm/pull/44677) - chore: move PR template to PULL_REQUEST_TEMPLATE/general.md, add rust.md by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44693](https://github.com/BerriAI/litellm/pull/44693) - fix(router): retry a /v1/messages stream the provider drops before the first content chunk by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44276](https://github.com/BerriAI/litellm/pull/44276) - fix(lens): persist final coverage with source diagnostics by [@​moe-berri](https://github.com/moe-berri) in [#​44682](https://github.com/BerriAI/litellm/pull/44682) - test(integration): point the scratch upgraded proxy's read replica at the scratch database by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44613](https://github.com/BerriAI/litellm/pull/44613) - fix(lens): show investigation findings for agent traces by [@​moe-berri](https://github.com/moe-berri) in [#​44696](https://github.com/BerriAI/litellm/pull/44696) - fix(build): rebuild the Rust bridge when uv sync sees Rust sources change by [@​nate-berri](https://github.com/nate-berri) in [#​44708](https://github.com/BerriAI/litellm/pull/44708) - fix(lens): preserve span timestamps in investigation evidence by [@​moe-berri](https://github.com/moe-berri) in [#​44702](https://github.com/BerriAI/litellm/pull/44702) - ci: split slow unit shards and build the Rust bridge once per run by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44622](https://github.com/BerriAI/litellm/pull/44622) - test(rust_bridge): expect PartRow start_time and end_time as required LENS_CONTENT fields by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44719](https://github.com/BerriAI/litellm/pull/44719) - chore(deps): bump langgraph-sdk from 0.4.2 to 0.4.4 by [@​dependabot](https://github.com/dependabot)\[bot] in [#​44713](https://github.com/BerriAI/litellm/pull/44713) - feat: add Reka as an OpenAI-compatible provider by [@​anniejeng](https://github.com/anniejeng) in [#​44278](https://github.com/BerriAI/litellm/pull/44278) - feat(ui): support native decisions endpoint in decision playground by [@​moyai-berriai](https://github.com/moyai-berriai)\[bot] in [#​44664](https://github.com/BerriAI/litellm/pull/44664) - fix(lens): reconstruct native coding agent conversations by [@​moe-berri](https://github.com/moe-berri) in [#​44711](https://github.com/BerriAI/litellm/pull/44711) - fix(responses): honor request cache controls on chat completions bridged to the Responses API by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44676](https://github.com/BerriAI/litellm/pull/44676) - fix(deps): bump source-map-js, smol-toml, mako, multidict and werkzeug for OSV advisories by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44728](https://github.com/BerriAI/litellm/pull/44728) - test(integration): rename translation runner run to assert_translation by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44741](https://github.com/BerriAI/litellm/pull/44741) - feat(lens): show investigation names in findings table by [@​moe-berri](https://github.com/moe-berri) in [#​44753](https://github.com/BerriAI/litellm/pull/44753) - feat(ui): configure cache-aware auto routing by [@​tin-berri](https://github.com/tin-berri) in [#​43396](https://github.com/BerriAI/litellm/pull/43396) - fix(lens): bound result recovery and preserve partial results by [@​moe-berri](https://github.com/moe-berri) in [#​44692](https://github.com/BerriAI/litellm/pull/44692) - feat(prometheus): cap series per metric for every labeled metric by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44420](https://github.com/BerriAI/litellm/pull/44420) - feat(lens): type to filter the traces agent dropdown by [@​ishaan-berri](https://github.com/ishaan-berri) in [#​44745](https://github.com/BerriAI/litellm/pull/44745) - test(lens): use current worker protocol in billing integration by [@​moe-berri](https://github.com/moe-berri) in [#​44760](https://github.com/BerriAI/litellm/pull/44760) - feat(model_prices): add chatgpt subscription rows for the gpt-6 family by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44758](https://github.com/BerriAI/litellm/pull/44758) - test(e2e): assert the sibling-replica cooldown through the router by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44706](https://github.com/BerriAI/litellm/pull/44706) - test(integration): basic translation cases for the bedrock_invoke route by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44752](https://github.com/BerriAI/litellm/pull/44752) - fix(gemini): stop replaying thinking block signatures to Gemini by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44661](https://github.com/BerriAI/litellm/pull/44661) - fix(ci): namespace claude session ids in tracing seeds and allowlist /v1/logs on backend by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44761](https://github.com/BerriAI/litellm/pull/44761) - fix(proxy): judge the free-model budget waiver by the group an alias routes to by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44638](https://github.com/BerriAI/litellm/pull/44638) - test(integration): basic translation cases for the vertex_ai route by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44751](https://github.com/BerriAI/litellm/pull/44751) - fix(vertex_ai): apply regional endpoint uplift on image generation cost path by [@​nate-berri](https://github.com/nate-berri) in [#​44679](https://github.com/BerriAI/litellm/pull/44679) - refactor(tests): extract Rust cache test split from [#​44714](https://github.com/BerriAI/litellm/issues/44714) by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44780](https://github.com/BerriAI/litellm/pull/44780) - feat(proxy): cap batch file records, daily batch uploads, and per-file downloads by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​43632](https://github.com/BerriAI/litellm/pull/43632) - feat(lens): add copy link button to trace header by [@​ishaan-berri](https://github.com/ishaan-berri) in [#​44749](https://github.com/BerriAI/litellm/pull/44749) - test(integration): keep scripted upstream connections open past the proxy's keepalive by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44695](https://github.com/BerriAI/litellm/pull/44695) - test(integration): basic translation cases for the bedrock_mantle route by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44750](https://github.com/BerriAI/litellm/pull/44750) - fix(chatgpt,github_copilot): refuse device-code login inside an event loop or worker thread by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​39585](https://github.com/BerriAI/litellm/pull/39585) - feat(bedrock): add glm 5.3 cross-region rows and nova 2.5 sonic by [@​berriai-litellm-provider-info-sync](https://github.com/berriai-litellm-provider-info-sync)\[bot] in [#​44710](https://github.com/BerriAI/litellm/pull/44710) - fix(responses): honor caller stream flag when provider forces SSE (internal copy of [#​34095](https://github.com/BerriAI/litellm/issues/34095)) by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​41235](https://github.com/BerriAI/litellm/pull/41235) - refactor(cache): remove dead Cache.\_native_cache runtime path by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44785](https://github.com/BerriAI/litellm/pull/44785) - fix(vertex_ai): add regional endpoint uplift to gemini-3.1-flash-image by [@​nate-berri](https://github.com/nate-berri) in [#​44673](https://github.com/BerriAI/litellm/pull/44673) - feat(ui): lead the Lens investigation detail with a run report by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44786](https://github.com/BerriAI/litellm/pull/44786) - fix(ui): polish Lens runs loading, reload, and time range menu by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44789](https://github.com/BerriAI/litellm/pull/44789) - refactor(types): move SpanAttributes, SpecialHeaders and AllowedModelRegion out of proxy.\_types by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44717](https://github.com/BerriAI/litellm/pull/44717) - refactor(types): import proxy-only types under TYPE_CHECKING in SDK modules by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44740](https://github.com/BerriAI/litellm/pull/44740) - refactor(logging): load enterprise alerting loggers lazily so import litellm skips proxy types by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44762](https://github.com/BerriAI/litellm/pull/44762) - perf(types): defer pydantic schema builds via shared LiteLLMBaseModel by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44720](https://github.com/BerriAI/litellm/pull/44720) - fix(router): price a model group from the deployments that serve it by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44732](https://github.com/BerriAI/litellm/pull/44732) - refactor: remove fresh tech debt from the 2026-10-05 window by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44821](https://github.com/BerriAI/litellm/pull/44821) - fix(azure): add gpt-6-sol priority processing prices by [@​berriai-litellm-provider-info-sync](https://github.com/berriai-litellm-provider-info-sync)\[bot] in [#​44829](https://github.com/BerriAI/litellm/pull/44829) - refactor(types): replace Any with proven types in 6 files by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44491](https://github.com/BerriAI/litellm/pull/44491) - test: deflake the JEV classifier select and two router tests that inherited leaked state by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44840](https://github.com/BerriAI/litellm/pull/44840) - refactor(rust): rename litellm-core to litellm-inference by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44802](https://github.com/BerriAI/litellm/pull/44802) - refactor(rust): extract inference-transcription crate by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44809](https://github.com/BerriAI/litellm/pull/44809) - refactor(rust): extract inference-responses crate by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44811](https://github.com/BerriAI/litellm/pull/44811) - refactor(rust): extract inference-messages crate by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44818](https://github.com/BerriAI/litellm/pull/44818) - refactor(rust): extract inference-chat crate by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44827](https://github.com/BerriAI/litellm/pull/44827) - refactor(rust): extract inference-ocr crate by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44832](https://github.com/BerriAI/litellm/pull/44832) - chore(rust): prune inference deps and rewrite layering docs by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44836](https://github.com/BerriAI/litellm/pull/44836) - fix(otel): name an Arize project on every OTel v2 Arize export by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44605](https://github.com/BerriAI/litellm/pull/44605) - fix(otel): gate the Arize OTel v2 exporter on operator credentials by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44596](https://github.com/BerriAI/litellm/pull/44596) - fix(otel): honor per-team Arize sampling rates in OTel v2 fan-out by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44595](https://github.com/BerriAI/litellm/pull/44595) - fix(model_prices): gemini deep research input limits, vertex flash retirement dates, azure data zone gpt-6.1-sol pricing by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44633](https://github.com/BerriAI/litellm/pull/44633) - refactor(types): replace Any with proven types in 157 files by [@​mateo-berri](https://github.com/mateo-berri) in [#​44798](https://github.com/BerriAI/litellm/pull/44798) - fix(sap): correctly handle cache_control by [@​yamaceay](https://github.com/yamaceay) in [#​39122](https://github.com/BerriAI/litellm/pull/39122) - fix(responses): stop SDK retries nesting under router retries, and unbreak CircleCI integration tests by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44791](https://github.com/BerriAI/litellm/pull/44791) - fix(cost-map): update together_ai Kimi-K3 and Qwen3.8-Flash prices to published rates by [@​berriai-litellm-provider-info-sync](https://github.com/berriai-litellm-provider-info-sync)\[bot] in [#​44864](https://github.com/BerriAI/litellm/pull/44864) - fix(ui): restore key activity search to the top of the tab and add model activity search by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44521](https://github.com/BerriAI/litellm/pull/44521) - refactor(rust): extract inference-testing crate by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44873](https://github.com/BerriAI/litellm/pull/44873) - fix(ui): read MCP submission rules from bare-array /config/list response by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44648](https://github.com/BerriAI/litellm/pull/44648) - chore(model_prices): remove malformed, duplicate and decommissioned palm cost map entries by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44880](https://github.com/BerriAI/litellm/pull/44880) - fix(azure): add model router flat fee to azure provider cost tracking by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44876](https://github.com/BerriAI/litellm/pull/44876) - chore(pricing): add azure model-router and whisper rows by [@​berriai-litellm-provider-info-sync](https://github.com/berriai-litellm-provider-info-sync)\[bot] in [#​44872](https://github.com/BerriAI/litellm/pull/44872) - fix(security): remove the publicly known master key from the repo by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44718](https://github.com/BerriAI/litellm/pull/44718) - chore(prices): add gemini/gemini-nano-banana-2.1 from the Gemini pricing page by [@​berriai-litellm-provider-info-sync](https://github.com/berriai-litellm-provider-info-sync)\[bot] in [#​44868](https://github.com/BerriAI/litellm/pull/44868) - fix(ci): skip the generated dashboard bundle in the master key guard by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44890](https://github.com/BerriAI/litellm/pull/44890) - fix(model_prices): correct supported_endpoints on gemini and vertex image rows by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44891](https://github.com/BerriAI/litellm/pull/44891) - feat(vertex-ai): add gemini-nano-banana-2.1 and Nano Banana 2 priority pricing by [@​berriai-litellm-provider-info-sync](https://github.com/berriai-litellm-provider-info-sync)\[bot] in [#​44892](https://github.com/BerriAI/litellm/pull/44892) - fix(lens): reuse trace reviews and consolidate findings across runs by [@​moe-berri](https://github.com/moe-berri) in [#​44778](https://github.com/BerriAI/litellm/pull/44778) - fix(lens): paginate visible conversation entries by [@​moe-berri](https://github.com/moe-berri) in [#​44878](https://github.com/BerriAI/litellm/pull/44878) - refactor(rust_…
….106.4 ) (#3677)
This PR contains the following updates:
| Package | Update | Change |
|---|---|---|
| [ghcr.io/berriai/litellm](https://images.chainguard.dev/directory/image/wolfi-base/overview) ([source](https://github.com/BerriAI/litellm)) | minor | `v1.105.1` → `v1.106.4` |
---
> ⚠️ **Warning**
>
> Some dependencies could not be looked up. Check the [Dependency Dashboard](issues/3566) for more information.
---
### Release Notes
<details>
<summary>BerriAI/litellm (ghcr.io/berriai/litellm)</summary>
### [`v1.106.4`](https://github.com/BerriAI/litellm/releases/tag/v1.106.4)
[Compare Source](https://github.com/BerriAI/litellm/compare/v1.106.3...v1.106.4)
#### Verify this release
```bash
cosign verify \
--key https://raw.githubusercontent.com/BerriAI/litellm/e82a3c646908ba0a6ee0beddc4310d7a6cac4065/cosign.pub \
ghcr.io/berriai/litellm:v1.106.4
```
Same command for `litellm-database`, `litellm-non_root` and the `docker.io/litellm/*` images
#### What's new in 1.106.4
3 PRs. 2 fixed. No action needed to upgrade
##### Changes
- **Deployment:** Fixed the proxy sometimes exiting on startup with an `ImportError` when it cannot reach the remote model cost map, broken since v1.104.0 ([#​46711](https://github.com/BerriAI/litellm/issues/46711)) <sup>Also in [1.104.7](https://github.com/BerriAI/litellm/releases/tag/v1.104.7), [1.105.5](https://github.com/BerriAI/litellm/releases/tag/v1.105.5)</sup>
- **Deployment:** Fixed image scans flagging brotli 1.2.0, the emSign ECC Root CA - G3 and unused lockfile packages ([#​48283](https://github.com/BerriAI/litellm/issues/48283), [#​48300](https://github.com/BerriAI/litellm/issues/48300))
***
Backport [#​48294](https://github.com/BerriAI/litellm/issues/48294) · [Full changelog](https://github.com/BerriAI/litellm/compare/v1.106.3...v1.106.4)
### [`v1.106.3`](https://github.com/BerriAI/litellm/releases/tag/v1.106.3)
[Compare Source](https://github.com/BerriAI/litellm/compare/v1.106.2...v1.106.3)
> \[!IMPORTANT]
> **The image-signing key has changed.** Starting with this release, every Docker image and Helm chart is signed with the new cosign key from commit [`e82a3c6`](https://github.com/BerriAI/litellm/commit/e82a3c646908ba0a6ee0beddc4310d7a6cac4065). Verifying with the previous key from commit `0112e53` will fail.
>
> Update anything that pins the old `cosign.pub`: `cosign verify` scripts, CI checks, and admission policies such as Kyverno, Sigstore policy-controller or Connaisseur.
>
> New `cosign.pub` SHA-256: `e3fe3e08c44b16407fe0b37c43b4f8b603b3a0f4ee74c29e647bc022d762c2c0`
#### Verify this release
```bash
cosign verify \
--key https://raw.githubusercontent.com/BerriAI/litellm/e82a3c646908ba0a6ee0beddc4310d7a6cac4065/cosign.pub \
ghcr.io/berriai/litellm:v1.106.3
```
Same command for `litellm-database`, `litellm-non_root` and the `docker.io/litellm/*` images
#### What's new in 1.106.3
4 PRs. 1 changed, 1 removed. 1 needs action
##### Action required
- **Deployment:** Verify image signatures with the `cosign.pub` from commit `e82a3c6`, the previous key no longer matches ([#​48103](https://github.com/BerriAI/litellm/issues/48103)) <sup>Also in [1.103.8](https://github.com/BerriAI/litellm/releases/tag/v1.103.8), [1.104.6](https://github.com/BerriAI/litellm/releases/tag/v1.104.6), [1.105.4](https://github.com/BerriAI/litellm/releases/tag/v1.105.4)</sup>
##### Changes
- **Deployment:** Removed the eMudhra root CA from every certificate bundle the SDK, proxy and Docker images trust ([#​47670](https://github.com/BerriAI/litellm/issues/47670), [#​47969](https://github.com/BerriAI/litellm/issues/47969), [#​48139](https://github.com/BerriAI/litellm/issues/48139)) <sup>Also in [1.103.8](https://github.com/BerriAI/litellm/releases/tag/v1.103.8), [1.104.6](https://github.com/BerriAI/litellm/releases/tag/v1.104.6), [1.105.4](https://github.com/BerriAI/litellm/releases/tag/v1.105.4)</sup>
- **Deployment:** Changed the cosign key that signs Docker images and Helm charts to the one from commit `e82a3c6` ([#​48103](https://github.com/BerriAI/litellm/issues/48103)) <sup>Also in [1.103.8](https://github.com/BerriAI/litellm/releases/tag/v1.103.8), [1.104.6](https://github.com/BerriAI/litellm/releases/tag/v1.104.6), [1.105.4](https://github.com/BerriAI/litellm/releases/tag/v1.105.4)</sup>
***
Backport [#​48078](https://github.com/BerriAI/litellm/issues/48078) · Backport [#​48142](https://github.com/BerriAI/litellm/issues/48142) · [Full changelog](https://github.com/BerriAI/litellm/compare/v1.106.2...v1.106.3)
### [`v1.106.2`](https://github.com/BerriAI/litellm/releases/tag/v1.106.2)
[Compare Source](https://github.com/BerriAI/litellm/compare/v1.106.1...v1.106.2)
#### Verify Docker Image Signature
All LiteLLM Docker images are signed with [cosign](https://docs.sigstore.dev/cosign/overview/). Every release is signed with the same key introduced in [commit `0112e53`](https://github.com/BerriAI/litellm/commit/0112e53046018d726492c814b3644b7d376029d0).
**Verify using the pinned commit hash (recommended):**
A commit hash is cryptographically immutable, so this is the strongest way to ensure you are using the original signing key:
```bash
cosign verify \
--key https://raw.githubusercontent.com/BerriAI/litellm/0112e53046018d726492c814b3644b7d376029d0/cosign.pub \
ghcr.io/berriai/litellm:v1.106.2
```
**Verify using the release tag (convenience):**
Tags are protected in this repository and resolve to the same key. This option is easier to read but relies on tag protection rules:
```bash
cosign verify \
--key https://raw.githubusercontent.com/BerriAI/litellm/v1.106.2/cosign.pub \
ghcr.io/berriai/litellm:v1.106.2
```
Expected output:
```
The following checks were performed on each of these signatures:
- The cosign claims were validated
- The signatures were verified against the specified public key
```
***
#### What's Changed
- fix(count_tokens): backport Bedrock Mantle 404 fallback to stable/1.106.x by [@​mateo-berri](https://github.com/mateo-berri) in [#​47742](https://github.com/BerriAI/litellm/pull/47742)
- fix(batches)!: backport P0 batch-cost, pricing and translation fixes to stable/1.106.x for v1.106.2 by [@​mrinal-berri](https://github.com/mrinal-berri) in [#​47736](https://github.com/BerriAI/litellm/pull/47736)
- fix(proxy): backport LIT-145 spend and cost fixes to stable/1.106.x for v1.106.2 by [@​kerry-berri](https://github.com/kerry-berri) in [#​47695](https://github.com/BerriAI/litellm/pull/47695)
**Full Changelog**: <https://github.com/BerriAI/litellm/compare/v1.106.1...v1.106.2>
### [`v1.106.1`](https://github.com/BerriAI/litellm/releases/tag/v1.106.1)
[Compare Source](https://github.com/BerriAI/litellm/compare/v1.106.0...v1.106.1)
#### Verify this release
```bash
cosign verify \
--key https://raw.githubusercontent.com/BerriAI/litellm/0112e53046018d726492c814b3644b7d376029d0/cosign.pub \
ghcr.io/berriai/litellm:v1.106.1
```
Same command for `litellm-database`, `litellm-non_root` and the `docker.io/litellm/*` images
#### What's new in 1.106.1
9 PRs. 6 fixed. 1 needs action
##### Action required
- **Vector stores:** Run `db_scripts/strip_vector_store_internal_metadata.py` to fix stores created before this release ([#​47021](https://github.com/BerriAI/litellm/issues/47021))
##### Changes
- **Anthropic:** Fixed signed thinking blocks and `thinking.block_binding` being dropped on multi-turn requests ([#​41362](https://github.com/BerriAI/litellm/issues/41362), [#​46342](https://github.com/BerriAI/litellm/issues/46342)) <sup>Also in [1.105.2](https://github.com/BerriAI/litellm/releases/tag/v1.105.2)</sup>
- **Chat completions:** Fixed tool calls landing on `choices[1]` with gpt-5.4+ models, broken since v1.97.0 ([#​46099](https://github.com/BerriAI/litellm/issues/46099)) <sup>Also in [1.105.2](https://github.com/BerriAI/litellm/releases/tag/v1.105.2)</sup>
- **Batches and files:** Fixed 403 when team keys retrieve, cancel or download their own batches ([#​47067](https://github.com/BerriAI/litellm/issues/47067)) <sup>Also in [1.105.2](https://github.com/BerriAI/litellm/releases/tag/v1.105.2)</sup>
- **Vector stores:** Fixed HTTP 400 on metadata updates caused by keys the proxy added to the store ([#​47021](https://github.com/BerriAI/litellm/issues/47021))
- **Token counting:** Fixed undercounted PDFs, system prompts and tools, and rejected Google `countTokens` requests ([#​47389](https://github.com/BerriAI/litellm/issues/47389), [#​47446](https://github.com/BerriAI/litellm/issues/47446), [#​47454](https://github.com/BerriAI/litellm/issues/47454)) <sup>Also in [1.104.4](https://github.com/BerriAI/litellm/releases/tag/v1.104.4), [1.105.2](https://github.com/BerriAI/litellm/releases/tag/v1.105.2)</sup>
- **MCP:** Fixed `Tool not found` when calling a bare tool name on `/{server}/mcp`, broken since v1.103.0 ([#​46225](https://github.com/BerriAI/litellm/issues/46225))
***
Backport [#​47517](https://github.com/BerriAI/litellm/issues/47517) · [Full changelog](https://github.com/BerriAI/litellm/compare/v1.106.0...v1.106.1)
### [`v1.106.0`](https://github.com/BerriAI/litellm/releases/tag/v1.106.0)
[Compare Source](https://github.com/BerriAI/litellm/compare/v1.105.5...v1.106.0)
#### Verify Docker Image Signature
All LiteLLM Docker images are signed with [cosign](https://docs.sigstore.dev/cosign/overview/). Every release is signed with the same key introduced in [commit `0112e53`](https://github.com/BerriAI/litellm/commit/0112e53046018d726492c814b3644b7d376029d0).
**Verify using the pinned commit hash (recommended):**
A commit hash is cryptographically immutable, so this is the strongest way to ensure you are using the original signing key:
```bash
cosign verify \
--key https://raw.githubusercontent.com/BerriAI/litellm/0112e53046018d726492c814b3644b7d376029d0/cosign.pub \
ghcr.io/berriai/litellm:v1.106.0
```
**Verify using the release tag (convenience):**
Tags are protected in this repository and resolve to the same key. This option is easier to read but relies on tag protection rules:
```bash
cosign verify \
--key https://raw.githubusercontent.com/BerriAI/litellm/v1.106.0/cosign.pub \
ghcr.io/berriai/litellm:v1.106.0
```
Expected output:
```
The following checks were performed on each of these signatures:
- The cosign claims were validated
- The signatures were verified against the specified public key
```
***
#### What's Changed
- test(mcp): fire MCP client test deadlines on conditions instead of wall-clock time by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44166](https://github.com/BerriAI/litellm/pull/44166)
- fix(cost-map): add OpenAI TTS and GPT-5.x deprecation dates by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44175](https://github.com/BerriAI/litellm/pull/44175)
- fix(tracing): unify ClickHouse storage configuration by [@​yujonglee-berri](https://github.com/yujonglee-berri) in [#​43941](https://github.com/BerriAI/litellm/pull/43941)
- feat(docker): one-command quickstart that starts the gateway, Postgres, and the admin UI by [@​misbahsy](https://github.com/misbahsy) in [#​43673](https://github.com/BerriAI/litellm/pull/43673)
- fix(lens): simplify the example investigation preview by [@​moe-berri](https://github.com/moe-berri) in [#​44123](https://github.com/BerriAI/litellm/pull/44123)
- chore(helm): drop migrationJob values the chart never reads by [@​Bernedotcom2312](https://github.com/Bernedotcom2312) in [#​42141](https://github.com/BerriAI/litellm/pull/42141)
- fix(ui): register tencent in the Add Model provider dropdown by [@​jimaldon](https://github.com/jimaldon) in [#​40924](https://github.com/BerriAI/litellm/pull/40924)
- test: remove 130 legacy tests owned by stronger unit proofs by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44157](https://github.com/BerriAI/litellm/pull/44157)
- feat(mcp)!: disable stdio MCP servers by default by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​44066](https://github.com/BerriAI/litellm/pull/44066)
- fix(proxy): exit when DATABASE_URL is set but the Prisma toolchain is missing by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44207](https://github.com/BerriAI/litellm/pull/44207)
- fix(proxy): preserve lifespan state across all entrypoints by [@​tin-berri](https://github.com/tin-berri) in [#​44214](https://github.com/BerriAI/litellm/pull/44214)
- fix(oci): resolve the GenAI endpoint realm from the compartment OCID instead of hardcoding oraclecloud.com by [@​fede-kamel](https://github.com/fede-kamel) in [#​43180](https://github.com/BerriAI/litellm/pull/43180)
- fix(chatgpt): preserve requested service tier in Responses calls by [@​SteffanWolter](https://github.com/SteffanWolter) in [#​40108](https://github.com/BerriAI/litellm/pull/40108)
- fix(ui): leave unset callback select params out of the save payload by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44213](https://github.com/BerriAI/litellm/pull/44213)
- fix(proxy-extras): hand libpq a root cert, not Prisma's sslcert, when the migration job builds indexes by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44203](https://github.com/BerriAI/litellm/pull/44203)
- feat: add Laya gateway and OSS classifier providers by [@​tin-berri](https://github.com/tin-berri) in [#​43626](https://github.com/BerriAI/litellm/pull/43626)
- fix(auto-router): count usage savings by selected UTC request day by [@​tin-berri](https://github.com/tin-berri) in [#​44115](https://github.com/BerriAI/litellm/pull/44115)
- fix(lens): preserve framework agent names and GenAI message content by [@​moe-berri](https://github.com/moe-berri) in [#​44218](https://github.com/BerriAI/litellm/pull/44218)
- feat(jwt): auto_register_map_existing_key maps JWT to the user's existing virtual key by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​42375](https://github.com/BerriAI/litellm/pull/42375)
- feat(ui): select Laya for OSS classification by [@​tin-berri](https://github.com/tin-berri) in [#​43768](https://github.com/BerriAI/litellm/pull/43768)
- fix(otel): tolerate non-dict callback_settings.otel and ignore bare EXCLUDED_SERVICES env by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44086](https://github.com/BerriAI/litellm/pull/44086)
- fix(proxy): carry key, team and project tags into pass-through spend logs by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​42662](https://github.com/BerriAI/litellm/pull/42662)
- feat(ui): add System One (Jev) tab to the playground by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44043](https://github.com/BerriAI/litellm/pull/44043)
- chore: bump litellm-proxy-extras 0.4.104 -> 0.4.105 by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44234](https://github.com/BerriAI/litellm/pull/44234)
- chore(prices): add xAI grok-voice-transcribe-1.0 deprecation date by [@​berriai-litellm-provider-info-sync](https://github.com/berriai-litellm-provider-info-sync)\[bot] in [#​44237](https://github.com/BerriAI/litellm/pull/44237)
- fix(exa): fall back to highlights/summary when text is missing by [@​wakqasahmed](https://github.com/wakqasahmed) in [#​42213](https://github.com/BerriAI/litellm/pull/42213)
- fix(scim): apply path-less group PATCH ops instead of storing them under an empty metadata key by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​43978](https://github.com/BerriAI/litellm/pull/43978)
- fix(lens): run investigations with configured wildcard models by [@​moe-berri](https://github.com/moe-berri) in [#​44233](https://github.com/BerriAI/litellm/pull/44233)
- fix(ui): shrink the sidebar logo so it stops outweighing page titles by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​44247](https://github.com/BerriAI/litellm/pull/44247)
- feat(tracing): support claude agent sdk traces with agent name, logo and chat content by [@​ishaan-berri](https://github.com/ishaan-berri) in [#​44248](https://github.com/BerriAI/litellm/pull/44248)
- fix(ui): make model leaderboard chart bars wide and readable by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44249](https://github.com/BerriAI/litellm/pull/44249)
- test: repair stale and polluting tests red on scheduled main CI by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​44229](https://github.com/BerriAI/litellm/pull/44229)
- feat(traces): type queries and align read access with log visibility by [@​yujonglee-berri](https://github.com/yujonglee-berri) in [#​44228](https://github.com/BerriAI/litellm/pull/44228)
- chore(ui): untrack tsconfig.tsbuildinfo and gitignore \*.tsbuildinfo by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44261](https://github.com/BerriAI/litellm/pull/44261)
- refactor(mcp): extract upstream preparation and support modern clients by [@​joshua-berri](https://github.com/joshua-berri) in [#​44232](https://github.com/BerriAI/litellm/pull/44232)
- fix(lens): default to traces and reopen span details by [@​moe-berri](https://github.com/moe-berri) in [#​44260](https://github.com/BerriAI/litellm/pull/44260)
- fix(bedrock): surface unrecognized converse-stream event frames instead of an empty turn by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44112](https://github.com/BerriAI/litellm/pull/44112)
- perf(proxy): batch daily model usage writes instead of upserting per request by [@​ishaan-berri](https://github.com/ishaan-berri) in [#​44243](https://github.com/BerriAI/litellm/pull/44243)
- test(integration): opt the config pass-through spend-log case into auth by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​44265](https://github.com/BerriAI/litellm/pull/44265)
- test: fix three order-dependent and timing-flaky tests by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​44271](https://github.com/BerriAI/litellm/pull/44271)
- fix(ui): make the Lens traces refresh button always clickable by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44252](https://github.com/BerriAI/litellm/pull/44252)
- feat(lens): add sample previews and improve setup and worker feedback by [@​moe-berri](https://github.com/moe-berri) in [#​44268](https://github.com/BerriAI/litellm/pull/44268)
- chore(openrouter): sync prices, limits and deprecation dates from the models API by [@​berriai-litellm-provider-info-sync](https://github.com/berriai-litellm-provider-info-sync)\[bot] in [#​44287](https://github.com/BerriAI/litellm/pull/44287)
- refactor(traces): type the ClickHouse query help response by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44285](https://github.com/BerriAI/litellm/pull/44285)
- fix(proxy): stop queued registry read-throughs spending the resync budget by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​44277](https://github.com/BerriAI/litellm/pull/44277)
- feat(lens): show investigation progress as one staged bar with time left by [@​ishaan-berri](https://github.com/ishaan-berri) in [#​44301](https://github.com/BerriAI/litellm/pull/44301)
- refactor(ui): compose dashboard pages with shared layouts by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44306](https://github.com/BerriAI/litellm/pull/44306)
- fix(proxy): share ownership permissions for spend logs and traces by [@​yujonglee-berri](https://github.com/yujonglee-berri) in [#​44239](https://github.com/BerriAI/litellm/pull/44239)
- fix(proxy-extras): retry P3009 when a peer already recovered the named migration row by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44283](https://github.com/BerriAI/litellm/pull/44283)
- fix(bedrock): price amazon nova 2 pro preview at the standard tier by [@​berriai-litellm-provider-info-sync](https://github.com/berriai-litellm-provider-info-sync)\[bot] in [#​44302](https://github.com/BerriAI/litellm/pull/44302)
- feat: add Bespoke Nimble gateway and OSS classifier support by [@​tin-berri](https://github.com/tin-berri) in [#​44246](https://github.com/BerriAI/litellm/pull/44246)
- fix(anthropic): stop repeating streamed thinking text in the signature chunk by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44127](https://github.com/BerriAI/litellm/pull/44127)
- fix(responses): drop bridge-minted reasoning items from OpenAI replays by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44132](https://github.com/BerriAI/litellm/pull/44132)
- feat(bedrock): drop lookaround regex patterns from tool schemas for Converse models that reject them by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44138](https://github.com/BerriAI/litellm/pull/44138)
- feat(bedrock): serve gpt-5.6+ chat completions natively by default, with chat_completions/ opt-in for gpt-oss and grok by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44307](https://github.com/BerriAI/litellm/pull/44307)
- feat(lens): issue briefs with problem, user goal, outcome and test cases by [@​ishaan-berri](https://github.com/ishaan-berri) in [#​44311](https://github.com/BerriAI/litellm/pull/44311)
- fix(responses): keep gpt-5.4/5.5 tool calls on chat and merge bridged tool calls into one choice by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44295](https://github.com/BerriAI/litellm/pull/44295)
- fix(cost): price rule-only model names at the deployment's rate by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44144](https://github.com/BerriAI/litellm/pull/44144)
- fix(auto-router): attribute router-day savings without spend logs or a session by [@​tin-berri](https://github.com/tin-berri) in [#​44226](https://github.com/BerriAI/litellm/pull/44226)
- test(integration): let run.py select cells by pytest node id by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44319](https://github.com/BerriAI/litellm/pull/44319)
- fix(bedrock): honor unsupported reasoning effort levels for OpenAI GPT models by [@​Aasif-Multani](https://github.com/Aasif-Multani) in [#​44183](https://github.com/BerriAI/litellm/pull/44183)
- feat(lens): add preset watch-for checks to investigation setup by [@​ishaan-berri](https://github.com/ishaan-berri) in [#​44313](https://github.com/BerriAI/litellm/pull/44313)
- fix(proxy): use OpenAI workload identity federation tokens on /openai_passthrough by [@​6matt](https://github.com/6matt) in [#​44140](https://github.com/BerriAI/litellm/pull/44140)
- feat(interactions): durable cross-pod settlement for background interaction billing by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​41955](https://github.com/BerriAI/litellm/pull/41955)
- fix(streaming): keep the finish_reason of the provider's last content chunk in the logged response by [@​joeym82956](https://github.com/joeym82956) in [#​44318](https://github.com/BerriAI/litellm/pull/44318)
- fix(bedrock): send the anthropic-workspace-id header to Bedrock Mantle by [@​kohjunhao](https://github.com/kohjunhao) in [#​44173](https://github.com/BerriAI/litellm/pull/44173)
- fix(guardrails): run unified guardrails on /v1/images/edits by [@​michelligabriele](https://github.com/michelligabriele) in [#​44195](https://github.com/BerriAI/litellm/pull/44195)
- feat(scaleway): add rerank support by [@​ankit373](https://github.com/ankit373) in [#​44160](https://github.com/BerriAI/litellm/pull/44160)
- fix(vector_stores): return managed file ids from vector store file list by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​43800](https://github.com/BerriAI/litellm/pull/43800)
- fix(proxy): return 4xx instead of 500 for missing required params, invalid pagination and unknown ids by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​43787](https://github.com/BerriAI/litellm/pull/43787)
- fix(guardrails): encrypt guardrail litellm_params secrets at rest by [@​yucheng-berri](https://github.com/yucheng-berri) in [#​43627](https://github.com/BerriAI/litellm/pull/43627)
- fix(search_tools): encrypt search tool litellm_params at rest by [@​yucheng-berri](https://github.com/yucheng-berri) in [#​43631](https://github.com/BerriAI/litellm/pull/43631)
- feat(roi): add GitLab sources and branch cost attribution by [@​moe-berri](https://github.com/moe-berri) in [#​44324](https://github.com/BerriAI/litellm/pull/44324)
- fix(model-prices): mark azure us/eu responses-only models as mode responses by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44323](https://github.com/BerriAI/litellm/pull/44323)
- revert(cost): revert "fix(cost): price rule-only model names at the deployment's rate" ([#​44144](https://github.com/BerriAI/litellm/issues/44144)) by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44335](https://github.com/BerriAI/litellm/pull/44335)
- fix(proxy): stamp the client alias on a copy of each streamed chunk so pricing sees the deployment model by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44341](https://github.com/BerriAI/litellm/pull/44341)
- revert(responses): revert "fix(responses): keep gpt-5.4/5.5 tool calls on chat and merge bridged tool calls into one choice" ([#​44295](https://github.com/BerriAI/litellm/issues/44295)) by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44344](https://github.com/BerriAI/litellm/pull/44344)
- refactor(dashboard): migrate to zod 4 and openai 6 by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44345](https://github.com/BerriAI/litellm/pull/44345)
- feat(ui): move LiteAdmin into the header with a docked side panel by [@​tin-berri](https://github.com/tin-berri) in [#​44293](https://github.com/BerriAI/litellm/pull/44293)
- fix(responses): merge bridged tool calls into the same choice as the text by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44346](https://github.com/BerriAI/litellm/pull/44346)
- feat(ui): improve trace inspection and ROI estimation by [@​moe-berri](https://github.com/moe-berri) in [#​44351](https://github.com/BerriAI/litellm/pull/44351)
- refactor(ui): reorganize Lens dashboard components by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44354](https://github.com/BerriAI/litellm/pull/44354)
- refactor(ui): inject Lens backends as services instead of a fake HTTP client by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44369](https://github.com/BerriAI/litellm/pull/44369)
- refactor: clean up fresh tech debt from 2026-10-02 by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44362](https://github.com/BerriAI/litellm/pull/44362)
- feat(traces): tracing development seed by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44363](https://github.com/BerriAI/litellm/pull/44363)
- refactor(types): replace Any with proven types in 4 files by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44370](https://github.com/BerriAI/litellm/pull/44370)
- build(deps): suppress unfixed braces GHSA-vfj7-8cjw-p6xm to clear osv-scan by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44347](https://github.com/BerriAI/litellm/pull/44347)
- fix(azure): add MAI-Image max_input_tokens from models sold directly page by [@​berriai-litellm-provider-info-sync](https://github.com/berriai-litellm-provider-info-sync)\[bot] in [#​44375](https://github.com/BerriAI/litellm/pull/44375)
- fix(proxy): emit postgres service spans only on real DB reads in auth cache helpers by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44148](https://github.com/BerriAI/litellm/pull/44148)
- fix(otel): nest cache spans under their operation and name service spans by purpose by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44150](https://github.com/BerriAI/litellm/pull/44150)
- fix(otel): name postgres service spans by operation and table by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44240](https://github.com/BerriAI/litellm/pull/44240)
- perf(proxy): stop prompt-cache eligibility from tokenizing the whole conversation by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44221](https://github.com/BerriAI/litellm/pull/44221)
- fix(proxy): keep the database error when the log_db_metrics failure hook raises by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44383](https://github.com/BerriAI/litellm/pull/44383)
- feat(azure): add azure_ai/kimi-k2-thinking from Azure Kimi pricing page by [@​berriai-litellm-provider-info-sync](https://github.com/berriai-litellm-provider-info-sync)\[bot] in [#​44382](https://github.com/BerriAI/litellm/pull/44382)
- feat(harness): add Harness.TOOL_LOOP, a minimal in-process tool-calling loop by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44391](https://github.com/BerriAI/litellm/pull/44391)
- feat(decisions): add unified /v1/decisions endpoint for Jev-compatible providers by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44236](https://github.com/BerriAI/litellm/pull/44236)
- fix(lens): paginate trace reads within ClickHouse limits by [@​moe-berri](https://github.com/moe-berri) in [#​44384](https://github.com/BerriAI/litellm/pull/44384)
- feat(lens): live dot field timeline and full-screen traces view by [@​ishaan-berri](https://github.com/ishaan-berri) in [#​44390](https://github.com/BerriAI/litellm/pull/44390)
- feat(ui): make LiteAdmin enterprise-only by [@​tin-berri](https://github.com/tin-berri) in [#​44399](https://github.com/BerriAI/litellm/pull/44399)
- fix(bedrock): stop emitting Converse cachePoint blocks for Kimi K3 by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44292](https://github.com/BerriAI/litellm/pull/44292)
- test(cost): pin OpenAI reported web search count with mixed actions by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44414](https://github.com/BerriAI/litellm/pull/44414)
- fix(ci): give the pass-through auth regression test a real FastAPI app by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44403](https://github.com/BerriAI/litellm/pull/44403)
- fix(azure): add azure_ai/flux.2-pro input token limit from the models sold directly page by [@​berriai-litellm-provider-info-sync](https://github.com/berriai-litellm-provider-info-sync)\[bot] in [#​44405](https://github.com/BerriAI/litellm/pull/44405)
- fix(lens): batch run reads and reset trace pagination by [@​moe-berri](https://github.com/moe-berri) in [#​44398](https://github.com/BerriAI/litellm/pull/44398)
- fix(lens): preserve full trace access and expose investigation failures by [@​moe-berri](https://github.com/moe-berri) in [#​44406](https://github.com/BerriAI/litellm/pull/44406)
- feat: add make lens-dev for one-command lens local dev by [@​ishaan-berri](https://github.com/ishaan-berri) in [#​44413](https://github.com/BerriAI/litellm/pull/44413)
- fix(bedrock): honor the per-request timeout on Converse and Invoke streaming (internal copy of [#​38210](https://github.com/BerriAI/litellm/issues/38210)) by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44134](https://github.com/BerriAI/litellm/pull/44134)
- feat(traces): render curated SQL examples from shared files by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44388](https://github.com/BerriAI/litellm/pull/44388)
- fix(model_prices): registry audit 2026-10-03, add cohere embed v5, grok-imagine-video-1.5-lite and openrouter gpt-image rows by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44376](https://github.com/BerriAI/litellm/pull/44376)
- refactor(traces): extract snapshot cache by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44424](https://github.com/BerriAI/litellm/pull/44424)
- feat(lens): always-on investigations with findings and investigations tables by [@​ishaan-berri](https://github.com/ishaan-berri) in [#​44418](https://github.com/BerriAI/litellm/pull/44418)
- feat(docker): add a Windows quickstart in PowerShell, and a styled terminal for both quickstarts by [@​misbahsy](https://github.com/misbahsy) in [#​44310](https://github.com/BerriAI/litellm/pull/44310)
- feat(proxy): serve a Codex-native model catalog with per-model service_tiers from /v1/models by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44136](https://github.com/BerriAI/litellm/pull/44136)
- refactor(types): replace Any with proven types in 9 files by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44389](https://github.com/BerriAI/litellm/pull/44389)
- fix(proxy): treat Postgres connection exhaustion as DB unavailable, not poison spend-log rows by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44270](https://github.com/BerriAI/litellm/pull/44270)
- fix(proxy): treat Postgres connection exhaustion as backpressure, not poison rows by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44266](https://github.com/BerriAI/litellm/pull/44266)
- fix(lens): release budget reservations when the analysis model call fails by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44431](https://github.com/BerriAI/litellm/pull/44431)
- fix(mcp): preserve upstream tool schemas and parameter headers by [@​joshua-berri](https://github.com/joshua-berri) in [#​44425](https://github.com/BerriAI/litellm/pull/44425)
- fix(bedrock_mantle): route Claude chat completions to the native Messages endpoint by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​43646](https://github.com/BerriAI/litellm/pull/43646)
- fix(tracing): preserve spend identity and gateway correlation by [@​yujonglee-berri](https://github.com/yujonglee-berri) in [#​44421](https://github.com/BerriAI/litellm/pull/44421)
- feat(roi): measure shipping velocity, quality, and recorded spend by [@​moe-berri](https://github.com/moe-berri) in [#​44426](https://github.com/BerriAI/litellm/pull/44426)
- fix(azure): set text-embedding max input to 8192 from the models sold directly page by [@​berriai-litellm-provider-info-sync](https://github.com/berriai-litellm-provider-info-sync)\[bot] in [#​44449](https://github.com/BerriAI/litellm/pull/44449)
- feat(ui): show invitation and reset password links in a copyable field by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​44454](https://github.com/BerriAI/litellm/pull/44454)
- feat(lens): coordinate worker releases and bundled installs by [@​moe-berri](https://github.com/moe-berri) in [#​44428](https://github.com/BerriAI/litellm/pull/44428)
- fix(auth): resolve hidden model_group_alias entries in the zero-cost budget check by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​43741](https://github.com/BerriAI/litellm/pull/43741)
- fix(vertex_ai): forward system and tools to partner model count_tokens by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​43900](https://github.com/BerriAI/litellm/pull/43900)
- fix(traces): reject conflicting spend aliases and unrelated HTTP siblings by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44456](https://github.com/BerriAI/litellm/pull/44456)
- chore(cost-map): add azure_ai/kimi-k2-thinking retirement date from the Azure retired models page by [@​berriai-litellm-provider-info-sync](https://github.com/berriai-litellm-provider-info-sync)\[bot] in [#​44455](https://github.com/BerriAI/litellm/pull/44455)
- fix(health): probe Bedrock Mantle Claude deployments over the Anthropic Messages API by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44419](https://github.com/BerriAI/litellm/pull/44419)
- fix(tests): match the OS bind error in the owned-proxy port-race retry by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44462](https://github.com/BerriAI/litellm/pull/44462)
- feat(anthropic): workload identity federation and pluggable identity sources by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44448](https://github.com/BerriAI/litellm/pull/44448)
- test(integration): exact four-part translation cases on a shared fake provider and shared YAML deployment by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44451](https://github.com/BerriAI/litellm/pull/44451)
- feat(roi): default people and branch lists to matched accounts by [@​moe-berri](https://github.com/moe-berri) in [#​44465](https://github.com/BerriAI/litellm/pull/44465)
- feat(enterprise): bundle LiteAdmin Slack with native gateway login by [@​tin-berri](https://github.com/tin-berri) in [#​44444](https://github.com/BerriAI/litellm/pull/44444)
- fix(caching): count tool_call cache_control marks in the injection census by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​43556](https://github.com/BerriAI/litellm/pull/43556)
- chore(cost-map): sync openrouter prices from the models API by [@​berriai-litellm-provider-info-sync](https://github.com/berriai-litellm-provider-info-sync)\[bot] in [#​44466](https://github.com/BerriAI/litellm/pull/44466)
- fix(lens): preserve approved worker digests and harden its image by [@​moe-berri](https://github.com/moe-berri) in [#​44467](https://github.com/BerriAI/litellm/pull/44467)
- feat: improve Lens dev seeding and live UI by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44468](https://github.com/BerriAI/litellm/pull/44468)
- test(ui): wait for step search value to settle in TraceDrawer test by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44470](https://github.com/BerriAI/litellm/pull/44470)
- feat(ui): share trace drawer as a closable SidePanel and polish Lens by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44473](https://github.com/BerriAI/litellm/pull/44473)
- fix(lens): align source setup with available worker images by [@​moe-berri](https://github.com/moe-berri) in [#​44476](https://github.com/BerriAI/litellm/pull/44476)
- feat(sdk): add run_tool_loop and arun_tool_loop helpers by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44381](https://github.com/BerriAI/litellm/pull/44381)
- feat(lens): guide setup through the first investigation by [@​moe-berri](https://github.com/moe-berri) in [#​44475](https://github.com/BerriAI/litellm/pull/44475)
- test(ci): fix six CircleCI test regressions on main by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44429](https://github.com/BerriAI/litellm/pull/44429)
- fix(proxy-extras): log v1 migration failures at ERROR so LITELLM_LOG=ERROR shows them by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44202](https://github.com/BerriAI/litellm/pull/44202)
- feat(ui): inline Lens settings tab and investigation editor by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44479](https://github.com/BerriAI/litellm/pull/44479)
- feat(mcp): hand listed-tool description and input schema to pre-call hooks per caller by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​41162](https://github.com/BerriAI/litellm/pull/41162)
- refactor: clean up fresh tech debt from 2026-10-03 by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44484](https://github.com/BerriAI/litellm/pull/44484)
- fix(otel): read the registered v2 logger without importing the proxy by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44485](https://github.com/BerriAI/litellm/pull/44485)
- test(mcp): keep the SSO assertion round trip from matching its refresh token inside random ciphertext by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44494](https://github.com/BerriAI/litellm/pull/44494)
- test(ci): pin the ROI estimator flag and the prompt-cache counter in two drifted tests by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44499](https://github.com/BerriAI/litellm/pull/44499)
- refactor(ui): move TraceView into components/lens/traces by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44501](https://github.com/BerriAI/litellm/pull/44501)
- refactor(ui): rename view_logs to logs and split request, audit and detail by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44505](https://github.com/BerriAI/litellm/pull/44505)
- refactor(ui): share CopyButton between Lens traces and logs by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44513](https://github.com/BerriAI/litellm/pull/44513)
- refactor(lens): storage-independent trace reads, shared keyset pager, typed read failures by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44422](https://github.com/BerriAI/litellm/pull/44422)
- fix(ci): send the Lens preview body as selection in the tracing endpoint tests by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44525](https://github.com/BerriAI/litellm/pull/44525)
- fix(azure): set gpt-realtime-2.1 retirement dates from the retirement schedule by [@​berriai-litellm-provider-info-sync](https://github.com/berriai-litellm-provider-info-sync)\[bot] in [#​44526](https://github.com/BerriAI/litellm/pull/44526)
- fix(ci): repair the security sweep and Lens billing integration tests for ROI, JWKS, and release identity changes by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44529](https://github.com/BerriAI/litellm/pull/44529)
- fix(dashboard): route dev API calls on Accept and fail fast on non-JSON 2xx by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44528](https://github.com/BerriAI/litellm/pull/44528)
- chore(cost-map): sync openrouter prices from the models API by [@​berriai-litellm-provider-info-sync](https://github.com/berriai-litellm-provider-info-sync)\[bot] in [#​44533](https://github.com/BerriAI/litellm/pull/44533)
- refactor(ui): route dashboard URL state through nuqs parsers by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44537](https://github.com/BerriAI/litellm/pull/44537)
- fix(logging): deduplicate streaming failure callbacks by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44442](https://github.com/BerriAI/litellm/pull/44442)
- feat(proxy): add LITELLM_FIPS_MODE startup gate with provider assertion and loud password migration failure by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​42700](https://github.com/BerriAI/litellm/pull/42700)
- fix(guardrails): run the end-of-stream post_call scan when the client disconnects mid-stream by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​43839](https://github.com/BerriAI/litellm/pull/43839)
- refactor(mcp): fix Sequence/list return mismatch and collapse record_listed_tools wrapper by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44556](https://github.com/BerriAI/litellm/pull/44556)
- chore(cost-map): add azure retirement dates for gpt-4o-realtime-preview-2024-10-01 and jamba-instruct by [@​berriai-litellm-provider-info-sync](https://github.com/berriai-litellm-provider-info-sync)\[bot] in [#​44567](https://github.com/BerriAI/litellm/pull/44567)
- feat(ui): add persistent columns and loading skeletons to Lens runs by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44579](https://github.com/BerriAI/litellm/pull/44579)
- fix(ui): remove Top models by task card from Model Leaderboard by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44502](https://github.com/BerriAI/litellm/pull/44502)
- refactor(rust): track ClickHouse migrations in a checksummed ledger by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44580](https://github.com/BerriAI/litellm/pull/44580)
- refactor(ui): extract shared timeline and time-range controls by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44584](https://github.com/BerriAI/litellm/pull/44584)
- fix(azure): set gpt-4o-transcribe retirement date from the retirement schedule by [@​berriai-litellm-provider-info-sync](https://github.com/berriai-litellm-provider-info-sync)\[bot] in [#​44585](https://github.com/BerriAI/litellm/pull/44585)
- ci: move Postgres, MCP and Redis suites to CircleCI integration by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44453](https://github.com/BerriAI/litellm/pull/44453)
- chore!: retire the integrated ROI calculator by [@​moe-berri](https://github.com/moe-berri) in [#​44477](https://github.com/BerriAI/litellm/pull/44477)
- fix(caching): key response cache by router model group in litellm_metadata by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44542](https://github.com/BerriAI/litellm/pull/44542)
- fix(auto-router): align preview and serving default resolution by [@​moyai-berriai](https://github.com/moyai-berriai)\[bot] in [#​44436](https://github.com/BerriAI/litellm/pull/44436)
- fix(mcp): bind OAuth clients to their upstream issuer by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​37777](https://github.com/BerriAI/litellm/pull/37777)
- feat(guardrails): add llm shield pii redaction and rehydration guardrail by [@​ninadphalak](https://github.com/ninadphalak) in [#​42645](https://github.com/BerriAI/litellm/pull/42645)
- refactor(tracing): generate existing HTTP request models from Rust schemas by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44591](https://github.com/BerriAI/litellm/pull/44591)
- ci: run unit selections from GHA test-path and drop the CircleCI unit jobs by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44461](https://github.com/BerriAI/litellm/pull/44461)
- fix(ui): make mobile sidebar and top bar responsive by [@​moe-berri](https://github.com/moe-berri) in [#​44603](https://github.com/BerriAI/litellm/pull/44603)
- refactor(types): replace Any with proven types in 137 files by [@​mateo-berri](https://github.com/mateo-berri) in [#​44478](https://github.com/BerriAI/litellm/pull/44478)
- fix(ui): restore inline Lens onboarding and responsive layout by [@​moe-berri](https://github.com/moe-berri) in [#​44604](https://github.com/BerriAI/litellm/pull/44604)
- test(integration): anthropic-route basic translation cases for six Claude models on messages, chat completions and responses by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44607](https://github.com/BerriAI/litellm/pull/44607)
- fix(responses): report truncated bridged output as incomplete and echo request params by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44460](https://github.com/BerriAI/litellm/pull/44460)
- fix(proxy): enforce internal-user model creation prohibition by [@​moyai-berriai](https://github.com/moyai-berriai)\[bot] in [#​44438](https://github.com/BerriAI/litellm/pull/44438)
- chore(harness): drop the unused Any import left in harness options by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44625](https://github.com/BerriAI/litellm/pull/44625)
- fix(harness): drop the unused Any import that fails ruff on main by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44627](https://github.com/BerriAI/litellm/pull/44627)
- refactor(proxy): rename management/teams/access.py to authz.py by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​44624](https://github.com/BerriAI/litellm/pull/44624)
- feat(tracing)!: return only data from SQL queries by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44609](https://github.com/BerriAI/litellm/pull/44609)
- fix(openai/realtime): drop model from upstream URL for intent=transcription by [@​michelligabriele](https://github.com/michelligabriele) in [#​43854](https://github.com/BerriAI/litellm/pull/43854)
- fix(gemini): add priority tier audio input price to three Gemini rows by [@​berriai-litellm-provider-info-sync](https://github.com/berriai-litellm-provider-info-sync)\[bot] in [#​44632](https://github.com/BerriAI/litellm/pull/44632)
- fix(cost): bill per-second transcription models outside chat modes by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44458](https://github.com/BerriAI/litellm/pull/44458)
- fix(ui): explain why team member reset spend is unavailable instead of hiding it by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44629](https://github.com/BerriAI/litellm/pull/44629)
- test(integration): bedrock_converse-route basic translation cases for six Claude models on messages, chat completions and responses by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44646](https://github.com/BerriAI/litellm/pull/44646)
- test(integration): run the anthropic /v1/responses basic translation cases now that the bridge echoes request params by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44615](https://github.com/BerriAI/litellm/pull/44615)
- feat(lens): restore compact navigation with live investigation review by [@​ishaan-berri](https://github.com/ishaan-berri) in [#​44472](https://github.com/BerriAI/litellm/pull/44472)
- feat(proxy): limit which models an end user can call by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​43904](https://github.com/BerriAI/litellm/pull/43904)
- fix(sso): let CLI and Claude Code gateway sign-in through on DISABLE_ADMIN_UI nodes by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44620](https://github.com/BerriAI/litellm/pull/44620)
- feat(ui): declare shared search operators and flush queries on blur by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44665](https://github.com/BerriAI/litellm/pull/44665)
- fix(lens): make tool steps and conversations readable by [@​moe-berri](https://github.com/moe-berri) in [#​44645](https://github.com/BerriAI/litellm/pull/44645)
- fix(ui): send null instead of $0 when a team's member default budget is cleared by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​44644](https://github.com/BerriAI/litellm/pull/44644)
- test(integration): azure_ai-route basic translation cases for six Claude models on messages, chat completions and responses by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44658](https://github.com/BerriAI/litellm/pull/44658)
- test(integration): basic translation cases for the openai_responses route by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44663](https://github.com/BerriAI/litellm/pull/44663)
- feat(lens): analyze trace workspaces with confined Python and compaction by [@​moe-berri](https://github.com/moe-berri) in [#​44640](https://github.com/BerriAI/litellm/pull/44640)
- fix(vertex-ai): add deprecation_date to gemini-3.1-flash-lite-image by [@​berriai-litellm-provider-info-sync](https://github.com/berriai-litellm-provider-info-sync)\[bot] in [#​44639](https://github.com/BerriAI/litellm/pull/44639)
- fix(bedrock): add priority and flex prices for Grok 4.3, 4.6, 4.7 and Kimi K3 by [@​berriai-litellm-provider-info-sync](https://github.com/berriai-litellm-provider-info-sync)\[bot] in [#​44670](https://github.com/BerriAI/litellm/pull/44670)
- test(integration): pin the Claude-tokenizer recount for streamed claude-sonnet-5 no-usage cases by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44643](https://github.com/BerriAI/litellm/pull/44643)
- feat(proxy): embed enterprise LiteAdmin MCP in LiteLLM images by [@​tin-berri](https://github.com/tin-berri) in [#​44610](https://github.com/BerriAI/litellm/pull/44610)
- test(integration): basic translation cases for the azure route by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44672](https://github.com/BerriAI/litellm/pull/44672)
- test(integration): assert the Messages API health probe on Mantle Claude by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44612](https://github.com/BerriAI/litellm/pull/44612)
- refactor(rust): rename litellm-framing crate to litellm-framer by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44683](https://github.com/BerriAI/litellm/pull/44683)
- test(integration): basic translation cases for the gemini route by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44662](https://github.com/BerriAI/litellm/pull/44662)
- test(integration): basic translation cases for the openai route by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44667](https://github.com/BerriAI/litellm/pull/44667)
- refactor(rust): derive string enum serde through strum and serde_with by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44675](https://github.com/BerriAI/litellm/pull/44675)
- fix(spend-tracking): stop caching failed spend-log metadata lookups as confirmed misses by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​43560](https://github.com/BerriAI/litellm/pull/43560)
- fix(proxy): let a listed team alias win over a same-named key alias in the customer model check by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44677](https://github.com/BerriAI/litellm/pull/44677)
- chore: move PR template to PULL_REQUEST_TEMPLATE/general.md, add rust.md by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44693](https://github.com/BerriAI/litellm/pull/44693)
- fix(router): retry a /v1/messages stream the provider drops before the first content chunk by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44276](https://github.com/BerriAI/litellm/pull/44276)
- fix(lens): persist final coverage with source diagnostics by [@​moe-berri](https://github.com/moe-berri) in [#​44682](https://github.com/BerriAI/litellm/pull/44682)
- test(integration): point the scratch upgraded proxy's read replica at the scratch database by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44613](https://github.com/BerriAI/litellm/pull/44613)
- fix(lens): show investigation findings for agent traces by [@​moe-berri](https://github.com/moe-berri) in [#​44696](https://github.com/BerriAI/litellm/pull/44696)
- fix(build): rebuild the Rust bridge when uv sync sees Rust sources change by [@​nate-berri](https://github.com/nate-berri) in [#​44708](https://github.com/BerriAI/litellm/pull/44708)
- fix(lens): preserve span timestamps in investigation evidence by [@​moe-berri](https://github.com/moe-berri) in [#​44702](https://github.com/BerriAI/litellm/pull/44702)
- ci: split slow unit shards and build the Rust bridge once per run by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44622](https://github.com/BerriAI/litellm/pull/44622)
- test(rust_bridge): expect PartRow start_time and end_time as required LENS_CONTENT fields by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44719](https://github.com/BerriAI/litellm/pull/44719)
- chore(deps): bump langgraph-sdk from 0.4.2 to 0.4.4 by [@​dependabot](https://github.com/dependabot)\[bot] in [#​44713](https://github.com/BerriAI/litellm/pull/44713)
- feat: add Reka as an OpenAI-compatible provider by [@​anniejeng](https://github.com/anniejeng) in [#​44278](https://github.com/BerriAI/litellm/pull/44278)
- feat(ui): support native decisions endpoint in decision playground by [@​moyai-berriai](https://github.com/moyai-berriai)\[bot] in [#​44664](https://github.com/BerriAI/litellm/pull/44664)
- fix(lens): reconstruct native coding agent conversations by [@​moe-berri](https://github.com/moe-berri) in [#​44711](https://github.com/BerriAI/litellm/pull/44711)
- fix(responses): honor request cache controls on chat completions bridged to the Responses API by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44676](https://github.com/BerriAI/litellm/pull/44676)
- fix(deps): bump source-map-js, smol-toml, mako, multidict and werkzeug for OSV advisories by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44728](https://github.com/BerriAI/litellm/pull/44728)
- test(integration): rename translation runner run to assert_translation by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44741](https://github.com/BerriAI/litellm/pull/44741)
- feat(lens): show investigation names in findings table by [@​moe-berri](https://github.com/moe-berri) in [#​44753](https://github.com/BerriAI/litellm/pull/44753)
- feat(ui): configure cache-aware auto routing by [@​tin-berri](https://github.com/tin-berri) in [#​43396](https://github.com/BerriAI/litellm/pull/43396)
- fix(lens): bound result recovery and preserve partial results by [@​moe-berri](https://github.com/moe-berri) in [#​44692](https://github.com/BerriAI/litellm/pull/44692)
- feat(prometheus): cap series per metric for every labeled metric by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44420](https://github.com/BerriAI/litellm/pull/44420)
- feat(lens): type to filter the traces agent dropdown by [@​ishaan-berri](https://github.com/ishaan-berri) in [#​44745](https://github.com/BerriAI/litellm/pull/44745)
- test(lens): use current worker protocol in billing integration by [@​moe-berri](https://github.com/moe-berri) in [#​44760](https://github.com/BerriAI/litellm/pull/44760)
- feat(model_prices): add chatgpt subscription rows for the gpt-6 family by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44758](https://github.com/BerriAI/litellm/pull/44758)
- test(e2e): assert the sibling-replica cooldown through the router by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44706](https://github.com/BerriAI/litellm/pull/44706)
- test(integration): basic translation cases for the bedrock_invoke route by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44752](https://github.com/BerriAI/litellm/pull/44752)
- fix(gemini): stop replaying thinking block signatures to Gemini by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44661](https://github.com/BerriAI/litellm/pull/44661)
- fix(ci): namespace claude session ids in tracing seeds and allowlist /v1/logs on backend by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44761](https://github.com/BerriAI/litellm/pull/44761)
- fix(proxy): judge the free-model budget waiver by the group an alias routes to by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44638](https://github.com/BerriAI/litellm/pull/44638)
- test(integration): basic translation cases for the vertex_ai route by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44751](https://github.com/BerriAI/litellm/pull/44751)
- fix(vertex_ai): apply regional endpoint uplift on image generation cost path by [@​nate-berri](https://github.com/nate-berri) in [#​44679](https://github.com/BerriAI/litellm/pull/44679)
- refactor(tests): extract Rust cache test split from [#​44714](https://github.com/BerriAI/litellm/issues/44714) by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44780](https://github.com/BerriAI/litellm/pull/44780)
- feat(proxy): cap batch file records, daily batch uploads, and per-file downloads by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​43632](https://github.com/BerriAI/litellm/pull/43632)
- feat(lens): add copy link button to trace header by [@​ishaan-berri](https://github.com/ishaan-berri) in [#​44749](https://github.com/BerriAI/litellm/pull/44749)
- test(integration): keep scripted upstream connections open past the proxy's keepalive by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44695](https://github.com/BerriAI/litellm/pull/44695)
- test(integration): basic translation cases for the bedrock_mantle route by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​44750](https://github.com/BerriAI/litellm/pull/44750)
- fix(chatgpt,github_copilot): refuse device-code login inside an event loop or worker thread by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​39585](https://github.com/BerriAI/litellm/pull/39585)
- feat(bedrock): add glm 5.3 cross-region rows and nova 2.5 sonic by [@​berriai-litellm-provider-info-sync](https://github.com/berriai-litellm-provider-info-sync)\[bot] in [#​44710](https://github.com/BerriAI/litellm/pull/44710)
- fix(responses): honor caller stream flag when provider forces SSE (internal copy of [#​34095](https://github.com/BerriAI/litellm/issues/34095)) by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​41235](https://github.com/BerriAI/litellm/pull/41235)
- refactor(cache): remove dead Cache.\_native_cache run…
TLDR
Problem this solves:
How it solves it:
User Flow
Before: a developer whose Agents SDK app runs Claude with extended thinking saves a tool-call turn and finds the thinking text doubled under one signature
LitellmModel("anthropic/claude-sonnet-4-5"),Reasoning(effort="medium")) streams a Claude turn that thinks, then callsget_weatherandget_local_timefor Parisresult.to_input_list()to JSON and open the reasoning item:thinking_blocks[0].thinkingis twice as long as the thinking they watched stream, the same text repeated once, under one signaturePOST https://api.anthropic.com/v1/messages; Claude accepts it, so every later turn pays input tokens for the thinking twice and any UI of the saved history shows it twicePOST https://litellm-domain/v1/responseswith"stream": true, returns areasoningitem whoseencrypted_contentholds the doubled textAfter: the same turn saves the thinking text exactly once
LitellmModel("anthropic/claude-sonnet-4-5"),Reasoning(effort="medium")) streams a Claude turn that thinks, then callsget_weatherandget_local_timefor Parisresult.to_input_list()to JSON and open the reasoning item:thinking_blocks[0].thinkingequals the thinking they watched stream, once, with the same signaturePOST https://api.anthropic.com/v1/messages; Claude accepts it and the history carries each thought oncePOST https://litellm-domain/v1/responseswith"stream": true, returns areasoningitem whoseencrypted_contentholds the text onceLinear ticket
Resolves LIT-9150
Pre-Submission checklist
Please complete all items before asking a LiteLLM maintainer to review your PR
uv run pytest tests/unit/<your_test_file>.py -v. Leave the suites (make test-unit-*,make test-unit) to CI: it finishes in ~15 minutes where a laptop takes an hour or more@greptileaito re-request a review after pushing changes)Delays in PR merge?
If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).
Screenshots / Proof of Fix
Last updated: 775fe9f. QA and /live-pr-risk ran on 1955444, the last commit with a product diff; the commits since are the audit's tests-only commits 6e40e47 and 775fe9f
QA ran at 1955444, the last commit with a product diff. Before = merge base 3f39fef, After = 1955444. Each leg is its own worktree and venv, its own proxy booted with
litellm --config proxy_config.yaml --host 127.0.0.1 --port <random port> --num_workers 2and no database, real Anthropic calls toclaude-sonnet-4-5, OpenAI Agents SDK 0.22.3 on openai 3.20.0. The scripts and curls below are identical for both legs; only the proxy's commit differsproxy_config.yamlCase 1: Agents SDK
LitellmModel("anthropic/claude-sonnet-4-5"), in processThe app streams a Paris tool-call turn, saves
result.to_input_list()to JSON, reloads it, and runs a Rome turn on top.thinking charsis the savedprovider_data.thinking_blocks[0].thinkingthe SDK will replay to Claude;streamedis the reasoning the user watched arrivecase1_sdk_litellm_model.py
Before (3f39fef): the saved block is the streamed thinking twice, and the Rome turn replays the doubled block
After (1955444): the saved block equals the streamed thinking, once, still signed, and the Rome turn succeeds on it
Case 2: Agents SDK
OpenAIResponsesModelthrough the proxy,POST http://127.0.0.1:<port>/v1/responseswith"stream": trueSame script, with
set_default_openai_client(AsyncOpenAI(base_url="http://127.0.0.1:<port>/v1", api_key=os.environ["LITELLM_MASTER_KEY"])),model="claude-sonnet-4-5", and the savedreasoningitem'sencrypted_contentdecoded withjson.loadsinstead of readingprovider_dataBefore (3f39fef)
After (1955444)
Case 3:
POST /v1/chat/completionsstream with thinking enabledBefore (3f39fef): the signed delta repeats the whole thinking text
After (1955444): the signed delta carries the signature only
Case 4:
POST /v1/messagesstream with thinking enabled (unchanged surface)Before (3f39fef)
After (1955444)
Observations from the legs
/v1/responsesputs reasoning text incontent, notsummary; pre-existing, left alone/v1/responsesemits an empty message item before tool calls; pre-existing, left alone/live-pr-risk at 1955444
Two proxies A/B'd from this PR's merge base 3f39fef and from 1955444, each with its own native Postgres (schema migrated by the proxy's own boot),
--num_workers 2,store_prompts_in_spend_logs: true, a recording reverse proxy in front ofapi.anthropic.comfor the Responses bridge scenario, and real calls to Anthropic, Bedrock, and Vertex AI. Nine scenarios ran on both sides in the same order with the same inputs; spend rows were read back fromGET /spend/logsafter each one. Then the top of the list ran a third time on 1955444 merged withorigin/main(d729f97, five commits of drift,uv.lockincluded): the merge applied clean, and all nine scenarios on the merged tree matched the head side (signed deltas with no text, the spend log'sthinking_blocksholding the text once, the Responses bridge replaying a 354-char block equal to the streamed text, non-stream unchanged, the deleted key answering 401)Breaking
POST /key/deletegot 401 on the base side and 200 on the head side. The two workers each keep an in-memory copy of the key foruser_api_key_cache_ttl(60 s default), the delete clears only the worker that served it, and the follow-up call landed on the other worker on the head side. The diff toucheslitellm/llms/anthropic/chat/handler.pyand its test only, so this is the rig's two-worker setup, not this PRBackward incompatible
thinking_blocksdelta on streamed chat chunks now carriesthinking: ""plus the signature, where it repeated the whole text. Observed on direct Anthropic (408 chars to 0), Bedrock invoke (445 to 0), Vertex AI Claude (429 to 0), and the router fallback replay (429 to 0), each with the per-delta blocks unchanged and the spend log's rebuiltthinking_blocksholding the text once (816 to 439, 890 to 469, 858 to 435, 858 to 437). A client that reads only the signed delta and ignores the deltas before it now gets no text; every client that accumulates deltas (the Agents SDK, the Responses bridge,stream_chunk_builder) gets the text once instead of twice. This is the change the PR ships (1955444); the alternative, text in the signed delta too, is the bug itselfprovider_specific_fields.thinking_blocks(spend logresponse, and whatstream_chunk_builderhands a caller) now mirrors the signature-only chunk instead of the full block: direct chat 408 chars to 0,/v1/messages519 to 0, Bedrock invoke 445 to 0, Vertex 429 to 0, pass-through 343 to 0, fallback 429 to 0. The message's ownthinking_blocks, the field LiteLLM documents for replay, carries the text once in every one of those rows. Left as is: merging the text back into the signature chunk's copy would put the text on the wire twice for every client that accumulates deltas, which is what this PR removes; a consumer ofprovider_specific_fields.thinking_blockson a rebuilt stream was not found in the repo. Open for the reviewer's callRegression risk
claude_platform, and Azure AI Anthropic deployments reuse this streaming iterator and were traced, not run: the rig has no deployment for them (no Snowflake orclaude_platformcredentials on this box; Azure AI and mantle credentials exist but no deployment was added in this run). Exercising them is onemodel_listentry each in the rig config and a rerun of the direct-stream scenario/v1/messagesadapter for non-Anthropic models does not run this iterator (traced, not on the changed path)Dependency graph
ModelResponseIteratorsignature chunk (the changed code): unit tests intests/unit/llms/anthropic/chat/test_anthropic_chat_handler.py(per-chunk shape and the next-turn replay); verified live, direct Anthropic streamCustomStreamWrapperto client chunks: verified live on direct, Bedrock invoke, Vertex, and fallback streamsstream_chunk_builderto the spend logresponseand to callers: verified live, spend rows of every stream scenario/v1/messagesroute and its spend log: verified live (signature_delta carried no text before or after; the spend row's block halves)/anthropic/v1/messagespass-through logging handler (rebuilds the response with the iterator): verified live, spend row halvesLiteLLMCompletionResponsesConfig, streaming iterator): verified live with the recorder; the Rome turn replays a 264-char signed block that equals the streamed text, where the base replayed 588 chars (the text twice) on three later requestsLitellmModelin process: verified live in the QA legs aboveclaude_platform, Azure AI Anthropic: traced, untested livetests/llm_translation/test_anthropic_completion.py,tests/local_testing/test_streaming.py,tests/llm_translation/test_bedrock_completion.py,-k "thinking or reasoning or signature"): 15 passed, 1 skipped (test_bedrock_thinking_in_assistant_message[False], the suite's own skip)Not verified
claude_platformstreaming (no credentials on this box)Audit round 1 of 3
/audit of tip
19554443c80fe1dba329996aa03fa3f5cb65c613against merge base3f39fef52f85b9cc161abba6bb47ca6740801200. Rig: owned Postgres, Redis, and a scripted Anthropic-wire upstream, one 2-worker proxy per leg, built fromtests/integration/providers/test_anthropic_thinking_signature_stream_wire.pyandtest_anthropic_thinking_signature_logging_wire.py(--seed 44127 --order-seed 44127, head run twice with identical selections and results). T is the scripted thinking text, S its signature. "red" on the merge base is the fix-sensitive thinking assertion (doubled T under one S), "green" on both legs is a controltest_openai_sdk_sync_stream_accumulates_the_thinking_oncetest_openai_sdk_async_stream_accumulates_the_thinking_oncetest_signature_chunk_carries_no_thinking_text_on_every_anthropic_wire_provider[anthropic]test_non_streaming_completion_keeps_the_signed_thinking_block_intact..._on_every_anthropic_wire_provider[bedrock_invoke]..._on_every_anthropic_wire_provider[claude_platform]..._on_every_anthropic_wire_provider[vertex_ai]..._on_every_anthropic_wire_provider[snowflake]..._on_every_anthropic_wire_provider[azure_ai]test_responses_stream_encrypts_the_thinking_oncetest_responses_non_stream_encrypts_the_signed_block_as_receivedtest_chat_stream_spend_row_stores_the_thinking_oncetest_cache_hit_replays_the_answer_from_one_upstream_call_and_never_doubles_the_thinkingtest_native_messages_stream_through_the_anthropic_sdk_logs_the_thinking_oncetest_native_messages_stream_on_bedrock_mantle_logs_the_thinking_oncetest_adapter_messages_stream_on_snowflake_logs_the_thinking_oncetest_anthropic_passthrough_stream_relays_the_frames_and_logs_the_thinking_oncetest_bedrock_invoke_passthrough_stream_relays_the_frames_and_logs_the_thinking_once:streamRawPredictpass-through, Claude Sonnet 5.5test_two_signed_blocks_each_keep_their_own_text_through_a_replaytest_redacted_block_before_a_signed_block_replays_each_oncetest_signature_only_block_without_thinking_deltas_is_relayed_as_istest_replaying_the_accumulated_turn_sends_the_thinking_once_with_its_signaturetest_two_identical_requests_with_no_cache_each_land_their_own_spend_rowtest_unusable_signature_values_yield_no_signed_block_and_keep_the_stream_intact[integer]..._keep_the_stream_intact[list]..._keep_the_stream_intact[empty]test_five_kilobyte_signature_is_relayed_verbatim_without_thinking_texttest_duplicate_signature_deltas_never_repeat_the_thinking_texttest_non_string_thinking_delta_is_ignored_and_the_signed_block_still_lands_oncetest_upstream_authentication_error_reaches_the_caller_and_leaves_the_proxy_healthytest_unauthenticated_stream_is_refused_before_the_upstream_is_calledtest_mid_thinking_upstream_aborts_in_a_mixed_burst_leave_every_completed_call_logged_oncetest_worker_sigkill_mid_burst_leaves_the_sibling_streaming_signed_thinking_onceLitellmModelreplay, Claude Sonnet 5.5Live rows ran once per leg against the real Anthropic API and Vertex AI with adaptive thinking (
reasoning_effort: high,display: summarized). On the merge base the Agents SDK saved the thinking text twice under one signature and sent it back doubled on turn 2 (the API still answered 200); on the tip it saved and replayed the text once. H6 (Bedrock Mantle on/v1/chat/completions) was dropped from the inventory because that surface never touches the Anthropic iteratorThree pre-existing behaviors outside the diff were ticketed internally: a cached stream replay drops
thinking_blocksandreasoning_contenton both legs, the Anthropic pass-through logging handler only runs when the upstream hostname isapi.anthropic.com, and the router pass-through keeps theinvoke/route marker of abedrock/invoke/...deployment in the upstream model pathRerun at 775fe9f (tests only, tightens the cache cell's replay assertion to exactly an empty tuple or one signed block): H14 ran again on both legs, green on the tip and red on the merge base at the first-stream thinking assertion as before (the stream module: 27 passed on the tip, 20 failed and 7 passed on the base, the same red set as the first base run); every other row keeps its result from the run at 6e40e47
Type
🐛 Bug Fix
Caveats (if any)
Low
/v1/messagesadapter already turned a signed block into asignature_deltaand dropped that chunk's text, so its output is unchangedprovider_specific_fields.thinking_blockschange on rebuilt streams is recorded under Backward incompatible in the /live-pr-risk section above, with the decision to leave itFinal Attestation
The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR
1955444 passes /live-pr-risk
Note
Medium Risk
Changes streamed chat
thinking_blocksshape on the signature chunk (intentional contract shift); clients that read only that delta lose inline text, though downstream accumulation and logging paths are heavily tested.Overview
Fixes doubled extended-thinking text when Claude streams a
signature_delta: the Anthropic chat stream iterator no longer re-emits the full accumulated thinking in that chunk and instead attaches a signature-onlythinking_blocksentry (thinking: ""plus the signature). Per-delta thinking streams unchanged viareasoning_contentand unsigned thinking blocks, so accumulators (stream_chunk_builder, Agents SDK, Responses bridge) rebuild one signed block with the text once.Tests add shared wire fixtures and broad integration coverage (chat/messages/responses, multiple providers, spend logging, replay, cache, chaos) plus updated unit expectations and a round-trip replay test.
Reviewed by Cursor Bugbot for commit 775fe9f. Bugbot is set up for automated code reviews on this repo. Configure here.