Skip to content

fix(llm): add disable_cache_control option for Anthropic-compatible proxies (#700) - #29

Open
chethanuk wants to merge 1 commit into
mainfrom
fix/issue-700-unrecognizedproperty-cachecontrol
Open

chethanuk wants to merge 1 commit into
mainfrom
fix/issue-700-unrecognizedproperty-cachecontrol

Conversation

@chethanuk

Copy link
Copy Markdown
Owner

Summary

Fixes alibaba#700 — Anthropic-compatible gateways/proxies that reject the
cache_control field now have a config option to suppress it.

OCR adds cache_control: {"type": "ephemeral"} to the last system
block and last tool definition to enable Anthropic prompt caching. Some
Bedrock-style proxies and custom gateways do not support this field and
return a 400 error:

unrecognizedProperty=cache_control

Changes

  • internal/llm/client.go — ClientConfig.DisableCacheControl field;
    buildAnthropicParams guards both CacheControl assignments behind
    !c.cfg.DisableCacheControl. No effect on OpenAI/OpenAI-Responses clients.
  • internal/llm/resolver.go — ResolvedEndpoint.DisableCacheControl,
    llmFileConfig.disable_cache_control, and
    providerEntryConfig.disable_cache_control (JSON-tagged). Threaded through
    both config-file resolution paths.
  • internal/llm/client_test.go — TestBuildAnthropicParams_DisableCacheControl:
    asserts no cache_control appears on any system block or tool when enabled.
  • pages/src/content/docs/en/configuration.md — new section explaining
    the option, with CLI and JSON examples.

How to use

ocr config set providers.anthropic.disable_cache_control true

Or in ~/.opencodereview/config.json:

{
  "providers": {
    "anthropic": { "disable_cache_control": true }
  }
}

Works for custom_providers too.

How to test

go test ./internal/llm/... -run TestBuildAnthropicParams_DisableCacheControl -v
# runs successfully; asserts all CacheControl.Type fields are empty

go test ./... -count=1   # all packages pass

Closes alibaba#700

…baba#700)

Some Anthropic-compatible gateways reject the cache_control field that
OCR injects for prompt caching, causing 400 errors with
unrecognizedProperty=cache_control.

Add a disable_cache_control boolean (false by default) to:
- ResolvedEndpoint / ClientConfig (runtime)
- llmFileConfig.llm section (legacy config)
- providerEntryConfig (providers / custom_providers)

When true, cache_control is omitted from system blocks and tool
definitions in buildAnthropicParams. All other providers are unaffected.

Users can enable it via:
  ocr config set providers.anthropic.disable_cache_control true

Adds TestBuildAnthropicParams_DisableCacheControl to verify the field is
absent from all system blocks and tools when the option is set.
@codeant-ai

codeant-ai Bot commented Aug 12, 2026 •

Copy link
Copy Markdown

🤖 CodeAnt AI — Review Status

Status Commit Started (UTC) Finished (UTC)
✅ Reviewed your PR ed8355c Aug 12, 2026 · 15:56 15:59

@codeant-ai

codeant-ai Bot commented Aug 12, 2026

Copy link
Copy Markdown

Thanks for using CodeAnt! 🎉

We're free for open-source projects. if you're enjoying it, help us grow by sharing.

Share on X ·
Reddit ·
LinkedIn

@coderabbitai

coderabbitai Bot commented Aug 12, 2026 •

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: e928444e-000a-4275-ba2d-208762a36225

📥 Commits

Reviewing files that changed from the base of the PR and between f8f0851 and ed8355c.

📒 Files selected for processing (4)
  • internal/llm/client.go
  • internal/llm/client_test.go
  • internal/llm/resolver.go
  • pages/src/content/docs/en/configuration.md

📝 Walkthrough

Walkthrough

The change adds disable_cache_control configuration for provider and legacy endpoints. Endpoint resolution and client construction propagate the setting. Anthropic request construction omits cache-control metadata from system blocks and tools when enabled. Documentation and tests cover the setting.

Changes

Cache-control configuration

Layer / File(s) Summary
Endpoint configuration and resolution
internal/llm/resolver.go, pages/src/content/docs/en/configuration.md
Configuration structs and resolved endpoints now carry DisableCacheControl for provider and legacy endpoints. Documentation describes CLI and JSON configuration.
Anthropic request handling
internal/llm/client.go, internal/llm/client_test.go
The client propagates DisableCacheControl. Anthropic system blocks and tools omit cache-control metadata when enabled. Tests cover the disabled-cache-control behavior.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Sequence Diagram(s)

sequenceDiagram
  participant Configuration
  participant ResolvedEndpoint
  participant NewLLMClient
  participant AnthropicRequest
  Configuration->>ResolvedEndpoint: resolve disable_cache_control
  ResolvedEndpoint->>NewLLMClient: forward DisableCacheControl
  NewLLMClient->>AnthropicRequest: omit cache_control when enabled
Loading

Suggested reviewers: lizhengfeng101, zhouzhih, stay-foolish-forever

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly summarizes the main change: adding a disable_cache_control option for Anthropic-compatible proxies.
Description check ✅ Passed The description explains the issue, implementation, usage, testing, and related issue, but omits the template's Type of Change and Checklist sections.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@codeant-ai codeant-ai Bot added the size:L This PR changes 100-499 lines, ignoring generated files label Aug 12, 2026
@codeant-ai

codeant-ai Bot commented Aug 12, 2026

Copy link
Copy Markdown

User description

Summary

Fixes alibaba#700 — Anthropic-compatible gateways/proxies that reject the
cache_control field now have a config option to suppress it.

OCR adds cache_control: {"type": "ephemeral"} to the last system
block and last tool definition to enable Anthropic prompt caching. Some
Bedrock-style proxies and custom gateways do not support this field and
return a 400 error:

unrecognizedProperty=cache_control

Changes

  • internal/llm/client.go — ClientConfig.DisableCacheControl field;
    buildAnthropicParams guards both CacheControl assignments behind
    !c.cfg.DisableCacheControl. No effect on OpenAI/OpenAI-Responses clients.
  • internal/llm/resolver.go — ResolvedEndpoint.DisableCacheControl,
    llmFileConfig.disable_cache_control, and
    providerEntryConfig.disable_cache_control (JSON-tagged). Threaded through
    both config-file resolution paths.
  • internal/llm/client_test.go — TestBuildAnthropicParams_DisableCacheControl:
    asserts no cache_control appears on any system block or tool when enabled.
  • pages/src/content/docs/en/configuration.md — new section explaining
    the option, with CLI and JSON examples.

How to use

ocr config set providers.anthropic.disable_cache_control true

Or in ~/.opencodereview/config.json:

{
  "providers": {
    "anthropic": { "disable_cache_control": true }
  }
}

Works for custom_providers too.

How to test

go test ./internal/llm/... -run TestBuildAnthropicParams_DisableCacheControl -v
# runs successfully; asserts all CacheControl.Type fields are empty

go test ./... -count=1   # all packages pass

Closes alibaba#700


CodeAnt-AI Description

Allow Anthropic-compatible proxies that reject cache-control fields to process reviews

What Changed

  • Added a disable_cache_control setting for Anthropic and custom providers
  • When enabled, requests omit cache-control fields from system prompts and tool definitions
  • Documented configuration through the command line and config file
  • OpenAI and OpenAI-Responses providers remain unchanged

Impact

✅ Fewer Anthropic proxy request failures
✅ Compatible with gateways that reject cache-control
✅ Clearer configuration for prompt-cache compatibility

💡 Usage Guide

Checking Your Pull Request

Every time you make a pull request, our system automatically looks through it. We check for security issues, mistakes in how you're setting up your infrastructure, and common code problems. We do this to make sure your changes are solid and won't cause any trouble later.

Talking to CodeAnt AI

Got a question or need a hand with something in your pull request? You can easily get in touch with CodeAnt AI right here. Just type the following in a comment on your pull request, and replace "Your question here" with whatever you want to ask:

@codeant-ai ask: Your question here

This lets you have a chat with CodeAnt AI about your pull request, making it easier to understand and improve your code.

Example

@codeant-ai ask: Can you suggest a safer alternative to storing this secret?

Preserve Org Learnings with CodeAnt

You can record team preferences so CodeAnt AI applies them in future reviews. Reply directly to the specific CodeAnt AI suggestion (in the same thread) and replace "Your feedback here" with your input:

@codeant-ai: Your feedback here

This helps CodeAnt AI learn and adapt to your team's coding style and standards.

Example

@codeant-ai: Do not flag unused imports.

Retrigger review

Ask CodeAnt AI to review the PR again, by typing:

@codeant-ai: review

Check Your Repository Health

To analyze the health of your code repository, visit our dashboard at https://app.codeant.ai. This tool helps you identify potential issues and areas for improvement in your codebase, ensuring your repository maintains high standards of code health.

Comment thread internal/llm/resolver.go
Models []string `json:"models,omitempty"`
AuthHeader string `json:"auth_header,omitempty"`
TimeoutSec int `json:"timeout_sec,omitempty"` // per-request HTTP timeout in seconds
DisableCacheControl bool `json:"disable_cache_control,omitempty"` // omit cache_control from Anthropic requests

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggestion: The newly accepted disable_cache_control field exists only in the resolver's private config structs. The public cmd/opencodereview.Config and ProviderEntry types do not define it, so the documented ocr config set providers.anthropic.disable_cache_control true command is rejected as an unknown provider field, and an existing manually-added setting is discarded the next time the CLI rewrites config.json. Add the field to the config-writing models and provider-field handling. [incomplete implementation]

Severity Level: Major ⚠️
- ❌ Documented providers.anthropic command always fails.
- ❌ Config rewrites silently remove manually added settings.
- ⚠️ Anthropic proxies continue rejecting cache_control fields.

Fix in Cursor Fix in VSCode Claude

Prompt for AI Agent 🤖
This is a comment left during a code review.

**Path:** internal/llm/resolver.go
**Line:** 257:257
**Comment:**
	*Incomplete Implementation: The newly accepted `disable_cache_control` field exists only in the resolver's private config structs. The public `cmd/opencodereview.Config` and `ProviderEntry` types do not define it, so the documented `ocr config set providers.anthropic.disable_cache_control true` command is rejected as an unknown provider field, and an existing manually-added setting is discarded the next time the CLI rewrites config.json. Add the field to the config-writing models and provider-field handling.

Validate the correctness of the flagged issue. If correct, How can I resolve this? If you propose a fix, implement it and please make it concise.
Once fix is implemented, also check other comments on the same PR, and ask user if the user wants to fix the rest of the comments as well. if said yes, then fetch all the comments validate the correctness and implement a minimal fix
👍 | 👎

Comment thread internal/llm/resolver.go
Protocol: protocol,
AuthHeader: authHeader,
Source: "provider:" + cfg.Provider,
ExtraBody: extraBody,

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggestion: extra_body is forwarded as unrestricted JSON overrides after the SDK request parameters are built. A provider configuration containing keys such as model, messages, tools, or input can replace the values assembled from the current ChatRequest, causing requests to use stale or attacker-controlled payload data instead of the current conversation. Reject reserved request-field collisions or restrict extra_body to vendor-specific fields. [api mismatch]

Severity Level: Major ⚠️
- ❌ Configured messages can replace the active review conversation.
- ❌ Configured tools can disable or alter tool execution.
- ⚠️ Configured model can ignore the selected request model.

Fix in Cursor Fix in VSCode Claude

Prompt for AI Agent 🤖
This is a comment left during a code review.

**Path:** internal/llm/resolver.go
**Line:** 433:433
**Comment:**
	*Api Mismatch: `extra_body` is forwarded as unrestricted JSON overrides after the SDK request parameters are built. A provider configuration containing keys such as `model`, `messages`, `tools`, or `input` can replace the values assembled from the current `ChatRequest`, causing requests to use stale or attacker-controlled payload data instead of the current conversation. Reject reserved request-field collisions or restrict extra_body to vendor-specific fields.

Validate the correctness of the flagged issue. If correct, How can I resolve this? If you propose a fix, implement it and please make it concise.
Once fix is implemented, also check other comments on the same PR, and ask user if the user wants to fix the rest of the comments as well. if said yes, then fetch all the comments validate the correctness and implement a minimal fix
👍 | 👎

Comment thread internal/llm/resolver.go
AuthHeader: authHeader,
Source: "provider:" + cfg.Provider,
ExtraBody: extraBody,
ExtraHeaders: extraHeaders,

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggestion: Configuration-file extra_headers are copied directly into the resolved endpoint without applying the reserved-header checks used by ParseExtraHeaders. A config entry containing Authorization, X-Api-Key, Content-Type, or User-Agent is therefore forwarded to the SDK and can override authentication or request metadata. Validate these maps with the same reserved-header rules before returning the endpoint. [security]

Severity Level: Major ⚠️
- ❌ Anthropic requests can use incorrect authentication headers.
- ⚠️ Configured Content-Type or User-Agent can corrupt request metadata.
- ⚠️ Manually edited provider configurations bypass reserved-header protection.

Fix in Cursor Fix in VSCode Claude

Prompt for AI Agent 🤖
This is a comment left during a code review.

**Path:** internal/llm/resolver.go
**Line:** 434:434
**Comment:**
	*Security: Configuration-file `extra_headers` are copied directly into the resolved endpoint without applying the reserved-header checks used by `ParseExtraHeaders`. A config entry containing `Authorization`, `X-Api-Key`, `Content-Type`, or `User-Agent` is therefore forwarded to the SDK and can override authentication or request metadata. Validate these maps with the same reserved-header rules before returning the endpoint.

Validate the correctness of the flagged issue. If correct, How can I resolve this? If you propose a fix, implement it and please make it concise.
Once fix is implemented, also check other comments on the same PR, and ask user if the user wants to fix the rest of the comments as well. if said yes, then fetch all the comments validate the correctness and implement a minimal fix
👍 | 👎

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size:L This PR changes 100-499 lines, ignoring generated files

Projects

None yet

Development

Successfully merging this pull request may close these issues.

unrecognizedProperty cache_control

1 participant