Repository navigation
feat(console): proxy keyless Zen requests to new inference per model - #53717
Merged
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Issue for this PR
N/A
Type of change
What does this PR do?
Free Zen models already end up in new inference (every
allowAnonymousmodel routes to aninf.*provider), but keyless requests first pass through legacy Zen's own free-tier checks and provider selection. That means two sets of rate limits and two hops for every free request. This lets us move free models off legacy one at a time.public-key requests forling-3.1-flash-freeandnemotron-3.5-lightning-freeare forwarded straight to new inference, like migrated workspaces already are. These two low-traffic models go first; more follow once they look healthy.authorization,x-api-key,x-goog-api-keyare removed), so new inference treats them as anonymous free-tier callers. The client IP still travels asx-zen-ip; OpenCode session and user-agent headers pass through unchanged.ANONYMOUS_PROXY_MODELSinhandler.ts.New inference already enforces the free-tier checks, the per-IP limits, and the Muse Spark/fledge country blocks (anomalyco/opencode-console#3180).
How did you verify your code works?
bun typecheckinpackages/console/app. No unit test:proxyInferencedepends on SST resources and the database, and the console app has no harness for them. After deploy, watch free-tier status codes for both models in the inference Honeycomb dataset.Screenshots / recordings
N/A
Checklist