Skip to content

feat(console): proxy keyless Zen requests to new inference per model - #53717

Merged
vimtor merged 2 commits into
devfrom
zen-free-proxy
Oct 7, 2026
Merged

vimtor merged 2 commits into
devfrom
zen-free-proxy

Conversation

@vimtor

@vimtor vimtor commented Oct 7, 2026 •

Copy link
Copy Markdown
Collaborator

Issue for this PR

N/A

Type of change

  • Bug fix
  • New feature
  • Refactor / code improvement
  • Documentation

What does this PR do?

Free Zen models already end up in new inference (every allowAnonymous model routes to an inf.* provider), but keyless requests first pass through legacy Zen's own free-tier checks and provider selection. That means two sets of rate limits and two hops for every free request. This lets us move free models off legacy one at a time.

  • Keyless and public-key requests for ling-3.1-flash-free and nemotron-3.5-lightning-free are forwarded straight to new inference, like migrated workspaces already are. These two low-traffic models go first; more follow once they look healthy.
  • Forwarded keyless requests carry no credentials (authorization, x-api-key, x-goog-api-key are removed), so new inference treats them as anonymous free-tier callers. The client IP still travels as x-zen-ip; OpenCode session and user-agent headers pass through unchanged.
  • Keyed requests and every other model behave exactly as before. Rollback is removing a model from ANONYMOUS_PROXY_MODELS in handler.ts.

New inference already enforces the free-tier checks, the per-IP limits, and the Muse Spark/fledge country blocks (anomalyco/opencode-console#3180).

How did you verify your code works?

bun typecheck in packages/console/app. No unit test: proxyInference depends on SST resources and the database, and the console app has no harness for them. After deploy, watch free-tier status codes for both models in the inference Honeycomb dataset.

Screenshots / recordings

N/A

Checklist

  • I have tested my changes locally (typecheck only; see above)
  • I have not included unrelated changes in this PR

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant