Repository navigation
Conversation
|
Thanks for your contribution! This PR doesn't have a linked issue. All PRs must reference an existing issue. Please:
See CONTRIBUTING.md for details. |
The prepared model request only carried generation values set by hooks, so a model's configured limit.output never reached the provider. On Bedrock the request was sent without inferenceConfig.maxTokens and the service applied a much lower implicit budget; with adaptive thinking the whole budget could be consumed by reasoning, truncating responses at exactly 4,096 output tokens despite limit.output=128000. Project maxTokens with explicit precedence: hook/context override, model defaults, route defaults, then resolved.limit.output. Each protocol already lowers generation.maxTokens into its native field, so no protocol changes are needed. Closes anomalyco#46595 Related: anomalyco#47398
49e5841 to
5de9656
Compare
|
Automated PR Cleanup Thank you for contributing to opencode. Due to the high volume of PRs from users and AI agents, we periodically close older PRs using automated criteria so maintainers can focus review time on the most active and community-supported contributions. This PR was closed because it matched the following cleanup criteria:
PRs created within the last month are not affected by this cleanup. If you believe this PR was closed incorrectly, or if you are still actively working on it, please leave a comment explaining why it should be reopened. A maintainer can review and reopen it if appropriate. Thanks again for taking the time to contribute. |
Issue for this PR
Closes #46595
Type of change
What does this PR do?
SessionModelRequest.preparebuilt the model request withgeneration: undefinedunless asession.contexthook set a generation value. The model's resolvedlimit.outputwas never projected into the request, so no protocol received amaxTokens:inferenceConfig.maxTokens; the service applied its implicit budget and, with adaptive thinking enabled, long-reasoning turns finished asmax_tokensat exactly 4096 output tokens, often with reasoning only and no visible text (Bedrock: configured output limit never sent, long reasoning turns truncate at 4096 tokens #46595).max_tokensfor the same reason (v2 (opencode2): max_tokens never sent for @ai-sdk/openai-compatible models, limit.output ignored, thinking turns truncate at provider default #47398).The fix always emits
generationwithmaxTokensresolved in this precedence order:session.contexthook value →model.defaults.generation.maxTokens→model.route.defaults.generation.maxTokens→resolved.limit.output. Hook and provider defaults keep winning; the catalog limit only fills the gap. Each protocol already serializedgeneration.maxTokensinto its native field (inferenceConfig.maxTokens,max_tokens,max_output_tokens,generationConfig.maxOutputTokens), so no protocol changes are needed.This is a re-submission of #46601, rebased on current
v2(the original was auto-closed by the compliance bot because the checklist section was missing). TheSessionModelRequest.context optionstest block that the original PR extended was removed upstream in #46639, so the regression test now lives in theSessionRunnerLLMscenario harness.Status
v2(437cdc03a, 526 commits). The bug is still present there:SessionModelRequest.prepareemitsgeneration: undefinedunless a hook sets an option, solimit.outputis never sent. Fix ported to the refactoredprepare(modelis now theResolved, hook overrides live onevent.options). Regression scenario updated to theevent.optionshook surface introduced in 2.0.x.global.anthropic.claude-fable-5-1, adaptive thinking): 146 turns in two weeks ended withfinish=lengthat exactly 4096 output tokens; zero turns exceeded 4096. WithinferenceConfig.maxTokenspresent the same model streams >6k-token turns without truncation.How did you verify your code works?
packages/core/test/session-runner.test.ts("projects the resolved output limit into the request while preserving hook overrides"): assertsrequest.generationequals{ maxTokens: <limit.output> }with no hooks, and that a hook-providedmaxTokens/temperaturewin over the catalog limit. Confirmed it fails onv2without the source change and passes with it.generationwasundefined(session-compaction.test.ts,session-runner.test.ts).bun run test test/session-compaction.test.ts test/session-model-request.test.ts test/session-runner.test.tsinpackages/core: 228 pass, 0 fail (after rebase). Confirmed the new scenario fails onv2HEAD without the source change (2 fail) and passes with it.bun typecheckacross the monorepo (33 packages) clean.Screenshots / recordings
Not a UI change.
Checklist