Repository navigation
Conversation
added 3 commits
October 5, 2026 19:11
Rejecting response_format with "no constrained decoding" breaks OpenAI-compatible clients that rely on json_object/json_schema even when the model can emit valid JSON from a prompt. Accept those wire types and prepend a best-effort system hint; schema is not token-level enforced. Unknown response_format types still return 400.
After soft-accepting json_object/json_schema, the old "no constrained decoding" message was misleading for remaining unknown types.
Soft-accepting json_object/json_schema is enough when clients already request JSON in the prompt. The injected schema wrapper was redundant and could distort the output shape.
KarrAcaRn
pushed a commit
to KarrAcaRn/FreeToken-ByAI
that referenced
this pull request
Oct 6, 2026
FlashML-org#604 rejected (accepts JSON response formats without enforcing them); FlashML-org#605 deferred (pre-sm70 only); drafts FlashML-org#614, FlashML-org#615, FlashML-org#616 deferred with what next already covers. Assisted-by: Claude Opus 5.5
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
response_formattypesjson_objectandjson_schemainstead of returning HTTP 400no constrained decoding.response_format.typevalues rejected, with a clearer error message.OpenAI-compatible clients (structured outputs / JSON mode) currently break against FreeToken even when the model can emit valid JSON from a prompt. This unblocks that wire format without claiming full guided decoding.
Test plan
POST /v1/chat/completionswithoutresponse_formatstill returns 200 (no regression).POST /v1/chat/completionswith"response_format": {"type": "json_object"}returns 200 and JSON content."type": "json_schema"(+ simple object schema) returns 200 with top-level fields matching the schema (best-effort).response_format.typestill returns 400 with the updated error text.json_objectstill completes.Verified in production against FreeToken 0.1.3 / Qwen3.6-35B-A3B (chat completions via an OpenAI gateway): baseline,
json_object, andjson_schemaall returned HTTP 200.