[JunyiXu-nv]

Stacked on #21059 ([None][fix] serve: enforce parallel_tool_calls=false by stopping after the first DSML tool call) and #21112 ([None][fix] Kimi-K3 tool parser: read calls whose close tags are missing): the first commits below belong to those PRs (the base is #21059's head with #21112's head merged in, as an integration base that is not part of the reviewable change) and disappear from this diff when they merge; this PR's own change is the last 2 commits, the first of which is a one-line test-fixture update that belongs in #21059 (see below).

Description

A request for at most one tool call per turn (parallel_tool_calls: false on Chat Completions and Responses, disable_parallel_tool_use on Anthropic Messages) is honoured by stopping generation on the text that ends one call and keeping that text in the output (BaseToolParser.single_call_stops, #21059). The kimi_k3 tool parser declared no such text, so a Kimi K3 request with parallel_tool_calls: false was accepted, logged once as not enforced, and still received every call the model wrote.

Kimi K3 writes all of a turn's calls in one tools section, and every call block ends with its own close tag, <|close|>call<|sep|>. That tag is the single-call stop:

• The K3 tokenizer writes it as the three tokens <|close|>, call, <|sep|> (ids 163588, 10257, 163589), which is how SamplingParams tokenizes the stop text, so the stop matches on token ids. Checked against the Kimi-K3 and Kimi-K3.1-Pro tokenizers: in a rendered two-call assistant turn the id sequence matches exactly twice, once per call.
• The stop leaves the tools section open. The parser already delivers the complete calls of an open section, so the one call reaches the client whole in both the streamed and the non-streamed views.
• Every parallel_tool_calls: false turn with a call now ends with the section open by design, so finish() notes a stream that ended on a call's close tag at debug level instead of warning. A section left open any other way (prose after the call, a next call cut off, a value cut off) still warns.

docs/source/features/disagg-serving.md lists kimi_k3 with the parsers whose single-call stop is enforced; the Kimi K3 deployment guide gains a short paragraph on parallel_tool_calls: false, outside its list of Kimi-specific chat extensions since the field is OpenAI's generic one.

Behavior change: a Kimi K3 request that sets parallel_tool_calls: false (Chat Completions, Responses) or disable_parallel_tool_use: true (Anthropic Messages) now ends generation after the model's first tool call and reports that one call; finish_reason is tool_calls and no stop_reason is reported for the stop the server added. Requests that omit the field, or set it to true, are unchanged. The stop is a stop sequence like any other: generation ends on the first occurrence of the token triple <|close|> call <|sep|> wherever the model writes it, and the server hides its own stop, so should the model emit those control tokens outside a tools section (inside its thinking, say), the turn ends there with finish_reason stop, truncated reasoning and no tool call. For K3 this is less likely than for formats whose closing tag is plain text, since all three pieces are control tokens the model is trained to emit only as markup, but it is part of the design. The warning the parser logged for a stream that ended before <|close|>tools<|sep|> is now a debug message when the stream ended on a call's close tag (trailing whitespace ignored); every other early end keeps the warning. This is a feature for clients that send parallel_tool_calls: false; no failure gate applies.

Also included, as its own commit: the fake engine promise in test_single_tool_call_stop.py's _chat_server gains a request_id, which openai_chat reads since #20936 (raw_request.state.executor_client_id = promise.request_id). Without it every chat and Anthropic case of that file fails on current main with HTTP 400. That commit belongs in #21059 and disappears from this stack once #21059 carries it.

Second-lander notes: #21624 (GLM single-call stop, also on #21059) edits the same test that lists the parsers declaring single_call_stops (renamed here to test_the_parsers_that_declare_single_call_stops, so the name stays right once both land), the same disagg-serving.md sentence, and the same parametrize lists in test_single_tool_call_stop.py. Whichever of the two merges second rebases; the resolution is the union of the rows and {deepseek_v32, deepseek_v4, deepseek_v41, glm4, glm47, kimi_k3}.

Test Coverage

CPU tests with small inline inputs, run from the branch with the compiled artifacts of a recent main (an import shim declares the C++ ops added since, none of which these tests call):

• tests/unittest/llmapi/apps/test_tool_parsers.py::TestKimiK3ToolParser:
  • test_the_single_call_stop_is_the_call_close_tag: the declared stop is ("<|close|>call<|sep|>",).
  • test_a_generation_stopped_after_its_first_call_delivers_that_call_quietly[at_the_stop|newline_after_the_call]: a generation ending on the kept stop, or cut right after the newline following the call, delivers the text before the section and exactly that call, whole and streamed over every token-aligned split, with no warning logged and the early end noted at debug (captured with a handler on the TRT-LLM logger).
  • test_a_generation_stopped_after_its_first_call_reads_the_same_through_the_pipeline: with the kimi_k3 reasoning parser in front, as served: the reasoning, the response text, then the one call, for every chunking of the generation.
  • test_a_section_left_open_any_other_way_still_warns[prose_after_the_call|next_call_cut_off|next_call_value_cut_off]: the warning stays for every other early end (prose after the call, a second call cut in its header, a second call cut inside its value).
  • test_the_parsers_that_declare_single_call_stops: kimi_k3 joins the DSML parsers; no other parser declares a stop.
• tests/unittest/llmapi/apps/test_single_tool_call_stop.py: kimi_k3 rows in the stop lookup, in the chat, Anthropic and Responses "configures the stop only when asked and possible" tests (the real openai_chat, anthropic_messages and openai_responses handlers), and test_a_kimi_k3_chat_reply_stopped_after_its_first_call_reports_that_call (what a chat client receives: finish_reason tool_calls, no stop_reason, the one call). The tokenizer-gated test is not extended: the K3 checkpoints ship a tiktoken tokenizer with remote code, not a tokenizer.json; the tokenization claim above was checked locally against both checkpoints.
• tests/unittest/llmapi/apps/test_responses_stream_final_consistency.py: test_a_kimi_k3_generation_stopped_after_its_first_call_reports_exactly_that_call (streamed through the Responses processor: the stream and its final snapshot report the same one call and message text) and the non-streamed counterpart (reasoning, message, function_call items, status completed).

Every new test fails without its production change: with the property removed, the stop and parser-set tests fail; with finish() always warning, the quiet test fails; with it always at debug, or logging nothing, the "still warns" tests fail (and the quiet test for the no-log case); with the wrong stop text, the stop tests fail; with the trailing-whitespace tolerance of the quiet-close check removed, the newline_after_the_call case fails.

The same three statements were also checked as properties over generated inputs (Hypothesis, 300 examples each, text before the section, tool names and argument values from the test fixtures, cut positions): a generation ending on the kept stop reads the same whole and streamed over every token-aligned split, delivers exactly that call, and logs no warning; a section left open any other way (a later call cut at a token boundary, prose after the call) still warns; through the kimi_k3 reasoning parser, any two-way cut of the stopped generation reads like the whole. No counter-example was found within those bounds (not a proof), and each of the five planted bugs above made at least one property fail. The harness is not part of the PR.

The whole test_tool_parsers.py file passes (1224 tests), as do test_single_tool_call_stop.py (41 passed, 4 skipped for want of model files) and test_responses_stream_final_consistency.py. GPU runs are left to CI.

Reviewers by CODEOWNERS: @NVIDIA/trt-llm-runtime-devs (tensorrt_llm/serve, tests/unittest/llmapi), @NVIDIA/trt-llm-doc-owners (docs).

PR Checklist

Please review the following before submitting your PR:

• PR description clearly explains what and why. If using CodeRabbit's summary, please make sure it makes sense.

• PR Follows TRT-LLM CODING GUIDELINES to the best of your knowledge.

• Test cases are provided for new code paths (see test instructions)

• Any new dependencies have been scanned for license and vulnerabilities

• CODEOWNERS updated if ownership changes

• Documentation updated as needed

• Update tava architecture diagram if there is a significant design change in PR.

• The reviewers assigned automatically/manually are appropriate for the PR.

• Please check this after reviewing the above items as appropriate for this PR.

🤖 Generated with Claude Code