Skip to content

[Bug]: trtllm-serve /v1/chat/completions rejects the standard OpenAI field prompt_cache_key with 400 #20367

Open

Description

@lucifer1004

System Info

TensorRT-LLM main @ 15f060c, trtllm-serve (any model, any GPU).

Reproduction

curl -s localhost:8000/v1/chat/completions -H 'Content-Type: application/json' -d '{
  "model": "<model>",
  "messages": [{"role": "user", "content": "hi"}],
  "prompt_cache_key": "session-1234"
}'

or with the OpenAI Python client: client.chat.completions.create(..., prompt_cache_key="session-1234").

Expected behavior

The request is served. prompt_cache_key is part of OpenAI's Chat Completions request body (a hint for prompt-cache routing), and clients built on it may set it on every request of a session.

Actual behavior

The request fails validation with extra_forbidden ("Extra inputs are not permitted") for prompt_cache_key, because ChatCompletionRequest inherits extra="forbid" from OpenAIBaseModel and has no such field. ResponsesRequest already accepts it (it switched to extra="allow", and the comment there names prompt_cache_key as one of the fields real clients send), so the same client works against /v1/responses but not against /v1/chat/completions.

Additional notes

KV cache reuse in TensorRT-LLM needs no client hint, and cache_salt is the field that isolates cached prefixes, so the field can be accepted and ignored, the same way parallel_tool_calls and user are. PR to follow.

Activity

  1. added
    OpenAI APItrtllm-serve's OpenAI-compatible API: endpoint behavior, req/resp formats, feature parity.
    on Oct 15, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    OpenAI APItrtllm-serve's OpenAI-compatible API: endpoint behavior, req/resp formats, feature parity.

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions