[Bug]: Responses API streaming for gpt-oss drops text and item events when a response carries several tokens #22109

System Info

• TensorRT-LLM main at e8e94448b (the affected code is the same at 6edbe3834), trtllm-serve with a gpt-oss (Harmony) checkpoint, POST /v1/responses with "stream": true.
• Every gpt-oss config under examples/configs (curated/gpt-oss-120b-latency.yaml, curated/gpt-oss-120b-throughput.yaml and database/openai/gpt-oss-120b/*) sets stream_interval: 20. Speculative decoding (e.g. Eagle3) also puts several tokens into one response, at any stream_interval.
• Found while reading the Responses streaming path and reproduced on CPU with the repository code (snippet below). No GPU run.

Who can help?

No response

Information

• The official example scripts
• My own modified scripts

Tasks

• An officially supported task in the examples folder (such as GLUE/SQuAD, ...)
• My own task or dataset (give details below)

Reproduction

_generate_streaming_event_harmony (tensorrt_llm/serve/responses_utils.py) feeds all tokens of a response to the Harmony parser in one call and then reads parser.last_content_delta and parser.state. Both describe only the last token processed, so when a response carries N tokens:

• the content deltas of the other N-1 tokens are never streamed;
• when a message ends inside a response (for example the analysis <|end|> followed by the final channel's header), the parser is past EXPECT_START by the end of the response. The reasoning item then gets no done events, the output index is not advanced, no response.output_item.added is sent for the message, and the answer deltas are attributed to the reasoning item.

The snippet drives the processor the server uses, with the tokens of one short gpt-oss answer grouped 1, 2 and 20 per response:

import json
from types import SimpleNamespace

from tensorrt_llm.executor.result import CompletionOutput
from tensorrt_llm.serve.harmony_adapter import get_harmony_adapter
from tensorrt_llm.serve.openai_protocol import ResponsesRequest
from tensorrt_llm.serve.responses_utils import ResponsesStreamingProcessor

REASONING = "The user asks where the order is. I should look it up and answer briefly."
ANSWER = "Your order #W2378156 shipped on Monday and should arrive by Thursday."
tokens = get_harmony_adapter().encoding.encode(
    f"<|channel|>analysis<|message|>{REASONING}<|end|>"
    f"<|start|>assistant<|channel|>final<|message|>{ANSWER}<|return|>",
    allowed_special="all")


def stream(tokens_per_response):
    request = ResponsesRequest(model="gpt-oss", input="Where is my order?", stream=True)
    processor = ResponsesStreamingProcessor(request, request.to_sampling_params(), "gpt-oss",
                                            use_harmony=True)
    result = SimpleNamespace(outputs=[CompletionOutput(index=0)], _done=False)
    output = result.outputs[0]
    events = []
    for start in range(0, len(tokens), tokens_per_response):
        output._last_token_ids_len = len(output.token_ids)
        output.token_ids.extend(tokens[start:start + tokens_per_response])
        result._done = start + tokens_per_response >= len(tokens)
        events += [json.loads(frame.split("data: ", 1)[1])
                   for frame in processor.process_single_output(result)]
    return events


for n in (1, 2, 20):
    events = stream(n)
    reasoning = "".join(e["delta"] for e in events if e["type"] == "response.reasoning_text.delta")
    text = "".join(e["delta"] for e in events if e["type"] == "response.output_text.delta")
    added = [(e["output_index"], e["item"]["type"]) for e in events
             if e["type"] == "response.output_item.added"]
    print(f"{n:2d} tokens/response: reasoning={reasoning!r}")
    print(f"{'':19}text={text!r}")
    print(f"{'':19}output_item.added={added}")

Output on main:

 1 tokens/response: reasoning='The user asks where the order is. I should look it up and answer briefly.'
                   text='Your order #W2378156 shipped on Monday and should arrive by Thursday.'
                   output_item.added=[(0, 'reasoning'), (1, 'message')]
 2 tokens/response: reasoning='The asks the is I look up answer.'
                   text=' orderW815 shipped Monday should by.'
                   output_item.added=[(0, 'reasoning')]
20 tokens/response: reasoning='.'
                   text=' by'
                   output_item.added=[(0, 'reasoning')]

Expected behavior

The streamed events describe the same output whatever the number of tokens per response: the whole reasoning and answer arrive as deltas, and each item is announced and closed.

actual behavior

With 20 tokens per response, as in the shipped gpt-oss configs, the streamed answer is ' by' and the streamed reasoning is '.', and the message item is never announced. The terminal response.completed snapshot is built from the whole output and is correct, so a client that renders the stream shows a different answer than the final snapshot.

additional notes

• The Chat Completions Harmony path is not affected: HarmonyStreamState.process_token_batch handles each token of a batch.
• [TRTLLM-8489][feat] Support stream interval for Responses API #10109 ("Support stream interval for Responses API") targeted this path and was closed in the August backlog cleanup.
• [#16377][fix] Keep Harmony streaming output independent of token batch size #22043 is about a different batch-size dependence in harmony_adapter.py (a token the parser rejects drops the rest of the batch).
• I have a fix with a CPU regression test and will open a PR.