[Mgluhovskoi]

Description

Integrate the Beam 52-layer NVFP4 baseline with standard LLM() and trtllm-serve paths: ModelingV2 routing and engine adapter, registered beam_nvfp4 checkpoint loader, tokenizer, Harmony parser, serving documentation and the public GSM8K evaluator.

Part 3/4 of TRTLLM-17024, implementing TRTLLM-17362. Depends on #21471 and #21472.

The runnable target is introduced here alongside its engine-admission tests and named TestModelingV2BeamFullModelNvfp4Sm103Tp4 accuracy gate. Earlier parts contain directly constructed components only. Component corrections are inherited, and the loader guard accepts the documented checkpoint_format="beam_nvfp4" path while rejecting incompatible explicit loaders.

The baseline is TP4 on four GB300s, NVFP4 experts, BF16 KV, eager execution and overlap disabled. Speculative decoding, FP8 KV, disaggregated serving and other optimizations are outside this baseline.

Review order

#21471 → #21472 → #21473 → #21474. All target main; later PRs are cumulative.

Incremental part-3 diff

Validation

At 62a184ba5383e9449f6ee906b13517bf045b6215:

• 120 host-side component, engine-contract and static ModelingV2 checks pass, including the named-gate requirement. Native dependencies are stubbed in this host check.
• 79 public-evaluator/gate-wiring CPU tests pass.
• Actual parser module passes 9 one-shot cases and 646 streaming splits; 97 attention-call keyword bindings match the shared interface.
• Pre-commit and AST test-list validation pass.

The named gate starts trtllm-serve with modeling_v2: require and runs the pinned public first-100 GSM8K protocol: at least 95 correct, with execution/parsing/truncation failures rejected. It is a manual four-GB300 qualification with a 45-minute cap, not an L0 or full-dataset CI run. Authorized checkpoint and dataset inputs must be staged externally; missing inputs fail.

The gate has not been executed on a matching GPU build for this corrected extraction. Native accuracy/timing qualification is required before scheduling it in LR/nightly or claiming this baseline qualified. Remaining human review, matching-build GPU validation and required CI/code-owner approval remain outstanding. No checkpoint weights or private evidence are included.

Dev Engineer Review

Adds Beam support across the PyTorch backend and the ModelingV2 serving path. The changes include Beam model components and operators, TP4 NVFP4 checkpoint loading and routing, tokenizer and Harmony parser support, serving documentation, and a pinned GSM8K evaluator.

The engine route accepts a narrow four-rank SM 10.3 NVFP4 configuration and refuses unsupported serving options. The documented baseline uses BF16 KV cache and disables overlap scheduling. Speculative decoding, FP8 KV cache, and disaggregated serving are outside its scope.

The evaluator supports managed-server and existing-endpoint modes, resumable records, model identity checks, and explicit failure verdicts. The manual gate requires at least 95 correct answers on the pinned first 100 GSM8K questions. The supplied context says matching-build GPU qualification remains outstanding. It also reports one successful PR job and one failed merge-request pipeline; investigation and a rerun remain outstanding.

QA Engineer Review

The change adds tests for Beam operators and components, checkpoint layouts and loading, routing and engine contracts, diagnostic behavior, Harmony parsing, evaluator protocol and records, and the accuracy gate. The l0_cpu.yml list includes component-contract, engine-contract, baseline-gate, diagnostic-mode, loading-contract, and serving-parser tests. It does not include tests/unittest/tools/test_nvfp4_serving_gsm8k_first100.py. The manual GSM8K gate is listed in manual_gb300_beam_baseline_gsm8k_first100.yml for the four-GB300 profile, with a 45-second test-list timeout.

Coverage verdict: needs follow-up. CPU contract tests cover key evaluator and serving paths, but they do not establish GPU compatibility. Run the manual accuracy gate on a matching GPU build and investigate the reported failed merge-request pipeline.

Per-File QA Perspective

Documentation and evaluator

• docs/source/deployment-guide/deployment-guide-for-beam-on-trtllm.md: Verify the documented serving configuration and manual accuracy criteria against runtime behavior.
• docs/source/deployment-guide/index.rst: Adds the Beam guide to the deployment documentation tree.
• examples/llm-api/eval_gsm8k.py: Verify CLI validation, endpoint and managed-server modes, runner arguments, and exit codes.
• examples/llm-api/gsm8k_eval/README.md: Check that usage and failure documentation matches evaluator behavior.
• examples/llm-api/gsm8k_eval/__init__.py: Adds package documentation; no runtime behavior change is described.
• examples/llm-api/gsm8k_eval/manifest.json: Verify the pinned dataset, question and exemplar selection, and recorded hashes.
• examples/llm-api/gsm8k_eval/protocol.py: Verify manifest validation, prompt construction, selection, and scoring.
• examples/llm-api/gsm8k_eval/records.py: Verify reply classification, ledger validation and resume behavior, and verdict rules.
• examples/llm-api/gsm8k_eval/runner.py: Verify endpoint and managed-server execution, model consistency, resume identity, and interruption handling.
• examples/llm-api/gsm8k_eval/server.py: Verify server startup, address reporting, process cleanup, and optional GPU memory polling.

ModelingV2 operators and Beam model

• tensorrt_llm/_torch/_experimental/modeling_v2/__init__.py: Verify the resolver documentation reflects its import behavior.
• tensorrt_llm/_torch/_experimental/modeling_v2/catalog/activation/beam_logit_softcap.py, flashinfer_silu_and_mul.py, and swiglu_fp32.py: Verify operator formulas, dtype and shape constraints, and new SM 103 cases.
• tensorrt_llm/_torch/_experimental/modeling_v2/catalog/attention/beam_neox_rope.py, beam_nope_query_scale.py, beam_output_gate.py, and thop_attention.py: Verify RoPE and query scaling, output gating, attention windows, and FMHA case configurations.
• tensorrt_llm/_torch/_experimental/modeling_v2/catalog/gemm/cublas_mm.py, catalog/moe/beam_dispatch_combine.py, beam_grouped_topk_router.py, catalog/norm/beam_rmsnorm_fp32.py, flashinfer_rmsnorm.py, and catalog/residual/beam_depth_residual.py: Verify projection coverage, routing and accumulation order, normalization, and residual scaling.
• tensorrt_llm/_torch/_experimental/modeling_v2/catalog/residual/__init__.py: Adds package metadata; no runtime behavior change is described.
• tensorrt_llm/_torch/_experimental/modeling_v2/models/beam/README.md: Check documented target constraints and unsupported serving features against implementation.
• tensorrt_llm/_torch/_experimental/modeling_v2/models/beam/__init__.py and the Beam component package __init__.py files: These add package metadata; no runtime behavior change is described.
• tensorrt_llm/_torch/_experimental/modeling_v2/models/beam/dense_block_bf16__sm_103__tp1/block.py, moe_bf16__sm_103__tp1/moe.py, moe_expert_bf16__sm_103__tp1/expert.py, moe_router_bf16__sm_103__tp1/router.py, and sparse_block_bf16__sm_103__tp1/block.py: Verify supported configurations, weight-loading guards, attention and routing behavior, and CUDA device checks.
• tensorrt_llm/_torch/_experimental/modeling_v2/models/beam/small_model_prefill_bf16__sm_103__tp1/model.py and generation.py: Verify cache lifecycle, prefix reuse, weight-change detection, and greedy generation.
• tensorrt_llm/_torch/_experimental/modeling_v2/models/beam/full_model_nvfp4__sm_103__tp4/_attention.py, component.py, component_bf16.py, dispatch_combine.py, expert.py, layout.py, layout_bf16.py, loader.py, and loader_bf16.py: Verify attention metadata and cache checks, rank-local checkpoint reads, quantization layout, collective behavior, and load-failure handling.
• tensorrt_llm/_torch/_experimental/modeling_v2/models/beam/full_model_nvfp4__sm_103__tp4/model.py and modeling.py: Verify full-model execution, prefill/decode dispatch, request restrictions, and logit selection.
• tensorrt_llm/_torch/_experimental/modeling_v2/models/beam/full_model_nvfp4__sm_103__tp4/weights.py and models/beam/routing.py: Verify checkpoint-loader integration and engine admission/refusal behavior.
• tensorrt_llm/_torch/_experimental/modeling_v2/models/__init__.py: Verify importing the package loads Beam routing without unintended registration or import effects.
• tensorrt_llm/_torch/models/checkpoints/__init__.py, tensorrt_llm/_torch/models/checkpoints/beam/checkpoint_loader.py, and tokenizer.py: Verify public exports, beam_nvfp4 registration, and tokenizer loading.
• tensorrt_llm/_torch/models/checkpoints/beam/__init__.py: Adds package metadata; no runtime behavior change is described.
• tensorrt_llm/_torch/models/modeling_beam.py: Verify model computation, configuration validation, and strict checkpoint mapping and loading.
• tensorrt_llm/_torch/pyexecutor/config_utils.py: Verify Beam config construction preserves the checkpoint path and supplied settings.
• tensorrt_llm/llmapi/reasoning_parser.py: Verify complete and streamed Harmony channel parsing, including termination markers.

Tests and test lists

• tests/integration/defs/accuracy/modeling_v2/test_beam.py: Adds the manual first-100 GSM8K gate. It is listed in the manual GB300 test list.
• tests/integration/test_lists/test-db/manual_gb300_beam_baseline_gsm8k_first100.yml: Adds the manual gate selector for the four-GB300 profile. Verify its selectors and timeout against the test.
• tests/integration/test_lists/test-db/l0_cpu.yml: Adds CPU component-contract, engine-contract, baseline-gate, diagnostic, loading, and serving tests. The evaluator protocol/records test is not listed.
• tests/unittest/tools/test_nvfp4_serving_gsm8k_first100.py: Covers protocol, scoring, dataset checks, reply handling, and ledger behavior with synthetic data. It is not listed in l0_cpu.yml.
• tests/unittest/tools/test_beam_baseline_accuracy_gate.py: Covers gate arguments, failure propagation, and checkpoint-path validation. It is listed in l0_cpu.yml.
• tests/unittest/_torch/modeling_v2/models/beam/test_component_contracts.py and test_engine_contracts.py: Cover cache and checkpoint contracts, request validation, engine-option refusal, and checkpoint-loader routing. Both are listed in l0_cpu.yml.
• tests/unittest/_torch/modeling_v2/models/beam/test_loading_contracts.py and test_diagnostic_mode.py: Cover loading failures, weight reload behavior, and rank agreement. Both are listed in l0_cpu.yml.
• tests/unittest/_torch/modeling_v2/models/beam/full_model_nvfp4__sm_103__tp4/test_serving.py: Covers parser registration and pretrained config loading. It is listed in l0_cpu.yml.
• tests/unittest/_torch/modeling_v2/models/beam/full_model_nvfp4__sm_103__tp4/test_batching.py: Covers batched attention metadata and logit-row selection. No test-list entry is shown.
• tests/unittest/_torch/modeling_v2/models/beam/checkpoint_layout_bf16__sm_103__tp4/test_checkpoint_layout.py and checkpoint_layout_nvfp4__sm_103__tp4/test_checkpoint_layout_nvfp4.py: Cover layout partitioning, checkpoint metadata, rank slices, and CUDA loading paths. No test-list entries are shown.
• tests/unittest/_torch/modeling_v2/models/beam/_beam_component_support.py: Adds shared test fixtures and evidence helpers; it is not a test module listed in a test list.
• tests/unittest/_torch/modeling_v2/models/beam/dense_block_bf16__sm_103__tp1/_dense_block_reference.py: Adds a Torch reference for dense-block calculations; no test-list entry is shown.
• tests/unittest/_torch/modeling_v2/models/beam/dense_block_bf16__sm_103__tp1/test_dense_block.py: Covers dense-block reference comparisons, restrictions, execution paths, and KV-cache writes. No test-list entry is shown.
• tests/unittest/_torch/modeling_v2/models/beam/moe_bf16__sm_103__tp1/_moe_reference.py: Adds a Torch reference for MoE routing and expert computations; no test-list entry is shown.
• tests/unittest/_torch/modeling_v2/models/beam/moe_bf16__sm_103__tp1/test_moe.py: Covers MoE routing, numerical behavior, configuration, weights, and execution paths. No test-list entry is shown.
• tests/unittest/_torch/modeling_v2/models/beam/moe_dispatch_combine_bf16__sm_103__tp1/_dispatch_combine_cases.py: Adds generated fixtures and a golden dispatch/combine implementation; no test-list entry is shown.
• tests/unittest/_torch/modeling_v2/models/beam/moe_dispatch_combine_bf16__sm_103__tp1/moe/test_modeling_v2_beam_dispatch_combine.py and test_dispatch_combine.py: Cover operator certification and component routing, output ordering, and input immutability. No test-list entries are shown.
• tests/unittest/_torch/modeling_v2/models/beam/moe_expert_bf16__sm_103__tp1/_expert_cases.py: Adds numerical controls, fixtures, and a golden expert implementation; no test-list entry is shown.
• tests/unittest/_torch/modeling_v2/models/beam/moe_expert_bf16__sm_103__tp1/activation/test_modeling_v2_swiglu_fp32.py: Covers SwiGLU reference and precision behavior, with validation guards. No test-list entry is shown.
• tests/unittest/_torch/modeling_v2/models/beam/moe_expert_bf16__sm_103__tp1/test_expert.py: Covers expert loading, validation, numerical stages, and execution behavior. No test-list entry is shown.
• tests/unittest/_torch/modeling_v2/models/beam/moe_router_bf16__sm_103__tp1/_router_cases.py: Adds router fixtures and comparison helpers; no test-list entry is shown.
• tests/unittest/_torch/modeling_v2/models/beam/moe_router_bf16__sm_103__tp1/moe/test_modeling_v2_beam_grouped_topk_router.py and test_router.py: Cover router reference comparisons, routing fixtures, validation, and numerical controls. No test-list entries are shown.
• tests/unittest/_torch/modeling_v2/models/beam/small_model_prefill_bf16__sm_103__tp1/test_beam_logit_softcap.py: Covers softcap operator behavior and validation. No test-list entry is shown.
• tests/unittest/_torch/modeling_v2/models/beam/small_model_prefill_bf16__sm_103__tp1/test_cached_generation.py: Covers generation lifecycle, cache writes, prefix reuse, and weight-change detection. No test-list entry is shown.
• tests/unittest/_torch/modeling_v2/models/beam/small_model_prefill_bf16__sm_103__tp1/test_small_model_prefill.py: Covers prefill validation, execution order, attention behavior, and request isolation. No test-list entry is shown.
• tests/unittest/_torch/modeling_v2/models/beam/sparse_block_bf16__sm_103__tp1/_sparse_block_reference.py: Adds a Torch reference for sparse-block calculations; no test-list entry is shown.
• tests/unittest/_torch/modeling_v2/models/beam/sparse_block_bf16__sm_103__tp1/test_sparse_block.py: Covers sparse-block output, routing, configuration, and cache behavior. No test-list entry is shown.
• tests/unittest/_torch/modeling_v2/attention/test_modeling_v2_beam_neox_rope.py, test_modeling_v2_beam_nope_query_scale.py, and test_modeling_v2_beam_output_gate.py: Cover operator references, shape and dtype behavior, precision, and validation guards. No test-list entries are shown.
• tests/unittest/_torch/modeling_v2/attention/test_modeling_v2_thop_attention.py: Extends attention reference and paged-context coverage for Beam sliding and full attention. No test-list entry is shown.
• tests/unittest/_torch/modeling_v2/residual/test_modeling_v2_beam_depth_residual.py: Covers residual coefficients, reference output, and validation guards. No test-list entry is shown.
• tests/unittest/_torch/modeling_v2/models/beam/checkpoint_layout_bf16__sm_103__tp4/norm/test_modeling_v2_beam_rmsnorm_fp32.py: Covers RMSNorm reference precision, output properties, and validation. No test-list entry is shown.

[Mgluhovskoi]

/bot run

[Mgluhovskoi]

/bot run

[tensorrt-cicd]

PR_Github #83521 [ run ] triggered by Bot. Commit: 079c4d6 Link to invocation

[tensorrt-cicd]

PR_Github #83515 [ run ] completed with state ABORTED. Commit: d482337

Link to invocation

[Mgluhovskoi]

/bot run

[Mgluhovskoi]

/bot run

[tensorrt-cicd]

PR_Github #83527 [ run ] triggered by Bot. Commit: 4b02f8e Link to invocation

[tensorrt-cicd]

PR_Github #83521 [ run ] completed with state ABORTED. Commit: 079c4d6

Link to invocation

[tensorrt-cicd]

PR_Github #83532 [ run ] triggered by Bot. Commit: 4b02f8e Link to invocation

[tensorrt-cicd]

PR_Github #83527 [ run ] completed with state ABORTED. Commit: 4b02f8e

Link to invocation

[Mgluhovskoi]

/bot run

[tensorrt-cicd]

PR_Github #83535 [ run ] triggered by Bot. Commit: cd11961 Link to invocation

[tensorrt-cicd]

PR_Github #83532 [ run ] completed with state ABORTED. Commit: 4b02f8e

Link to invocation

[sunnyqgg]

I re-checked the Blockers from my review of 2026-11-04 at cd11961. None of the 14 open Blockers has a reply or a code change since 2ab3329. At cd11961, three of them still fail pre-merge CI:

• CPU-Build_Docs: eval_gsm8k.py has no example metadata.
• l0_cpu test_modeling_v2_ci_selection.py::test_stock_code_does_not_reach_modeling_v2: checkpoints/beam/checkpoint_loader.py:23.
• A10 api_stability test_serve_cli: beam_harmony_text is missing from trtllm_serve_cli.yaml:437.

Resolved or accepted:

• model.py:129 now accepts the KV-cache estimator's full-context retention on sliding layers while the op keeps the 512 window, so the guide config gets past estimation (5c27f5c, from [TRTLLM-17361][test] Add the consolidated Beam StairCase test suite #21472)

Still open, so my request for changes stands:

• eval_gsm8k.py breaks the docs build (CPU-Build_Docs): unchanged, see its thread.
• Stock checkpoint_loader.py imports a ModelingV2 target (stock-import test fails): unchanged, see its thread.
• Structured output on a Beam server returns empty content: unchanged, see its thread.
• Quoted channel delimiters truncate or alter the final answer: unchanged, see its thread.
• Postprocess workers cannot load the Beam tokenizer; requests hang: unchanged, see its thread.
• BeamFastTokenizer truncates prompts from the right: unchanged, see its thread.
• Second interrupt during stop() leaves the managed server running: unchanged, see its thread.
• Accepted CLI inputs or replies abort the run; torn ledger lines block resume: unchanged, see its thread.
• stop() can SIGKILL a reused PID: unchanged, see its thread.
• --server-env values (credentials) recorded in plaintext in the run identity: unchanged, see its thread.
• Unbounded poller.wait() in teardown; unmeasured GPU cost reported as 0: unchanged, see its thread.
• api_stability reference lacks beam_harmony_text (A10 test_serve_cli fails): Still fails at cd11961. beam_harmony_text is registered (reasoning_parser.py:1415), so it is part of the --reasoning_parser Choice (commands/serve.py:1377-1379). tests/unittest/api_stability/references/trtllm_serve_cli.yaml:437 still omits it. test_serve_cli compares type exactly (test_serve_cli.py:208), and unittest/api_stability runs in pre-merge A10 (l0_a10.yml:262), so that stage fails. Add 'beam_harmony_text' to that Choice, right after 'auto'.
• Evaluator sends reasoning_effort kwargs to every endpoint (HTTP 400 on other templates): Unchanged at cd11961. gsm8k_eval/protocol.py:78 still sends chat_template_kwargs: {"reasoning_effort": 128} on every request. trtllm-serve rejects caller kwargs that the active template does not read unless TRTLLM_ALLOW_UNUSED_CHAT_TEMPLATE_KWARGS=1 is set (inputs/chat_template_guard.py:153-166), and reasoning_effort is not always allowed. Against a model whose template does not reference it (e.g. Llama-3, Qwen3), every question gets HTTP 400 and the run exits 1. Yet gsm8k_eval/README.md:30-31 offers any OpenAI-compatible endpoint. Make these kwargs a per-run option recorded in the run identity, or give the protocol a Beam-specific id and document that it needs a template that reads reasoning_effort.
• Pre-header text differs between streamed and non-streamed parsing: Still reproduces at cd11961. Text before the first header is emitted as content when streamed, but parse() folds it into _header (reasoning_parser.py:1484-1490). '\n<|channel|>final<|message|>Paris.<|return|>' streams as '\nParis.' but parses to 'Paris.'. With a template that prefills <|channel|>, 'final<|message|>42 is the answer.<|return|>' streamed as ['f', 'inal...'] gives content 'f', while parse() gives '42 is the answer.'. Treat pre-header text the same way in both paths: hold it until a delimiter or finish() decides it. Add the leading-newline and prefilled-header replies to REPLIES in test_serving.py.

[tensorrt-cicd]

PR_Github #83535 [ run ] completed with state SUCCESS. Commit: cd11961
/LLM/main/L0_MergeRequest_PR pipeline #69056 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

• Please check the failed tests and fix your PR
• If you cannot view the failures, ask the CI triggerer to share details
• Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

[BowenFu]

Could you update the tests to import _attention and dispatch_combine from models.beam after moving those modules? This import still targets full_model_nvfp4__sm_103__tp4, which no longer contains _attention, so the dense-block suite selected by l0_b300.yml fails during collection. The MoE suite has the same stale dispatch_combine import.

[Mgluhovskoi]

/bot run

[tensorrt-cicd]

PR_Github #83584 [ run ] triggered by Bot. Commit: 4da602a Link to invocation

[tensorrt-cicd]

PR_Github #83584 [ run ] completed with state SUCCESS. Commit: 4da602a
/LLM/main/L0_MergeRequest_PR pipeline #69099 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

• Please check the failed tests and fix your PR
• If you cannot view the failures, ask the CI triggerer to share details
• Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

[sunnyqgg]

I re-checked the Blockers from my review of 2026-11-04 at 37f775f. At 37f775f the three Blockers that failed pre-merge CI (docs build, stock-import test, api_stability reference) are fixed and the guide config gets past KV-cache estimation, but the other eleven Blockers have no reply and their code is unchanged since 2ab3329.

Resolved or accepted:

• examples/llm-api/eval_gsm8k.py:1-3 now has :title/:order/:section Customization metadata, so the docs build indexes it (4da602a).
• checkpoints/beam/checkpoint_loader.py:57-69 now delegates to the selected model's load_checkpoint() without importing a ModelingV2 target, so the stock-import test no longer flags it (81a1d6f).
• full_model_nvfp4__sm_103__tp4/model.py:129 accepts the estimator's full-context retention on sliding layers while the op keeps the 512 window, so the guide config gets past KV-cache estimation (5c27f5c, from [TRTLLM-17361][test] Add the consolidated Beam StairCase test suite #21472).
• trtllm_serve_cli.yaml:437 now includes beam_harmony_text, matching the registered parsers (2beb087).

Still open, so my request for changes stands:

• Structured output on a Beam server returns empty content: unchanged, see its thread.
• Quoted channel delimiters truncate or alter the final answer: unchanged, see its thread.
• Postprocess workers cannot load the Beam tokenizer; requests hang: unchanged, see its thread.
• BeamFastTokenizer truncates prompts from the right: unchanged, see its thread.
• Second interrupt during stop() leaves the managed server running: unchanged, see its thread.
• Accepted CLI inputs or replies abort the run; torn ledger lines block resume: unchanged, see its thread.
• stop() can SIGKILL a reused PID: unchanged, see its thread.
• --server-env values (credentials) recorded in plaintext in the run identity: unchanged, see its thread.
• Unbounded poller.wait() in teardown; unmeasured GPU cost reported as 0: unchanged, see its thread.
• Evaluator sends reasoning_effort kwargs to every endpoint (HTTP 400 on other templates): Still open at 37f775f. gsm8k_eval/protocol.py:78 still sends chat_template_kwargs: {"reasoning_effort": 128} on every request. trtllm-serve rejects kwargs the active template does not read unless TRTLLM_ALLOW_UNUSED_CHAT_TEMPLATE_KWARGS=1 is set (inputs/chat_template_guard.py:153-166), and reasoning_effort is not always allowed. Against a model whose template does not reference it (e.g. Llama-3, Qwen3), every question gets HTTP 400 and the run exits 1, while gsm8k_eval/README.md:30-31 offers any OpenAI-compatible frontend. Make these kwargs a per-run option recorded in the run identity, or give the protocol a Beam-specific id and document that it needs a template that reads reasoning_effort.
• Pre-header text differs between streamed and non-streamed parsing: Still reproduces at 37f775f. Text before the first header is emitted as content when streamed, but parse() folds it into _header (reasoning_parser.py:1484-1490). '\n<|channel|>final<|message|>Paris.<|return|>' streams as '\nParis.' but parses to 'Paris.'. With a template that prefills <|channel|>, 'final<|message|>42 is the answer.<|return|>' streamed as ['f', 'inal...'] gives content 'f', while parse() gives '42 is the answer.'. Hold pre-header text until a delimiter or finish() decides it, the same way in both paths, and add the leading-newline and prefilled-header replies to REPLIES (test_serving.py:27).