[JunyiXu-nv]

Stacked on #21448 ([TRTLLM-17261][refactor] Add the trtllm-serve router skeleton and move /health, /version and /v1/models into ops.system), which is itself stacked on #21406 ([TRTLLM-17261][test] Add route, middleware-order and shim-identity snapshots and a CPU test seam for trtllm-serve): the first commits below belong to those PRs and disappear from this diff when they merge; this PR's own change is the last 4 commits.

@coderabbitai summary

Description

Move the eight VisualGen request and response models of trtllm-serve (ImageGenerationRequest, ImageEditRequest, ImageObject, ImageGenerationResponse, MediaReferenceItem, VideoGenerationRequest, VideoJob, VideoJobList) out of tensorrt_llm/serve/openai_protocol.py into the new package tensorrt_llm/serve/api/visual_gen/ (protocol.py), unchanged, so that VisualGen protocol changes land in the package that CI selection will later claim for VisualGen-only pull requests. openai_protocol keeps every old name importable through a lazy module __getattr__: importing it no longer loads the VisualGen models or tensorrt_llm.visual_gen, and the re-export is part of openai_protocol's own interface, so it never warns and has no removal release. The same PR gives tensorrt_llm/serve/api/visual_gen/ to the VisualGen team in .github/CODEOWNERS and adds an api-stability gate for the pinned VisualGen models under a visual_gen/ path, because once the package is claimed, the existing serve API gate would not run on VisualGen-only PRs.

Part of the trtllm-serve refactor (epic TRTLLM-17056, task TRTLLM-17264: VisualGen isolation). Builds on #21448 (the router skeleton creates tensorrt_llm/serve/api/) and #21406 (the shim identity table this PR adds rows to). Should follow #21408 ([TRTLLM-17264][refactor] Merge the two VisualGen validation-error renderers), which is independent of this change.

Behavior

No behavior change. OpenAPI component names may change with the move; they are not a contract.

Until the VisualGen handlers move out of openai_server.py, every LLM role still imports the new module through openai_server.py's image-model imports; the import-isolation tests arrive with the PRs that move the handlers and let CI selection claim serve/api/visual_gen/.

Moves

Pure-move check

## Move spec

```yaml
moves:
- from: tensorrt_llm/serve/openai_protocol.py
  to: tensorrt_llm/serve/api/visual_gen/protocol.py
  group: B
  symbols:
  - ImageGenerationRequest
  - ImageEditRequest
  - ImageObject
  - ImageGenerationResponse
  - MediaReferenceItem
  - VideoGenerationRequest
  - VideoJob
  - VideoJobList
shims:
- module: tensorrt_llm/serve/openai_protocol.py
  style: lazy-names
  warn: false
  names:
  - ImageGenerationRequest
  - ImageEditRequest
  - ImageObject
  - ImageGenerationResponse
  - MediaReferenceItem
  - VideoGenerationRequest
  - VideoJob
  - VideoJobList
rewrite_internal_imports: true

Pure-move check: base d3aacdd, head 18a8b2f, tools 517fc468330a

Symbol	From	To	Result
ImageEditRequest	tensorrt_llm/serve/openai_protocol.py	tensorrt_llm/serve/api/visual_gen/protocol.py	verbatim
ImageGenerationRequest	tensorrt_llm/serve/openai_protocol.py	tensorrt_llm/serve/api/visual_gen/protocol.py	verbatim
ImageGenerationResponse	tensorrt_llm/serve/openai_protocol.py	tensorrt_llm/serve/api/visual_gen/protocol.py	verbatim
ImageObject	tensorrt_llm/serve/openai_protocol.py	tensorrt_llm/serve/api/visual_gen/protocol.py	verbatim
MediaReferenceItem	tensorrt_llm/serve/openai_protocol.py	tensorrt_llm/serve/api/visual_gen/protocol.py	verbatim
VideoGenerationRequest	tensorrt_llm/serve/openai_protocol.py	tensorrt_llm/serve/api/visual_gen/protocol.py	verbatim
VideoJobList	tensorrt_llm/serve/openai_protocol.py	tensorrt_llm/serve/api/visual_gen/protocol.py	verbatim
VideoJob	tensorrt_llm/serve/openai_protocol.py	tensorrt_llm/serve/api/visual_gen/protocol.py	verbatim

8 definitions moved verbatim; 0 differences; shim exports 8 names

</details>

The eight models are moved verbatim; the only hand edits in `openai_protocol.py` are the removal of the moved block's section comment and of the three imports only it used (`UploadFile`, `AliasChoices`, `MediaRole`). The lazy re-export block written at the end of `openai_protocol.py`:

```python
# Old import path of names that moved to
# tensorrt_llm.serve.api.visual_gen.protocol.
# They are re-exported lazily (PEP 562): importing this module does not import
# their new module.
_LAZY_NAMES = {
    "ImageEditRequest": "tensorrt_llm.serve.api.visual_gen.protocol",
    "ImageGenerationRequest": "tensorrt_llm.serve.api.visual_gen.protocol",
    "ImageGenerationResponse": "tensorrt_llm.serve.api.visual_gen.protocol",
    "ImageObject": "tensorrt_llm.serve.api.visual_gen.protocol",
    "MediaReferenceItem": "tensorrt_llm.serve.api.visual_gen.protocol",
    "VideoGenerationRequest": "tensorrt_llm.serve.api.visual_gen.protocol",
    "VideoJob": "tensorrt_llm.serve.api.visual_gen.protocol",
    "VideoJobList": "tensorrt_llm.serve.api.visual_gen.protocol",
}


def __getattr__(name):
    module_name = _LAZY_NAMES.get(name)
    if module_name is None:
        raise AttributeError(f"module {__name__!r} has no attribute {name!r}")
    value = getattr(importlib.import_module(module_name), name)
    globals()[name] = value  # cache: later access skips __getattr__
    return value


def __dir__():
    return sorted(set(globals()) | set(_LAZY_NAMES))

One observable difference for star-imports: from tensorrt_llm.serve.openai_protocol import * no longer brings in the eight names (the module has no __all__, and lazily re-exported names are invisible to a star-import); nothing in the repository star-imports them, and explicit imports keep working.

Coordination with work in flight: #21226 ([TRTLLM-16807][feat] Support combined Omni references in video serving) edits VideoGenerationRequest.image_reference, video_reference and audio_reference inside the moved block. If it lands first, this move is regenerated on top of it right before merging; if it lands second, it re-applies that hunk to tensorrt_llm/serve/api/visual_gen/protocol.py (same text, new file). #21408 adds import lines next to the ones rewritten here; whichever lands second resolves a trivial import-block conflict.

This re-export is openai_protocol's permanent interface: no FutureWarning, no removal release, and the eight names have identity-only rows in the VisualGen shim identity table (tests/unittest/serve/api/visual_gen/test_shim_identity_visual_gen.py: no warning check, no ban on the old path). When openai_protocol.py is later split into a package with a lazy facade, the eight names become rows of that facade's name-to-module table and this block disappears into it. No shim is kept and no old path is deleted: openai_protocol keeps its path. Code inside the repository imports the new module (every importer is rewritten in this PR). History: git blame -C -C tensorrt_llm/serve/api/visual_gen/protocol.py attributes the moved lines to their original commits.

CI bookkeeping

• Multi-GPU table: none. tensorrt_llm/serve/api/visual_gen/ is deliberately not added: VisualGen serving is to be claimed by CI selection for VisualGen-only PRs, and a prefix there would drag it into the multi-GPU stages. tensorrt_llm/serve/openai_server.py is in the table and its import lines change, so this PR runs the multi-GPU stages (approval label needed).
• CODEOWNERS: /tensorrt_llm/serve/api/visual_gen @NVIDIA/trt-llm-torch-visual-gen-devs, in the VisualGen section (after the /tensorrt_llm/serve rule, so it wins). Effect: VisualGen serving files (protocol.py now, the handlers later) need the VisualGen team's approval and no longer need runtime-devs.
• legacy-files.txt: tensorrt_llm/serve/api/visual_gen/protocol.py added (legacy-formatted group, because its code comes from openai_protocol.py, which is in that group); derived configs regenerated with scripts/legacy_utils.py gen-configs (ruff-legacy.toml, the managed blocks of pyproject.toml and .pre-commit-config.yaml); ruff-legacy-baseline.json: none (openai_protocol.py has no baseline entry, and ruff check --config ruff-legacy.toml passes on both files).
• Test lists: tests/integration/test_lists/test-db/l0_cpu.yml gains unittest/api_stability/visual_gen/test_visual_gen_serve_api.py right after unittest/api_stability. The new file is also collected by the directory entry on unclaimed PRs (about one second twice); its own entry is what keeps it running on VisualGen-only PRs, whose CI selection keeps only entries containing visual_gen/.
• Docs: tensorrt_llm/serve/scripts/BENCHMARKING_VISUAL_GEN.md links the new file. The published module path tensorrt_llm.serve.scripts.benchmark_visual_gen does not change.

The VisualGen api-stability gate

tests/unittest/api_stability/visual_gen/test_visual_gen_serve_api.py loads the checks of test_serve_api.py and runs them on the models of references/trtllm_serve_api.yaml that resolve, through tensorrt_llm.serve.openai_protocol, to classes defined in tensorrt_llm.serve.api.visual_gen.protocol (today ImageGenerationRequest, ImageGenerationResponse, MediaReferenceItem, VideoGenerationRequest); it also checks that importing openai_protocol does not load the VisualGen module, that the eight names are listed by dir(), and that unknown names still raise AttributeError. It lives under tests/unittest/api_stability/ so that the API review committee stays its owner (the alternative, tests/unittest/_torch/visual_gen/, would have made the VisualGen team the only owner of a gate on their own models). The reference YAML is unchanged, so no api-compatible label is needed.

Test Coverage

All checks ran on a CPU-only host from this branch's worktree (PYTHONPATH overlay over a prebuilt wheel); GPU end-to-end tests are left to CI.

• Pure-move check on the pushed head: see "Moves" (8 definitions moved verbatim, 0 differences, 8 lazily re-exported names).
• pytest -m cpu_only tests/unittest/api_stability/visual_gen/test_visual_gen_serve_api.py → 9 passed (the 7 inherited checks on the VisualGen models + 2 lazy re-export tests). Mutation proofs, run and reverted: renaming a pinned model in protocol.py turns the gate red (7 errors: "Model 'ImageGenerationResponse' is in the stability reference YAML but not in tensorrt_llm.serve.openai_protocol"); an eager VisualGen import added to tensorrt_llm/serve/__init__.py fails the probe with the message naming the modules that hold the classes.
• pytest tests/unittest/api_stability/test_serve_api.py → 7 passed, unchanged: the reference YAML resolves through the lazy re-export.
• pytest -m cpu_only tests/unittest/_torch/visual_gen/test_trtllm_serve_endpoints.py tests/unittest/_torch/visual_gen/test_visual_gen_utils.py tests/unittest/_torch/visual_gen/test_visual_gen_params.py → 323 passed, the same count as on the base branch.
• Shim identity tables (tests/unittest/serve/test_shim_identity.py, tests/unittest/serve/api/visual_gen/test_shim_identity_visual_gen.py) → 11 passed with the 8 new identity-only rows.
• Harmony tests with -rs → 79 passed, no "not importable" skip; tests/unittest/llmapi/apps/test_serve_lazy_import.py → 5 passed; route snapshot (test_serve_routes.py, test_routes_visual_gen.py) → 36 passed, no route change; tests/unittest/serve and tests/unittest/serve/api/visual_gen/ → 112 passed (layering test included).
• Serve CPU seam suite (PYTHONPATH=tests python3 -m test_common.serve.cpu_tests, every test-db entry it selects): green except two environmental failures that reproduce identically on the base branch — test_render_standalone.py::TestCpuOnlyStartup::test_the_launcher_loads_a_cuda_stub_only_when_the_driver_is_missing (the trtllm-render console script is not installed in the overlay) and OpenEngine test_frontend_lifecycle.py child-process waits timing out on a heavily loaded host (14/14 on a quieter re-run).
• Fresh-interpreter checks: importing tensorrt_llm.serve.openai_protocol does not load tensorrt_llm.serve.api.visual_gen.protocol; the first read of each of the 8 names emits no FutureWarning or DeprecationWarning; each name is the same object as in the new module; unknown names raise AttributeError; dir() lists all 8.
• Property check of the lazy re-export (Hypothesis, 3 properties × 200 examples: identity and caching, unknown names, dir()), with a 6-mutation gate that every property catches (local harness, not committed).
• git grep for old-path imports of the 8 names under tensorrt_llm/ and tests/: none; MediaRole, UploadFile, AliasChoices in openai_protocol.py: none; no test patches these names.
• python scripts/legacy_utils.py check-configs passes; python scripts/legacy_utils.py lint-precommit on both protocol modules passes; pre-commit passes twice on every changed file with no further modification.
• python scripts/test_to_stage_mapping.py --tests "unittest/api_stability/visual_gen/test_visual_gen_serve_api.py" → the CPU-Generic stages; the file contains pytest.mark.cpu_only.
• CI test-selection dry run on this range: scope: None, no narrowed stages (the changed openai_protocol.py and openai_server.py are unclaimed, so the full suite runs).
• The two-FastAPI-version check is not needed: no route is added or moved.

PR Checklist

Please review the following before submitting your PR:

• PR description clearly explains what and why. If using CodeRabbit's summary, please make sure it makes sense.

• PR Follows TRT-LLM CODING GUIDELINES to the best of your knowledge.

• Test cases are provided for new code paths (see test instructions)

• Any new dependencies have been scanned for license and vulnerabilities

• I have run the self-review checklist on the PR

🤖 Generated with Claude Code