Skip to content

[https://nvbugs/6308608][fix] Apply one local media input policy to chat and VisualGen and rename its variables - #21528

Open
BoyueZ wants to merge 23 commits into
NVIDIA:mainfrom
BoyueZ:fix/6308608-local-media-policy-followup
Open

BoyueZ wants to merge 23 commits into
NVIDIA:mainfrom
BoyueZ:fix/6308608-local-media-policy-followup

Conversation

@BoyueZ

@BoyueZ BoyueZ commented Nov 3, 2026 •

Copy link
Copy Markdown
Collaborator

Description

Follow-up to #21183 (NVBug 6308608), from the review discussion there.

trtllm-serve had two unrelated rules for a request that names a file on the server:

After this change each direction has one rule and one variable named after it:

  • Input, TRTLLM_MEDIA_ALLOWED_INPUT_DIR (was TRTLLM_MEDIA_ALLOWED_LOCAL_PATH): the directory from which a request may have the server read media. Unset refuses every local path. A VisualGen format='path' reference is now read at the HTTP boundary through the same policy as a chat media part (read_allowed_local_media): refused with HTTP 400 unless the variable names a directory holding it, and then only a regular file of at most 200 MiB, with at most 512 MiB across the path references of one request. It reaches the engine as bytes, like a multipart upload.
  • Output, TRTLLM_MEDIA_DISABLE_PATH_RESPONSE=1 (was TRTLLM_DISALLOW_LOCAL_MEDIA_PATH): turns off response_format='path', as before, and nothing else. It accepts only 1, 0 or empty; a VisualGen server refuses to start on any other value.

Renames: both former names shipped in release candidates (TRTLLM_DISALLOW_LOCAL_MEDIA_PATH since 1.3, TRTLLM_MEDIA_ALLOWED_LOCAL_PATH in v1.4.0rc3). The former input name is no longer read, so local paths stay refused until the new one is set. A VisualGen server started with TRTLLM_DISALLOW_LOCAL_MEDIA_PATH set to anything other than 0 or empty refuses to start and names TRTLLM_MEDIA_DISABLE_PATH_RESPONSE=1; 0 or empty only logs a warning. The check runs at the top of launch_visual_gen_server, before the model loads, and again in OpenAIServer.__init__ for VisualGen servers only, so other server roles never read the variable. The VisualGen benchmark recognizes the server's refusal of a path response by its wording, so it works against servers on either side of the rename.

Reading files at the boundary: a request with a multipart upload or a format='path' reference is parsed on the media-load pool, which bounds how many requests read at once; any other request is parsed inline, as before. trtllm-serve forwards --num_media_load_workers to the VisualGen server, and the server unpublishes and shuts the pool down when it stops. The video routes clear the reference fields of their own params once generate_async has taken its snapshot, so a queued or running job does not keep a second copy of its media.

open_allowed_local_media also checks the location of the file it opened, not only the path it resolved: on Linux it reads /proc/self/fd/<fd> right after the open and refuses the request unless that location is under the allowed directory as well, before the type and size checks. O_NOFOLLOW covers only the last path component, so a directory on the path that became a symlink between resolution and open was followed before. Platforms without /proc/self/fd keep the previous behavior.

The offline Python API reads files as before: VisualGenParams with MediaRef(format='path') and the media loaders' async_load still read the caller's own files. MediaRef's repr now shows a bytes payload by its size. The trtllm-serve docs, the VisualGen docs, the VisualGen serve README, the benchmark docs, the benchmark skill (its server now starts with TRTLLM_MEDIA_ALLOWED_INPUT_DIR) and the Release 1.4 notes describe the policy.

Breaking change for VisualGen deployments that send format='path' references

Before this change such a reference was read from any path on the server, with no size limit, unless TRTLLM_DISALLOW_LOCAL_MEDIA_PATH=1. Now it is refused with HTTP 400 unless the server is started with TRTLLM_MEDIA_ALLOWED_INPUT_DIR=<directory>, and then only regular files inside that directory and within the size limits are read; a larger file has to be sent as a multipart upload. Deployments whose clients send local paths, including benchmark_visual_gen.py workloads with format: path references, must set the variable to the directory that holds their media. url, base64 and multipart uploads are unaffected, and so is response_format='path'.

Renamed variables: a deployment that set TRTLLM_MEDIA_ALLOWED_LOCAL_PATH must set TRTLLM_MEDIA_ALLOWED_INPUT_DIR instead. A VisualGen deployment that set TRTLLM_DISALLOW_LOCAL_MEDIA_PATH=1 must unset it and set TRTLLM_MEDIA_DISABLE_PATH_RESPONSE=1 instead.

Test Coverage

All in l0_cpu except tests/unittest/llmapi/test_config_database.py, which no L0 list includes; its new case passed in a local run of the CI image.

  • tests/unittest/llmapi/apps/test_chat_utils.py::TestRequestLocalMediaPaths: the existing policy cases under the new variable name; a directory on the path swapped for a symlink between resolution and open is refused, and a non-regular file reached that way gets the same location refusal with either descriptor directory (Linux; skipped without /proc/self/fd). The shared open/read function is covered here, so the VisualGen tests below check the wiring.
  • tests/unittest/_torch/visual_gen/test_visual_gen_utils.py:
    • TestPathReferencePolicy: a format='path' reference is refused without the directory, read to bytes at the boundary inside it (bare path and file://), and unaffected by the output switch.
    • TestReferenceBounds: the per-request 512 MiB budget across path references.
    • TestPathResponseSwitch: an unrecognized value refused at startup and treated as off; 0 or empty allowing path responses; the former name set to disable refusing to start; set to 0 or empty only warning; the startup check silent by default and logging a disabled switch.
  • tests/unittest/_torch/visual_gen/test_trtllm_serve_endpoints.py:
    • the existing response_format='path' tests under the new variable name;
    • TestPathReferenceRoutes: /v1/videos and /v1/videos/sync refuse a format='path' reference without the directory before reaching the engine, and with it hand the engine the file's bytes and drop the route's own copy;
    • a request that reads files is parsed on the media-load pool, and a cancelled request keeps its worker until its parse ends; a request that reads no file is parsed inline;
    • the server unpublishes and shuts down the media-load pool when it stops;
    • TestPathResponseStartupCheck: a VisualGen server refuses the former name before creating its pools, and an LLM server ignores it.
  • tests/unittest/_torch/visual_gen/test_visual_gen_params.py: a bytes reference payload shown by size in the repr.
  • tests/unittest/llmapi/test_visual_gen_startup_shutdown.py: launch_visual_gen_server refuses the former name before loading the model, and passes the media-load pool size to the server.
  • tests/unittest/llmapi/test_config_database.py (not in an L0 list): --num_media_load_workers reaches launch_visual_gen_server.

PR Checklist

Please review the following before submitting your PR:

  • PR description clearly explains what and why. If using CodeRabbit's summary, please make sure it makes sense.

  • PR Follows TRT-LLM CODING GUIDELINES to the best of your knowledge.

  • Test cases are provided for new code paths (see test instructions)

  • If PR introduces API changes, an appropriate PR label is added - either api-compatible or api-breaking. For api-breaking, include BREAKING in the PR title.

  • Any new dependencies have been scanned for license and vulnerabilities

  • CODEOWNERS updated if ownership changes

  • Documentation updated as needed

  • Update tava architecture diagram if there is a significant design change in PR.

  • The reviewers assigned automatically/manually are appropriate for the PR.

  • Please check this after reviewing the above items as appropriate for this PR.

Dev Engineer Review

The server uses TRTLLM_MEDIA_ALLOWED_INPUT_DIR to restrict chat media and VisualGen format="path" inputs. VisualGen reads path references at the HTTP boundary and passes their bytes to the engine. The policy checks regular files, directory containment, and size limits.

TRTLLM_MEDIA_DISABLE_PATH_RESPONSE controls response_format="path" separately. Startup rejects the retired TRTLLM_DISALLOW_LOCAL_MEDIA_PATH variable. On Linux, the server checks the opened file location through /proc/self/fd. VisualGen request parsing runs in the media-load pool, which shuts down with the server. URL, base64, multipart inputs, and offline Python API reads remain outside the HTTP input policy.

QA Engineer Review

Tests cover local-media restrictions, VisualGen path reads, path-response controls, startup validation, byte limits, symlink replacement, cancellation while parsing, worker-pool shutdown, and payload-safe parameter representation. No test-list files changed. The modified integration test file has entries in CI and manual-QA lists, but the changed environment-setting line is setup code rather than a listed test. Test execution results were not supplied. Coverage verdict: sufficient for the stated policy and lifecycle changes.

Per-File QA Perspective

  • .claude/skills/trtllm-visualgen-bench/SKILL.md — Verify benchmark guidance names the input-directory setting for server-side path references.
  • .claude/skills/trtllm-visualgen-bench/workloads/README.md — Verify path-reference guidance matches the server’s directory restriction.
  • docs/source/commands/trtllm-serve/trtllm-serve.rst — Verify the documented setting and file restrictions match chat and VisualGen behavior.
  • docs/source/models/visual-generation.md — Verify input-path policy and output-path response controls are documented separately.
  • examples/serve/openai_chat_client_for_multimodal.py — Verify the example names the current input-directory setting.
  • examples/visual_gen/serve/README.md — Verify the input and output path settings are distinct and accurate.
  • tensorrt_llm/inputs/media_io.py — Verify directory containment, regular-file checks, size limits, descriptor closing, and the Linux opened-file location check.
  • tensorrt_llm/serve/chat_utils.py — Verify local-media policy documentation matches enforced behavior.
  • tensorrt_llm/serve/openai_protocol.py — Verify the format="path" API description matches accepted server inputs.
  • tensorrt_llm/serve/openai_server.py — Verify startup policy checks, path-response rejection, worker-thread parsing for image routes, and media-pool shutdown.
  • tensorrt_llm/serve/openai_video_routes.py — Verify both video routes parse VisualGen parameters off the event loop.
  • tensorrt_llm/serve/scripts/BENCHMARKING_VISUAL_GEN.md — Verify benchmark instructions use the current path-input setting.
  • tensorrt_llm/serve/scripts/benchmark_visual_gen.py — Verify the path-response failure hint recognizes the new error and setting.
  • tensorrt_llm/serve/visual_gen_utils.py — Verify path references use the input policy, other formats remain unchanged, and the response switch behaves as documented.
  • tensorrt_llm/visual_gen/params.py — Verify offline Python API reads remain separate from serve-boundary reads and byte payloads are not exposed in representations.
  • tests/integration/defs/accuracy/test_glm53_flash.py — Verify the integration fixture uses the current input-directory setting. The file has CI and manual-QA entries; the changed fixture setup is not a separate listed test.
  • tests/unittest/_torch/visual_gen/test_trtllm_serve_endpoints.py — Covers path-response rejection, cancellation while parsing, and media-pool shutdown. Integration-list status is not applicable to this unit test.
  • tests/unittest/_torch/visual_gen/test_visual_gen_params.py — Covers byte-payload size representation while preserving visible path references. Integration-list status is not applicable.
  • tests/unittest/_torch/visual_gen/test_visual_gen_utils.py — Covers path-input policy, byte reads, request byte limits, response-switch values, logging, and retired-setting startup validation. Integration-list status is not applicable.
  • tests/unittest/llmapi/apps/_test_openai_chat_multimodal.py — Verify the server fixture uses the current input-directory setting. Integration-list status is not applicable.
  • tests/unittest/llmapi/apps/_test_openai_chat_vlm_guided_decoding.py — Verify the server fixture uses the current input-directory setting. Integration-list status is not applicable.
  • tests/unittest/llmapi/apps/_test_openai_mmencoder.py — Verify the server fixture uses the current input-directory setting. Integration-list status is not applicable.
  • tests/unittest/llmapi/apps/_test_trtllm_serve_multimodal_example.py — Verify the server setup uses the current input-directory setting. Integration-list status is not applicable.
  • tests/unittest/llmapi/apps/test_chat_utils.py — Covers local-media restrictions and rejects a Linux directory-symlink swap. Integration-list status is not applicable.
  • tests/unittest/pytest.ini — Verify the thread-leak exclusion matches media-load worker threads without hiding unrelated threads.

…e request local media policy

trtllm-serve had two unrelated rules for a request that names a file on
the server. A chat media part followed TRTLLM_MEDIA_ALLOWED_LOCAL_PATH and
was refused unless the file lies under that directory, while a VisualGen
reference with format='path' was read from anywhere on the server unless
TRTLLM_DISALLOW_LOCAL_MEDIA_PATH=1, the switch that also turns off
response_format='path'.

Give each direction one rule. A VisualGen path reference is now read at
the HTTP boundary through read_allowed_local_media, the same policy as a
chat media part: refused unless TRTLLM_MEDIA_ALLOWED_LOCAL_PATH names a
directory holding it, and then only a regular file within the media size
limit, handed to the engine as bytes like a multipart upload.
TRTLLM_DISALLOW_LOCAL_MEDIA_PATH keeps governing response_format='path'
only. The offline Python API still reads its caller's own files.

Since parsing a VisualGen request can now read a file, the four routes
run parse_visual_gen_params on a worker thread, as they already do for
generate_async. The VisualGen docs, the serve README and the benchmark
docs describe the policy, and the tests cover the boundary under each
setting.

Signed-off-by: berryz <berryz@nvidia.com>
…at they govern

TRTLLM_MEDIA_ALLOWED_LOCAL_PATH and TRTLLM_DISALLOW_LOCAL_MEDIA_PATH read
as two settings of one thing, while each governs one direction of local
file handling. Rename them after their job: TRTLLM_MEDIA_ALLOWED_INPUT_DIR
is the directory from which a request may have the server read media, and
TRTLLM_MEDIA_DISABLE_PATH_RESPONSE=1 turns off response_format='path'.

TRTLLM_MEDIA_ALLOWED_INPUT_DIR replaces its former name outright: that name
was introduced on main by NVIDIA#21183 and has not shipped in a release.
TRTLLM_DISALLOW_LOCAL_MEDIA_PATH has shipped, so a server started with it
set refuses to start and names the variable to set instead. Keeping it as
an alias would carry the outdated name on, and ignoring it would silently
turn path responses back on for a deployment that relied on it. The server
reads the switch once at startup and logs when path responses are disabled,
so an unrecognized value shows when the server starts rather than at the
first request that asks for a path response. The VisualGen benchmark
recognizes the server's refusal of a path response by its wording rather
than by the variable name, so it works against servers on either side of
the rename.

Code, docs, examples and tests use the new names.

Signed-off-by: berryz <berryz@nvidia.com>
… was opened

open_allowed_local_media resolved the request's path, checked that it lies
under TRTLLM_MEDIA_ALLOWED_INPUT_DIR, then opened it with O_NOFOLLOW. That
flag guards only the last path component, so a directory on the path that
became a symlink between the resolution and the open was still followed,
and the open file could lie outside the directory.

After the open, read /proc/self/fd/<fd> to learn where the open file is
and refuse the request unless that location is under the allowed directory
as well. Platforms without /proc/self/fd keep the previous behavior. A test
swaps a directory for a symlink between the two steps and expects the
refusal; it is skipped where /proc/self/fd is absent.

Signed-off-by: berryz <berryz@nvidia.com>
@coderabbitai

coderabbitai Bot commented Nov 3, 2026 •

Copy link
Copy Markdown
Contributor

Explain Change →

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • Resume reviews
  • Trigger review

Walkthrough

The server now reads local path references only from a configured directory and controls generated path responses with a separate setting. It checks opened-file containment and enforces a 512 MiB request-level limit for path-reference bytes. VisualGen parameter parsing in image and video routes now runs in worker threads.

Changes

VisualGen serving

Layer / File(s) Summary
Local path-reference input policy
.claude/skills/trtllm-visualgen-bench/*, docs/source/commands/trtllm-serve/trtllm-serve.rst, docs/source/models/visual-generation.md, examples/serve/openai_chat_client_for_multimodal.py, examples/visual_gen/serve/README.md, tensorrt_llm/inputs/media_io.py, tensorrt_llm/serve/{openai_protocol.py,visual_gen_utils.py,chat_utils.py}, tensorrt_llm/visual_gen/params.py, tensorrt_llm/serve/scripts/BENCHMARKING_VISUAL_GEN.md, tests/integration/defs/accuracy/test_glm53_flash.py, tests/unittest/_torch/visual_gen/{test_visual_gen_utils.py,test_visual_gen_params.py}, tests/unittest/llmapi/apps/*
Path references are read as bytes only when they resolve to regular files under TRTLLM_MEDIA_ALLOWED_INPUT_DIR. The opened descriptor is checked for containment. Request parsing limits the total bytes in path references to 512 MiB. MediaRef representations show the byte count instead of payload contents.
Generated path-response policy
docs/source/models/visual-generation.md, examples/visual_gen/serve/README.md, tensorrt_llm/serve/visual_gen_utils.py, tensorrt_llm/serve/openai_server.py, tensorrt_llm/serve/scripts/benchmark_visual_gen.py, tests/unittest/_torch/visual_gen/{test_visual_gen_utils.py,test_trtllm_serve_endpoints.py}
TRTLLM_MEDIA_DISABLE_PATH_RESPONSE controls generated path responses. Startup rejects the former variable name. Endpoint errors, benchmark hints, tests, and documentation use the response-only setting.
Worker-thread parameter parsing
tensorrt_llm/serve/{openai_server.py,openai_video_routes.py}, tests/unittest/_torch/visual_gen/test_trtllm_serve_endpoints.py, tests/unittest/pytest.ini
Image-generation, image-edit, and video routes run parse_visual_gen_params in worker threads. The server shuts down the media-load executor. Endpoint tests cover cancellation while the media-load worker remains occupied.

Estimated code review effort: 3 (Moderate) | ~25 minutes

Change: Bug fix

Sequence Diagram(s)

sequenceDiagram
  participant VisualGenRequest
  participant visual_gen_utils
  participant media_io
  VisualGenRequest->>visual_gen_utils: Submit path reference
  visual_gen_utils->>media_io: read_allowed_local_media
  media_io->>media_io: Check descriptor containment and read bytes
  media_io-->>visual_gen_utils: Return reference bytes
  visual_gen_utils-->>VisualGenRequest: Build normalized reference lists
Loading

Suggested reviewers: junyixu-nv, luyiyun1021

Merge Risk: Low · previous review 96bbf

Local media loading later in the same process may fail after server shutdown. Clear the shared executor reference during teardown; the previously identified reference-growth and logging risks are addressed.

Pre-merge checks: Passed 4 · Failed 1

Failed Failed checks: 1 warning

Check name Status Explanation Resolution
Docstring Coverage Warning Docstring coverage is 70.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff with cyclomatic complexity of at least 2. Documenta… Write docstrings for the functions missing them to satisfy the coverage threshold.
Passed Passed checks: 4 passed

Check name Status Explanation
Title check Passed The title names the main change: one local-media input policy for chat and VisualGen, plus the variable rename. The changeset renames TRTLLM_MEDIA_ALLOWED_LOCAL_PATH to TRTLLM_MEDIA_ALLOWED_INPUT_DIR,…
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
Description check Passed The description follows the repository template closely. It has a Description section that explains the problem (two unrelated rules for server-side files), the solution (one input variable TRTLLM_MED…
Full details: Title check

Explanation

The title names the main change: one local-media input policy for chat and VisualGen, plus the variable rename. The changeset renames TRTLLM_MEDIA_ALLOWED_LOCAL_PATH to TRTLLM_MEDIA_ALLOWED_INPUT_DIR, applies the same policy to VisualGen path references, and splits the output switch into TRTLLM_MEDIA_DISABLE_PATH_RESPONSE. The title uses the required NVBugs ID and [fix] type format. It does not mention the output-switch split or the breaking change, which is expected for a short title.

Full details: Description check

Explanation

The description follows the repository template closely. It has a Description section that explains the problem (two unrelated rules for server-side files), the solution (one input variable TRTLLM_MEDIA_ALLOWED_INPUT_DIR and one output switch TRTLLM_MEDIA_DISABLE_PATH_RESPONSE), the renames and their startup effects, and a clearly flagged breaking change with migration steps. It has a Test Coverage section that lists specific test files and cases, and a PR Checklist with the last item checked. The title and ticket line were not in the supplied description, and the template's title format is a separate matter. Overall the description is mostly complete and specific.

Full details: Docstring Coverage

Explanation

Docstring coverage is 70.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff with cyclomatic complexity of at least 2. Documentation-only edits and deletions within surviving functions are included; unchanged diff context and fully deleted functions are excluded. Analyzed 30 functions across 18 files.

Resolution

Write docstrings for the functions missing them to satisfy the coverage threshold.

Finishing Touches

Generate unit tests (beta)

  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

@coderabbitai coderabbitai Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • Fix comments: Fix CodeRabbit comments on this PR

Comment thread tensorrt_llm/serve/visual_gen_utils.py Resolved
Comment thread tensorrt_llm/serve/visual_gen_utils.py Resolved
@BoyueZ
BoyueZ marked this pull request as draft November 3, 2026 06:41
@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #82785 [ run ] triggered by Bot. Commit: ad5cad9 Link to invocation

@sunnyqgg sunnyqgg left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Moving VisualGen format='path' reads to the HTTP boundary under TRTLLM_MEDIA_ALLOWED_INPUT_DIR is the right fix, and the post-open /proc/self/fd check closes the directory-swap race. This review has 1 Blocker, 2 Possibly blocking, 8 Required and 11 Nice-to-have items: inline comments, plus the bullets below for points with no diff line to anchor on.

The blocker is the retired-variable startup refusal. It applies to every server role, refuses any value including 0, and fires only after the model has loaded.

  • [Required] Missing copyright headers: tensorrt_llm/serve/visual_gen_utils.py (+93/-49) and tests/unittest/llmapi/apps/test_chat_utils.py (+30) are substantially modified but have no NVIDIA SPDX header. CODING_GUIDELINES.md:910 requires one "that includes the year of its latest meaningful modification", and AGENTS.md says to update the year on modified files. Please add the 2026 header.
  • [Required] Breaking-change process: the description calls this a breaking change, but the PR has no api-breaking label, no BREAKING in the title, and no release note. The PR template says "an appropriate PR label is added - either api-compatible or api-breaking. For api-breaking, include BREAKING in the PR title". The only release note an upgrading operator finds (docs/source/release-notes.md:85) still says to set TRTLLM_DISALLOW_LOCAL_MEDIA_PATH=1, which now stops the server from starting. Please add BREAKING: and the label, and a [BREAKING CHANGE] entry under Release 1.4 -> API Changes covering:
    • format='path' now needs TRTLLM_MEDIA_ALLOWED_INPUT_DIR (regular files only, 200 MiB per file, 512 MiB per request);
    • both renames;
    • the startup refusal.
      #17490, which introduced the variable, did both.
  • [Required] Description vs. code:
    • /v1/images/edits is listed as taking format='path' references, but ImageEditRequest has no *_reference fields and its image field already refused paths.
    • "The offline Python API is unchanged", but the MediaRef/VisualGenParams repr changed.
    • "Adds no copy", but the bytes now live in the route-side params until the job ends (see the inline comment).
    • Test Coverage omits the tests from the later commits (TestReferenceBounds, the repr test, the cancellation and shutdown tests) and the 512 MiB budget.
  • [Nice to have] +1 to coderabbitai's resolved thread on tensorrt_llm/serve/visual_gen_utils.py:182 about bounding reads before they happen. PATH_REFERENCE_MAX_TOTAL_BYTES is checked only after each file is fully read (_build_reference_lists, L241-247), so a refused request may already hold ~512 + 200 MiB. The format == "path" test is also repeated after _reference_transport already branched on it. Passing the remaining budget into read_allowed_local_media and checking it against fstat before reading fixes both.
  • [Nice to have] Question: is a release/1.3 backport planned for this NVBug? release/1.3 still reads VisualGen format='path' references from anywhere on the server (params.py:444). If so, backporting the input-dir policy while keeping the shipped TRTLLM_DISALLOW_LOCAL_MEDIA_PATH honoured there seems safer than cherry-picking the rename and the startup refusal into a patch release.

Comment thread tensorrt_llm/serve/openai_server.py Resolved
Comment thread tensorrt_llm/serve/visual_gen_utils.py Resolved
Comment thread tensorrt_llm/inputs/media_io.py Resolved
Comment thread tensorrt_llm/inputs/media_io.py Resolved
@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #82785 [ run ] completed with state SUCCESS. Commit: ad5cad9
/LLM/main/L0_MergeRequest_PR pipeline #68356 completed with status: 'UNSTABLE'

CI Report

⚠️ Multi-GPU Label Required:
Multi-GPU tests require the ci: full pre-merge approved label on this PR. Either:

  • Wait for the PR to be fully approved — the label is added automatically once approval is complete. Having unresolved open comments is fine, or
  • If needed, ask a member of NVIDIA/trt-llm-ci-approvers to add the label manually.
    Then re-trigger CI with the same bot command (no rebase needed).

⚠️ Action Required:

  • Please check the failed tests and fix your PR
  • If you cannot view the failures, ask the CI triggerer to share details
  • Once fixed, request an NVIDIA team member to trigger CI again

Link to invocation

…itch value

TRTLLM_MEDIA_DISABLE_PATH_RESPONSE accepted any value and read everything
other than "1" as allowed, with a warning. A server now refuses to start
when the switch holds anything other than "1", "0" or empty, and a request
that still sees such a value treats path responses as off.

Signed-off-by: berryz <berryz@nvidia.com>
…n VisualGen only, before the model loads

The startup refusal of TRTLLM_DISALLOW_LOCAL_MEDIA_PATH ran in
OpenAIServer.__init__ for every server role, refused any value including
"0", and ran only after the model had loaded. The check now runs at the
top of launch_visual_gen_server, before the model is built, and again in
OpenAIServer.__init__ for VisualGen servers before the thread pools are
created. A value of "0" or empty asked for the default, so it only logs a
warning; any other value still refuses to start and names
TRTLLM_MEDIA_DISABLE_PATH_RESPONSE=1.

Signed-off-by: berryz <berryz@nvidia.com>

@sunnyqgg sunnyqgg left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The Blockers from my review of 2026-11-05 are resolved or answered at a3ced03:

  • serve.py:967 now checks the retired variable at the top of launch_visual_gen_server, before the model loads; openai_server.py:682 runs it for VisualGen only and before the pools; 0/empty only warn (a3ced03)

The last CI run (ad5cad9) ended UNSTABLE, and the data shows no run on a3ced03 yet; L0 needs to pass on the new head before merge.

The Possibly-blocking and Required items are not covered: path-reference bytes kept on the route side, the fixed 8-worker parse pool, missing SPDX headers, no BREAKING label or release note, and the 200/512 MiB limits not documented. All are unchanged at H.

@crazydemo crazydemo left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review summary - Approve

Reviewed the full diff; no blocking or major issues found.

Automated review by NVCortex Lite, run by @crazydemo.

BoyueZ added 11 commits November 9, 2026 11:25
…aseMediaIO.clear_executor

Server teardown read BaseMediaIO._executor directly and set_executor took
None only for that. BaseMediaIO.clear_executor(executor) now stops
publishing a pool only when it is still the published one, and
set_executor takes an Executor again.

Signed-off-by: berryz <berryz@nvidia.com>
…e the engine has its snapshot

The video routes read format='path' references into params before
generate_async, which snapshots params for the engine. The route kept its
own params, and with it the reference bytes, until the job finished: the
async route hands params to the background task and the sync route holds
it across the wait for the result. Both routes now clear the reference
fields right after generate_async returns, which also frees multipart
uploads early.

The test generator snapshots params as the engine does, and route-level
tests send a format='path' reference to both video routes, with and
without TRTLLM_MEDIA_ALLOWED_INPUT_DIR.

Signed-off-by: berryz <berryz@nvidia.com>
…the request reads files

Every VisualGen request was parsed on the media-load pool, including one
with nothing to read, so it could wait behind slow format='path' reads,
and a VisualGen server always sized that pool at 8. The parse now moves to
the pool only for a request with a multipart upload or a format='path'
reference (request_reads_files); any other request is parsed inline, as
before this PR. trtllm-serve forwards --num_media_load_workers to the
VisualGen server.

_parse_visual_gen_params moves from the video-routes mixin to
OpenAIServer, next to _reject_disabled_path, as both image routes use it
too.

Signed-off-by: berryz <berryz@nvidia.com>
…fuses the former variable before loading the model

Signed-off-by: berryz <berryz@nvidia.com>
…o modified files

Signed-off-by: berryz <berryz@nvidia.com>
…h' references

An HTTP format='path' reference is read only up to 200 MiB per file and
512 MiB across the path references of one request. The VisualGen model
page, the serve example README and the MediaReferenceItem.format
description now say so and point a larger file to a multipart upload.

Signed-off-by: berryz <berryz@nvidia.com>
…LLM_MEDIA_ALLOWED_INPUT_DIR

Four bundled /v1/videos workloads send format: path references from the
run's reference_media/, which the server now reads only under
TRTLLM_MEDIA_ALLOWED_INPUT_DIR. Serve step C now exports it as the parent
of the vg-bench-* runs before any of its launch commands, so a reused
server also reads the media of later runs.

Signed-off-by: berryz <berryz@nvidia.com>
…trings in line with the code

_build_reference_lists takes a VideoGenerationRequest and returns lists of
MediaRef, and MediaRef.__repr_args__ declares its return type.

check_path_response_policy and the comment on the former variable now state
the contract only. parse_visual_gen_params says it reads uploads and
format='path' references and blocks for those. prepare_reference_slots, the
generate_async comment and the test helper no longer describe serve and the
Python API as sharing one resolution path, and the __repr_args__ comment no
longer claims that a printed VisualGenParams never carries media.

Signed-off-by: berryz <berryz@nvidia.com>
… its type and size

The post-open location check ran after the regular-file and size checks, so
a file reached through a directory swapped mid-request was refused as "not
a regular file" or as too large rather than with the location refusal. It
now runs right after the fstat. It also has its own constant,
_FD_LINK_DIR, instead of testing _DESCRIPTOR_DIR, so how decoders reach
the descriptor no longer decides whether the check runs.

Signed-off-by: berryz <berryz@nvidia.com>
…for the local media policy

Covers that VisualGen format='path' references now need
TRTLLM_MEDIA_ALLOWED_INPUT_DIR and are capped at 200 MiB per file and
512 MiB per request, both variable renames, and that a VisualGen server
refuses to start while the former path-response variable is set.

Signed-off-by: berryz <berryz@nvidia.com>
…edia-policy-followup

Signed-off-by: berryz <berryz@nvidia.com>

# Conflicts:
#	docs/source/release-notes.md
@BoyueZ

BoyueZ commented Nov 9, 2026

Copy link
Copy Markdown
Collaborator Author

/bot run

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #83448 [ run ] triggered by Bot. Commit: b6fb5eb Link to invocation

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #83448 [ run ] completed with state SUCCESS. Commit: b6fb5eb
/LLM/main/L0_MergeRequest_PR pipeline #68975 completed with status: 'UNSTABLE'

CI Report

⚠️ Multi-GPU Label Required:
Multi-GPU tests require the ci: full pre-merge approved label on this PR. Either:

  • Wait for the PR to be fully approved — the label is added automatically once approval is complete. Having unresolved open comments is fine, or
  • If needed, ask a member of NVIDIA/trt-llm-ci-approvers to add the label manually.
    Then re-trigger CI with the same bot command (no rebase needed).

⚠️ Action Required:

  • Please check the failed tests and fix your PR
  • If you cannot view the failures, ask the CI triggerer to share details
  • Once fixed, request an NVIDIA team member to trigger CI again

Link to invocation

@zhaoyangwang-nvidia zhaoyangwang-nvidia left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approve with nits.

os.close(fd)


def log_allowed_local_media_path() -> None:

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A deployment that still sets TRTLLM_MEDIA_ALLOWED_LOCAL_PATH now gets a 400 on every local media request with nothing in the server log. Could log_allowed_local_media_path warn when the former name is set and name TRTLLM_MEDIA_ALLOWED_INPUT_DIR, as check_path_response_policy does for the output-side rename?

…irectory variable is set

A deployment that still sets TRTLLM_MEDIA_ALLOWED_LOCAL_PATH has every
local media request refused. log_allowed_local_media_path now names the
new variable in a warning when the former one is set.

Signed-off-by: berryz <berryz@nvidia.com>
@BoyueZ

BoyueZ commented Nov 11, 2026

Copy link
Copy Markdown
Collaborator Author

/bot run

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #84049 [ run ] triggered by Bot. Commit: 9c1b13a Link to invocation

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #84049 [ run ] completed with state SUCCESS. Commit: 9c1b13a
/LLM/main/L0_MergeRequest_PR pipeline #69518 completed with status: 'UNSTABLE'

CI Report

⚠️ Multi-GPU Label Required:
Multi-GPU tests require the ci: full pre-merge approved label on this PR. Either:

  • Wait for the PR to be fully approved — the label is added automatically once approval is complete. Having unresolved open comments is fine, or
  • If needed, ask a member of NVIDIA/trt-llm-ci-approvers to add the label manually.
    Then re-trigger CI with the same bot command (no rebase needed).

⚠️ Action Required:

  • Please check the failed tests and fix your PR
  • If you cannot view the failures, ask the CI triggerer to share details
  • Once fixed, request an NVIDIA team member to trigger CI again

Link to invocation

@BoyueZ

BoyueZ commented Nov 12, 2026 •

Copy link
Copy Markdown
Collaborator Author

@NVIDIA/trt-llm-models-devs @NVIDIA/trt-llm-torch-visual-gen-devs Just kind reminder.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

8 participants