Repository navigation
[TRTLLM-385][doc] Add Configuration Schema for LlmArgs, AutoDeploy, and VisualGeneration Classes - #14159
Open
arysef wants to merge 7 commits into
Open
[TRTLLM-385][doc] Add Configuration Schema for LlmArgs, AutoDeploy, and VisualGeneration Classes#14159arysef wants to merge 7 commits into
arysef wants to merge 7 commits into
Conversation
Signed-off-by: asefidi <asefidi@nvidia.com>
- Generate trtllm-serve-autodeploy-config.schema.json from
tensorrt_llm._torch.auto_deploy.llm_args.LlmArgs so users running with
--backend _autodeploy get an IDE-validatable schema for that backend's
YAML surface.
- Strip json_schema_extra "type" strings ("Union[...]", "Literal[...]")
that previously left the serve config schema invalid under JSON Schema
Draft 2020-12 and broke Draft202012Validator.check_schema.
- Widen env_overrides values to match the runtime str-coercion validator
so unquoted scalar YAML (e.g. TRTLLM_ENABLE_PDL: 1) validates in
existing configs.
- Read tensorrt_llm/version.py directly (no package import) to embed the
trtllm version in SCHEMA_BASE_URL, pinning each release's schemas
immutably; docs example URLs stay on /latest/ with a pin-note.
- Add tests for the autodeploy schema (minimal, aliases, wrong-backend,
typo) and for unquoted env_overrides scalars.
- Tighten the "Configuring with YAML Files" section to a single
schema-directive block with the AutoDeploy schema linked inline.
Signed-off-by: asefidi <asefidi@nvidia.com>
Pre-commit's ruff-format hook reformatted the two python files (mostly line-length and dict/string-literal wrapping). No behavior change; the schema test suite still passes (219 tests). Signed-off-by: asefidi <asefidi@nvidia.com>
arysef
changed the title
arysef
changed the title
QiJune
reviewed
May 18, 2026
| _widen(branch) | ||
|
|
||
|
|
||
| def _add_schema_metadata( |
Collaborator
There was a problem hiding this comment.
This does not affect runtime, but there is one schema coverage gap around disaggregated serving. trtllm-serve <model> --config supports a serve-level disagg_cluster key: serve.py pops it from the merged config and converts it to DisaggClusterConfig. The disaggregated README also documents putting disagg_cluster in context/generation server configs.
Since this schema is advertised for trtllm-serve --config and uses additionalProperties=false, a valid worker config containing disagg_cluster: would be flagged as invalid by IDEs. Could we either add this serve-level key to the schema, or document that the schema only covers the LLM args portion and not serve-level disaggregated options?
Collaborator
Author
There was a problem hiding this comment.
Added a schema for the disagg orchestrator and expanded the schema for disagg_cluster/worker
…ve schemas - Generate trtllm-serve-disagg-config.schema.json from tensorrt_llm.llmapi.disagg_utils dataclasses (via TypeAdapter) plus the TorchLlmArgs surface. The orchestrator YAML accepted by trtllm-serve disaggregated --config is validated end-to-end: TorchLlmArgs fields at the top level (inherited into each server block), disagg-specific fields (hostname, port, schedule_style, max_retries, perf_metrics_max_requests, node_id, conditional_disagg_config, otlp_config, disagg_cluster) at the top level, and nested context_servers / generation_servers (DisaggServerBlock) with the same TorchLlmArgs surface plus num_instances, urls, and router. - backend is widened to enum: [pytorch, _autodeploy, tensorrt, trt] to match what the legacy TRT-backend disagg test configs use; free_gpu_memory_fraction is exposed as a CLI alias the runtime propagates into each worker's KvCacheConfig. - Add disagg_cluster to the serve and autodeploy schemas as well: trtllm-serve serve pops disagg_cluster from the parsed YAML before LLM construction (serve.py), so workers can join a disagg cluster via the regular config. Use the same DisaggClusterConfig shape so the worker-side and orchestrator-side cluster blocks stay in lockstep. - Add unit tests (minimal yaml, inherited torch args, gen-only via num_instances=0, top-level typo, nested typo, bad schedule_style, disagg_cluster on serve + autodeploy, missing cluster_uri) and parametrize over the existing examples/disaggregated/*.yaml configs. - Rephrase the docs schema-directive paragraph so each schema is tied to its subcommand surface: trtllm-serve serve --config, --backend _autodeploy, and trtllm-serve disaggregated --config each get their own schema link. Signed-off-by: asefidi <asefidi@nvidia.com>
CI's pre-commit hook flagged two short lines that fit on one line each (spec_from_file_location call and RuntimeError message). Apply the reformat so the hook passes. Signed-off-by: asefidi <asefidi@nvidia.com>
Contributor
📝 WalkthroughWalkthroughThis PR establishes JSON Schema generation for TensorRT-LLM's ChangesSchema Generation and IDE Support
🎯 3 (Moderate) | ⏱️ ~25 minutes 🚥 Pre-merge checks | ✅ 3 | ❌ 2❌ Failed checks (1 warning, 1 inconclusive)
✅ Passed checks (3 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 2
🧹 Nitpick comments (1)
tests/unittest/llmapi/test_trtllm_serve_config_schemas.py (1)
287-287: 💤 Low valueUnnecessary parentheses around string literal.
The parentheses around
"trtllm_schema_assets.py"serve no purpose and can be removed.♻️ Proposed fix
- extension_path = REPO_ROOT / "docs" / "source" / "_ext" / ("trtllm_schema_assets.py") + extension_path = REPO_ROOT / "docs" / "source" / "_ext" / "trtllm_schema_assets.py"🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@tests/unittest/llmapi/test_trtllm_serve_config_schemas.py` at line 287, The expression assigning extension_path contains unnecessary parentheses around the string literal; update the assignment for the variable extension_path (in tests/unittest/llmapi/test_trtllm_serve_config_schemas.py) to remove the redundant parentheses so the path concatenation uses "trtllm_schema_assets.py" directly instead of ("trtllm_schema_assets.py").
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@scripts/generate_trtllm_serve_schemas.py`:
- Around line 316-358: The DisaggServerBlock schema is built from
server_block_properties before _add_schema_metadata widens env_overrides, so
context_servers/generation_servers still restrict env_overrides to strings;
update the construction so server_block_properties includes the same relaxed
env_overrides schema (e.g., allow string, number, boolean or any scalar) before
assigning defs["DisaggServerBlock"] — modify the block that prepares
server_block_properties (the dict used to build DisaggServerBlock) to include
the widened env_overrides definition (the same shape produced by
_add_schema_metadata) or call _add_schema_metadata on the cloned properties
prior to creating defs["DisaggServerBlock"] so env_overrides accepts non-string
scalar values.
- Around line 407-413: The schema for properties["node_id"] documents a valid
range [0, 1023] but does not enforce it; update the node_id schema to include
numeric bounds by adding "minimum": 0 and "maximum": 1023 (while preserving the
existing "type": ["integer","null"], "default": None, and description) so that
static validation fails for out-of-range integers; locate this in the
properties["node_id"] block and add the two keys.
---
Nitpick comments:
In `@tests/unittest/llmapi/test_trtllm_serve_config_schemas.py`:
- Line 287: The expression assigning extension_path contains unnecessary
parentheses around the string literal; update the assignment for the variable
extension_path (in tests/unittest/llmapi/test_trtllm_serve_config_schemas.py) to
remove the redundant parentheses so the path concatenation uses
"trtllm_schema_assets.py" directly instead of ("trtllm_schema_assets.py").
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Enterprise
Run ID: 01baa3bb-b4a6-4d47-83af-8ecfd904ef8d
📒 Files selected for processing (5)
docs/source/_ext/trtllm_schema_assets.pydocs/source/commands/trtllm-serve/trtllm-serve.rstdocs/source/conf.pyscripts/generate_trtllm_serve_schemas.pytests/unittest/llmapi/test_trtllm_serve_config_schemas.py
…e_id Signed-off-by: asefidi <asefidi@nvidia.com>
Signed-off-by: asefidi <asefidi@nvidia.com>
Collaborator
Author
|
/bot run |
Collaborator
|
PR_Github #58313 [ run ] triggered by Bot. Commit: |
Collaborator
|
PR_Github #58313 [ run ] completed with state
|
|
@arysef this pull request has had no activity for more than 120 days and currently has merge conflicts. Is it still active? Please resolve the conflicts or post an update. Otherwise, it will be closed after another 60 days of inactivity. |
Draft
1 task done
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary by CodeRabbit
New Features
trtllm-serveconfigurations, enabling IDE and YAML editor schema validation with autocomplete and real-time validation across all configuration typesDocumentation
trtllm-servedocumentation with schema references and code examples to help users set up IDE supportTests
Description
Generates config schema to make basic static configuration checks easier for users. Adds pages to the trtllm docs to allow users to source the schema from the latest releases.
Test Coverage
PR Checklist
Please review the following before submitting your PR:
PR description clearly explains what and why. If using CodeRabbit's summary, please make sure it makes sense.
PR Follows TRT-LLM CODING GUIDELINES to the best of your knowledge.
Test cases are provided for new code paths (see test instructions)
If PR introduces API changes, an appropriate PR label is added - either
api-compatibleorapi-breaking. Forapi-breaking, includeBREAKINGin the PR title.Any new dependencies have been scanned for license and vulnerabilities
CODEOWNERS updated if ownership changes
Documentation updated as needed
Update tava architecture diagram if there is a significant design change in PR.
The reviewers assigned automatically/manually are appropriate for the PR.
Please check this after reviewing the above items as appropriate for this PR.
GitHub Bot Help
To see a list of available CI bot commands, please comment
/bot help.