You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
[None][test] Refresh B200 VisualGen perf sanity recipes from the October sweep - #21439
Refresh the five covered FLUX.2-dev, Wan2.2 I2V and LTX2 B200 sanity cases using the best observed October sweep recipes within the existing model/mode, logical GPU budget and usable CI checkpoint set. #21929 has merged and removed Wan2.1. Refresh the five remaining scheduled B200 cases without restoring the removed recipe or selector.
The measured source was main dddb74c217dadabd6e9ba04ef22d48da53a73e01. Each point had one measured request, so these are observed minima under the tested constraints, not universal optima. Existing CI prompts/reference images remain unchanged; exact CI latency can differ. Update pytest names and scheduled selectors together, using measured warmup shapes and explicit LTX compile options. All refreshed B200 LTX cases pin the native BF16 file rather than a directory containing multiple precision variants. GPU allocations, client shapes/steps/counts, timeouts, KPI extraction and regression thresholds are unchanged.
LTX checkpoint and stage selection
The old ltx2_nvfp4 case pointed to the general LTX-2 directory without enabling quantization. The staged native LTX-2/ltx-2-19b-dev-fp4.safetensors header declares 1,176 transformer-layer nvfp4 entries; video/audio VAE tensors remain BF16. The metadata converter at the #21929 snapshot (23512d112e) still recognizes only float8_e4m3fn, not that nvfp4 format (checked with an isolated execution of the actual converter). The overall one-stage sweep minimum (5.25 s) used a separate ModelOpt static NVFP4 checkpoint and is not claimed reproducible using this native file.
Use the measured FP8-blockwise recipe with the explicit staged BF16 file instead: the best observed one-stage recipe among the usable checkpoints in the October staging set (6.20 s). No loader workaround, unmeasured on-load NVFP4 substitution or cache mutation is introduced.
The LTX-2 directory also contains the upscaler and distilled LoRA. The loader discovers them automatically, so omitting YAML fields does not reliably keep a run single-stage. Move the one-stage case into its own YAML with TLLM_LTX2_FORCE_ONE_STAGE_PIPELINE=1; explicitly set it to 0 for the two-stage family. This isolates server environments and preserves intended coverage without changing runtime code or adding scheduled B200 cases.
Platform scope and integration
This branch is rebased onto main 64f7213093, which includes merged #21929. The redundant private copy of the Wan2.1-removal commit is dropped, so the PR contains only the recipe refresh and obsolete-waiver cleanup. The current shared ltx2.yaml also covers VR200, so keep its existing recipes unchanged for VR200 and scope it to that GPU. Place the sweep-derived B200 settings in ltx2_blackwell.yaml and ltx2_single_stage_blackwell.yaml. This preserves VR200 recipes while refreshing the five B200 selectors. No VR200 performance claim is made.
Waiver consistency repair
Removed the obsolete waiver for the retired FP8 FLUX.2 selector. The replacement BF16 recipe remains unwaived. Local execution of the affected VisualGen parameter-generation and waiver-membership checks reproduces the stale-entry error before cleanup and passes afterward; these scoped checks do not replace full CI collection or the five-case B200 performance run. NVBug 6759009 and its separate perf-sanity investigation are unchanged. The branch also incorporates main's existing CPU fixture/API corrections; no unrelated executor code is changed by this PR.
Baselines and validation
Nov11 local integration checks (head a2620c2e9e0010e43042611ba2878e7562cb44c3, rebased onto main 64f7213093; not a new GPU run):
The five scheduled B200 selectors resolve to exactly one server case each using the actual selector/parser definitions; GPU counts are 4/8/8/8/8 and the original timeouts remain 120/30/30/30/30. GPU identity and asset files are test substitutes, so this is not runtime certification.
The five refreshed B200 serve configs validate against the checked-in VisualGen JSON Schema.
Parameter generation now includes 26 names across both platform families (previously 16); the scheduled B200 case count remains five. Actual parser GPU gates skip the VR200-only family on B200 and the B200-only families on VR200.
Existing VR200 LTX recipe contents and server names are unchanged, with the shared family scoped to VR200; both refreshed LTX families are B200-only.
README YAML and command examples match the updated BF16 FLUX.2 recipe and B200-only LTX family; all concrete README selectors resolve through the actual parser.
All applicable pre-commit hooks and git diff --check passed.
The final eight-file diff preserves the merged Wan2.1 removal and all unrelated upstream waiver changes. Only the retired FP8 FLUX.2 waiver is removed; the replacement BF16 case is not waived.
KPI extraction and regression implementation files are unchanged; no baseline data is modified.
Historical preparation evidence (October sweep / original PR preparation; not rerun against the updated PR head):
Changed settings require separate CI baselines. Do not rewrite old timing evidence or accept a known regression by installing a slower baseline. Fastest-recipe KPI cases complement fixed controls/investigations; they do not cover every regression, including single-GPU FP8-blockwise LTX (NVBug 7030044) and eight-GPU BF16/TRTLLM Wan5B (NVBug 7030393).
The five updated recipes were matched to saved sweep configs after asset-path translation; one-stage selection follows the historical staged-checkpoint constraint above.
Inspected native BF16/NVFP4 file headers and auxiliary asset paths without loading weights. Exercised the then-current one-stage variant-selection method under the override, and confirmed the same directory otherwise discovered both auxiliary assets. Asset staging and runtime stage selection must be revalidated in target CI; these checks were not repeated on Nov11.
Existing client workloads remain unchanged; Wan2.1 stays removed and existing VR200 recipes are preserved; no runtime implementation or public schema changes.
The previous #69231 request was accepted, but single-GPU failures blocked the multi-GPU prerequisite, so none of the five performance cases ran. Results from prior or superseded heads do not validate this rebased head; its target run is still required.
Before merge: run the updated cases in target B200 CI and review their new baselines. The updated five-case B200 performance run and baseline review are pending; local parser/schema checks and ordinary PR CI do not replace that acceptance. Native NVFP4 metadata support or staging the separately measured ModelOpt checkpoint remains a follow-up.
Dev Engineer Review
Refreshes five B200 VisualGen sanity recipes for FLUX.2-dev, Wan2.2 A14B, and LTX2.
Adds separate B200 LTX2 two-stage and one-stage recipes. Keeps ltx2.yaml for VR200.
Replaces the obsolete FLUX.2 FP8 recipe with an unwaived BF16 recipe with CUDA graphs.
Adds explicit LTX2 model paths, FA4 attention, compilation shapes, and stage-selection environment variables.
No runtime code changes are included. The main risk is unvalidated GPU compatibility and performance for the new settings. The required five-case B200 run and baseline review remain pending.
QA Engineer Review
Modifies l0_b200_visual_gen_perf_sanity.yml with five scheduled entries:
FLUX.2 BF16.
Wan2.2 A14B with Sage attention quantization.
LTX2 BF16 two-stage I2V.
LTX2 FP8 block-scale two-stage T2V.
LTX2 FP8 block-scale one-stage I2V.
Updates pytest selectors and removes the obsolete FLUX.2 waiver.
Adds or updates README examples, selectors, KPI observations, checkpoint constraints, and stage-selection guidance.
The five cases are listed in the B200 CI test database.
Reported CI results are mixed: #83724 passed, #84058 failed, #69231 failed, #69525 was aborted, and #84125 has no reported completion result.
Coverage verdict: needs follow-up. The five-case B200 performance run and baseline review are not confirmed complete.
Per-File QA Perspective
tests/integration/defs/perf/README_test_visual_gen_perf_sanity.md: Updates selectors, recipe examples, KPI observations, checkpoint constraints, and LTX2 stage guidance. QA should verify that documented selectors match the CI test-list entries.
tests/integration/test_lists/test-db/l0_b200_visual_gen_perf_sanity.yml: Replaces five scheduled B200 entries and preserves the 120-second FLUX.2 and 30-second non-FLUX.2 timeouts. QA should verify that every selector resolves to its intended recipe.
tests/integration/test_lists/waives.txt: Removes the FLUX.2 FP8 waiver. QA should verify that the BF16 replacement passes without a waiver.
tests/scripts/perf-sanity/visual_gen/flux2_blackwell.yaml: Changes FLUX.2 from dynamic FP8 to BF16 and enables CUDA graphs. QA should verify server startup, compilation, and the 1024×1024 single-frame request.
tests/scripts/perf-sanity/visual_gen/ltx2.yaml: Removes B200 from the shared LTX2 metadata. QA should verify that VR200 selection remains unchanged and B200 selection uses the dedicated recipes.
tests/scripts/perf-sanity/visual_gen/ltx2_blackwell.yaml: Adds B200 BF16 I2V and FP8 block-scale T2V two-stage recipes. QA should verify FA4 attention, parallelism, compilation, checkpoint paths, and forced two-stage execution.
tests/scripts/perf-sanity/visual_gen/ltx2_single_stage_blackwell.yaml: Adds a B200 FP8 block-scale I2V recipe with TLLM_LTX2_FORCE_ONE_STAGE_PIPELINE=1. QA should verify the one-stage execution path.
tests/scripts/perf-sanity/visual_gen/wan22_i2v_a14b_blackwell.yaml: Adds INT8 Q/K and FP8 V attention quantization, async Ulysses, CUDA graphs, and explicit compilation shapes. QA should verify the 1280×720, 81-frame workload.
No actionable comments were generated in the recent review.
Walkthrough
This change updates B200 VisualGen performance-sanity recipes for FLUX.2, Wan 2.2, and LTX-2. It refreshes the scheduled test cases, removes a FLUX.2 skip entry, and revises the README with recipe names, examples, measurements, and configuration guidance.
The FLUX.2 recipe changes to BF16 with CUDA graphs enabled. The Wan 2.2 recipe adds attention quantization and enables CUDA graphs. Two Blackwell LTX-2 recipes are added for two-stage and one-stage workloads. The generic LTX-2 recipe no longer lists B200 as supported.
Refresh scheduled test cases tests/integration/test_lists/test-db/l0_b200_visual_gen_perf_sanity.yml, tests/integration/test_lists/waives.txt
The five scheduled cases select the updated FLUX.2, Wan 2.2, and LTX-2 recipes. The FLUX.2 skip declaration is removed.
Align recipe guidance with configurations tests/integration/defs/perf/README_test_visual_gen_perf_sanity.md
The README updates recipe names and examples. It documents five B200 sweep-derived recipes, their measured KPIs and constraints, and guidance for selecting recipes and maintaining baselines.
NVIDIA/TensorRT-LLM#21929: Removes the Wan2.1 VisualGen performance-sanity recipe and test-list entry, affecting the same scheduled coverage set.
NVIDIA/TensorRT-LLM#21524: Changes the performance-sanity infrastructure that runs the selectors and updates related VisualGen guidance.
NVIDIA/TensorRT-LLM#21488: Changes the shared LTX-2 recipe naming and the related B200 test IDs and README guidance.
Merge Risk: Low · up to a2620
This updates B200 VisualGen performance-sanity recipes and does not change runtime code. The five-case B200 run and baseline review are still pending. Until they complete, the new recipes are unvalidated on hardware, and regression checking may be weaker for the renamed cases.
Pre-merge checks: 5
Check name
Status
Explanation
Title check
The title clearly identifies the test change and the B200 VisualGen performance recipe refresh.
Description check
The description clearly explains the purpose, affected workloads, recipe changes, validation results, limitations, and required pre-merge B200 performance run. It does not reproduce the template headi…
Docstring Coverage
No eligible functions found in the changed files. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff with cyclomatic complexity of at least 2. Documentat…
Linked Issues check
Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check
Check skipped because no linked issues were found for this pull request.
Full details: Description check
Explanation
The description clearly explains the purpose, affected workloads, recipe changes, validation results, limitations, and required pre-merge B200 performance run. It does not reproduce the template headings or checklist, but it provides the required information in a complete, detailed form.
Full details: Docstring Coverage
Explanation
No eligible functions found in the changed files. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff with cyclomatic complexity of at least 2. Documentation-only edits and deletions within surviving functions are included; unchanged diff context and fully deleted functions are excluded. Analyzed 0 functions across 0 files. (7 skipped: 7 unsupported.)
Finishing Touches
Generate unit tests (beta)
Create a new PR
Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts
Included review availability: This review used your included allowance. Your plan provides up to 12 included reviews per hour; 11 remain after this review.
The reason will be displayed to describe this comment to others. Learn more.
Review summary - Approve
Reviewed the full diff; no blocking or major issues found.
Automated review by NVCortex Lite, run by @LarryXFly.
This branch has not been deployed
No deployments
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
LarryXFly
left a comment
Refresh the five covered FLUX.2-dev, Wan2.2 I2V and LTX2 B200 sanity cases using the best observed October sweep recipes within the existing model/mode, logical GPU budget and usable CI checkpoint set. #21929 has merged and removed Wan2.1. Refresh the five remaining scheduled B200 cases without restoring the removed recipe or selector.
The measured source was main
dddb74c217dadabd6e9ba04ef22d48da53a73e01. Each point had one measured request, so these are observed minima under the tested constraints, not universal optima. Existing CI prompts/reference images remain unchanged; exact CI latency can differ. Update pytest names and scheduled selectors together, using measured warmup shapes and explicit LTX compile options. All refreshed B200 LTX cases pin the native BF16 file rather than a directory containing multiple precision variants. GPU allocations, client shapes/steps/counts, timeouts, KPI extraction and regression thresholds are unchanged.LTX checkpoint and stage selection
The old
ltx2_nvfp4case pointed to the generalLTX-2directory without enabling quantization. The staged nativeLTX-2/ltx-2-19b-dev-fp4.safetensorsheader declares 1,176 transformer-layernvfp4entries; video/audio VAE tensors remain BF16. The metadata converter at the #21929 snapshot (23512d112e) still recognizes onlyfloat8_e4m3fn, not thatnvfp4format (checked with an isolated execution of the actual converter). The overall one-stage sweep minimum (5.25 s) used a separate ModelOpt static NVFP4 checkpoint and is not claimed reproducible using this native file.Use the measured FP8-blockwise recipe with the explicit staged BF16 file instead: the best observed one-stage recipe among the usable checkpoints in the October staging set (6.20 s). No loader workaround, unmeasured on-load NVFP4 substitution or cache mutation is introduced.
The
LTX-2directory also contains the upscaler and distilled LoRA. The loader discovers them automatically, so omitting YAML fields does not reliably keep a run single-stage. Move the one-stage case into its own YAML withTLLM_LTX2_FORCE_ONE_STAGE_PIPELINE=1; explicitly set it to0for the two-stage family. This isolates server environments and preserves intended coverage without changing runtime code or adding scheduled B200 cases.Platform scope and integration
This branch is rebased onto main
64f7213093, which includes merged #21929. The redundant private copy of the Wan2.1-removal commit is dropped, so the PR contains only the recipe refresh and obsolete-waiver cleanup. The current sharedltx2.yamlalso covers VR200, so keep its existing recipes unchanged for VR200 and scope it to that GPU. Place the sweep-derived B200 settings inltx2_blackwell.yamlandltx2_single_stage_blackwell.yaml. This preserves VR200 recipes while refreshing the five B200 selectors. No VR200 performance claim is made.Waiver consistency repair
Removed the obsolete waiver for the retired FP8 FLUX.2 selector. The replacement BF16 recipe remains unwaived. Local execution of the affected VisualGen parameter-generation and waiver-membership checks reproduces the stale-entry error before cleanup and passes afterward; these scoped checks do not replace full CI collection or the five-case B200 performance run. NVBug 6759009 and its separate perf-sanity investigation are unchanged. The branch also incorporates main's existing CPU fixture/API corrections; no unrelated executor code is changed by this PR.
Baselines and validation
Nov11 local integration checks (head
a2620c2e9e0010e43042611ba2878e7562cb44c3, rebased onto main64f7213093; not a new GPU run):git diff --checkpassed.Historical preparation evidence (October sweep / original PR preparation; not rerun against the updated PR head):
Changed settings require separate CI baselines. Do not rewrite old timing evidence or accept a known regression by installing a slower baseline. Fastest-recipe KPI cases complement fixed controls/investigations; they do not cover every regression, including single-GPU FP8-blockwise LTX (NVBug 7030044) and eight-GPU BF16/TRTLLM Wan5B (NVBug 7030393).
The previous #69231 request was accepted, but single-GPU failures blocked the multi-GPU prerequisite, so none of the five performance cases ran. Results from prior or superseded heads do not validate this rebased head; its target run is still required.
Before merge: run the updated cases in target B200 CI and review their new baselines. The updated five-case B200 performance run and baseline review are pending; local parser/schema checks and ordinary PR CI do not replace that acceptance. Native NVFP4 metadata support or staging the separately measured ModelOpt checkpoint remains a follow-up.
Dev Engineer Review
ltx2.yamlfor VR200.QA Engineer Review
l0_b200_visual_gen_perf_sanity.ymlwith five scheduled entries:#83724passed,#84058failed,#69231failed,#69525was aborted, and#84125has no reported completion result.Per-File QA Perspective
tests/integration/defs/perf/README_test_visual_gen_perf_sanity.md: Updates selectors, recipe examples, KPI observations, checkpoint constraints, and LTX2 stage guidance. QA should verify that documented selectors match the CI test-list entries.tests/integration/test_lists/test-db/l0_b200_visual_gen_perf_sanity.yml: Replaces five scheduled B200 entries and preserves the 120-second FLUX.2 and 30-second non-FLUX.2 timeouts. QA should verify that every selector resolves to its intended recipe.tests/integration/test_lists/waives.txt: Removes the FLUX.2 FP8 waiver. QA should verify that the BF16 replacement passes without a waiver.tests/scripts/perf-sanity/visual_gen/flux2_blackwell.yaml: Changes FLUX.2 from dynamic FP8 to BF16 and enables CUDA graphs. QA should verify server startup, compilation, and the 1024×1024 single-frame request.tests/scripts/perf-sanity/visual_gen/ltx2.yaml: Removes B200 from the shared LTX2 metadata. QA should verify that VR200 selection remains unchanged and B200 selection uses the dedicated recipes.tests/scripts/perf-sanity/visual_gen/ltx2_blackwell.yaml: Adds B200 BF16 I2V and FP8 block-scale T2V two-stage recipes. QA should verify FA4 attention, parallelism, compilation, checkpoint paths, and forced two-stage execution.tests/scripts/perf-sanity/visual_gen/ltx2_single_stage_blackwell.yaml: Adds a B200 FP8 block-scale I2V recipe withTLLM_LTX2_FORCE_ONE_STAGE_PIPELINE=1. QA should verify the one-stage execution path.tests/scripts/perf-sanity/visual_gen/wan22_i2v_a14b_blackwell.yaml: Adds INT8 Q/K and FP8 V attention quantization, async Ulysses, CUDA graphs, and explicit compilation shapes. QA should verify the 1280×720, 81-frame workload.