[taianz-nv]

Description

Remove the exact WAN pipeline-parallel waiver for NVBug 6644465. The shared CUTLASS DSL 4.6.1 regression was fixed on main by #18145, merged on 2026-08-30; current main uses CUTLASS DSL 4.7.0.

The sibling MiniMax-M3 issue NVBug 6670052 has no waiver entry, so no additional test-list change is needed here.

Validation

• NSC job 1905270, ToT e1e6bcca3b, 8x B200, CUTLASS DSL 4.7.0, FA4 4.0.0b19:
  • test_split_kv_matches_no_split: 2/2 passed (auto and forced num_splits=8).
  • Exact waived WAN case passed; cosine similarity to the HF reference was 0.999632.
• NSC job 1905495, B200, same ToT/kernel stack:
  • test_sparse_decode_matches_msa_kernel passed, directly exercising the fmha_sm100 sparse GQA/combine path implicated by NVBug 6670052.
• Weekly rc26 regression on 2026-09-04: all seven exact minimax_m3_fp4 B200 perf cases passed (job 1155).

The NSC manual run used ToT Python and pinned kernel dependencies with an rc25 precompiled native wheel. A test-only import fallback was used solely for the post-rc25 start_coordinator_watchdog binding, which these tests do not exercise; it is not included in this PR. The required 8x B200 pre-merge stage will be requested on this PR.

Checklist

• Change is limited to the obsolete waiver.
• git diff --check passes.
• Pre-commit hooks for waives.txt pass, including duplicate and AST-entry validation.
• Commit is signed off.

Dev Engineer Review

• Removed one obsolete waiver from tests/integration/test_lists/waives.txt.
• The change restores execution of the exact WAN pipeline-parallel test without changing source code, APIs, or defaults.
• Scope is limited to the intended waiver removal. The reported WAN and sparse decode validation passed.

QA Engineer Review

• Modified tests/integration/test_lists/waives.txt.
• Removed the waiver for unittest/_torch/visual_gen/multi_gpu/test_wan_pipeline_parallel.py::TestWanPipelineParallel::test_cfg2_attn2d2x1_ulysses2_pvae8.
• The change enables the test after the CUTLASS DSL regression fix. WAN, related sparse decode tests, and seven minimax_m3_fp4 performance cases passed.
• Coverage verdict: sufficient.

Per-File QA Perspective

• tests/integration/test_lists/waives.txt: Removes the exact test waiver. QA should confirm that the WAN pipeline-parallel test runs and remains stable in CI; no test implementation changed.

[taianz-nv]

/bot run --extra-stage "DGX_B200-8_GPUs-PyTorch-1"

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: d3f79906-cba3-4ea8-9060-c379798f60d8

📥 Commits

Reviewing files that changed from the base of the PR and between d8d7d38 and 0232b93.

📒 Files selected for processing (1)

• tests/integration/test_lists/waives.txt

💤 Files with no reviewable changes (1)

• tests/integration/test_lists/waives.txt

Included review availability: Your plan provides up to 12 included reviews per hour; 11 remain after this review.

---

Walkthrough

The integration test skip list no longer waives TestWanPipelineParallel::test_cfg2_attn2d2x1_ulysses2_pvae8.

Changes

Wan pipeline-parallel test

Layer / File(s)	Summary
Remove Wan pipeline waiver tests/integration/test_lists/waives.txt	Removed the waiver for the Wan pipeline-parallel test associated with NVIDIA bug 6644465.

Priority: ⬇️ Low

Estimated code review effort: 1 (Trivial) | ~2 minutes

Merge Risk: ⚪ Minimal · up to 0232b

The previously waived Wan pipeline-parallel test will run again; no current merge-blocking risk is identified.

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly identifies the NVBugs ticket, test-related change, and removal of the WAN pipeline-parallel waiver.
Description check	✅ Passed	The description explains the issue and solution, provides detailed validation results, and includes a relevant checklist. It uses "Validation" and "Checklist" instead of the template headings "Test Co…
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff. Analyzed 0 functions across 0…
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

_{Comment @coderabbitai help to get the list of available commands.}

[tensorrt-cicd]

PR_Github #72324 [ run ] triggered by Bot. Commit: 0232b93 Link to invocation

[tensorrt-cicd]

PR_Github #72324 [ run ] completed with state SUCCESS. Commit: 0232b93
/LLM/main/L0_MergeRequest_PR pipeline #59351 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

• Please check the failed tests and fix your PR
• If you cannot view the failures, ask the CI triggerer to share details
• Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

[taianz-nv]

/bot run

[tensorrt-cicd]

PR_Github #72366 [ run ] triggered by Bot. Commit: 0232b93 Link to invocation

[tensorrt-cicd]

PR_Github #72366 [ run ] completed with state FAILURE. Commit: 0232b93
/LLM/main/L0_MergeRequest_PR pipeline #59391 completed with status: 'UNSTABLE'

CI Report

⚠️ Multi-GPU Label Required:
Multi-GPU tests require the ci: full pre-merge approved label on this PR. Ask a member of NVIDIA/trt-llm-ci-approvers to add the label, then re-trigger CI with the same bot command (no rebase needed).

⚠️ Action Required:

• Please check the failed tests and fix your PR
• If you cannot view the failures, ask the CI triggerer to share details
• Once fixed, request an NVIDIA team member to trigger CI again

Link to invocation

[github-actions]

Automatically added "ci: full pre-merge approved" because this PR has satisfied the required GitHub review approvals. Unresolved review conversations and other required checks remain independent merge requirements.

[taianz-nv]

/bot run

[tensorrt-cicd]

PR_Github #72847 [ run ] triggered by Bot. Commit: 0232b93 Link to invocation

[tensorrt-cicd]

PR_Github #72847 [ run ] completed with state SUCCESS. Commit: 0232b93
/LLM/main/L0_MergeRequest_PR pipeline #59828 completed with status: 'SUCCESS'

CI Report

Link to invocation