[#20507][fix] Number Sm107a as 5 to match the prebuilt trtllm-gen archives by lucifer1004 · Pull Request #20518 · NVIDIA/TensorRT-LLM

lucifer1004 · 2026-10-18T00:06:46Z

Description

Fixes #20507.

The trtllm-gen FMHA target is compiled with TLLM_RUBIN_FEATURES, and the exported trtllm/gen/CudaArchDecl.h numbered CudaArch::Sm107a as 4. The prebuilt fmha/lib/<arch>/libTrtLlmGen.a and libTrtLlmGenFmhaLib.a were compiled with an enum that also has Sm105a: there 4 is Sm105a and 5 is Sm107a. So on SM107 every CudaArch value that TllmGenFmhaKernel passes into the archives (through options.mCudaArch, for example in warmupOneKernel and the FmhaAutoTuner paths) is read as Sm105a. Archive functions that switch on the architecture then fail; for example getMaxNumTmemCols throws Unsupported CUDA arch: 105a.

This PR sets Sm107a = 5 so both sides use the same numbering. Other values are unchanged (0..3 already match), and Sm107a is only defined when TLLM_RUBIN_FEATURES is set, so non-Rubin builds are unaffected.

Test Coverage

New CPU-only gtest cpp/tests/unit_tests/kernels/trtllmGenCudaArchAbiTest.cpp. It links only the prebuilt libTrtLlmGen.a (no libtensorrt_llm, no GPU) and checks that the archive's getMaxNumTmemCols returns 512 for Sm100a, Sm100f and Sm103a and 576 for Sm107a. It is in cpp/tests/unit_tests/kernels, so the existing cpp/test_unit_tests.py::test_unit_tests[kernels-*] stages pick it up.
Built the test locally with g++ 13 against googletest v1.15.2 and the x86_64 archive, with the same compile definitions as trtllm_gen_fmha_interface. With this change it passes. With the old header it fails with C++ exception with description "Unsupported CUDA arch: 105a".
I have not run a full CMake build of the test target.

PR Checklist

PR description clearly explains what and why.
PR follows the TRT-LLM coding guidelines to the best of my knowledge.
Test cases are provided for new code paths.
No public API changes, no new dependencies, no CODEOWNERS changes.
Please check this after reviewing the above items as appropriate for this PR.

Dev Engineer Review

Sm107a is explicitly assigned value 5 under TLLM_RUBIN_FEATURES. This matches the prebuilt FMHA archive numbering and preserves the preceding enum values.
The new ABI test links directly to the FMHA interface and processor-specific libTrtLlmGen.a archive. CMake excludes the test on Windows.
The author reports a local test pass and an expected failure with the old enum value. A full CMake build and CI test result are not confirmed.

QA Engineer Review

Added trtllmGenCudaArchAbiTest.cpp and registered the test in cpp/tests/unit_tests/kernels/CMakeLists.txt.
MaxNumTmemColsMatchesArchive checks that Sm100a, Sm100f, and Sm103a return 512, and Sm107a returns 576 from the archive.
No integration test-list entry was added. This is a unit test, not an integration test.
Coverage verdict: needs follow-up because a full CMake build and CI test execution are not confirmed.

Per-File QA Perspective

cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha/trtllmGen_fmha_export/trtllm/gen/CudaArchDecl.h: Changes Rubin Sm107a to value 5. Verify that SM107 archive calls use the intended architecture and that non-Rubin enum values remain unchanged.
cpp/tests/unit_tests/kernels/CMakeLists.txt: Adds a non-Windows test target with direct FMHA archive linkage. Verify archive discovery and linkage on supported platforms.
cpp/tests/unit_tests/kernels/trtllmGenCudaArchAbiTest.cpp: Checks archive TMEM limits for four architecture values. The test is not listed in the integration CI or manual-QA lists.

…en archives The prebuilt trtllm-gen FMHA archives use 4 for Sm105a and 5 for Sm107a. With TLLM_RUBIN_FEATURES the exported CudaArchDecl.h numbered Sm107a as 4, so the archives treated it as Sm105a. Add a CPU-only gtest that checks the numbering against libTrtLlmGen.a. Signed-off-by: Zihua Wu <13583761+lucifer1004@users.noreply.github.com>

coderabbitai · 2026-10-18T00:11:22Z

No actionable comments were generated in the recent review.

Recent review info

Run configuration

Source mode: Auto
Repository: NVIDIA/TensorRT-LLM
Branch: main
File: .coderabbit.yaml
Commit: ba2dddf
Layers: Repository YAML
Review profile: CHILL
Plan: Enterprise
Run ID: e467fa23-5ccb-4a41-a06e-e00b8e20c8ff

Commits

Reviewing files that changed from the base of the PR and between ce6662c and 5843f6f.

Files ignored due to path filters: 1

cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha/trtllmGen_fmha_export/trtllm/gen/CudaArchDecl.h is excluded by !**/gen/**

Files selected for processing: 1

cpp/tests/unit_tests/kernels/CMakeLists.txt

Included review availability: This review used your included allowance. Your plan provides up to 12 included reviews per hour; 11 remain after this review.

Walkthrough

Adds a non-Windows CPU test target linked directly to the FMHA interface and prebuilt archive. The test checks getMaxNumTmemCols for four CUDA architecture values.

Changes

CUDA architecture archive ABI test

Layer / File(s)	Summary
Configure and check archive ABI `cpp/tests/unit_tests/kernels/CMakeLists.txt`, `cpp/tests/unit_tests/kernels/trtllmGenCudaArchAbiTest.cpp`	CMake adds a non-Windows CPU test target with direct archive linkage. The test expects `512` for `Sm100a`, `Sm100f`, and `Sm103a`, and `576` for `Sm107a`.

Priority: Normal

Estimated code review effort: 2 (Simple) | ~8 minutes

Change: Bug fix · Severity of issue fixed: Medium

Suggested reviewers: qijune

Merge Risk: Minimal · up to 5843f

No actionable merge-blocking risk was identified in the reviewed change.

Pre-merge checks: 5

Check name	Status	Explanation
Description check		The description includes complete Description, Test Coverage, and PR Checklist sections. It clearly explains the issue, solution, test scope, local validation, and remaining limitation that a full CMa…
Title check		The title follows the required [issue][type] format and clearly summarizes the primary change: aligning Sm107a numbering with the prebuilt trtllm-gen archives.
Linked Issues check		Issue `#20507` requires matching `CudaArch` values across the FMHA interface and the prebuilt archives. The PR reports `Sm107a = 5` under `TLLM_RUBIN_FEATURES`. The CPU-only test links the FMHA interfac…
Out of Scope Changes check		The changed CMake file registers the ABI test, and the test directly validates the `#20507` archive boundary. These changes support the linked issue. No unrelated implementation, dependency, or configur…
Docstring Coverage		Docstring coverage is inconclusive: files from earlier reviews still need to be rechecked within the check's analysis limits. Not blocking.

Full details: Description check

Explanation

The description includes complete Description, Test Coverage, and PR Checklist sections. It clearly explains the issue, solution, test scope, local validation, and remaining limitation that a full CMake build was not run.

Full details: Linked Issues check

Explanation

Issue #20507 requires matching CudaArch values across the FMHA interface and the prebuilt archives. The PR reports Sm107a = 5 under TLLM_RUBIN_FEATURES. The CPU-only test links the FMHA interface directly to libTrtLlmGen.a and verifies that Sm107a returns the archive's 576-TMEM-column result. It also verifies the unchanged Sm100a, Sm100f, and Sm103a mappings. The exported header is excluded from direct inspection, but the reported passing test and its failure with the old header provide independent behavior evidence.

Full details: Out of Scope Changes check

Explanation

The changed CMake file registers the ABI test, and the test directly validates the #20507 archive boundary. These changes support the linked issue. No unrelated implementation, dependency, or configuration change is shown.

Finishing Touches

🧪 Generate unit tests (beta)

Create a new PR

Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

_{Comment @coderabbitai help to get the list of available commands.}

yuxianq · 2026-10-18T00:15:32Z

@yunruis, could you help to review this PR as trt-llm-torch-attention-devs? Thanks~

Assignment explanation

Components	kernels/trtllm-gen (100%)
Tags	None identified
PR effort	3 points — The implementation change is tiny, but reviewing binary-contract compatibility requires more reasoning than a mechanical enum edit. Missing test and build diffs add verification effort.
Importance	Important — The enum change affects a sensitive architecture ABI with prebuilt FMHA archives. Incorrect numbering can cause architecture misinterpretation; the visible change is restricted to TLLM_RUBIN_FEATURES builds.
Affinity (0–1)	0.580 component/tag match; 0.435 after PIC preference
Outstanding points at selection	53 → 56 (+3)
Points already reserved on this PR	0
Effective weight	1
Completed review points (preceding 168 hours)	16
Combined points after assignment	72
Demand-derived target	71.00 points (soft share)
Additional fair share before assignment	2.00 points
Share explanation	Whole PR / qualification or coverage constraint exceeds a soft share; assignment is allowed.
Selection rule	affinity with 0.3-share guard
Assignment scope (yunruis)	all

Affinity and workload are snapshots at selection, not live counters. Existing reservations on this PR are counted once.

yunruis

CudaArchDecl.h is exported from trtllm-gen, so the problem will be appear if export again.
I think the right and clean way is to sync the revision Sm107a=5 in trtllm-gen too.

lucifer1004 · 2026-10-19T15:29:35Z

@yunruis thanks for the review. Agreed: CudaArchDecl.h is exported from trtllm-gen, so the clean fix is to make Sm107a = 5 there too, otherwise the next export brings back 4. That source is not in this repository, so could you (or whoever owns the trtllm-gen export) sync the value on that side? Until then, the new TrtllmGenCudaArchAbiTest.MaxNumTmemColsMatchesArchive gtest guards this repo: it asks the prebuilt libTrtLlmGen.a about CudaArch::Sm107a (576 TMEM columns, distinct from the other archs), so a re-export with the old numbering would fail that test rather than silently dispatch SM107 as SM105.

lucifer1004 · 2026-10-19T15:29:37Z

/bot run

tensorrt-cicd · 2026-10-19T15:45:22Z

PR_Github #78962 [ run ] triggered by Bot. Commit: 70fbb9b Link to invocation

tensorrt-cicd · 2026-10-19T19:29:44Z

PR_Github #78962 [ run ] completed with state SUCCESS. Commit: 70fbb9b
/LLM/main/L0_MergeRequest_PR pipeline #65051 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

lucifer1004 · 2026-10-19T20:28:00Z

CI #78962 failed outside this PR. The only hard failure is on DGX_B200-PyTorch-6: unittest/_torch/thop/parallel/test_cute_dsl_moe.py::test_nvfp4_gather_grouped_gemm_act_fusion_blackwell[1024-1-32-128-relu2] and [8192-8-1-256-relu2] had about 5.0% of their elements outside the atol/rtol 1e-4 tolerance, just over the 5% limit (see #16281 (comment)). That is the CuTe DSL NVFP4 MoE path, which does not use the trtllm-gen FMHA headers this PR touches; the Sm107a value is also only compiled under TLLM_RUBIN_FEATURES. Both cases passed in the other recent runs on B200 / GB300, and the same test function hit a similar overshoot (5.4% of elements) with a different parametrization on #20368 earlier today. The remaining stages were aborted by fail-fast.

Re-running.

lucifer1004 · 2026-10-19T20:28:04Z

/bot run

tensorrt-cicd · 2026-10-19T20:36:24Z

PR_Github #79028 [ run ] triggered by Bot. Commit: 70fbb9b Link to invocation

tensorrt-cicd · 2026-10-19T21:28:29Z

PR_Github #79028 [ run ] completed with state SUCCESS. Commit: 70fbb9b
/LLM/main/L0_MergeRequest_PR pipeline #65102 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

lucifer1004 · 2026-10-19T22:28:03Z

CI #79028 did not get to run tests: the CPU-Generic-arm-1 pod could not be scheduled and then could not pull the SBSA test image ("Unable to pull Docker image", Kubernetes pod launch timeout), so that stage was aborted and fail-fast cancelled the other test stages. Both builds passed. This is an infrastructure failure, not related to this PR. Re-running.

lucifer1004 · 2026-10-19T22:28:07Z

/bot run

tensorrt-cicd · 2026-10-19T22:34:32Z

PR_Github #79041 [ run ] triggered by Bot. Commit: 70fbb9b Link to invocation

tensorrt-cicd · 2026-10-20T01:49:10Z

PR_Github #79041 [ run ] completed with state FAILURE. Commit: 70fbb9b
/LLM/main/L0_MergeRequest_PR pipeline #65113 completed with status: 'UNSTABLE'

CI Report

⚠️ Multi-GPU Label Required:
Multi-GPU tests require the ci: full pre-merge approved label on this PR. Either:

Wait for the PR to be fully approved — the label is added automatically once approval is complete. Having unresolved open comments is fine, or
If needed, ask a member of NVIDIA/trt-llm-ci-approvers to add the label manually.
Then re-trigger CI with the same bot command (no rebase needed).

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

Link to invocation

Signed-off-by: Zihua Wu <13583761+lucifer1004@users.noreply.github.com>

lucifer1004 · 2026-10-20T18:41:08Z

The last CI run didn't get to the multi-GPU stages. No test failures were counted in it. A CPU arm unit test, tools/test_periodic_junit.py::test_hang_dumps_traceback, failed and went through a rerun; that marked the pipeline unstable and blocked the multi-GPU stages. The test has passed in its recent runs elsewhere, and it does not touch anything this PR changes (the Sm107a arch enum value and its CPU gtest).

I merged current main (557ddca) so the multi-GPU stages pick up the recent waives, including the GPT-OSS eagle3 accuracy case (https://nvbugs/6962175). The PR diff is unchanged. Re-running.

lucifer1004 · 2026-10-20T18:41:10Z

/bot run

tensorrt-cicd · 2026-10-20T18:49:09Z

PR_Github #79358 [ run ] triggered by Bot. Commit: ce6662c Link to invocation

tensorrt-cicd · 2026-10-20T22:04:12Z

PR_Github #79358 [ run ] completed with state SUCCESS. Commit: ce6662c
/LLM/main/L0_MergeRequest_PR pipeline #65361 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

lucifer1004 · 2026-10-21T18:36:13Z

The last run failed for infrastructure reasons and no test failure was reported: the DGX_B200 stages could not reach the Slurm login node, and fail-fast then interrupted the remaining stages. The B200 stages are running normally again on other PRs, so re-running.

lucifer1004 · 2026-10-21T18:36:14Z

/bot run

tensorrt-cicd · 2026-10-21T19:00:52Z

PR_Github #79678 [ run ] triggered by Bot. Commit: ce6662c Link to invocation

tensorrt-cicd · 2026-10-22T04:58:11Z

PR_Github #79678 [ run ] completed with state FAILURE. Commit: ce6662c
/LLM/main/L0_MergeRequest_PR pipeline #65601 completed with status: 'UNSTABLE'

CI Report

⚠️ Multi-GPU Label Required:
Multi-GPU tests require the ci: full pre-merge approved label on this PR. Either:

Wait for the PR to be fully approved — the label is added automatically once approval is complete. Having unresolved open comments is fine, or
If needed, ask a member of NVIDIA/trt-llm-ci-approvers to add the label manually.
Then re-trigger CI with the same bot command (no rebase needed).

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

Link to invocation

lucifer1004 · 2026-10-23T08:30:08Z

The last run reported no test failures, but it stopped before the multi-GPU stages. On A10, disaggregated/test_workers.py::test_workers_kv_cache_aware_router timed out once and passed when it was rerun. The pipeline was therefore marked unstable, and both multi-GPU stages were blocked. That test also times out intermittently on other PRs, and it does not exercise the trtllm-gen arch numbering this PR changes. Re-running.

lucifer1004 · 2026-10-23T08:30:10Z

/bot run

tensorrt-cicd · 2026-10-23T08:37:47Z

PR_Github #80118 [ run ] triggered by Bot. Commit: ce6662c Link to invocation

tensorrt-cicd · 2026-10-23T10:09:14Z

PR_Github #80118 [ run ] completed with state FAILURE. Commit: ce6662c
/LLM/main/L0_MergeRequest_PR pipeline #65934 completed with status: 'UNSTABLE'

CI Report

⚠️ Multi-GPU Label Required:
Multi-GPU tests require the ci: full pre-merge approved label on this PR. Either:

Wait for the PR to be fully approved — the label is added automatically once approval is complete. Having unresolved open comments is fine, or
If needed, ask a member of NVIDIA/trt-llm-ci-approvers to add the label manually.
Then re-trigger CI with the same bot command (no rebase needed).

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

Link to invocation

lucifer1004 · 2026-10-24T00:35:32Z

Correction to my previous comment: the multi-GPU stages were not blocked by the A10 timeout. They are gated on the ci: full pre-merge approved label, which is added automatically once the PR has its required approvals. All single-GPU stages passed in the last two runs; I'll re-run CI once the label is in place.

Signed-off-by: Zihua Wu <13583761+lucifer1004@users.noreply.github.com>

lucifer1004 · 2026-10-29T14:13:06Z

/bot run

tensorrt-cicd · 2026-10-29T14:20:55Z

PR_Github #81434 [ run ] triggered by Bot. Commit: 5843f6f Link to invocation

tensorrt-cicd · 2026-10-29T18:29:36Z

PR_Github #81434 [ run ] completed with state SUCCESS. Commit: 5843f6f
/LLM/main/L0_MergeRequest_PR pipeline #67110 completed with status: 'UNSTABLE'

CI Report

⚠️ Multi-GPU Label Required:
Multi-GPU tests require the ci: full pre-merge approved label on this PR. Either:

Wait for the PR to be fully approved — the label is added automatically once approval is complete. Having unresolved open comments is fine, or
If needed, ask a member of NVIDIA/trt-llm-ci-approvers to add the label manually.
Then re-trigger CI with the same bot command (no rebase needed).

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

Link to invocation

lucifer1004 requested review from a team as code owners October 18, 2026 00:06

lucifer1004 requested review from heyuhhh, lfr-0531 and rosong11 October 18, 2026 00:06

github-actions Bot assigned lucifer1004 Oct 18, 2026

yuxianq requested review from yunruis and removed request for heyuhhh and lfr-0531 October 18, 2026 00:15

This was referenced Oct 18, 2026

[#20506][fix] Size the generation-only attention workspace without the context requests #20520

Closed

[#20515][fix] Pass the environment's nvcc to the trtllm-gen FMHA JIT #20521

Open

yunruis approved these changes Oct 19, 2026

View reviewed changes

lucifer1004 mentioned this pull request Oct 20, 2026

[Bug] test_nvfp4_gather_grouped_gemm_act_fusion_blackwell flaky on B300: ~0.05% mismatch vs rtol 1e-4, varying parametrizations #16281

Open

Merge main into upstream/sm107a-abi

ce6662c

Signed-off-by: Zihua Wu <13583761+lucifer1004@users.noreply.github.com>

Merge main into upstream/sm107a-abi

5843f6f

Signed-off-by: Zihua Wu <13583761+lucifer1004@users.noreply.github.com>

Conversation

lucifer1004 commented Oct 18, 2026 • edited by coderabbitai Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Test Coverage

PR Checklist

Dev Engineer Review

QA Engineer Review

Per-File QA Perspective

Uh oh!

coderabbitai Bot commented Oct 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Walkthrough

Changes

Uh oh!

yuxianq commented Oct 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

yunruis left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

lucifer1004 commented Oct 19, 2026

Uh oh!

lucifer1004 commented Oct 19, 2026

Uh oh!

tensorrt-cicd commented Oct 19, 2026

Uh oh!

tensorrt-cicd commented Oct 19, 2026

Uh oh!

lucifer1004 commented Oct 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

lucifer1004 commented Oct 19, 2026

Uh oh!

tensorrt-cicd commented Oct 19, 2026

Uh oh!

tensorrt-cicd commented Oct 19, 2026

Uh oh!

lucifer1004 commented Oct 19, 2026

Uh oh!

lucifer1004 commented Oct 19, 2026

Uh oh!

tensorrt-cicd commented Oct 19, 2026

Uh oh!

tensorrt-cicd commented Oct 20, 2026

Uh oh!

lucifer1004 commented Oct 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

lucifer1004 commented Oct 20, 2026

Uh oh!

tensorrt-cicd commented Oct 20, 2026

Uh oh!

tensorrt-cicd commented Oct 20, 2026

Uh oh!

lucifer1004 commented Oct 21, 2026

Uh oh!

lucifer1004 commented Oct 21, 2026

Uh oh!

tensorrt-cicd commented Oct 21, 2026

Uh oh!

tensorrt-cicd commented Oct 22, 2026

Uh oh!

lucifer1004 commented Oct 23, 2026

Uh oh!

lucifer1004 commented Oct 23, 2026

Uh oh!

tensorrt-cicd commented Oct 23, 2026

Uh oh!

tensorrt-cicd commented Oct 23, 2026

Uh oh!

lucifer1004 commented Oct 24, 2026

Uh oh!

lucifer1004 commented Oct 29, 2026

Uh oh!

tensorrt-cicd commented Oct 29, 2026

Uh oh!

tensorrt-cicd commented Oct 29, 2026

Uh oh!

lucifer1004 commented Oct 18, 2026 •

edited by coderabbitai Bot

Loading

coderabbitai Bot commented Oct 18, 2026 •

edited

Loading

yuxianq commented Oct 18, 2026 •

edited

Loading

yunruis left a comment •

edited

Loading

lucifer1004 commented Oct 19, 2026 •

edited

Loading

lucifer1004 commented Oct 20, 2026 •

edited

Loading