[yizhang-nv]

Dev Engineer Review

The inspected checkout contains only three modified files, and all changes reformat imports. No behavior or API changes are visible. The supplied change summary describes additional files and functional changes that are not present in the inspected diff.

QA Engineer Review

The three modified files are attention unit tests. Their test logic is unchanged; only imports were reformatted. No matching entries were found by filename search in tests/integration/test_lists/test-db/ or tests/integration/test_lists/qa/. Test execution was not established. Coverage verdict: needs follow-up because the supplied broader change summary does not match the inspected checkout.

Per-File QA Perspective

• tests/unittest/_torch/attention/test_attention.py: Covers attention backends and KV-cache paths. Only import formatting changed; no matching CI or manual-QA list entry was found by filename search.
• tests/unittest/_torch/attention/test_flashinfer_attention.py: Covers FlashInfer attention. Only import formatting changed; no matching CI or manual-QA list entry was found by filename search.
• tests/unittest/_torch/attention/test_vanilla_attention.py: Covers vanilla attention. Only import formatting changed; no matching CI or manual-QA list entry was found by filename search.

Description

Migrate attention/kernel fixtures and the local performance harness to KVCacheManagerV2 while preserving workloads, parameter sets, numerical assertions, and performance/accuracy gates. The fixtures handle physical page allocation, shared per-layer views, global HELIX capacity, and the current DSA interface. Sparse request histories remain identical across page layouts. GPU-only HELIX fixtures disable host cache to avoid an unmatched host-sizing collective in the rank-zero reference.

DeepSeek prims-ts removes the explicit V1 override to use the default manager. The original post-warmup delegated-backend-call assertion and GSM8K accuracy gate are retained. Temporary manager-type assertions and JUnit diagnostic properties have been removed after validation. This branch was rebased with git pull --rebase origin main onto fced9ff9849c317a031e13b1e403e1fe61927dc4, where V2 is enabled by default. Candidate 2df698293c7c656ad4e3b7021287950771a4c1d8 contains eight T2-only commits, 19 files, +174/-135. Parent #17495 and sibling commits are not part of the PR increment. The previous frozen-parent validation is historical evidence and is not counted as a pass on this baseline.

Low-level HELIX/K3 helper callers remain in scope. DeepSeek/Qwen HELIX model E2E and Kimi model E2E are unchanged. Explicit V1/V2 comparison parameters remain as controls. Selectors and original assertions/decorators were audited against the GPU-validated afc1fa27 baseline; no CI/QA selector, waiver, golden, or runtime product change is included.

Test Coverage

The latest rebase preserves main's modeling_v2 infrastructure update: three owned MLA tests now live under tests/unittest/_torch/modeling_v2/models/deepseek_v3/r1_0528_nvfp4__sm_103__dep4/attention/. All T2-added/deleted lines are identical after mapping those upstream renames; main's other test and operator changes remain intact. Syntax, signed-commit verification, diff checks and repository pre-commit hooks passed. GPU tests were not rerun after this rebase. All GPU results below remain evidence for the stated afc1fa27 baseline and original run SHAs, not a pass claim for the new main baseline. Rebase evidence is in tmp/t2-rebase-20261013/.

The scoped matrix on afc1fa27 has finished execution, with three unchanged-gate failures still unresolved. All runs use task-owned containers, caches and GPU allocations through tekit-run; these local runs preceded the CI request recorded below. Skips are not passes. Scoped local execution does not claim that complete CI stages passed.

GPU execution on afc1fa2	Actual resources	Current result
Eight scoped attention files selected by l0_h100	H100, 1 GPU	993 unique passed, 180 skipped, combining the complete suite and complete fixed-DSA regression.
CPU-marked DSA cases	x86 and ARM, no visible GPU	8 unique cases passed on each architecture. CUDA drivers remain mounted; this is not a driver-less CPU-host result.
Eight scoped attention files selected by l0_b200	B200, 1 GPU	1169 unique passed, 4 skipped, combining the complete suite and complete fixed-DSA regression.
Eight scoped attention files selected by l0_b300	B300 PCIe, 1 GPU	1168 unique passed, 4 skipped, 1 failed, combining the complete suite and complete fixed-DSA regression. Sparse failure remains.
Eight scoped attention files selected by l0_gb300_multi_gpus	GB300 ARM, 4 NVLink GPUs visible	1169 unique passed, 4 skipped, combining the complete suite, 12-case MSA dependency supplement and complete fixed-DSA regression.
Complete DSA file after radix fixture correction	H100 / B200 / B300 PCIe / B300 NVLink / GB300	142 passed, 19 skipped on H100; 157 passed, 4 skipped on each Blackwell platform, all exit 0. Each full-file total includes eight CPU cases, reported separately from GPU attention totals. All 20 formerly hidden custom-TopK cases now pass with original assertions.
Low-level HELIX MHA/MLA/zero-KV	GB300 ARM and B300 NVLink, 4 NVLink GPUs visible, original 2 MPI ranks	53/53 passed on each, including the zero-KV negative control. Final B300 NVLink run used clean 9fae1e00, configured /usr/bin/python, and exited 0.
modeling_v2 attention	B300 PCIe, 1 GPU; supplemental B300 NVLink, 4 GPUs visible	78/78 passed on each.
Kimi K3 MLA kernel helper caller	B200, 1 GPU	4/4 passed; model E2E remains outside scope.
DeepSeek prims-ts, l0_b200 and QA selector	B200, 1 GPU	1319 GSM8K samples, 64.632% accuracy > unchanged 61.537% gate. Actual KVCacheManagerV2, NoneType transceiver, 12,120 post-warmup delegated MLA generation calls. No explicit V2 override.
Historical sparse FP8 numerical failure	B300 PCIe, 1 GPU	Still fails: test_generation_page_sparse_mha[float16_h256_fp8_kv_fp8_output], 1/2048 elements at (0, 980); absolute/relative error 0.125 exceeds 0.08/0.04. Byte-identical original-main V1 fixture on the same GPU reproduces the identical failure.
Full local attention performance harness	B200, 1 GPU	5 passed, 11 missing-golden skips, 2 failed. Context timing remains 0.738384 ms > 0.674377 ms and 0.136160 ms > 0.124740 ms. The original-main V1 harness on the same quiet GPU also fails at 0.737312/0.136160 ms. Original three fresh-process repetitions and goldens unchanged.
Full local attention performance harness	H100 PCIe and B300 PCIe, 1 GPU each	18 measured bootstrap skips on each: exact device-name keys lack goldens. H100 measurements are earlier build-venv diagnostics; PCIe B300 uses the configured runtime. No device-name remapping or performance pass claimed.
Rubin trtllm_fp8_rubin benchmark, local only	Actual SM107 ARM NVIDIA VR NVL72 ES, 1 GPU	All six original batches 16/30/60/120/200/300 completed with finite-output checks; sequence 30080, heads 128, qlen 1, warmup 10, iterations 50. 0.071/0.101/0.236/0.478/0.485/0.719 ms; actual V2 in every constructor, exit 0. No golden/CI-baseline claim.
Eight scoped attention files selected by l0_dgx_b300	B300, 4 NVLink GPUs	1169 unique passed, 4 skipped, combining the complete suite and complete fixed-DSA regression. The separate complete configured-runtime HELIX result is above; an earlier build-venv HELIX attempt was intentionally stopped after 25 cases and is not counted as a complete pass.

Each attention row covers 1173 unique GPU cases. The original 419 explicit V1 control cases are retained and counted separately in the matrix: H100 has 411 passing controls and eight architecture skips; each Blackwell platform has 419 passing controls. These are not V2 route evidence. The four remaining Blackwell skips are the existing CuTe DSL FP4 reference-mismatch skips; all 180 H100 skips are existing architecture gates. No skipped case is reported as a pass.

The empty Custom topk not available skips were traced to the DSA mock omitting the radix scratch tensors required by the production metadata contract. Both original selectors catch the resulting bare AssertionError at indexer.py:1987; the omission also exists in the tested main baseline. The mock now initializes the row capacity and calls the production radix allocator. No assertion, reference calculation, or runtime code changed. All pre-fix skips remain separately recorded.

The same-GPU original-main sparse/performance controls show that the T2 migration is not required to reproduce those failures. They do not establish a root cause or waive the unchanged gates. Existing explicit V1 control parameters are also reported separately from V2 route evidence.

Runtime: NGC 26.09, PyTorch 2.14.0a0+b2c75dd062.nv26.09, CUDA 13.4, NCCL 2.31.2. Initial full suites ran at clean 9c728acc; the subsequently validated 9fae1e00 adds only the two DSA fixture initialization lines. The affected file was rerun in full: GB300 and B300 NVLink at clean 9fae1e00; H100, B200 and B300 PCIe using final fixture bytes (SHA256 0c35bc2aafdc49467010f07522df151e563f61ea03f8d989db300c6e5e40b541) against the unchanged clean product checkout. Native artifacts come from official build 63637, source 5fe59f05c14642dd77e23e12c8363b52ec618fd3; native/build inputs were verified identical to the candidate. x86 wheel SHA256 b530cc62c1cb5baad8ec78c43c63f59cc806ecbf47f2d238a64f9d9f45bfc8c3; ARM wheel 66222bd7bc8c1aeb431a7c53b4b06b4f18d841cf2fd3dfbc8c6bbed83505451d. These are not the old frozen-parent artifacts. The earlier NGC26.08 local compile is not used by these runs; the redundant NGC26.09 compile was stopped after artifact equivalence was established.

The final diagnostic cleanup at f644b4dd removes three JUnit properties, the temporary manager-type assertion and its unused import/fixture argument. Syntax, repository commit hooks, and an AST comparison against the preceding commit passed. Model configuration, workload, GSM8K accuracy gate and the original delegated-backend-call assertion are unchanged. GPU validation was not repeated for this cleanup; the results and observed route properties above retain their original validated source SHAs. prims-cleanup-verification.json records the final delta and GitHub comparison.

The canonical x86 runs use the configured /usr/bin/python runtime. Old build-venv attempts are retained as history. Twelve MSA cases skipped in incomplete runtime environments were rerun after restoring the official wheel's fmha_sm100 package; package provenance and unchanged native hashes were checked, and all twelve now pass on B200, B300 PCIe, B300 NVLink and GB300.

B300 NVLink HELIX used NVLink P2P; the runtime reported that NVLS multicast was unavailable because the node's IMEX fabric was not provisioned. The startup PMIx output-descriptor epoll errors are retained in stderr; all 53 cases, JUnit, pytest and the outer runner completed successfully. This is correctness evidence for the observed topology, not an NVLS performance claim.

Full nodeid/parameter/registration matrix, source and native manifests, route properties, JUnit outcomes, and execution notes are retained under /home/scratch.yizhan_sw_1/tekit-kvcm-v2-attention/tmp/t2-main-20261012/; final-validation.json is the canonical deduplicated result and executed-matrix.json maps runs to CI/QA registrations at validation time. Complete stdout/stderr logs are separate under /home/scratch.yizhan_sw_1/logs/2026-10-12/<node>/t2_*. The draft remains open for review and disposition of the sparse numerical failure, two performance gate failures, and remaining original skip/missing-golden limits. No assertion or golden is relaxed.

PR Checklist

• Scope, source baseline, preserved assertions and current validation limits are documented.
• Signed commits and repository hooks passed.
• No public API, dependency, ownership or runtime implementation changes.
• Execute the scoped afc1fa27 matrix and preserve complete results and control comparisons.
• Resolve or explicitly disposition the unchanged-gate failures and remaining acceptance limits.

GitHub Bot Help

On 2026-10-13, the user requested that local diagnosis stop at the current state and CI check the candidate. Posted /bot run --disable-fail-fast for head 2df698293c7c656ad4e3b7021287950771a4c1d8, using the standard CBTS-selected scope. The PR remains draft and the three local failures remain unresolved; no gate, golden, or waiver changed. The two local performance cases are not registered in CI, so a CI pass will not resolve those failures. Actual stage selection and results must be checked before claiming coverage of the sparse case or any GPU combination.

[yizhang-nv]

/bot run --disable-fail-fast

[tensorrt-cicd]

PR_Github #77490 [ run ] triggered by Bot. Commit: 2df6982 Link to invocation

[tensorrt-cicd]

PR_Github #77490 [ run ] completed with state SUCCESS. Commit: 2df6982
/LLM/main/L0_MergeRequest_PR pipeline #63878 completed with status: 'UNSTABLE'

CI Report

⚠️ Multi-GPU Label Required:
Multi-GPU tests require the ci: full pre-merge approved label on this PR. Either:

• Wait for the PR to be fully approved — the label is added automatically once approval is complete. Having unresolved open comments is fine, or
• If needed, ask a member of NVIDIA/trt-llm-ci-approvers to add the label manually.
  Then re-trigger CI with the same bot command (no rebase needed).

⚠️ Action Required:

• Please check the failed tests and fix your PR
• If you cannot view the failures, ask the CI triggerer to share details
• Once fixed, request an NVIDIA team member to trigger CI again

Link to invocation

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

• Configuration used: Repository: NVIDIA/TensorRT-LLM/.coderabbit.yaml
• Review profile: CHILL
• Plan: Enterprise
• Run ID: 2d226389-659c-40a3-b4ff-13302dd651db

📥 Commits

Reviewing files that changed from the base of the PR and between b22283a and 2df6982.

📒 Files selected for processing: 19

• tests/integration/defs/accuracy/test_llm_api_pytorch.py
• tests/microbenchmarks/attention_perf/attention_perf_harness.py
• tests/microbenchmarks/bench_fp4_mla_decode.py
• tests/unittest/_torch/attention/backend_case.py
• tests/unittest/_torch/attention/multi_gpu/helix_test_utils.py
• tests/unittest/_torch/attention/multi_gpu/test_mha_helix.py
• tests/unittest/_torch/attention/multi_gpu/test_mla_helix.py
• tests/unittest/_torch/attention/sparse/dsa/test_dsa_indexer.py
• tests/unittest/_torch/attention/sparse/test_sparse_mha.py
• tests/unittest/_torch/attention/sparse/test_sparse_mqa_gqa.py
• tests/unittest/_torch/attention/test_attention.py
• tests/unittest/_torch/attention/test_flashinfer_attention.py
• tests/unittest/_torch/attention/test_prims_ts_attention_backend.py
• tests/unittest/_torch/attention/test_vanilla_attention.py
• tests/unittest/_torch/modeling/test_kimi_k3_mla.py
• tests/unittest/_torch/modeling_v2/attention/test_modeling_v2_thop_attention.py
• tests/unittest/_torch/modeling_v2/models/deepseek_v3/r1_0528_nvfp4__sm_103__dep4/attention/test_modeling_v2_load_paged_kv_cache_for_mla.py
• tests/unittest/_torch/modeling_v2/models/deepseek_v3/r1_0528_nvfp4__sm_103__dep4/attention/test_modeling_v2_mla_rope_append_paged_kv_assign_q.py
• tests/unittest/_torch/modeling_v2/models/deepseek_v3/r1_0528_nvfp4__sm_103__dep4/attention/test_modeling_v2_mla_rope_generation.py

💤 Files with no reviewable changes: 1

• tests/integration/defs/accuracy/test_llm_api_pytorch.py

Included review availability: This review used your included allowance. Your plan provides up to 12 included reviews per hour; 11 remain after this review.

Walkthrough

Attention tests and performance harnesses migrate to KVCacheManagerV2. The changes update cache configuration defaults, request setup, physical-page initialization, cache-capacity checks, and DSA indexer and attention calls.

Changes

KV Cache Manager V2 Test Migration

Layer / File(s)	Summary
V2 defaults and benchmark harnesses tests/integration/defs/accuracy/test_llm_api_pytorch.py, tests/microbenchmarks/attention_perf/attention_perf_harness.py, tests/microbenchmarks/bench_fp4_mla_decode.py, tests/unittest/_torch/attention/backend_case.py, tests/unittest/_torch/attention/test_attention.py	Default test settings and benchmark paths now use V2 cache managers. The DSA harness uses DSACacheManagerV2, prepares indexer intermediates with pre_indexer_proj, and calls _forward_dsa_attn.
Distributed and sparse cache setup tests/unittest/_torch/attention/multi_gpu/*, tests/unittest/_torch/attention/sparse/*	Helix and sparse attention tests create V2 cache managers and seed request data through allocated pages. Helix and generation tests check cache capacity; the DSA scatter test uses strided views and remapped physical slot indices.
Attention backend fixtures tests/unittest/_torch/attention/test_flashinfer_attention.py, tests/unittest/_torch/attention/test_prims_ts_attention_backend.py, tests/unittest/_torch/attention/test_vanilla_attention.py	Backend tests use V2 cache managers and updated configuration imports. The Qwen context case now selects the V2 manager.
Modeling attention cache fixtures tests/unittest/_torch/modeling/test_kimi_k3_mla.py, tests/unittest/_torch/modeling_v2/attention/test_modeling_v2_thop_attention.py, tests/unittest/_torch/modeling_v2/models/deepseek_v3/.../attention/*	Modeling tests use V2 managers, adapt multi-layer cache views, and resize request cache maps to reserve decode capacity.

Priority: ➖ Normal

Estimated code review effort: 3 (Moderate) | ~25 minutes

Change: Other

Possibly related PRs

• NVIDIA/TensorRT-LLM#20098: It migrates other test fixtures from KVCacheManager to KVCacheManagerV2 and updates their cache configuration usage.

Merge Risk: ⚪ Minimal · up to 2df69

No specific migration failure is established. Confirm the affected GPU test outcomes before treating the rebased head as validated.

Pre-merge checks: 4 · 1

Failed checks: 1 warning

Check name	Status	Explanation	Resolution
Docstring Coverage		Docstring coverage is 35.42% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 48 functions across 18 files.	Write docstrings for the functions missing them to satisfy the coverage threshold.

Passed checks: 4 passed

Check name	Status	Explanation
Title check		The title clearly identifies the migration of attention fixtures and prims-ts validation to KV cache V2. It follows the repository format and summarizes the main change.
Description check		The description includes the required Description, Test Coverage, PR Checklist, and GitHub Bot Help sections. It clearly explains the migration, validation scope, test results, unresolved failures, an…
Linked Issues check		Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check		Check skipped because no linked issues were found for this pull request.

• Fix all pre-merge checks with AI

✨ Finishing Touches: 💡 1

🛠️ Fix failing CI checks: 💡

• Commit to this branch
• Create a new PR

🧪 Generate unit tests (beta)

• Create a new PR

• Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

_{Comment @coderabbitai help to get the list of available commands.}

[yuxianq]

@yihwang-nv, could you help to review this PR as trt-llm-torch-attention-devs? Thanks~

Assignment explanation

Components	integration/attention-tests (100%)
Tags	algorithm:dense (50%); algorithm:sparse (50%); specific-algorithm:dsa (100%); representation:mha (40%); representation:mla (60%); phase:prefill (40%); phase:decode (60%); features:paged-kv (60%); features:context-parallelism (30%); features:speculative-decoding (10%); context:llm (100%)
PR effort	6 points — Review requires checking physical-page versus logical-history equivalence, layer-strided scatter coverage, and HELIX global versus rank-local capacity. These interacting fixture contracts, plus incomplete remapping hunks, exceed a routine manager substitution.
Retained GitHub review requests	litaotju: preserve direct code-owner review request
Affinity (0–1)	0.461 component/tag match
Outstanding points at selection	47 → 53 (+6)
Points already reserved on this PR	0
Effective weight	1
Completed review points (preceding 168 hours)	31
Combined points after assignment	84
Demand-derived target	78.00 points (soft share)
Additional fair share before assignment	0.00 points
Share explanation	Whole PR / qualification or coverage constraint exceeds a soft share; assignment is allowed.
Selection rule	balance-first maintenance

Affinity and workload are snapshots at selection, not live counters. Existing reservations on this PR are counted once.

[github-actions]

Automatically added "ci: full pre-merge approved" because this PR has satisfied the required GitHub review approvals. Unresolved review conversations and other required checks remain independent merge requirements.

[yizhang-nv]

/bot run --disable-fail-fast

[tensorrt-cicd]

PR_Github #77764 [ run ] triggered by Bot. Commit: 2df6982 Link to invocation

[tensorrt-cicd]

PR_Github #77764 [ run ] completed with state SUCCESS. Commit: 2df6982
/LLM/main/L0_MergeRequest_PR pipeline #64105 completed with status: 'SUCCESS'

CI Report

Link to invocation