[SimengLiu-nv]

@coderabbitai summary

Description

Enable safe shared-pool KV reuse and optional asynchronous loading in the Mooncake store connector.

• Publish only computed blocks and acknowledge saves once queued; recompute an affected request after a failed prefix load even if the store continues to advertise its keys. Recomputed pages remain eligible for saving.
• Share compatible full-head keys across attention-DP owners and support speculative-prefill lookahead with whole-block local reuse.
• Enable disaggregated completion reservation and reject unsafe aggregated attention-DP producer/both configurations at startup.
• Add optional soft pins, per-block eviction groups, prefix/load diagnostics, and asynchronous loads with allocation fences and bounded host staging.

Asynchronous loading remains off by default. Upgrade guidance requires draining old writers and resetting affected pool contents because unchanged keys cannot identify previously published invalid KV.

Dependency: #21655; the current diff includes its unmerged runtime changes.

Test Coverage

The Mooncake connector, common, settings, and real-pool suites cover publication, recovery, key compatibility, speculative reuse, startup gates, eviction options, staging, and asynchronous load lifecycle. test_llm_api_connector_mooncake.py covers model-level reuse, including attention-DP consumers.

• Passed: scoped pre-commit and static checks; 24 source-isolated ADP role/transceiver guard cases; 18 source-isolated recovery/scheduler cases, including synchronous and asynchronous persistent-hit fallback, recomputed saves, and request isolation. The three new regression cases fail on the previous code.
• Pending: native, real-pool, DMA-lifetime, multi-rank, speculative, and GPU integration tests. The source-isolated guard checks do not validate transfers.

PR Checklist

Please review the following before submitting your PR:

• PR description clearly explains what and why. If using CodeRabbit's summary, please make sure it makes sense.

• PR Follows TRT-LLM CODING GUIDELINES to the best of your knowledge.

• Test cases are provided for new code paths (see test instructions)

• If PR introduces API changes, an appropriate PR label is added - either api-compatible or api-breaking. For api-breaking, include BREAKING in the PR title.

• Any new dependencies have been scanned for license and vulnerabilities

• CODEOWNERS updated if ownership changes

• Documentation updated as needed

• Update tava architecture diagram if there is a significant design change in PR.

• The reviewers assigned automatically/manually are appropriate for the PR.

• Please check this after reviewing the above items as appropriate for this PR.

GitHub Bot Help

To see a list of available CI bot commands, please comment /bot help.

[SimengLiu-nv]

/bot run --disable-fail-fast

[tensorrt-cicd]

PR_Github #82583 [ run ] triggered by Bot. Commit: fa14298 Link to invocation

[tensorrt-cicd]

PR_Github #82583 [ run ] completed with state SUCCESS. Commit: fa14298
/LLM/main/L0_MergeRequest_PR pipeline #68176 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

• Please check the failed tests and fix your PR
• If you cannot view the failures, ask the CI triggerer to share details
• Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation