[peihu-nv]

@coderabbitai summary

Description

Generic retention foundation only; serving integration remains. K3’s integrated implementation is in internal Rubin-advance MR !10793; the required merge-back and main port are still pending.

Private turn checkpoints can accumulate across cache owners, while a local per-worker limit cannot safely reclaim checkpoints used by another owner. This adds a model-independent retention catalog, optional job-scoped HTTP coordinator, and a transfer-lifecycle adapter. The adapter pins before lookup or transfer, prepares an immutable private-key manifest before saving, and publishes ownership only after the producer confirms successful completion and retirement. Collection preserves referenced or pinned checkpoints and never includes shared attention pages.

The policy retains the last N distinct turns in first successful publication order, not logical conversation order. Uncertain transfers leave durable pins; ambiguous deletion quarantines the endpoint. The coordinator is intended for a trusted job network and must restart with a fresh epoch and Store namespace.

Draft scope: core and lifecycle adapter only. This is a companion foundation for #19171, but has no code dependency on its unmerged changes, so the diff targets main directly. The attention-only connector does not yet produce private checkpoints. K3/SWA producers, serving integration, native buffer retirement integration, and GPU/RDMA qualification remain follow-up work. No serving flag is added and no throughput or memory-saving result is claimed.

Expired-turn identities now survive ledger eviction, so late duplicate publications cannot displace newer turns. Catalog schema 2 requires a fresh job namespace; old catalogs are rejected because their expiry history cannot be recovered. Small expiry records persist for the job lifetime. Transfer pins and shared checkpoint references retain their existing behavior.

Test Coverage

• 58 focused CPU cases pass on macOS/Python 3.12 after the replay fix. The preceding 48-case version also passed on Linux/Python 3.12. Tests exercise real multiprocess filesystem locks and localhost HTTP coordination with a file-backed test Store.
• Added regressions cover expired replay across owners and RPC, restart, crash recovery, active transfer pins, and endpoints still owned by another conversation or newer turn.
• Coverage includes cross-owner limits, shared checkpoint references, in-flight loads versus collection, partial deletion, abandoned clients, cancellation, uncertain completion, failed publication, immutable manifests, and late publication ordering.
• Tested files: test_mooncake_store_retention.py, test_mooncake_store_retention_rpc.py, and test_mooncake_checkpoint_lifecycle.py, under tests/unittest/_torch/executor/. Their cpu_only markers place them under the existing CPU-Generic executor registration.
• Local commands use pytest -c /dev/null --noconftest -p no:cacheprovider with a test-only package bootstrap that bypasses TRT-LLM/CUDA initialization while importing the real changed modules. Standard package initialization and the full repository harness await CI.
• All applicable changed-file pre-commit checks pass, including vendor consistency and test-list validation. No real Store, GPU, RDMA, or end-to-end inference qualification is claimed.

PR Checklist

Please review the following before submitting your PR:

• PR description clearly explains what and why. If using CodeRabbit's summary, please make sure it makes sense.

• PR Follows TRT-LLM CODING GUIDELINES to the best of your knowledge.

• Test cases are provided for new code paths (see test instructions)

• If PR introduces API changes, an appropriate PR label is added - either api-compatible or api-breaking. For api-breaking, include BREAKING in the PR title.

• Any new dependencies have been scanned for license and vulnerabilities

• CODEOWNERS updated if ownership changes

• Documentation updated as needed

• Update tava architecture diagram if there is a significant design change in PR.

• The reviewers assigned automatically/manually are appropriate for the PR.

• Please check this after reviewing the above items as appropriate for this PR.

GitHub Bot Help

To see a list of available CI bot commands, please comment /bot help.

[peihu-nv]

/bot run

[tensorrt-cicd]

PR_Github #76785 [ run ] triggered by Bot. Commit: 494342c Link to invocation

[tensorrt-cicd]

PR_Github #76785 [ run ] completed with state FAILURE. Commit: 494342c
/LLM/main/L0_MergeRequest_PR pipeline #63313 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

• Please check the failed tests and fix your PR
• If you cannot view the failures, ask the CI triggerer to share details
• Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

[peihu-nv]

/bot run --disable-fail-fast

[tensorrt-cicd]

PR_Github #76808 [ run ] triggered by Bot. Commit: 41572ff Link to invocation

[tensorrt-cicd]

PR_Github #76808 [ run ] completed with state ABORTED. Commit: 41572ff
/LLM/main/L0_MergeRequest_PR pipeline #63331 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

• Please check the failed tests and fix your PR
• If you cannot view the failures, ask the CI triggerer to share details
• Once fixed, request an NVIDIA team member to trigger CI again

Link to invocation