[Wanli-Jiang]

@coderabbitai summary

Description

Split out of #21935; it applies to main on its own.

Under block_reuse_config.policy=per_conversation, LRU keeps a finished
sub-agent conversation as long as a main agent that comes back after
minutes of tool time, and at agentic concurrency the sub-agents' blocks push
the main agents out.

New block_reuse_config.finished_subagent_priority (default None: off) and
finished_subagent_min_idle_iterations (default 32). A request with a parent
conversation id (LlmRequest.py_subagent_parent_id) registers its conversation
as a child of the parent. The parent id comes only from an authenticated
sub-agent affinity hint, which the disaggregated edge and the OpenEngine gRPC
front end accept on context requests; without one nothing is demoted. When
the parent sends its next request, every child with no request in flight
whose last context ended at least the idle bound of scheduler steps earlier
gets finished_subagent_priority on the committed blocks it computed itself (C++ CommittedRetentionHandle, from
KvCache.plan_committed_retention()): never the prefix it reused, and never
a block another path extends. Below the cold-tier offload threshold (30)
those blocks are dropped instead of offloaded. A demoted child that comes
back gets the default priority again.

Every input of a demotion is rank-identical so that TP ranks evict the same
pages: the idle test counts scheduler steps rather than wall time, and a
parent keeps its children in arrival order (an earlier version decided per
rank and hung a TP2 context server).

Moving a demoted prefix below the offload threshold no longer erases from
the middle of the fallen-page deque once per page: _prepareFreeSlots makes
one stable pass per life cycle.

Evidence (Qwen3.8 FP8 AgentX trace, GB300):

• GPU replay of the trace arrivals at 1/4 scale, same node, both arm orders:
  block hit ratio 0.9327 -> 0.9409 (+0.82 pt), uncached tokens -12.1 %; on
  top of the cold-tier split of [None][perf] KV cache manager v2 for agentic traffic: host-path cost, in-forward snapshot restore and cold-tier sizing #21935, 0.9444-0.9447 -> 0.9467 (+0.23 pt,
  uncached -3.8..-4.2 %), with finished_subagent_priority=1.
• Formal 3600 s c208 with the cold-tier split: interactivity 117.3 -> 119.5
  (+1.9 %) at equal throughput (236.0K vs 237.0K tok/s), hit ratio 92.05 ->
  93.97 %; confounded by other code changes between the two runs.

Tests: TestRetention.test_committed_retention_handle_{demotes_only_owned_blocks,
stops_at_a_fork,steers_eviction},
test_per_conversation_policy_demotes_finished_subagent_when_parent_returns,
test_finished_subagent_demotion_is_rank_deterministic,
test_finished_subagent_priority_requires_per_conversation.

The new fields kv_cache_config.block_reuse_config.finished_subagent_priority and finished_subagent_min_idle_iterations are prototype and additive (label api-compatible); they are telemetry-captured, so this needs the telemetry / privacy code owners' review.

Test Coverage

• C++ CommittedRetentionHandle tests (demotes only owned blocks, stops at a fork, steers eviction); test_per_conversation_policy_demotes_finished_subagent_when_parent_returns, test_finished_subagent_demotion_is_rank_deterministic, test_finished_subagent_priority_requires_per_conversation, and the llm_args golden manifest test.
• At this head on GB300 (4 GPUs), on main 2722406 with the native code rebuilt:
  • tests/unittest/kv_cache_manager_v2_tests: 608 passed, 12 skipped.
  • tests/unittest/_torch/pyexecutor/kv_cache: 3031 passed; the 3 failures (test_kv_cache_v2_vocab_size.py::test_branch_only_enables_multimodal_keys_when_needed[qwen3_5_hybrid-*]) fail the same way on main.
  • tests/unittest/llmapi/test_llm_args.py and the serve schema tests: 919 passed; the 2 failures (test_runtime_sizes, test_model_kwargs_with_num_hidden_layers) need a model checkpoint the test machine does not have (HFValidationError) and fail the same way on main. api_stability: 99 passed.
  • scripts/generate_llm_args_golden_manifest.py and scripts/generate_trtllm_serve_schemas.py: no change.

PR Checklist

Please review the following before submitting your PR:

• PR description clearly explains what and why. If using CodeRabbit's summary, please make sure it makes sense.

• PR Follows TRT-LLM CODING GUIDELINES to the best of your knowledge.

• Test cases are provided for new code paths (see test instructions)

• If PR introduces API changes, an appropriate PR label is added - either api-compatible or api-breaking. For api-breaking, include BREAKING in the PR title.

• Any new dependencies have been scanned for license and vulnerabilities

• CODEOWNERS updated if ownership changes

• Documentation updated as needed

• Update tava architecture diagram if there is a significant design change in PR.

• The reviewers assigned automatically/manually are appropriate for the PR.

• Please check this after reviewing the above items as appropriate for this PR.