Skip to content

BlockRadixTree.clear() does not terminate when the tree holds a childless RootBlock #17908

Open

Description

@StephenLReed

System Info

  • TensorRT-LLM: 1.3.0rc23 (built from f20ea652); behaviour re-checked against main @ fbda11f5
  • GPU: NVIDIA GeForce RTX 5090 (sm_120), driver 595.71.05
  • PyTorch 2.11.0+cu130, CUDA 13.0, Python 3.12.13
  • Ubuntu 25.10, kernel 6.17.0-41-generic
  • Backend: PyTorch; single GPU, no TP/PP
  • Model where observed: Qwen3.6-35B-A3B-NVFP4 with FP8 KV cache (where model-specific)

Who can help?

No response

Reproduction

BlockRadixTree.clear() in tensorrt_llm/runtime/kv_cache_manager_v2/_block_radix_tree.py shrinks
self.next only as a side effect of its inner loop: detach_next() prunes a RootBlock when its
last child is detached. A RootBlock that is already childless has no last child to detach, so the
outer loop re-selects it indefinitely and clear() never returns. The trailing
assert not self.next is unreachable in that state.

A childless root is not exotic. RootBlock.__init__ ends with prev.next[self.key] = self, so
add_or_get_existing() publishes an empty root by construction, before its caller attaches a
block. Any interruption between those two steps leaves the tree permanently unclearable.

tree = BlockRadixTree(object(), 32)                        # life_cycles unused by clear()
tree.add_or_get_existing(ReuseScope(lora_id=0, salt=None))
tree.clear()                                               # never returns

clear() touches neither the life-cycle registry nor pages, so this needs no device and no engine.
(salt must be None: ReuseScope.to_bytes() asserts non-int fields are unset.)

Expected behavior

clear() removes every root and returns, leaving the tree empty.

actual behavior

clear() never returns when a childless RootBlock is present: it pins one core at 100% with no GPU activity, and KVCacheManager.shutdown() hangs behind it.

additional notes

Make each iteration of clear() take an explicit progress step — detach a childless root, or remove
one subtree — so termination does not depend on a callee's side effect, and centralise root-edge
bookkeeping so eager pruning and explicit teardown cannot diverge.

We have a patch with eight CPU-only regression tests (6 fail before, all pass after) and are opening
this Issue Request first per CONTRIBUTING.md.

Activity

  1. Samyra312007 commented on Sep 2, 2026

    @Samyra312007

    Respected Maintainer and Author, I would like to understand about this issue end to end and work on it. Is it open for contributors?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    KV-Cache Managementkv-cache management for efficient LLM inference

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions