Repository navigation

[TRTLLM-10061][feat] Prefix caching support for mamba hybrid models - #12185

Merged

VALLIS-NERIA merged 122 commits into

NVIDIA:mainfrom

VALLIS-NERIA:user/xiweny/linear_reuse_new

Apr 26, 2026

Merged

[TRTLLM-10061][feat] Prefix caching support for mamba hybrid models#12185
VALLIS-NERIA merged 122 commits into
NVIDIA:mainfrom
VALLIS-NERIA:user/xiweny/linear_reuse_new

Commits on Mar 5, 2026
- [None][feat] Wire KVCacheBlock to UnifiedBlockTree, replacing mPrevBlock/mNextBlocks with lookup-node pointers.
  SimengLiu-nv
  committed
  bdb3791
Commits on Mar 6, 2026
- Address comments.
  SimengLiu-nv
  committed
  bd4810a
- block allocation and reusing works for linear attention
  VALLIS-NERIA
  committed
  27574b9
- copy states during context shifts
  VALLIS-NERIA
  committed
  3543bbe
- fix corner cases
  VALLIS-NERIA
  committed
  36aa474
- temp stage: accuracy w/o reuse ok
  VALLIS-NERIA
  committed
  cd1a67b
- temp stage: accuracy with reuse ok
  VALLIS-NERIA
  committed
  94d4312
Commits on Mar 9, 2026
- fix merge conflicts
  VALLIS-NERIA
  committed
  d885842
- Merge remote-tracking branch 'origin/main' into pr-11919
  VALLIS-NERIA
  committed
  603d822
Commits on Mar 13, 2026
- temporary stage
  VALLIS-NERIA
  committed
  b398561
Commits on Mar 14, 2026
- fix multiple issues
  VALLIS-NERIA
  committed
  df7284a
- Merge remote-tracking branch 'origin/main' into user/xiweny/linear_reuse_new
  VALLIS-NERIA
  committed
  ce9674a
- use pre calculated buffers
  VALLIS-NERIA
  committed
  cab2412
Commits on Mar 17, 2026
- Merge remote-tracking branch 'origin/main' into user/xiweny/linear_reuse_new
  VALLIS-NERIA
  committed
  a1889b8
Commits on Mar 18, 2026
- scheduler support
  VALLIS-NERIA
  committed
  22e7fd2
- FIFO placeholder management
  VALLIS-NERIA
  committed
  6475692
- remove debug prints in module/op level
  VALLIS-NERIA
  committed
  3312fa9
- change memory layout to layer first
  VALLIS-NERIA
  committed
  9b73cbf
- fix scheduler
  VALLIS-NERIA
  committed
  efbb815
Commits on Mar 19, 2026
- auto choose mamba cache manager impl
  VALLIS-NERIA
  committed
  aa15395
- format code
  VALLIS-NERIA
  committed
  5bfda48
Commits on Mar 20, 2026
- fix unhandled kFORCE_CHUNK enum in switch statement
  VALLIS-NERIA
  committed
  f9e2ad0
- fix config of current implementation
  VALLIS-NERIA
  committed
  1810dba
- merge upstream main and resolve conflicts
  VALLIS-NERIA
  committed
  cf50425
- fix missing is_nemotron_hybrid/is_qwen3_hybrid imports
  VALLIS-NERIA
  committed
  4dd57bf
- remove some hacks
  VALLIS-NERIA
  committed
  b4e54e7
- [Agent fix] restore block reuse defaults and fix AutoDeploy mamba_layer_mask
  VALLIS-NERIA
  committed
  ee0b690
Commits on Mar 21, 2026
- Merge remote-tracking branch 'origin/main' into user/xiweny/linear_reuse_new
  VALLIS-NERIA
  committed
  782c46f
Commits on Mar 22, 2026
Commits on Mar 23, 2026
Commits on Mar 24, 2026
- refine mamba cache manager
  VALLIS-NERIA
  committed
  7dacd9e
- Merge remote-tracking branch 'origin/main' into user/xiweny/linear_reuse_new
  VALLIS-NERIA
  committed
  ab3bc32
- clean up unnecessary chagnes
  VALLIS-NERIA
  committed
  b31dd85
- fix
  VALLIS-NERIA
  committed
  81eb415
- add tests for scheduler
  VALLIS-NERIA
  committed
  12d8dda
- [TRTLLM-10061][feat] Add FORCE_CHUNK context chunking policy
  VALLIS-NERIA
  committed
  5d3a46e
- Merge remote-tracking branch 'origin' into user/xiweny/force_chunk_policy
  VALLIS-NERIA
  committed
  730d6ea
- add tests for scheduler
  VALLIS-NERIA
  committed
  6ed49f3
- improve comments
  VALLIS-NERIA
  committed
  807e9d3
- fix
  VALLIS-NERIA
  committed
  45f0fa5
- Merge branch 'main' into user/xiweny/force_chunk_policy
  VALLIS-NERIA
  authored
  5f54054
- fix kvcache manager ut
  VALLIS-NERIA
  committed
  a8dea92
- Merge remote-tracking branch 'origin/main' into user/xiweny/linear_reuse_new
  VALLIS-NERIA
  committed
  4717171
- Merge branch 'main' into user/xiweny/force_chunk_policy
  VALLIS-NERIA
  authored
  43469ec
Commits on Mar 25, 2026
Commits on Apr 3, 2026
- Merge remote-tracking branch 'origin/main' into user/xiweny/linear_reuse_new
  VALLIS-NERIA
  committed
  bf5599d
- remove warning at exit
  VALLIS-NERIA
  committed
  c1129e4
- rename
  VALLIS-NERIA
  committed
  82b7049
- [TRTLLM-10061][fix] Address review items for linear attention hybrid cache manager (By Agent)
  VALLIS-NERIA
  committed
  42cd76b
- fix style
  VALLIS-NERIA
  committed
  d616e76
- Merge remote-tracking branch 'origin/main' into user/xiweny/linear_reuse_new
  VALLIS-NERIA
  committed
  d8dbaf0
- [Agent fix] Add missing triton import in modeling_qwen3_next.py
  VALLIS-NERIA
  committed
  162a5eb
Commits on Apr 4, 2026
Commits on Apr 5, 2026
Commits on Apr 6, 2026
Commits on Apr 7, 2026
- fix attention DP sharding
  VALLIS-NERIA
  committed
  079f8bf
- Merge branch 'main' into user/xiweny/linear_reuse_new
  xinhe-nv
  authored
  0115a80
- Merge remote-tracking branch 'fork/user/xiweny/linear_reuse_new' into user/xiweny/linear_reuse_new
  VALLIS-NERIA
  committed
  67a3c57
- address commentes
  VALLIS-NERIA
  committed
  ad051e9
- fix the placeholder issue
  VALLIS-NERIA
  committed
  9a34a49
Commits on Apr 8, 2026
- Update l0_gb200_multi_gpus.yml
  VALLIS-NERIA
  authored
  daa6320
- address comments
  VALLIS-NERIA
  committed
  866dfcc
Commits on Apr 9, 2026
- address comments
  VALLIS-NERIA
  committed
  3915d7d
Commits on Apr 12, 2026
- fix
  VALLIS-NERIA
  committed
  9c2cc8b
Commits on Apr 13, 2026
- Merge branch 'main' into user/xiweny/linear_reuse_new
  VALLIS-NERIA
  committed
  ae33b90
- refine fix
  VALLIS-NERIA
  committed
  95e6539
- [None][fix] Disable spec decoding on Blackwell for MLA models
  sunnyqgg
  committed
  82758e5
- Merge pull request #7 from sunnyqgg/bug_6071081
  VALLIS-NERIA
  authored
  88d1a0a
Commits on Apr 15, 2026
- Merge branch 'main' into user/xiweny/linear_reuse_new
  VALLIS-NERIA
  authored
  35808b8
- Merge branch 'main' into user/xiweny/linear_reuse_new
  VALLIS-NERIA
  authored
  712ca7f
Commits on Apr 16, 2026
- Merge branch 'main' into user/xiweny/linear_reuse_new
  VALLIS-NERIA
  committed
  8f72945
- fix qwen context mamba states
  VALLIS-NERIA
  committed
  dd95fde
- Merge branch 'main' into user/xiweny/linear_reuse_new
  VALLIS-NERIA
  authored
  8ef6d25
Commits on Apr 17, 2026
- Merge branch 'main' into user/xiweny/linear_reuse_new
  VALLIS-NERIA
  authored
  b2ae280
- update docstring
  VALLIS-NERIA
  committed
  e317bf6
- Merge branch 'main' into user/xiweny/linear_reuse_new
  VALLIS-NERIA
  authored
  82e5486
- Revert "[None][fix] Disable spec decoding on Blackwell for MLA models"
  VALLIS-NERIA
  committed
  38eb890
- choose python mixed manager correctly
  VALLIS-NERIA
  committed
  819a320
Commits on Apr 19, 2026
- fix param
  VALLIS-NERIA
  committed
  82506db
Commits on Apr 20, 2026
- fix
  VALLIS-NERIA
  committed
  71061af
- update flaky test
  VALLIS-NERIA
  committed
  4644973
Commits on Apr 21, 2026
- Merge branch 'main' into user/xiweny/linear_reuse_new
  VALLIS-NERIA
  committed
  d8b7cb3
Commits on Apr 22, 2026
- Merge branch 'main' into user/xiweny/linear_reuse_new
  VALLIS-NERIA
  authored
  a234b66
Commits on Apr 23, 2026
- Merge branch 'main' into user/xiweny/linear_reuse_new
  VALLIS-NERIA
  authored
  0d14d65
- Update waives.txt
  VALLIS-NERIA
  authored
  0954318
- Update interface.py
  VALLIS-NERIA
  authored
  4fe263e
Commits on Apr 24, 2026
- Merge branch 'main' into user/xiweny/linear_reuse_new
  VALLIS-NERIA
  committed
  8a5953a
- fix pp sharding issue
  VALLIS-NERIA
  committed
  5b01cde
- Merge branch 'main' into user/xiweny/linear_reuse_new
  VALLIS-NERIA
  authored
  b83f354
- fix waive
  VALLIS-NERIA
  committed
  034dd8a
- Merge branch 'main' into user/xiweny/linear_reuse_new
  VALLIS-NERIA
  authored
  51f3a6d
Commits on Apr 25, 2026
- Merge branch 'main' into user/xiweny/linear_reuse_new
  VALLIS-NERIA
  authored
  440ab26
Commits on Apr 26, 2026
- Update waives.txt
  VALLIS-NERIA
  authored
  8a8f53c

Commits

Commits on Mar 5, 2026

Commits on Mar 6, 2026

Commits on Mar 9, 2026

Commits on Mar 13, 2026

Commits on Mar 14, 2026

Commits on Mar 17, 2026

Commits on Mar 18, 2026

Commits on Mar 19, 2026

Commits on Mar 20, 2026

Commits on Mar 21, 2026

Commits on Mar 22, 2026

Commits on Mar 23, 2026

Commits on Mar 24, 2026

Commits on Mar 25, 2026

Commits on Apr 3, 2026

Commits on Apr 4, 2026

Commits on Apr 5, 2026

Commits on Apr 6, 2026

Commits on Apr 7, 2026

Commits on Apr 8, 2026

Commits on Apr 9, 2026

Commits on Apr 12, 2026

Commits on Apr 13, 2026

Commits on Apr 15, 2026

Commits on Apr 16, 2026

Commits on Apr 17, 2026

Commits on Apr 19, 2026

Commits on Apr 20, 2026

Commits on Apr 21, 2026

Commits on Apr 22, 2026

Commits on Apr 23, 2026