TensorRT LLM Roadmap (Q3 2026)

This issue tracks planned TensorRT LLM development and releases for Q2 2026.
Plans are subject to change as the team iterates and receives feedback.
If you have suggestions for features, please open a feature request or comment below.

Q3 Roadmap

SW Architecture

• Default to KV Cache Manager V2
• Default to KV Cache Transceiver V2
• Consolidation of TorchSampler/TRTLLMSampler/spec_dec_sample

Performance

• Advanced Speculative Decoding incl. dflash and dynamic draft lengths
• SLM / prefill-only performance improvement
• KV Cache reuse for Hybrid models
• BOLT-optimized builds default for GB200 + GB300
• Kernels for MoE, MLA, FP4 attention
• XGrammar 2.0 integration

User + Dev experience

• Simplified Eagle configuration using one-model for all use cases
• weekly stable release cadence including improved testing
• Align CLI + YAML configs via Hydra
• Simplified MoE backend configs
• Static config validation using API schema

Workflows + Features

• ModelExpress P2P weight transfer for faster scale-out
• GPU Memory Service to persist weights for faster restart
• Agent SKILLs for modeling, kernels, CI, and more
• LLM.encode() API for prefill-only workload
• trtllm-serve embeddings endpoint with encoder support
• Chunked prefill for encoder support
• Helix parallelism x Spec Dec compatibility
• Helix parallelism support for short ISL (ISL<CP*block_size)
• Scoring API for ranking tasks
• Profiling endpoint to capture traces from live traffic
• Auto-research style performance optimization agentic harness
• Modeling Agent to autonomously add support for a new model architecture

Visual Gen

• Skip Softmax support for visual gen models
• Video Sparse Attention kernel
• Qwen-Image support
• Disaggregating LLM and diffusion components
• Cosmos 3 predict support
• FastVAE kernel in NVFP4, FP8
• Parallelism for NVL72 -- Ring Parallelism, Attention2D
• Multi node distribution to scale >8

Let us know how to improve or prioritize these features for your distributed and multi-GPU workloads! Contributions, code feedback, issue submissions, and discussion are welcome.