Repository navigation
[New Model]: Support Qwen3.5 model in torch backend #11674
Description
The model to consider.
Is there any plan to add support for Qwen3.5 with torch backend in the near future?
I've searched the issues list but didn't find any related discussion. Knowing if this is planned would help me decide whether to build my pipeline around this repository.
Thanks for your time!
The closest model TensorRT-LLM already supports.
No response
What's your difficulty of supporting the model you want?
No response
Before submitting a new issue...
- Make sure you already searched for relevant issues, and checked the documentation and examples for answers to frequently asked questions.
Reacted by Kanghwan
Activity
karljang commentedon Feb 24, 2026
Thank you for bringing this to our attention.
For your information,
Qwen/Qwen3.5-397B-A17Bseems to have some support from the AutoDeploy backend, as indicated in supported-models.md in the main branch. However, it’s worth noting that the AutoDeploy backend is still in its early stages.RobertLou commentedon Feb 25, 2026
Thank you for your response!
I've found some docs like TensorRT-LLM AutoDeploy README and AutoDeploy (Beta) , which helped me understand the basic workflow and usage.
However, I'm still curious about the underlying technical principles, such as:
How the automated engine selection and optimization strategies work internally
The methodology behind performance bottleneck analysis and kernel auto-tuning
Any design docs, architecture whitepapers, or technical deep-dives that explain the decision-making process during deployment
Would appreciate any pointers to more in-depth materials. Thanks again!
karljang commentedon Feb 25, 2026
Hi @bmarimuthu-nv ,
Could you assist @RobertLou by providing some additional information?
Thank you!
bmarimuthu-nv commentedon Feb 26, 2026
Hi @RobertLou ,
We just added a cookbook to help try out Qwen3.5 with AutoDeploy here: examples/auto_deploy/cookbooks/qwen_3.5_trtllm_cookbook.ipynb
Autodeploy uses TRTLLM Pytorch backend, so there are no "engines" usage in AutoDeploy.
And here's a good explanation on auto-deploy principles and working: https://developer.nvidia.com/blog/automating-inference-optimizations-with-nvidia-tensorrt-llm-autodeploy/
Just curious, what is your use case/how does your workflow look like? I can help provide more details on how auto-deploy can fit in and help your workflow!
RobertLou commentedon Feb 27, 2026
@bmarimuthu-nv Thanks a lot. We currently use TensorRT-LLM with the torch backend to deploy LLMs on GPUs (primarily H20 and A100, with a small number of H100) for internal company use. Previously, we mostly relied on models like DeepSeek R1 and Qwen 235B, but we are now looking to transition to more advanced, multimodal models such as Qwen3.5 and Kimi K2.5. While the torch backend offers competitive performance compared to vLLM and SGLang, we haven't yet tried or benchmarked its auto deploy capabilities. As a result, we still have some reservations about adopting it in production at this stage.
karljang commentedon Apr 3, 2026
Qwen3.5 PyTorch backend support has landed on main:
So it'll be available soon~, I'm closing this issue~
Thank you!