[New Model]: Qwen 3.5 support from huggingface's transformers v5.2+ #12321

The model to consider.

Release 1.3.0rc8 in the NGC (i.e. NVIDIA's dockers repo) catalog
https://catalog.ngc.nvidia.com/orgs/nvidia/teams/tensorrt-llm/containers/release/tags?version=1.3.0rc8
uses
tensorrt_llm 1.3.0rc8
transformers 4.57.3
(just run pip list after running (docker exec ...) to "docker exec" into the docker container once it's running.
You are not able to deploy Qwen3.5 models with this version of transformers, 4.X.

Qwen3.5 was supported from v5.2 onwards:
https://github.com/huggingface/transformers/releases/tag/v5.2.0

If you update transformers to v5.X + you immediately run into this error when trying to run trtllm-serve:

  File "/usr/local/lib/python3.12/dist-packages/tensorrt_llm/models/gpt/model.py", line 32, in <module>
    from .convert import (load_hf_gpt, load_weights_from_hf_model,
  File "/usr/local/lib/python3.12/dist-packages/tensorrt_llm/models/gpt/convert.py", line 32, in <module>
    from transformers import (AutoModelForCausalLM, AutoModelForVision2Seq,
ImportError: cannot import name 'AutoModelForVision2Seq' from 'transformers' (/ThirdParty/transformers/src/transformers/__init__.py)

This is still within the NGC docker.

AutoModelForVision2Seq was hard deprecated by any v5 of transformers:
https://github.com/huggingface/transformers/blob/779cd2d6923c9d264d46136e90c58f8bafade6ab/MIGRATION_GUIDE_V5.md?plain=1#L506

I see that the code on the main branch is still referring to this deprecated class on this commit snapshot b095c62 : and in these files:

Can we

• Update to using AutoModelForImageTextToText and remove any AutoModelForVision2Seq; we need to do that anyways
• Check if any existing models that need to be critical be supported break from this change (so far I only see a kosmos)
• Then check if a Docker container with the updated tensorRT-llm and updated transformers works when running

• trtllm-serve
• trtllm-eval
• trtllm-bench

• Finally be able to run Qwen3.5 models with trtllm-serve with this Docker container

The closest model TensorRT-LLM already supports.

No response

What's your difficulty of supporting the model you want?

No response

Before submitting a new issue...

• Make sure you already searched for relevant issues, and checked the documentation and examples for answers to frequently asked questions.