Repository navigation
[Bug] Bug when deploy Qwen3-VL-32B-Instruct #4098
Description
Checklist
- 1. I have searched related issues but cannot get the expected help.
- 2. The bug has not been fixed in the latest version.
- 3. Please note that if the bug-related issue you submitted lacks corresponding environment info and a minimal reproducible demo, it will be challenging for us to reproduce and resolve the issue, reducing the likelihood of receiving feedback.
Describe the bug
There is an error when deploy Qwen3-VL-32B-Instruct
lmdeploy - ERROR - base.py:55 - AttributeError: 'Qwen3VLConfig' object has no attribute 'hidden_size'
2025-11-03 21:40:56,729 - lmdeploy - ERROR - base.py:56 - check failed!
("Checking failed with error 'Qwen3VLConfig' object has no attribute 'hidden_size'", 'Please send issue to LMDeploy with error logs.')
Reproduction
model_path = "Qwen/Qwen2.5-VL-32B-Instruct" ; the development is same as the Official Documentation
Environment
Environment development is same as the Official DocumentationError traceback
Metadata
Metadata
Assignees
Labels
No labels
Activity
CUHKSZzxy commentedon Nov 4, 2025
What's your LMDeploy version? The PR for qwen3 vl support has not been merged yet. You may try the following PR and see whether it works or not
yangyuxaing commentedon Nov 4, 2025
Thank you for responding to me.
conda create -n lmdeploy python=3.10 -y
conda activate lmdeploy
pip install lmdeploy
export LMDEPLOY_VERSION=0.10.2
export PYTHON_VERSION=310
pip install https://github.com/InternLM/lmdeploy/releases/download/v${LMDEPLOY_VERSION}/lmdeploy-${LMDEPLOY_VERSION}+cu128-cp${PYTHON_VERSION}-cp${PYTHON_VERSION}-manylinux2014_x86_64.whl --extra-index-url https://download.pytorch.org/whl/cu128
PyTorch built with CUDA: 12.8
cuDNN version: 91002
GPU: NVIDIA GeForce RTX 5090
CUHKSZzxy commentedon Nov 4, 2025
That makes sense, lmdeploy v0.10.2 does not support qwen3-vl yet, while qwen2.5 vl has been supported.
You can try the PR I mentioned above, which contains the code for qwen3-vl.
yangyuxaing commentedon Nov 4, 2025
Thank you for your help, but it still doesn't work. There is a CUDA kernel compilation error. I suspect it might be related to the use of RTX 5090 GPUs. The instructions I use are:
pip uninstall lmdeploy -y
git clone https://github.com/InternLM/lmdeploy.git
cd lmdeploy
git fetch origin pull/4093/head:qwen3vl
git checkout qwen3vl
pip install -r requirements.txt
pip install -r requirements/build.txt
pip install -e .
CUHKSZzxy commentedon Nov 4, 2025
Are there any error logs? For qwen3 vl series, we currently support it in the pytorch backend, so we can use it without any compilations.
You can use the following commands, where
DISABLE_TURBOMIND=1 pip install -e .should be fine (will disable turbomind backend-related compilations). And also, no need to pip install requirements separately.yangyuxaing commentedon Nov 4, 2025
Thank you so much! I have solved this issue by following your instructions. I really appreciate your help.
CUHKSZzxy commentedon Nov 4, 2025
You're welcome, feel free to post issues if you encounter any other problems. qwen3-vl PR will be merged into main branch in the next release, and later you may simply use `pip install lmdepoy'.