Repository navigation
Support for the Qwen3 models in engine-flow #3813
Description
Similar PRs:
- [Model] Adding Qwen3 and Qwen3MoE sgl-project/sglang#4693
- Adding Qwen3 and Qwen3MoE huggingface/transformers#36878
We are most interested in the regular QWen (non moe) model. Happy to provide additonal details.
Edit: Now that the model is released, some more additional details:
- it would be great to have the qwen3 model in the
quantize_and_exportworkflow with fp8 precision - support for the modeling code of engine in qwen3 and qwen3_moe
Reacted by Rangehow, zhaochaochao, Jacky Hua, NinjaGah, smh, Kavin, Donghwan Lee, Kyungmin Lee, Nikola Borisov, Efim and 6 more
Activity
michaelfeil commentedon May 9, 2025
I have such implementation working internally. If someone from Nvidia will manage the review, @basetenlabs would be happy to contribute this for the engine flow.
plume commentedon Jun 5, 2025
@michaelfeil I'm really looking forward to this. Let's do our best!