[Bug]: Qwen4-Exp PLE table allocated in BF16 for NVFP4 checkpoints that declare ple_embedding_dtype float8_e4m3fn #21902

System Info

• TensorRT-LLM 1.4.0rc2 (nvcr.io/nvidia/tensorrt-llm/release:1.4.0rc2); _uses_scaled_fp8_ngram_table is unchanged on main.
• 2× NVIDIA RTX PRO 6000 Blackwell Server Edition, TP 2.

Who can help?

@Wanli-Jiang

Information

• The official example scripts
• My own modified scripts

Tasks

• An officially supported task in the examples folder (such as GLUE/SQuAD, ...)
• My own task or dataset (give details below)

Reproduction

Serve a ModelOpt NVFP4 checkpoint of Qwen3.8-Flash-Next whose PLE table is FP8 but excluded from quantization, for example dealignai/Qwen3.8-Flash-Next-ABLITERATED-NVFP4 or primitive-ai/Qwen3.8-Flash-Next-NVFP4, with TP 2 and TRTLLM_QWEN4_EXP_PLE_HOST_OFFLOAD=1.

These checkpoints list *.ple.* in quantization_config.ignore and declare the table with ple_embedding_dtype: float8_e4m3fn in text_config. _uses_scaled_fp8_ngram_table (tensorrt_llm/_torch/modules/qwen4_exp/ple.py) only reads quantization_config, so the table is preallocated in BF16.

Expected behavior

The PLE table is allocated as float8_e4m3fn, and the model loads.

actual behavior

• With PLE host offload, loading fails with PLE host-offload checkpoint dtype does not match its preallocated table: torch.float8_e4m3fn != torch.bfloat16.
• Without it, the BF16 table stays on the GPU and the model runs out of memory while loading.

additional notes

#18823 added the FP8 detection from quantization_config. #21837 also reads ple_embedding_dtype.

Before submitting a new issue...

• Make sure you already searched for relevant issues, and checked the documentation and examples for answers to frequently asked questions.