Question about minimum CUDA version for vector/block scaling modes in cublasLtMatmul #310

Activity

1. JanuszL

   added

   cuBLASLt

   on Apr 16, 2026
2. dali-automaton

   assigned

   hbabak

   on Apr 16, 2026
3. mkukula-nvidia commented on Apr 16, 2026

   

   mkukula-nvidia

   on Apr 16, 2026

   More actions

   Hi @Shmilkky,
   Both CUBLASLT_MATMUL_MATRIX_SCALE_VEC128_32F and CUBLASLT_MATMUL_MATRIX_SCALE_BLK128x128_32F were introduced in cuBLAS 12.9 / CUDA Toolkit 12.9.0 (May 2025) - that's your minimum required version.
   GPU architecture requirement: Hopper (sm_90) only, i.e., H100 or H200.
   One thing to note: cuBLAS 12.9.1 brought a targeted performance improvement for the 128×128 2D block scaling path on Hopper specifically, so 12.9.1 or later is recommended if performance matters to you.
4. Shmilkky commented on Apr 16, 2026

   

   Shmilkky

   on Apr 16, 2026

   Author

   More actions

   Hi @Shmilkky, Both CUBLASLT_MATMUL_MATRIX_SCALE_VEC128_32F and CUBLASLT_MATMUL_MATRIX_SCALE_BLK128x128_32F were introduced in cuBLAS 12.9 / CUDA Toolkit 12.9.0 (May 2025) - that's your minimum required version. GPU architecture requirement: Hopper (sm_90) only, i.e., H100 or H200. One thing to note: cuBLAS 12.9.1 brought a targeted performance improvement for the 128×128 2D block scaling path on Hopper specifically, so 12.9.1 or later is recommended if performance matters to you.

   Thanks for the clarification.

   So if I am using B200, does that mean I cannot use CUBLASLT_MATMUL_MATRIX_SCALE_VEC128_32F or CUBLASLT_MATMUL_MATRIX_SCALE_BLK128x128_32F?

   If so, does B200 support CUBLASLT_MATMUL_MATRIX_SCALE_VEC16_UE4M3 and CUBLASLT_MATMUL_MATRIX_SCALE_VEC32_UE8M0 instead?
5. mkukula-nvidia commented on Apr 16, 2026

   

   mkukula-nvidia

   on Apr 16, 2026

   More actions

   Yes, CUBLASLT_MATMUL_MATRIX_SCALE_VEC16_UE4M3 and CUBLASLT_MATMUL_MATRIX_SCALE_VEC32_UE8M0 are the correct and recommended scaling modes for Blackwell (sm_100).

   These correspond to the native 1D block-scaled data types that Blackwell's Tensor Cores accelerate directly in hardware:

   • VEC16_UE4M3: 16-element blocks with UE4M3 scale type, used with CUDA_R_4F_E2M1 (FP4) inputs
   • VEC32_UE8M0: 32-element blocks with UE8M0 scale type, used with MX-FP8 (CUDA_R_8F_E4M3/CUDA_R_8F_E5M2) inputs

   Both were introduced in CUDA 12.8.0. Unlike the Hopper modes (which are software-implemented using FP32 scale factors), these are natively supported by Blackwell hardware and will be significantly faster on Blackwell-series GPUs.

   See the LtNvfp4Matmul and LtMxfp8Matmul samples for reference usage.
6. Shmilkky commented on Apr 16, 2026

   

   Shmilkky

   on Apr 16, 2026

   Author

   More actions

   Yes, CUBLASLT_MATMUL_MATRIX_SCALE_VEC16_UE4M3 and CUBLASLT_MATMUL_MATRIX_SCALE_VEC32_UE8M0 are the correct and recommended scaling modes for Blackwell (sm_100).

   These correspond to the native 1D block-scaled data types that Blackwell's Tensor Cores accelerate directly in hardware:

   • VEC16_UE4M3: 16-element blocks with UE4M3 scale type, used with CUDA_R_4F_E2M1 (FP4) inputs
   • VEC32_UE8M0: 32-element blocks with UE8M0 scale type, used with MX-FP8 (CUDA_R_8F_E4M3/CUDA_R_8F_E5M2) inputs

   Both were introduced in CUDA 12.8.0. Unlike the Hopper modes (which are software-implemented using FP32 scale factors), these are natively supported by Blackwell hardware and will be significantly faster on Blackwell-series GPUs.

   See the LtNvfp4Matmul and LtMxfp8Matmul samples for reference usage.

   Thank you very much for the detailed clarification. This is very helpful.

   👍React with 👍1Reacted by mkukula-nvidia