Repository navigation
[None][feat] Add VR200 support for Mistral Large 4 modeling_v2 - #21604
Conversation
Signed-off-by: yt0428 <51468697+yt0428@users.noreply.github.com>
|
/bot skip |
GitHub Bot Help
Provide a user friendly way for developers to interact with a Jenkins server. Run See details below for each supported subcommand. Details
Launch build/test pipelines. All previously running jobs will be killed.
kill
Kill all running builds associated with pull request. skip
Skip testing for latest commit on pull request. reuse-pipeline
Reuse a previous pipeline to validate current commit. This action will also kill all currently running builds associated with the pull request. IMPORTANT NOTE: This is dangerous since lack of user care and validation can cause top of tree to break. |
|
/bot skip --comment "Format check only" |
|
PR_Github #82421 [ skip ] triggered by Bot. Commit: |
|
PR_Github #82421 [ skip ] completed with state |
…eat/mistral-large-4 Port of feat/mistral-large-4 3030ff5 (NVIDIA#21604) onto this branch. - Three sm_107 targets, tep, dep and tp: each is this branch's sm_103 target with NVIDIA#21604's changes. The FP8 block-scale linears run the Rubin MXFP8 GEMM (cute_dsl_fp8_block_scale_linear), their K128 weight scales converted once to the R128c4 K32 layout, and the indexer scores prefill tiles with a Triton FP8 MQA scorer (triton_fp8_mqa_logits). - Both entries sit under models/mistral_large4/ops/ beside the family's other entries, with their tests; l0_gr100 runs them on sm_107. - The Lamport process group's rendezvous address comes from get_local_ip. - Routing returns the sm_107 targets on compute capability (10, 7) and names sm107 in its refusal. - Accuracy: the gates share a layout base per target layout, and each architecture has its own gate class (Sm107Tep/Dep/Tp beside the sm_103 ones). test_accuracy_without_dspark runs the full and long_full profiles without the draft; artifacts go under the GPU architecture and record per-phase timings. RULER pins its attention-DP warm pairs for every long profile. - The vision, native DSpark and routing tests cover sm_107; the deployment guide lists VR200 and its validated scope. Signed-off-by: Yi Zhang <187001205+yizhang-nv@users.noreply.github.com>
…eat/mistral-large-4 Port of feat/mistral-large-4 3030ff5 (NVIDIA#21604) onto this branch. - Three sm_107 targets, tep, dep and tp: each is this branch's sm_103 target with NVIDIA#21604's changes. The FP8 block-scale linears run the Rubin MXFP8 GEMM (cute_dsl_fp8_block_scale_linear), their K128 weight scales converted once to the R128c4 K32 layout, and the indexer scores prefill tiles with a Triton FP8 MQA scorer (triton_fp8_mqa_logits). - Both entries sit under models/mistral_large4/ops/ beside the family's other entries, with their tests; l0_gr100 runs them on sm_107. - The Lamport process group's rendezvous address comes from get_local_ip. - Routing returns the sm_107 targets on compute capability (10, 7) and names sm107 in its refusal. - Accuracy: the gates share a layout base per target layout, and each architecture has its own gate class (Sm107Tep/Dep/Tp beside the sm_103 ones). test_accuracy_without_dspark runs the full and long_full profiles without the draft; artifacts go under the GPU architecture and record per-phase timings. RULER pins its attention-DP warm pairs for every long profile. - The vision, native DSpark and routing tests cover sm_107; the deployment guide lists VR200 and its validated scope. Signed-off-by: Yi Zhang <187001205+yizhang-nv@users.noreply.github.com>
Description
Mistral Large 4 modeling_v2 currently rejects VR200/SM107, and its existing DeepGEMM projection and prefill-indexer paths hit architecture feature guards on that GPU. Add dedicated SM107 TEP, TP, and DEP targets for four or eight ranks, with native three-token DSpark enabled or disabled.
The three target trees intentionally mirror their SM103 counterparts to follow modeling_v2's architecture-isolation contract. Their inference changes are concentrated in FP8 projection and prefill scoring; the existing checkpoint layout, NVFP4 routed experts, vision path, and native DSpark implementation are retained.
Test Coverage
Checks on the submitted files: all configured pre-commit and commit-message hooks passed, including Ruff lint/format, AST test-list validation, pinned-memory policy, and DCO validation. All 23 submitted files match their pre-hook hashes.
Recorded validation of the same inference/test sources:
modeling_v2="require"and checkpoint revisionbb6d00fe249b0e876f0cffc6b85014be02de3e68ofmistralai/Mistral-Large-4-1T-A52B-NVFP4.The six TEP4/TP4/DEP4 configurations at the 1M envelope, with DSpark ON/OFF, each passed
fullandlong_full(12 workloads). Full-panel scores below are percentages; each panel evaluated GSM8K (1,319), MMMU (900), GPQA Diamond (198), and RULER (1,800), plus functional/cache/coverage checks.All six long panels passed 183 RULER questions: 180 questions across 64K/128K and three fixed true-1M NIAH/VT witnesses. The longest completed prompt was 1,048,330 tokens. The 64K mean was 94.6296%; 128K means ranged from 93.1481% to 93.5185%; all three 1M witnesses were correct in each configuration. Long speculative execution, cache reuse, image-prefill, and required rank-coverage checks passed.
The other 18 configurations passed the complete 1,319-question GSM8K evaluation. All ON runs passed the acceptance-length gate. The six fresh OFF comparison arms within the ON core panels passed both the GSM8K threshold and the 1.5 percentage-point difference limit. Existing accuracy references and tolerances were unchanged.
Validation boundaries: the eight-GPU 1M configurations received GSM8K-only coverage; VR200 llguidance and disaggregated-serving recipes were not validated. The current expanded standalone GPU catalog/vision combinations were not all rerun: the FP8 catalog now defines 112 cells, while the 28 standalone cells listed above were executed. The separate end-to-end matrix covers all 24 requested configurations.
PR Checklist