You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Adds text-only serving performance coverage for Muse Spark Open NVFP4, introduced in #21042. Registers the released checkpoint and adds a TP4/EP4 recipe with TRTLLM attention/MoE, FP8 KV cache manager V2, NCCL all-reduce, and a longer cold-start timeout.
Adds 8192-input/1024-output cases at concurrency 1, 32, and 256 to the existing four-GPU condition in llm_perf_core.yml, with five measured requests per concurrent slot.
The max_seq_len: 49152 cap follows the validated Muse Spark deployment profile; 8192/1024 are the measured request lengths. Keeping that context cap preserves the serving/workspace and CUDA-graph profile used for the initial measurements. Reducing the cap would define a different serving profile and require a new baseline.
Hardware selection intentionally reuses condition 6. The TRTLLM attention and native NVFP4 MoE paths admit the SM100 family, including SM107; execution on every selected platform is not claimed by the GB300 measurements.
Validation:
Pre-commit hooks and DCO sign-off passed.
Targeted checks passed for checkpoint lookup, case parsing, configuration selection, fixed output lengths, and preservation of existing core conditions and cases.
Existing Muse Spark tokenizer tests: 20 passed.
All added serving performance cases passed on four GB300 GPUs; initial measurements were collected using a public PyTorch 26.09 container and a matching official main wheel.
Manual performance execution covers GB300 only. Standard-container performance validation on the other selected platforms remains follow-up coverage. Automated PR pre-merge CI has passed.
Dev Engineer Review
The change registers the Muse Spark Open NVFP4 checkpoint, adds model-specific serving settings, and extends the startup timeout to 3600 seconds. The settings select TRTLLM attention and MoE, NCCL all-reduce, FP8 KV cache manager V2, and a 49,152-token sequence limit.
The QA list adds three four-GPU TP4/EP4 serving cases with 8192-token inputs and 1024-token outputs, at concurrency 1, 32, and 256. The entries target the listed GPU architectures with compute capability 10.0–10.7. No material correctness issue is established by the supplied evidence. Standard CI validation remains pending.
QA Engineer Review
The manual QA list adds three performance cases for Muse Spark Open NVFP4. The cases cover the configured text-only serving path at three concurrency levels. The cases were reported to pass on four GB300 GPUs; standard CI container validation remains pending. No matching entries were found in tests/integration/test_lists/test-db/. Coverage verdict: needs follow-up until standard CI validation completes.
Per-File QA Perspective
tests/integration/defs/perf/_model_paths.py: Adds the model key and checkpoint directory. Verify that the serving test environment can resolve and load this checkpoint.
tests/integration/defs/perf/test_perf.py: Extends the startup timeout to 3600 seconds for this model. Verify the timeout accommodates cold checkpoint loading and executor warmup.
tests/integration/test_lists/qa/llm_perf_core.yml: Adds three manual QA performance entries for concurrency 1, 32, and 256. No corresponding test-db CI entries were found.
The change adds local model-path and PyTorch serving configuration for Muse Spark Open NVFP4, a model-specific server startup timeout, and three streaming-serving performance cases.
Changes
Muse Spark Open NVFP4 performance tests
Layer / File(s)
Summary
Model path and serving configuration tests/integration/defs/perf/_model_paths.py, tests/integration/defs/perf/pytorch_model_config.py
Adds the local model path and a pattern-matched PyTorch serving configuration with execution, sequence-length, CUDA graph, FP8 KV-cache, and MoE backend settings.
Serving test cases and timeout tests/integration/test_lists/qa/llm_perf_core.yml, tests/integration/defs/perf/test_perf.py
Adds three streaming-serving performance cases with different request and concurrency counts. Each uses TP4/EP4, an 8192/1024 workload, FP8 KV cache, and a 120-minute test timeout. The model-specific server startup timeout is 3600 seconds.
Run the new cases on VR200 or restrict their QA selection to validated hardware before relying on their results.
Pre-merge checks: 5
Check name
Status
Explanation
Title check
The title names the main change: adding Muse Spark Open NVFP4 core performance coverage. It follows the repository template with [None][test] and a short summary. The changeset adds perf test configur…
Docstring Coverage
Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff with cyclomatic complexity of at least 2. Documentat…
Linked Issues check
Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check
Check skipped because no linked issues were found for this pull request.
Description check
The description follows the repository template in substance. It has a Description section that explains the change and the reasoning for the 49,152 context cap and the condition 6 hardware reuse. It …
Full details: Title check
Explanation
The title names the main change: adding Muse Spark Open NVFP4 core performance coverage. It follows the repository template with [None][test] and a short summary. The changeset adds perf test configuration, model path mapping, and llm_perf_core.yml cases for this model, so the title matches the main change.
Full details: Docstring Coverage
Explanation
Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff with cyclomatic complexity of at least 2. Documentation-only edits and deletions within surviving functions are included; unchanged diff context and fully deleted functions are excluded. Analyzed 2 functions across 3 files. (1 skipped: 1 unsupported.)
Full details: Description check
Explanation
The description follows the repository template in substance. It has a Description section that explains the change and the reasoning for the 49,152 context cap and the condition 6 hardware reuse. It names the checkpoint registration, the TP4/EP4 recipe, the added 8192/1024 cases at concurrency 1, 32, and 256, and the validation performed. The Test Coverage section is not a separate heading in the author text, but the Validation list covers targeted checks, existing tokenizer tests, and GB300 runs. The PR title is not part of the description, so it is not assessed here. The PR checklist is not addressed, but the template marks it as a self-review step. Remaining gaps are the scope limits: performance runs cover GB300 only, and standard-container validation on other selected platforms is still pending.
Finishing Touches
Generate unit tests (beta)
Create a new PR
Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts
Tip
🎯 Customize your risk assessments
Guide how CodeRabbit assesses merge risk and security architecture risk. Highlight sensitive changes and explain your security boundaries using Merge Risk Instructions and Security Risk Instructions in your review settings. Security architecture risk assessments require the Advanced plan or above.
✅ Automatic approvals, with your rules
Let CodeRabbit approve pull requests once required threads are resolved, pre-merge checks pass, and your risk limits are met. Enable Automatic approvals in review settings, with overrides for teams or file paths.
CodeRabbit Verify
Run CodeRabbit Verify to test the changes in this pull request.
Included review availability: This review used your included allowance. Your plan provides up to 12 included reviews per hour; 11 remain after this review.
The reason will be displayed to describe this comment to others. Learn more.
Approving — the comments below are optional touch-ups, not blockers.
The test IDs, the per-model config and the startup timeout all follow the existing conventions in these files.
One scope point: these cases sit in the compute_capability 10.0–10.7 block, so they will also be scheduled on B200, GB200 and VR200 nodes, but the description says only GB300 was run. An 816B NVFP4 checkpoint plus FP8 KV at kv_frac:0.8 and con:256 leaves less headroom on B200 (less HBM per GPU than GB300). Please either confirm one B200 run before this leaves draft, or note that the other architectures are unvalidated so a first-run failure there isn't mistaken for a regression.
Automatically added "ci: full pre-merge approved" because this PR has satisfied the required GitHub review approvals. Unresolved review conversations and other required checks remain independent merge requirements.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
brnguyen2
left a comment
LarryXFly
left a comment
Adds text-only serving performance coverage for Muse Spark Open NVFP4, introduced in #21042. Registers the released checkpoint and adds a TP4/EP4 recipe with TRTLLM attention/MoE, FP8 KV cache manager V2, NCCL all-reduce, and a longer cold-start timeout.
Adds 8192-input/1024-output cases at concurrency 1, 32, and 256 to the existing four-GPU condition in
llm_perf_core.yml, with five measured requests per concurrent slot.The
max_seq_len: 49152cap follows the validated Muse Spark deployment profile; 8192/1024 are the measured request lengths. Keeping that context cap preserves the serving/workspace and CUDA-graph profile used for the initial measurements. Reducing the cap would define a different serving profile and require a new baseline.Hardware selection intentionally reuses condition 6. The TRTLLM attention and native NVFP4 MoE paths admit the SM100 family, including SM107; execution on every selected platform is not claimed by the GB300 measurements.
Validation:
Manual performance execution covers GB300 only. Standard-container performance validation on the other selected platforms remains follow-up coverage. Automated PR pre-merge CI has passed.
Dev Engineer Review
The change registers the Muse Spark Open NVFP4 checkpoint, adds model-specific serving settings, and extends the startup timeout to 3600 seconds. The settings select TRTLLM attention and MoE, NCCL all-reduce, FP8 KV cache manager V2, and a 49,152-token sequence limit.
The QA list adds three four-GPU TP4/EP4 serving cases with 8192-token inputs and 1024-token outputs, at concurrency 1, 32, and 256. The entries target the listed GPU architectures with compute capability 10.0–10.7. No material correctness issue is established by the supplied evidence. Standard CI validation remains pending.
QA Engineer Review
The manual QA list adds three performance cases for Muse Spark Open NVFP4. The cases cover the configured text-only serving path at three concurrency levels. The cases were reported to pass on four GB300 GPUs; standard CI container validation remains pending. No matching entries were found in
tests/integration/test_lists/test-db/. Coverage verdict: needs follow-up until standard CI validation completes.Per-File QA Perspective
tests/integration/defs/perf/_model_paths.py: Adds the model key and checkpoint directory. Verify that the serving test environment can resolve and load this checkpoint.tests/integration/defs/perf/pytorch_model_config.py: Adds model-label-specific serving settings. Verify text-only operation, KV-cache behavior, sequence-length handling, and TRTLLM backend selection.tests/integration/defs/perf/test_perf.py: Extends the startup timeout to 3600 seconds for this model. Verify the timeout accommodates cold checkpoint loading and executor warmup.tests/integration/test_lists/qa/llm_perf_core.yml: Adds three manual QA performance entries for concurrency 1, 32, and 256. No corresponding test-db CI entries were found.