Skip to content

[10/10] docs(recipes): add portable RTX PRO5000 draft training examples - #967

Open
jiapingW wants to merge 7 commits into
mainfrom
codex/pr942-11-portable-recipes
Open

jiapingW wants to merge 7 commits into
mainfrom
codex/pr942-11-portable-recipes

Conversation

@jiapingW

@jiapingW jiapingW commented Oct 14, 2026 •

Copy link
Copy Markdown
Collaborator

Motivation

The newly split training capabilities need portable, checked examples with explicit storage semantics.

Modifications

Add six RTX PRO5000 PerfectBlend recipes plus causal DSpark and DSpine draft configs. Use repository/working-directory-relative paths and configurable service examples. Scheduling examples use the method-independent two_window_bucketed and two_window_joint_balanced policies. Omit max_checkpoints in every new recipe; regression checks verify the resolved unlimited default of 0. Include config inventory/wiring checks and ignore local deployment artifacts.

Stack and dependencies

Layer 10/10 split from #942. Previous layer: #966 (codex/pr942-10-consumer-dispatch). Review the diff against this PR's base branch; it contains only this layer. The full stack index is maintained in #942.

Actual prerequisites: #958, #959, #960, #961, #963, #965, #966

The linear stack is the review/merge order. Model-specific optimizations are independently opt-in; the projection-checkpointing and attention layers expose shared capabilities. When merging a lower PR with squash, restack the remaining branches before merging them so already-reviewed commits do not reappear. The repository GPU test workflow only auto-runs for PRs targeting main; upper layers need that CI run when retargeted, while lint runs for all PRs.

Source: #942 at 211b87fc7d707e3a628cb95de5c73340af863c04, split onto main fdcc2eddf1b56b5b24a5a15d3abcb366adfeb5aa. The original branch is preserved.

Accuracy Test

  • Updated stack at e35167c693d6: 135 targeted regression tests passed, covering reference dispatch, capture metadata, configuration, recipes, and package architecture on H20 (Python 3.11.12 / PyTorch 2.13.0+cu130).
  • Repository pre-commit checks passed for the updated stack.

Layer command (run in this branch's checkout, PYTHONPATH=.):

CUDA_VISIBLE_DEVICES=0 OMP_NUM_THREADS=4 python -m unittest \
  tests.test_config.test_example_draft_config_wiring \
  tests.test_config.test_recipe_readme \
  tests.test_runtime.test_package_architecture -v

Qwen3 tokenizer regressions reuse the cached official tokenizer with HF_HUB_OFFLINE=1 and HF_HUB_CACHE pointing to that cache. No training job was launched.

Benchmark & Profiling

No new end-to-end throughput or downstream accuracy benchmark was run for this split. Model-side tests exercise numerical parity and gradients; historical measurements remain in #942 with their original scope and caveats.

Checklist

  • Format changed code.
  • Run the layer's regression tests and final-stack integration tests.
  • Keep feature tests/documentation with their owning layer.
  • Complete GitHub CI and any required downstream accuracy/performance validation before merge.

@jiapingW
jiapingW force-pushed the codex/pr942-11-portable-recipes branch from 97cf2d5 to eaa14f1 Compare October 14, 2026 09:00
@jiapingW
jiapingW force-pushed the codex/pr942-10-consumer-dispatch branch from ff15381 to cb2ca89 Compare October 14, 2026 09:00
@jiapingW jiapingW changed the title [11/11] docs(recipes): add portable RTX PRO5000 draft training examples [10/10] docs(recipes): add portable RTX PRO5000 draft training examples Oct 14, 2026
"top_k": 16,
"ce_weight": 0.1,
"alignment_weight": 0.5,
"alignment_warmup_steps": 146,

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
"alignment_warmup_steps": 146,
"alignment_warmup_steps": null,

Setting this to null lets the code-loss ramp follow the LR warmup, the default introduced in #1004.

@maocheng23
maocheng23 force-pushed the codex/pr942-10-consumer-dispatch branch from aa8d222 to 3aac14a Compare October 21, 2026 21:54
@maocheng23
maocheng23 force-pushed the codex/pr942-11-portable-recipes branch from e35167c to 2c146ee Compare October 21, 2026 21:54
@maocheng23
maocheng23 force-pushed the codex/pr942-11-portable-recipes branch from 2c146ee to e35167c Compare October 21, 2026 22:10
@maocheng23
maocheng23 force-pushed the codex/pr942-10-consumer-dispatch branch from 3aac14a to aa8d222 Compare October 21, 2026 22:10
@jiapingW
jiapingW force-pushed the codex/pr942-10-consumer-dispatch branch from aa8d222 to 7aad50d Compare October 22, 2026 04:47
Base automatically changed from codex/pr942-10-consumer-dispatch to main October 22, 2026 08:03

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants