[jiapingW]

Superseded by #963. Domino and DSpark now share training.cache_vocab_projection, training.flatten_vocab_projection, and the vocabulary projection helper. #965 has been retargeted to #963. This branch is retained for reference.

---

Motivation

DSpark vocabulary projection can use 2-D GEMMs and retain frozen results across objective recomputation.

Modifications

Add separate opt-in flatten/cache flags, DSpark-only validation, builder wiring, and explicit Liger kernel injection. Compare losses, metrics, all gradients, checkpoint keys, and resume contracts. Flattening can change floating-point rounding; caching trades memory for recomputation.

Stack and dependencies

Layer 8/11 split from #942. Previous layer: #963 (codex/pr942-07-domino-optimizations). Review the diff against this PR's base branch; it contains only this layer. The full stack index is maintained in #942.

Actual prerequisites: #960, #962

The linear stack is the review/merge order. Model-specific optimizations are independently opt-in; the projection-checkpointing and attention layers expose shared capabilities. When merging a lower PR with squash, restack the remaining branches before merging them so already-reviewed commits do not reappear. The repository GPU test workflow only auto-runs for PRs targeting main; upper layers need that CI run when retargeted, while lint runs for all PRs.

Source: #942 at 211b87fc7d707e3a628cb95de5c73340af863c04, split onto main fdcc2eddf1b56b5b24a5a15d3abcb366adfeb5aa. The original branch is preserved.

Accuracy Test

• This layer at 46c322216e4b: 5 tests run, zero failures/errors on H20 with Python 3.11.12 / PyTorch 2.13.0+cu130.
• Final stack at 97cf2d5065c8: 401 integration tests run, zero failures/errors; 2 skipped. Modules run in fresh processes because the existing checkpoint tests retain a process-global NCCL group that interferes with later CPU/Gloo tests. This is targeted regression coverage, not a full-suite or downstream task-accuracy claim.
• Repository pre-commit checks passed for the final stack; each layer was formatted with the repository's pinned Black/isort versions.

Layer command (run in this branch's checkout, PYTHONPATH=.):

CUDA_VISIBLE_DEVICES=0 OMP_NUM_THREADS=4 python -m unittest \
  tests.test_modeling.test_dspark_optimizations \
  tests.test_modeling.test_dspark_causal -v

Qwen3 tokenizer regressions reuse the cached official tokenizer with HF_HUB_OFFLINE=1 and HF_HUB_CACHE pointing to that cache. No training job was launched.

Benchmark & Profiling

No new end-to-end throughput or downstream accuracy benchmark was run for this split. Model-side tests exercise numerical parity and gradients; historical measurements remain in #942 with their original scope and caveats.

Checklist

• Format changed code.
• Run the layer's regression tests and final-stack integration tests.
• Keep feature tests/documentation with their owning layer.
• Complete GitHub CI and any required downstream accuracy/performance validation before merge.