Repository navigation
Fuse sparse mass factorization across tree levels - #1965
Open
adenzler-nvidia wants to merge 6 commits into
Open
adenzler-nvidia wants to merge 6 commits into
adenzler-nvidia wants to merge 6 commits into
Conversation
Gather descendant updates within each tree to remove per-level launches and atomic accumulation. Preserve sleeping factors and use typed array views so strided output buffers remain supported. Keep the existing dense dispatch, factor layout and triangular solves. Build depth metadata during model upload and retain the solve metadata needed by deterministic execution.
Choose a bounded power-of-two block size from the average number of factor entries per dependency level. This exposes more parallel work within wide trees without changing the factorization or solve layout. Use inclusive DOF subtree sizes to match the body convention and name the staged loads by their numerical roles. Cover sleeping and waking with a wider sparse tree.
Include the latest Newton solver cleanup so review and validation use the current upstream baseline.
Include the rendering fix so the upstream proposal and its validation use the current main revision.
Collaborator
|
Thanks @adenzler-nvidia! Clean design fusing sparse 1.
|
The fused kernel reads M directly and writes every sparse-block entry of L and D, and the block factorization already covers compact and dense blocks, so the copy of M into L and the reciprocal-diagonal launch on mixed models did nothing that is read later. Remove both, along with the now unused _qLDiag_div kernel. Pairs are grouped by level, so compare the entry against the level instead of loading M_rownnz to detect the diagonal.
Collaborator
Author
|
Thanks @thowell, both applied in 69b8dd3.
|
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Sparse mass factorization launches updates at each tree depth and atomically scatters descendant contributions. This change assigns one block to each sparse tree and world, gathers each factor entry's contributions, and synchronizes between levels inside the kernel.
Model setup groups factor entries by depth and selects the block width from the mean level width. The kernel uses MuJoCo's contiguous DOF subtrees and full ancestor-chain rows. It retains the reverse LDL factor format and existing triangular solves. Dense dispatch is unchanged, sleeping trees preserve their factors, and factor storage can be strided. One block per tree limits within-tree parallelism for very large trees.
Full-step times at 8,192 worlds on an exclusive NVIDIA L40, against main at
dbf69e1e:Medians of three alternating baseline/patched pairs of 1,000 steps, excluding setup/JIT; Warp 1.18.0, driver 615.71.10. Positive percentages mean less time. G1 and humanoid use the existing dense path and serve as controls. H2 is an assembled 81-DOF model with Sharpa hands in a zero-control contact rollout, not a policy task. G1 uses the shuffle-dance replay.
Both variants use Newton/CG iteration limits of 100/1,000 and 50 line-search iterations, with all overflow warnings enabled and treated as errors. H2 uses
nconmax=128,nccdmax=128, andnjmax=512to fit the 8,192-world run in memory. All 36 rollouts completed without overflow or non-finite worlds.Tests cover mixed dense/sparse models, wide trees, sleep/wake transitions under graph replay, deterministic mode, and strided factor storage. All pre-commit checks passed.
uv run pytest -n 8reported 3,718 passed, 42 skipped, and four failures in the existing Newton elliptic in-place Cholesky tests. All four reproduce on pristine main with the same unsupported conditional-graph memory-allocation error.