Skip to content

Implement canonical GPU constraint ordering and address remapping - #1924

Closed
huster-wgm wants to merge 2 commits into
google-deepmind:mainfrom
huster-wgm:fix/deterministic-constraint-order
Closed

huster-wgm wants to merge 2 commits into
google-deepmind:mainfrom
huster-wgm:fix/deterministic-constraint-order

Conversation

@huster-wgm

@huster-wgm huster-wgm commented Nov 8, 2026 •

Copy link
Copy Markdown

The CONSTRAINTS determinism flag currently calls an empty sorting hook, leaving row allocation and sparse storage order dependent on execution order. Implement canonical GPU ordering by world, constraint type and object while preserving within-object axes. Rebuild sparse Jacobian storage and Newton blocks, and remap contact and DOF-friction row addresses.

Eight regression cases cover one/two worlds, dense/sparse Jacobians and pyramidal/elliptic cones, including equality, joint-limit and multiple DOF-friction rows. They shuffle object groups, verify all associated mappings, check acceleration parity and exercise CUDA graph replay. The tests fail with the empty hook; the strengthened friction cases also caught an address-remapping omission in the prototype. Combined with the independent contact-ordering patch, the upstream suite passed 2585 tests with 41 skipped on an NVIDIA L20.

The two implementations are independent PRs. Contact-based trajectory repeatability also requires canonical contact ordering, since constraint object IDs include contact indices. Combined rigid-body replay tests are evidence for the tested models on the same GPU, not a guarantee for arbitrary flex models or different devices. Deterministic sorting is opt-in and carries extra GPU cost.

Companion implementation: #1923. Together, #1923 (contact ordering) and #1924 (constraint ordering and address remapping) replace #1775 and fill both previously empty ordering hooks in the model-level determinism pipeline. With the existing deterministic atomic path enabled via DeterminismType.ALL, the combined implementation achieved bitwise-identical native/Playground physical execution over 500 control steps each for G1, H2, A3, R1 and SIH2300 on an NVIDIA L20. A3 also matched across two independent processes on the final combined revision. The combined upstream suite passed 2585 tests with 41 skipped. This completes the missing ordering implementations for this validated rigid-body path; it is not a claim of universal flex, cross-GPU or global Warp RUN_TO_RUN determinism.

Tracks #1925 together with the companion ordering PR. Keep #1925 open until both #1923 and #1924 have landed and the combined behavior has been reviewed.

@johnnynunez

Copy link
Copy Markdown
Contributor

Heads-up on the row order: smooth._cfrc_ext_equality reads equality forces by position (connect k at row 3k, welds after them), so it needs every connect row before the weld rows. A sort by (world, type, id) leaves equality rows in XML id order.

With equalities listed weld, joint, connect, opt.deterministic=CONSTRAINT at 8bb30fd gives max|cfrc_ext - NONE| = 92.9 (max |cfrc_ext| is 537.6; CPU, dense and sparse), and force/torque sensors are off by as much. With connect listed first it is fine. Putting eq_type in the key before the id should fix it; #1422 orders them that way.

@thowell

thowell commented Nov 12, 2026

Copy link
Copy Markdown
Collaborator

@huster-wgm thanks for contributing to mujoco warp! closing since this functionality should be implemented by #1422.

@thowell thowell closed this Nov 12, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants