Solvent leg fails with "cutoff distance cannot be greater than half the periodic box size" and `SimulationNaNError` when using `box_shape="dodecahedron"` with `solvent_padding=1.2 nm

[MartinFuentetaja]

Description

When running RBFE calculations with RelativeHybridTopologyProtocol using a dodecahedron box and a solvent padding of 1.2 nm, many solvent-leg transformations fail, while the corresponding complex legs run without problems. Two different errors appeared, which turned out to share the same cause:

1. OpenMMException: NonbondedForce: The cutoff distance cannot be greater than half the periodic box size.
2. SimulationNaNError: Propagating replica 0 at state 2 resulted in a NaN!

Increasing solvent_padding from 1.2 nm to 1.5 nm (keeping everything else identical) resolved both errors.

Environment

• openfe: 1.11.1
• OpenMM: 8.2.0
• Python: 3.13.2
• OS / hardware: SLURM/GPU

Settings used

• solvation_settings.box_shape = "dodecahedron"
• solvation_settings.solvent_padding = 1.2 nm
• integrator_settings.timestep = 4 fs (default HMR)
• lambda_settings.lambda_windows = 11
• simulation_settings.production_length = 1 ns
• Nonbonded cutoff: default (0.9 nm)

What I observed

• The complex legs always ran fine.
• Many solvent legs failed, and they failed deterministically: all repeats of a given transformation failed.
• Reducing the timestep to 1 fs for the solvent leg did not help, so this is not a timestep issue.
• A plain MD simulation of a single ligand in water, using the same padding (1.2 nm) and box shape, also raised the "cutoff distance cannot be greater than half the periodic box size" error. So the problem does not require the alchemical part of the hybrid system.
• With solvent_padding = 1.5 nm the MD test ran fine, and the failing RBFE solvent legs also ran successfully.

Expected behavior

With a padding larger than the nonbonded cutoff (1.2 nm vs. 1.0 nm), I expected the box to be large enough for the simulation to run in the NPT ensemble without violating the minimum-image requirement.

Hypothesis (not verified)

The limiting box dimension of a dodecahedron is smaller than that of a cube built with the same padding, so for small solutes the box may be close to 2 × cutoff. Box fluctuations under the barostat may then push a dimension below 2 × cutoff, or the box may be too small from the start. I have not confirmed which of the two happens. I do not know whether the NaN is a direct consequence of the box problem or was a separate symptom of the same setup.

Workaround

Increase solvent_padding (1.5 nm worked for me), or use box_shape="cube".

Suggestion

It would be helpful to have a warning or validation error at setup time when the resulting box (especially for a dodecahedron) is close to or below 2 × the nonbonded cutoff, or to document in the solvent_padding / box_shape docs that the padding needed to satisfy the cutoff depends on the box shape. Also, the resulting SimulationNaNError is hard to trace back to the box size, so a clearer message would save users a lot of debugging time.