Skip to content

BUG: Preserve component layout when reconstructing and serializing images - #938

Open
jdotc1 wants to merge 2 commits into
ANTsX:mainfrom
jdotc1:fix/channel-first-reconstruction-20261023
Open

jdotc1 wants to merge 2 commits into
ANTsX:mainfrom
jdotc1:fix/channel-first-reconstruction-20261023

Conversation

@jdotc1

@jdotc1 jdotc1 commented Oct 23, 2026 •

Copy link
Copy Markdown
Contributor

Problem

With channels_first=True, multicomponent ANTsImage.numpy() places the component axis first. Consumers that assume components-last can fail reconstruction or silently alter serialized and NIfTI shapes.

Changes

  • Normalize component axes before new_image_like, pickle storage, .npy writing, and NIfTI conversion while preserving the in-memory layout.
  • Validate new_image_like input against the actual image.numpy() shape before moving axes and report the received and expected shapes.
  • Preserve channels_first through pickle and deepcopy using an optional seventh state element; still accept older six-element states.
  • Store an optional channels_first: true key in the existing .npy JSON sidecar and restore the flag on read, without changing the saved array layout.
  • Correct to_nibabel_nifti to produce (x, y, z, 1, components) with the right data for channels-first 3D vector images.
  • Add synthetic 2D/3D regressions covering both layouts, img + img, invalid shapes, legacy pickle state, deepcopy, .npy, and NIfTI round trips.

Validation

  • Focused regressions: 6 passed, 34 subtests passed.
  • Three fully affected test modules: 140 passed, 3 skipped, 61 subtests passed (6 warnings).
  • Broader test run excluding test_core_ants_transform.py: 278 passed, 3 skipped, 174 subtests passed; 7 failures/subfailures exclusively in unchanged displacement-transform I/O tests. The checked-out Python source calls the native transform-to-field API with None, which the reused antspyx 0.6.3 native extension does not accept.
  • An unfiltered run terminated in an unchanged transform-setter test with a native segmentation fault; no full-suite pass is claimed. compileall and git diff --check passed.
  • Tests used checked-out Python code and the prebuilt antspyx 0.6.3 extension, not a matching freshly built native extension. Hosted CI remains the authoritative integration gate.

Compatibility and scope

  • Old six-item pickles still load. Standard channels-last pickles still use six items; new channels-first component pickles use seven. Older ANTsPy versions accepting only six items may not load those new pickles.
  • .npy arrays remain components-last, and older readers can ignore the new optional sidecar key; old sidecars still load with the default channels-last presentation.
  • new_image_like now requires the documented image.numpy() axis layout; manually supplied components-last arrays on a channels-first image may need adaptation.
  • Previously malformed serialized files are not repaired, and pre-existing RGB numpy()/view() issues are outside scope.
  • Independent channels_first propagation defects in resample_image, smooth_image, and multicomponent indexing are reserved for a follow-up, as suggested in review.

…ages

An ANTsImage with channels_first=True exposes components on axis zero through numpy(). Reconstruction and serialization currently hand that array to a components-last reader. Valid arithmetic therefore raises, and NumPy/pickle round trips silently alter the spatial shape and component count.

Move the component axis to the last position before reconstruction and serialization. Preserve channels_first on new_image_like results. NumPy and pickle restore the existing canonical channels-last presentation while preserving physical data and geometry.

Add repository-style regression coverage for the corrected behavior.

@jig-mistry jig-mistry left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tried this on top of current main. It merges cleanly, the full test suite passes, and with a channels_first=True image, arithmetic, new_image_like, pickle and .npy now give the right shape and data. A few things I noticed while testing:

  1. to_nibabel_nifti has the same problem and isn't covered yet. It takes image.numpy() and treats the last axis as the components (nibabel_nifti_to_ants.py:147), so a channels_first image silently turns into a wrong NIfTI:

    s = ants.from_numpy(np.random.rand(4, 5, 6).astype('float32'))
    v = ants.merge_channels([s, s, s], channels_first=True)
    ants.to_nibabel_nifti(v).shape   # (3, 4, 5, 1, 6), should be (4, 5, 6, 1, 3)
  2. ANTsImage has no __deepcopy__, so copy.deepcopy also goes through __getstate__. With this change deepcopy(v) has the right data but comes back with channels_first=False, so its numpy() is (4, 5, 6, 3) instead of (3, 4, 5, 6), while v.clone() keeps the flag. Images sent to multiprocessing workers lose it the same way. Would it work to store channels_first as an optional 7th item (only written when it's True) and have __setstate__ accept 6 or 7 items? Existing pickles would still load. For .npy, the current reader ignores extra keys in the JSON sidecar, so a channels_first key could go there too.

  3. Probably for a follow-up: a few functions drop the flag, so their output's numpy() comes back components-last.

    • resample_image tries to keep it, but the multi-component branch loops with for image in images:, so channels_first=image.channels_first on line 75 of resample_image.py reads the flag of the last split channel, which is always False.
    • smooth_image and the multi-component branch of __getitem__ don't pass channels_first to merge_channels at all.
  4. Small one: the array is moved before the shape check. So passing a components-last (6, 7, 2) array to new_image_like on a (6, 7) channels_first image reports given array shape ((2, 6)) and image array shape ((6, 7)) do not match. That call worked on main, so checking the shape before the move and printing the expected image.numpy() shape would make the error easier to understand for anyone who hits it.

It might also be worth adding a deepcopy case and an image-with-image operator (img + img) to the tests.

@jdotc1

jdotc1 commented Nov 8, 2026

Copy link
Copy Markdown
Contributor Author

Thanks, @jig-mistry, for the careful testing and concrete reproductions. I pushed follow-up commit fca642c addressing the points in this PR:

  1. to_nibabel_nifti now normalizes channels-first vector data before adding the NIfTI singleton axis. A regression test checks both shape and voxel data for channels-first and channels-last inputs.
  2. Pickle and deepcopy preserve channels_first through an optional seventh state element, while __setstate__ still accepts old six-item states. The .npy sidecar likewise records/restores the flag without changing stored array order. One caveat: older ANTsPy versions may not read newly written seven-item channels-first pickles.
  3. Agreed on the independent resample_image, smooth_image, and indexing propagation issues. I am keeping those for a separate follow-up to keep this PR focused.
  4. new_image_like validates the incoming shape before the axis move and reports expected and received shapes. I also added the suggested deepcopy and img + img tests, plus legacy serialization coverage.

All three affected test modules pass locally (140 passed, 3 skipped, 61 subtests passed). The broader local suite is limited by unrelated transform/native-extension mismatches under the prebuilt 0.6.3 extension; the hosted CI run is in progress. Thanks again for strengthening the patch.

@jig-mistry

Copy link
Copy Markdown
Contributor

Thanks for the quick follow-up. I checked fca642c on its own and merged into current main, and the full test suite passes on both.

to_nibabel_nifti, pickle, deepcopy, multiprocessing and the .npy round trip now keep the layout and the data of a channels_first image. Old six-item states and .npy files without the new key still load as before, and new_image_like now reports the expected shape. The caveat matches what I see on main: it can't unpickle a channels_first image written by this branch, but channels_last pickles and the new .npy files still load there.

Looks good to me, and keeping the propagation issues for a separate PR makes sense.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants