Support OME-Zarr v0.5 (Zarr v3) datasets by talonchandler · Pull Request #519 · Biohub/waveorder

talonchandler · 2026-02-08T20:25:36Z

Fixes three bugs that prevent reconstruction of v0.5 datasets. Closes #518.

Fix KeyError: 'plate' when reading v0.5 plate metadata (nested inside "ome" key)
Preserve zarr version from input to output (v2 in → v2 out, v3 in → v3 out)
Convert torch tensors to numpy before writing to zarr (fixes torch.dtype incompatibility with zarr-python 3)
Add parametrized tests covering v0.4 and v0.5 for metadata extraction, version propagation, and end-to-end reconstruct

…fore writing

Soorya19Pradeep · 2026-02-09T18:00:19Z

Thank you for the fix @talonchandler . Here are the details of the issue I was encountering:

I was trying to perform phase and fluorescence reconstruction on a zarr. The zarr was produced from the conversion of MM ome tiffs to zarr format using the following PR:Prevent data loss during YX to ZYX rechunking for conversion iohub#368. This was done to prevent data loss in this specific dataset: /hpc/projects/intracellular_dashboard/virtual_stain_ft_infected/2026_01_29_A549_H2B_CAAX_DAPI_DENV_ZIKV/0-convert/.
The converted data was in zarr v3 format.
I created an updated environment using the main branch of waveOrder. This env has python 3.12.2 and iohub 0.2.3.post1. I used the environment to reconstruct and got the following error:

Traceback (most recent call last):
File "/home/soorya.pradeep/.local/bin/waveorder", line 8, in
sys.exit(cli())
^^^^^
File "/hpc/apps/anaconda/25.3.1/lib/python3.12/site-packages/click/core.py", line 1442, in call
return self.main(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/hpc/apps/anaconda/25.3.1/lib/python3.12/site-packages/click/core.py", line 1363, in main
rv = self.invoke(ctx)
^^^^^^^^^^^^^^^^
File "/hpc/apps/anaconda/25.3.1/lib/python3.12/site-packages/click/core.py", line 1828, in invoke
sub_ctx = cmd.make_context(cmd_name, args, parent=ctx)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/hpc/apps/anaconda/25.3.1/lib/python3.12/site-packages/click/core.py", line 1186, in make_context
self.parse_args(ctx, args)
File "/hpc/apps/anaconda/25.3.1/lib/python3.12/site-packages/click/core.py", line 1197, in parse_args
value, args = param.handle_parse_result(ctx, opts, args)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/hpc/apps/anaconda/25.3.1/lib/python3.12/site-packages/click/core.py", line 2416, in handle_parse_result
value = self.process_value(ctx, value)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/hpc/apps/anaconda/25.3.1/lib/python3.12/site-packages/click/core.py", line 2355, in process_value
value = self.callback(ctx, self, value)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/hpc/mydata/soorya.pradeep/waveorder/waveorder/cli/parsing.py", line 18, in _validate_and_process_paths
with open_ome_zarr(path, mode="r") as dataset:
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/soorya.pradeep/.local/lib/python3.12/site-packages/iohub/ngff/nodes.py", line 1996, in open_ome_zarr
root = _open_store(store_path, mode, version, synchronizer)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/soorya.pradeep/.local/lib/python3.12/site-packages/iohub/ngff/nodes.py", line 80, in _open_store
raise FileNotFoundError(
FileNotFoundError: Dataset directory not found at /hpc/projects/intracellular_dashboard/virtual_stain_ft_infected/2026_01_29_A549_H2B_CAAX_DAPI_DENV_ZIKV/0-convert/convert.zarr/0/2/000000.

Soorya19Pradeep · 2026-02-09T19:19:16Z

@talonchandler , you were right about the dependency issue. I just tried to do an editable install of the waveoder pulling from this PR and I got the following error:

ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
waveorder 3.0.1.dev4+g0db0ad23a requires iohub<0.4,>=0.2, but you have iohub 0.4.0.dev2+g73c7f2426 which is incompatible.

I changed my iohub to be an editable install as well.

talonchandler · 2026-02-09T20:12:43Z

Hi @Soorya19Pradeep, I just bumped waveorder's dependency on iohub and I think you will be able to complete your reconstruction. If you pip install -e a fresh environment on this branch you should get iohub==0.3.0a5.

See my test reconstruction here: /hpc/projects/intracellular_dashboard/virtual_stain_ft_infected/2026_01_29_A549_H2B_CAAX_DAPI_DENV_ZIKV/1-reconstruct/tc-test

The tests do not pass here yet, so I'm changing this back to draft.

srivarra · 2026-02-09T20:17:15Z

@talonchandler GitHub is having a lot of issues atm: https://www.githubstatus.com/

Soorya19Pradeep · 2026-02-10T04:40:08Z

Thank you for the fix Talon!
I tried it out. I was hopeful as I was able to lazy-load the images in napari from the environment with iohub==0.3.0a4. I am having an editable install of waveorder==3.0.1.dev5+ga28a97843 and 'not-editable' install of iohub==0.3.0a5. But I am getting the following error now:

File "/hpc/apps/anaconda/25.3.1/lib/python3.12/site-packages/click/core.py", line 2416, in handle_parse_result
value = self.process_value(ctx, value)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/hpc/apps/anaconda/25.3.1/lib/python3.12/site-packages/click/core.py", line 2355, in process_value
value = self.callback(ctx, self, value)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/hpc/mydata/soorya.pradeep/waveorder/waveorder/cli/parsing.py", line 18, in _validate_and_process_paths
with open_ome_zarr(path, mode="r") as dataset:
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/soorya.pradeep/.local/lib/python3.12/site-packages/iohub/ngff/nodes.py", line 2383, in open_ome_zarr
root = _open_store(store_path, mode, version)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/soorya.pradeep/.local/lib/python3.12/site-packages/iohub/ngff/nodes.py", line 84, in _open_store
raise RuntimeError(
RuntimeError: Cannot open Zarr root group at /hpc/projects/intracellular_dashboard/virtual_stain_ft_infected/2026_01_29_A549_H2B_CAAX_DAPI_DENV_ZIKV/0-convert/convert.zarr/0/2/zarr.json

talonchandler · 2026-02-10T05:59:24Z

Okay @Soorya19Pradeep I've now tried your reconstruct.sh and I saw it start. You can try again.

Tricky bug that I missed:

Your shell script used our typical .zarr/*/*/* pattern, which glob expands to all positions
In zarr v2, glob expansion skips .zattrs files because globs don't match dotfiles
In zarr v3, glob expansion catches zarr.json files, causing the issue you ran into above. waveorder now filters these out.
FWIW my test shell script used .zarr/*/*/*/ (trailing slash), which only matches directories, so did not recreate the bug. This made debugging challenging.

cc @srivarra it seems that you fixed this issue in biahub here, though not in iohub...I'll open an issue.

srivarra · 2026-02-10T19:00:02Z

@talonchandler @Soorya19Pradeep

Fixed the zarr v3 globbing issue here: Biohub/iohub/pull/372

srivarra · 2026-02-10T19:08:32Z

If this is being fixed in iohub, do we need it here?

I think we still need it here because this is the top-level click parsing. We should aim to deduplicate these code paths, but I think (?) we'll need to: (1) release iohub, (2) make a wo PR that depends on the new iohub release & deduplicates. @srivarra do you see a smoother path to a fix than this?

@talonchandler We can expose that snippit in utils.py of iohub and allow it to be reused.

It's probably fine where it is, no? You can still import from cli.parsing. I'm mostly thinking about the dependency management...we'll need to release iohub before we can import it here.

Ah yeah. It's good where it is then!

Soorya19Pradeep · 2026-02-10T19:53:51Z

Thank you @talonchandler! It is working now and the reconstructions looks good.

* Fix KeyError on v0.5 plate metadata and propagate version * Preserve zarr version in output and convert torch tensors to numpy before writing * Preserve zarr version for transfer function output * Add tests for zarr v3 reconstruction * bump iohub dependency * parse only directories * test parsing * fix test

talonchandler added 3 commits February 8, 2026 12:23

Fix KeyError on v0.5 plate metadata and propagate version

9aeaf17

Preserve zarr version in output and convert torch tensors to numpy be…

7943597

…fore writing

Preserve zarr version for transfer function output

676722d

talonchandler marked this pull request as draft February 8, 2026 20:29

Add tests for zarr v3 reconstruction

0db0ad2

talonchandler force-pushed the fix/zarr-v3-reconstruction branch from 1c14888 to 0db0ad2 Compare February 8, 2026 20:34

talonchandler marked this pull request as ready for review February 8, 2026 20:54

talonchandler requested review from Soorya19Pradeep and srivarra February 8, 2026 20:54

talonchandler marked this pull request as draft February 9, 2026 20:01

bump iohub dependency

a28a978

talonchandler marked this pull request as ready for review February 9, 2026 21:23

talonchandler changed the title ~~Fix reconstruction for OME-Zarr v0.5 (Zarr v3) datasets~~ Support OME-Zarr v0.5 (Zarr v3) datasets Feb 9, 2026

talonchandler added 3 commits February 9, 2026 21:36

parse only directories

1a5e1d8

test parsing

1a148f0

fix test

6ca474c

srivarra reviewed Feb 10, 2026

View reviewed changes

Soorya19Pradeep approved these changes Feb 10, 2026

View reviewed changes

Soorya19Pradeep requested a review from alxndrkalinin February 10, 2026 19:58

talonchandler merged commit 46d87df into main Feb 10, 2026
11 checks passed

ieivanov mentioned this pull request Feb 17, 2026

Zarr v2 -> Zarr v3 migration across computational imaging team's frameworks Biohub/biahub#199

Open

8 of 10 tasks

talonchandler deleted the fix/zarr-v3-reconstruction branch April 20, 2026 21:10

Conversation

talonchandler commented Feb 8, 2026

Uh oh!

Soorya19Pradeep commented Feb 9, 2026

Uh oh!

Soorya19Pradeep commented Feb 9, 2026

Uh oh!

talonchandler commented Feb 9, 2026

Uh oh!

srivarra commented Feb 9, 2026

Uh oh!

Soorya19Pradeep commented Feb 10, 2026

Uh oh!

talonchandler commented Feb 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

srivarra commented Feb 10, 2026

Uh oh!

srivarra Feb 10, 2026

Choose a reason for hiding this comment

Uh oh!

talonchandler Feb 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

srivarra Feb 10, 2026

Choose a reason for hiding this comment

Uh oh!

talonchandler Feb 10, 2026

Choose a reason for hiding this comment

Uh oh!

srivarra Feb 10, 2026

Choose a reason for hiding this comment

Uh oh!

Soorya19Pradeep commented Feb 10, 2026

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

talonchandler commented Feb 10, 2026 •

edited

Loading

talonchandler Feb 10, 2026 •

edited

Loading