Skip to content

Group.create_dataset does not match 2.x API #2631

Open
#2638
Open
Group.create_dataset does not match 2.x API#2631
#2638
Labels
bugPotential issues with the zarr-python library
Milestone
3.0.0

Description

@jni
jni
opened on Jan 3, 2025

Zarr version

3.0.0rc2.dev1+gc070940e

Numcodecs version

0.14.1

Python Version

3.11.11

Operating System

macOS 15.1.1

Installation

editable install from main into a conda env

Description

#2463 broke the signature of Group.create_dataset, because you can no longer pass a data kwarg to the function. This effectively defeats the purpose of the deprecated function.

Steps to reproduce

import numpy as np
import zarr

group = zarr.open('test-group.zarr')
arr = np.random.random((5, 5))
ds = group.create_dataset(
        'random',
        data=arr,
        shape=(5, 5),
        )

saved = np.asarray(ds)
np.testing.assert_equal(arr, saved)

On 7907852 (right before #2463), this outputs:

/Users/jni/projects/play/test-create-dataset.py:5: DeprecationWarning: Use Group.create_array instead.
  ds = group.create_dataset(

On main (c070940), this crashes with the following traceback:

/Users/jni/projects/play/test-create-dataset.py:5: DeprecationWarning: Use Group.create_array instead.
  ds = group.create_dataset(
Traceback (most recent call last):
  File "/Users/jni/projects/play/test-create-dataset.py", line 5, in <module>
    ds = group.create_dataset(
         ^^^^^^^^^^^^^^^^^^^^^
  File "/Users/jni/micromamba/envs/all/lib/python3.11/site-packages/typing_extensions.py", line 2853, in wrapper
    return arg(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^
  File "/Users/jni/projects/zarr-python/src/zarr/core/group.py", line 2413, in create_dataset
    return Array(self._sync(self._async_group.create_dataset(name, **kwargs)))
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/jni/projects/zarr-python/src/zarr/core/sync.py", line 185, in _sync
    return sync(
           ^^^^^
  File "/Users/jni/projects/zarr-python/src/zarr/core/sync.py", line 141, in sync
    raise return_result
  File "/Users/jni/projects/zarr-python/src/zarr/core/sync.py", line 100, in _runner
    return await coro
           ^^^^^^^^^^
  File "/Users/jni/projects/zarr-python/src/zarr/core/group.py", line 1168, in create_dataset
    return await self.create_array(name, shape=shape, **kwargs)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: AsyncGroup.create_array() got an unexpected keyword argument 'data'

Additional output

No response

Activity

  1. jni
    added
    bugPotential issues with the zarr-python library
    on Jan 3, 2025
  2. jni
    added this to the 3.0.0 milestone on Jan 3, 2025

  3. d-v-b commented on Jan 3, 2025

    @d-v-b
    d-v-b
    on Jan 3, 2025
    Contributor

    oops, looks like we need to add a step in Group.create_dataset where we write data to the array after creating it

  4. jni
    added a commit that references this issue on Jan 4, 2025

    Add failing test for zarr-developers#2631

    76d6240
  5. jhamman
    added a commit that references this issue on Jan 4, 2025

    Fix create_dataset with data kwarg (#2638)

    3194534
  6. jhamman
    closed this as completedin #2638on Jan 4, 2025

  7. QuLogic commented on Jan 20, 2025

    @QuLogic
    QuLogic
    on Jan 20, 2025
    Contributor

    I don't believe this is fully fixed. I ran a test build of probeinterface against Zarr 3.0.1, and it currently fails as:

    ______________________________ test_save_to_zarr _______________________________

tmp_path = PosixPath('/tmp/pytest-of-mockbuild/pytest-0/test_save_to_zarr0')

    def test_save_to_zarr(tmp_path):
        # Generate a dummy probe instance
        probe = generate_dummy_probe()
    
        # Define file path in the temporary directory
        folder_path = Path(tmp_path) / "probe.zarr"
    
        # Save the probe object to Zarr format
>       probe.to_zarr(folder_path=folder_path)

tests/test_probe.py:176: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../BUILDROOT/usr/lib/python3.13/site-packages/probeinterface/probe.py:1138: in to_zarr
    self.add_probe_to_zarr_group(zarr_group)
../BUILDROOT/usr/lib/python3.13/site-packages/probeinterface/probe.py:1104: in add_probe_to_zarr_group
    group.create_dataset(name=field_name, data=data, dtype=dtype, chunks=False)
/usr/lib64/python3.13/warnings.py:637: in wrapper
    return arg(*args, **kwargs)
/usr/lib/python3.13/site-packages/zarr/core/group.py:2398: in create_dataset
    return Array(self._sync(self._async_group.create_dataset(name, **kwargs)))
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

args = (<AsyncGroup file:///tmp/pytest-of-mockbuild/pytest-0/test_save_to_zarr0/probe.zarr>, 'x')
kwargs = {'chunks': False, 'data': array([ 0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0., 25., 25., 25.,
       25., 25., 25., 25., 25., 25., 25., 25., 25., 50., 50., 50., 50.,
       50., 50., 50., 50., 50., 50.]), 'dtype': dtype('float64')}

    @functools.wraps(arg)
    def wrapper(*args, **kwargs):
        warn(msg, category=category, stacklevel=stacklevel + 1)
>       return arg(*args, **kwargs)
E       TypeError: AsyncGroup.create_dataset() missing 1 required keyword-only argument: 'shape'

    and indeed the old signature does not require shape.

  8. jni commented on Jan 22, 2025

    @jni
    jni
    on Jan 22, 2025
    ContributorAuthor

    Interesting, for some reason, I thought that it did require shape in the old signature, even though it made no sense. (We've had that discussion a few times in the repo.) But indeed I just tested it in a new env and it works fine. 🤔 @QuLogic Do you think you'd have time to make a PR to fix that case also?

  9. jni
    reopened this on Jan 22, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugPotential issues with the zarr-python library

    Type

    No type

    Projects

    No projects

      Milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions