fix(sharding): reject a stored shard too short to hold its index on every path

[d-v-b]

This is a bugfix that ensures that the two codec pipelines do the same thing (raise an exception) when writing to a shard that's too smol. Full shard writes still clobber an existing shard, no matter its length

🤖 AI text below 🤖

A stored shard too short to hold its shard index, including a zero-length value, now raises a ValueError saying the shard is truncated or corrupt, on every read and partial-write path under either codec pipeline. Previously the asynchronous full-shard read and partial write treated a zero-length shard as missing (reading the fill value, or overwriting it), while every other path failed with a checksum mismatch or a decode error. zarrs and tensorstore also reject such a shard.

Changes

• refactor(sharding): tests shard readers and chunk byte slices with is None instead of truthiness. _ShardReader defines __len__, so shard_reader or _ShardReader.create_empty(...) replaced a loaded zero-chunk shard with an empty reader. The outcome was the same, but the check meant "was a shard loaded". This commit changes no behaviour.
• fix(sharding): adds _check_shard_index_size, which runs before the shard index is decoded. Every read and partial-write path decodes the index, so this one check covers all of them. _load_full_shard_maybe no longer treats a zero-length value as missing.

zarr-python never writes a shard shorter than its index: a shard with no chunks is deleted instead. A short value comes only from a truncated or interrupted write, an empty object created by some other tool, or another writer.

zarrs and tensorstore both reject a shard shorter than its index:

• zarrs sharding_codec.rs#L1273
• tensorstore shard_format.cc#L113

Tests

test_sharding_truncated_shard_raises checks the error for both pipelines, memory and local stores, both index locations, zero-length and 3-byte shards, and full read, partial read and partial write: 48 cases. All 48 fail without the fix. With it, tests/test_codecs, tests/test_array.py and tests/test_indexing.py pass (3857 passed, 770 skipped, 4 xfailed).

Bookkeeping

Mirrored from fork PR d-v-b#388. The changelog fragment is named after this PR.

🤖 Generated with Claude Code

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 94.69%. Comparing base (24ea324) to head (bd7544c).

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #4475   +/-   ##
=======================================
  Coverage   94.69%   94.69%           
=======================================
  Files          94       94           
  Lines       13597    13604    +7     
=======================================
+ Hits        12875    12882    +7     
  Misses        722      722           

Files with missing lines	Coverage Δ
src/zarr/codecs/sharding.py	95.73% <100.00%> (+0.04%)	⬆️

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[d-v-b]

doc build failure is due to rate limiting from other docs inventories, not due to this change. self-merging