Skip to content

Fix LZMA container decoding with custom filters - #886

Open
Golden007-prog wants to merge 1 commit into
zarr-developers:mainfrom
Golden007-prog:fix/lzma-filter-decoding
Open

Golden007-prog wants to merge 1 commit into
zarr-developers:mainfrom
Golden007-prog:fix/lzma-filter-decoding

Conversation

@Golden007-prog

@Golden007-prog Golden007-prog commented Oct 30, 2026 •

Copy link
Copy Markdown

Fixes #188.

LZMA can encode XZ and ALONE containers with custom filters, but decoding those containers forwards the filter list to lzma.decompress, which rejects explicit filters outside FORMAT_RAW. Read container filters from their headers by supplying the filter list only for raw streams. This preserves the configured format and keeps raw decoding's required explicit filter chain.

The regression covers custom XZ and ALONE filters plus a RAW control, using the existing buffer/array/output-buffer round-trip helper and registry-restored configurations. It also verifies standard-library decoding of the encoded bytes. On unchanged source, XZ and ALONE fail with Cannot specify filters except with FORMAT_RAW, while nine other cases pass.

Validation on Linux/WSL, Python 3.12 with the declared Meson/Cython build and unchanged c-blosc submodule:

  • 47 focused and related tests pass.
  • Full suite: 661 passed, 33 skipped, with skips for unavailable optional codecs/Zarr/CuPy and free-threaded/removal cases. An initial run lacked the external zstd executable; installing the official Ubuntu tool in the disposable test environment resolves both failures without changing tests or repository configuration.
  • Pinned Ruff 0.14.6 check and formatting, and git diff --check, pass.
  • The local full run covers all 26 statements in the LZMA module (100%). Hosted Codecov also reports 100.00% project coverage and full coverage of all modified coverable lines for commit a494015.
  • The Sphinx HTML documentation build succeeds with 28 warnings. A separate build with the original source/tests/release notes produces the identical 28 warnings; none are introduced by this change.
  • Hosted checks for commit a494015: 41 successful overall (39 check-runs and two legacy statuses), two skipped, no pending or failed checks. This includes the 32-bit test job and platform wheel builds.

TODO:

  • Unit tests and/or doctests in docstrings
  • Tests pass locally
  • Docstrings and API docs for any new/modified user-facing classes and functions
  • Changes documented in docs/release.md
  • Docs build locally
  • GitHub Actions CI passes
  • Test coverage to 100% (Codecov passes)

AI assistance: OpenAI Codex researched, implemented and validated this contribution.

@codecov

codecov Bot commented Oct 30, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 100.00%. Comparing base (54fc274) to head (a494015).

Additional details and impacted files
@@            Coverage Diff            @@
##              main      #886   +/-   ##
=========================================
  Coverage   100.00%   100.00%           
=========================================
  Files           27        27           
  Lines          914       915    +1     
=========================================
+ Hits           914       915    +1     
Files with missing lines Coverage Δ
src/numcodecs/lzma.py 100.00% <100.00%> (ø)
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

LZMA custom filter error

1 participant