Repository navigation
Drop numcodecs[crc32c] as dependency due to use of LGPL license #3510
Description
To whom it may concern,
We noticed that ZarrV3 introduced the following (non-optional) dependency: numcodecs[crc32c]
The Python package crc32c is licensed under LGPL. In general, I consider (L)GPL to be a license that is not well-suited for Python packages and it may create issues with MPL 2.0 which can be incompatible with (L)GPL based on how it is instantiated. Other problematic contexts are, e.g., redistribution of a software with closed-source extensions or plugins, where at least the distributor of the software is at risk of inadvertently making a mistake w.r.t. to having included (L)GPL licensed software. To the best of my knowledge, the exact scope or extent of LGPL within the context of Python is unclear, which is why it should be avoided.
Due to these reasons, I propose to:
- As a first step, to include numcodecs[crc32c] only as an optional dependency, such that it is convenient to not include crc32c. Note that crc32c itself is only an optional component within numcodec and it appears that numcodecs (without the optional crc32c) could be used instead.
- As a second step, to potentially drop the dependency on numcodecs altogether, see: a codec simplification plan #3162
Thank you for your attention.
Activity
d-v-b commentedon Oct 7, 2025
thanks for raising this concern! operationally we need crc32c for the sharding codec, which we have been socializing extensively as part of the core Zarr v3 spec / implementation. So I think we want
pip install zarrto include crc32c support of some kind, because users expect sharding to work out of the box.I see 2 independent things we could do. Either or both would address this issue, I think:
define a
zarr[minimal]install that only includes the minimal dependencies. We then define "minimal dependencies" to exclude the crc32c codec.use a different crc32c library. This one is apache 2.0 licensed, which should be more permissive than LGPL?
decryptofy commentedon Oct 7, 2025
Thank you. Both options would work for us. Apache 2.0 license is great, more permissive than LGPL, and better suited for a Python project due to the wording of the LGPL 2.1 that in essence, only applies to compiled languages with header files. I'm also under the impression that the Apache-licensed library uses a crc32c core (from Google) that is more actively maintained, i.e., yet another aspect that supports moving to this crc32c library.
jhamman commentedon Oct 9, 2025
see zarr-developers/numcodecs#792 for a fix.