Repository navigation
System.IO.Compression work planned for .NET 7 #62658
Description
This issue captures the planned work for .NET 7. This list is expected to change throughout the release cycle according to ongoing planning and discussions, with possible additions and subtractions to the scope.
Planned for .NET 7
- Add the ArchiveComment property for ZipArchive class #1547
- Detect password protected file with ZipArchive #57197
- [API Proposal]: APIs to support tar archives #65951
Backlog (roughly in priority order)
- Explore if we should consume the zlib-ng native cross-platform implementation to benefit ARM64.
- PR testing this: Add zlib-ng library, optimized for ARM64 #61883
- Improve the way our Zip APIs load entries in memory.
- System.IO.Compression: ZipArchive loads entire file in memory on .Dispose #1543
- System.IO.Compression: ZipArchiveEntry always stores uncompressed data in memory #1544
- Add support for adding new entries to large zip archives #49149
- ZipArchive.Open will read non seakable stream to memory & fail with bad error on large streams #59027
- Add Flush to ZipArchive #24149
- Explore if we should implement LZMA/7z or recommend an existing 3rd party package and contribute to it.
- Allow opening (raw) compressed archive entries in ZipArchiveEntry
- Add async compression APIs.
- Add stream-less compression APIs
- Ensure we provide full support for zip64 (reading, creating, updating).
- Add password to ZipArchive #1545
- Explore if we should implement ZStandard or recommend an existing 3rd party package and contribute to it.
- Clarify documentation ambiguity between some of our compression types:
DeflateStream,GZipStream,ZLibStream. - Explore if we should implement other algorithms or file formats, or recommend an existing 3rd party package and contribute to it.
- Continuous performance improvements.
- Improve documentation and add more examples.
Activity
danmoseley commentedon Dec 10, 2021
Thanks @carlossanlop . Are there opportunities for community contributions here? Perhaps there's something that's suitable for someone interested to get started on
carlossanlop commentedon Jan 3, 2022
Yes, I think I can mark a couple of issues as up-for-grabs, but I first need to add a comment with some context first.
6 remaining items
AraHaan commentedon Feb 8, 2022
More things I would like done for .NET 7 (Zlib):
For this it might be until P3~P5 or 6 at the latest (to give time for review, approval, PR, tests, and merge).
Genbox commentedon Feb 17, 2022
Finally, System.IO.Compression gets some much-needed attention!
Ask 1:
Can we revert this rather crazy input validation? It completely removes the ability to provide an alternative value, even though the related issue describes it as "a range check between 0 and 11" which out make much more sense.
Edit: It seems all other encoder/decoders have the same behavior. I respect that all are now doing the same thing, but I much rather have a range validation check than a strict map from enum to hardcoded level.
Every sine encoder/decoder that currently exists and all future ones should have sane input validation on the compression level to ensure there is no unexpected behavior.
Genbox commentedon Feb 17, 2022
Ask 2:
Can we change CompressionLevel to something a lot more user-friendly?
Current values are:
I propose we change it to:
If that is not possible for whatever reason, can we at least change "Optimal" to "Default"?
The main reason is that .NET devs and users alike seem to not understand what "optimal" means. I've had this discussion in 2018 and about 15 times since.
Genbox commentedon Feb 17, 2022
Ask 3:
Please expose more advanced configurations. Encoders/Decoders have a million options like providing a set of pre-defined wordlists, Huffman encoding specific options, window sizes, and much more. There is no way to squeeze all that into constructors as single values. Properties are not a solution as some of these values needs to be available before the encoder/decoder is used, so they need to come in the constructor.
There is an excellent proposal here, which would solve that problem altogether: #42820
Genbox commentedon Feb 17, 2022
Ask 4:
Zlib is great, but the implementation is dated.
There are several highly optimized versions out there:
Zlib-ng seems like a good next step:
AraHaan commentedon Feb 18, 2022
I been working on a span based zlib that calls into the same native version under the hood that provides more config. However it will need to be marked as ready for review so it can have a PR be created.
As for the PR I would have to place 2 related proposals into the same PR as well they depend on eachother on changes (new types would be added to expose all compression levels for all of the currently implemented compressors in the runtime for better configuration).
I would have made the options classes structs in that proposal but sadly C# does not allow structs to inherit from other structs.
As for other zlib native libraries, while they may be beneficial for performance, what prevents them from accidentally breaking something in the library they are working on. I noticed something similar when I was trying to implement the same faster stuff in my fully C# zlib library but it only broke on MacOS where all the compressed data was simply:
but on other OSs:
As such, something as trivial as an native dependency change could break someone even if it provides a performance boost unless tested extensively in anyone's possible situation that can happen.
jeffhandley commentedon Mar 4, 2022
This list in this issue has been edited to reflect changes from our ongoing planning and discussions. The headings have been updated to reflect what is still planned for .NET 7 and what is in the backlog. The list is still expected to change throughout the release cycle with periodic reviews of our plans.
AraHaan commentedon Mar 20, 2022
@jeffhandley any updates for what I proposed here for zlib (all linked here): #62658 (comment) ?
I really need them in .NET 7 at least if possible because span based compression apis should be a priority as well (I think) for those of us who hate stream based due to perf from loading a files as bytes, then create a stream, use that stream with compression apis to compress/decompress, do other stuff that expects spans, and then save the compressed/decompressed output. The stream part becomes a pain as with span based compression apis they can be avoided entirely.
jeffhandley commentedon Aug 9, 2022
@AraHaan I'm sorry I overlooked that question months back and didn't notice until now. The proposals you made for zlib are valuable, but did not make it into .NET 7.0. We will consider them while we do our .NET 8 planning. Thank you.