Repository navigation
System.IO.Compression: ZipArchiveEntry always stores uncompressed data in memory #1544
Description
If a ZipArchive is opened in Update mode, calling ZipArchiveEntry.Open will always result in the entry's data being decompressed and stored in memory.
This is because ZipArchiveEntry.Open calls ZipArchiveEntry.OpenInUpdateMode, which in return gets the value of the ZipArchiveEntry.UncompressedData property, which will decompress the data and store it in a MemoryStream.
This in its own is already fairly inconvenient - if I'm updating a large file (say 1GB) that's compressed in a zip archive, I would want to read it from the ZipArchiveEntry in smaller chunks, save it to a temporary file, and update the entry in the ZipArchive in a similar way (i.e. limiting the memory overhead and preferring temporary files instead).
This also means that as soon as a ZipArchive is opened in Update mode, even reading an ZipArchiveEntry which you'll never update incurs the additional cost.
A short-term fix may be to expose ZipArchiveEntry.OpenInReadMode, which will return a DeflateStream instead. If you're doing mixed reading/writing of entries in a single ZipArchive, this should already help you avoid some of the memory overhead.
Activity
ericstj commentedon Feb 27, 2019
I've noticed the same. In general ZipArchiveEntry.Open is very non-intuitive in its behavior.
For read only / write only you get a wrapped DeflateStream which doesn't tell you the length of the stream nor permit you to seek it. For read/write (update) ZipArchiveEntry will read and decompress the entire entry into memory (in fact, into a memory stream backed by a single contiguous managed array) so that you have a seek-able representation of the file. Once opened for update the file is then written back to the archive when the archive itself is closed.
I agree with @qmfrederik here that we need a better API. Rather than rely solely on the archive's open mode we should allow for the individual call's to Open to specify what kind of stream they want. We can then check that against how the archive was opened in case it is incompatible and throw. Consider the addition:
For an archive opened with ZipArchiveMode.Update we could allow FileAccess.Read, FileAccess.Write, or FileAccess.ReadWrite, where only the latter would do the MemoryStream expansion. Read and write would be have as they did today. In addition to solving the OP issue, this would address the case where someone is opening an archive for Update and simply adding a single file: we shouldn't need to copy that uncompressed data into memory just to write it to the archive.
We also can do better in the read case, we know the length of the data (assuming the archive is not inconsistent) and can expose that rather than throwing.
ericstj commentedon Feb 27, 2019
Another interesting consideration for this is something like the approach taken by System.IO.Packaging in desktop. It implemented an abstraction over the deflate stream that would change modes depending on how you interacted with it: https://referencesource.microsoft.com/#WindowsBase/Base/MS/Internal/IO/Packaging/CompressStream.cs,e0a52fedb240c2b8
Exclusive reads small seeks would operate on a deflate stream; same for exclusive writes. Large seeks or random access would fall back to "emulation mode" wherein it would decompress everything to stream that was partially backed by memory but would fallback to disk.
I don't really like this since it hides some very expensive operations behind synchronous calls, as well as introducing a potential trust boundary (temp file) behind something that is expected to be purely computation. I think it makes sense to keep Zip lower level and not try to hide this in the streams we return. Perhaps we could allow for the caller to provide a stream for temporary storage in the Update case.
carlossanlop commentedon Dec 16, 2019
Triage:
This would be nice to have.
twsouthwick commentedon Jul 24, 2020
@carlossanlop This is blocking many users from moving to .NET Core as writing large office files ends up hitting this by way of System.IO.Packaging->DocumentFormat.OpenXml
danmoseley commentedon Jul 24, 2020
Maybe we should mark this 6.0.0
Trysor commentedon Aug 20, 2020
@twsouthwick @danmosemsft Is there a way to work around this issue (for OpenXml or otherwise) as a temporary solution? Alternatively re-discuss it for 5.0.0.
danmoseley commentedon Aug 20, 2020
I do not have context on this area. That is @twsouthwick and @carlossanlop
37 remaining items
edwardneal commentedon Dec 6, 2024
Thanks @ericstj. Being able to open a ZipArchiveEntry in read-only mode would definitely help, but there's another problem behind it: when disposed of, a ZipArchive opened in Update mode will currently rewrite the Stream it was loaded from by loading every ZipArchiveEntry into memory and writing them all back out. Issue #1543 tracks this; PR #102704 is currently going through the review process and will fix this specific use case.
The idea of a secondary buffer stream is interesting. The WPF repo currently uses a GetSeekableStream method to work around the lack of seekable streams, and this would at least give them some degree of control over that. dotnet/wpf#2085 may be helpful here.
Besides this,
ZipArchiveEntry.Openonly allows a single writer stream. This might not be necessary depending on how we use a buffer stream, so we might be able to permit multiple simultaneous read-only streams.twsouthwick commentedon Dec 10, 2024
This would be great for the dotnet/open-xml-sdk. It is the main blocker for migration from .NET Framework->Core for those workloads.
We generally don't operate directly with the
ZipArchiveEntry, but rather throughSystem.IO.Packaging. If that gets updated, then things should just work for us. If a new API is added there to work with this new API, we can update to use that as well.twsouthwick commentedon Jul 21, 2025
@ericstj I wanted to check if this is going to get fixed for .net 10 with some of the other fixes I've seen go through. We've checked this scenario with the latest previews, but it appears to still be an issue.
ericstj commentedon Jul 21, 2025
There's been no change here yet, but it's on our radar as an important one to fix. cc @artl93
Kkamikadzee commentedon Jul 30, 2025
If someone stumbles upon this, the problem is still reproducible.
DocumentFormat.OpenXml 3.3.0
System.IO.Packaging 9.0.7
mkaszewiak commentedon Aug 27, 2025
Issue still reproducible in:
DocumentFormat.OpenXml 3.3.0
System.IO.Packaging 9.0.8, 10.0.0-preview.7.25380.108
iremyux commentedon Jan 12, 2026
#122032 addresses this issue by adding new
ZipArchiveEntry.Open(FileAccess)/OpenAsync(FileAccess)overloads.The existing parameterless
Open()behavior in Update mode is unchanged and still loads the entry into memory. However, usingOpen(FileAccess.Read) inUpdatemode allows callers to stream the entry’s data without eagerly decompressing it into memory, avoiding the memory overhead described here. This provides an opt-in solution for mixed read/write scenarios without breaking existing code.Add ZipArchiveEntry.Open(FileAccess) and OpenAsync(FileAccess, Cancel…
mortan commentedon Jan 12, 2026
Thanks for the fix and the update! I see that the change is tagged for the .NET 11 milestone. Is there any way to get this for .NET 8/10?
iremyux commentedon Jan 19, 2026
Hello @mortan , unfortunately we do not plan to backport this change to any .NET versions prior to 11.
mortan commentedon Jan 19, 2026
Thanks for the clarification!