Repository navigation
Regressions in System.IO.Compression.Gzip #73389
Description
Run Information
| Architecture | arm64 |
|---|---|
| OS | Windows 10.0.19041 |
| Baseline | 54c717a4ed822f46a23893479b8d4398596c041d |
| Compare | 1926f0939a855d6ef41593377c75ab4c31e9fcbb |
| Diff | Diff |
Regressions in System.IO.Compression.Gzip
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| Compress - Duration of single invocation | 2.85 ms | 3.00 ms | 1.05 | 0.00 | False |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.IO.Compression.Gzip*'Details
Payloads
Histogram
System.IO.Compression.Gzip.Compress(level: Fastest, file: "TestDocument.pdf")
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.9996544642857135 > 2.9892825729166663.
IsChangePoint: Marked as a change because one of 5/9/2022 6:32:22 AM, 8/2/2022 9:32:02 PM, 8/4/2022 5:17:15 AM falls between 7/26/2022 2:47:26 PM and 8/4/2022 5:17:15 AM.
IsRegressionStdDev: Marked as regression because -68.96288776311692 (T) = (0 -2997694.554906898) / Math.Sqrt((117543328.83053406 / (41)) + (7034096.18717764 / (4))) is less than -2.016692199226234 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (41) + (4) - 2, .025) and -0.052052518626114985 = (2849377.2904242603 - 2997694.554906898) / 2849377.2904242603 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
| Architecture | arm64 |
|---|---|
| OS | Windows 10.0.19041 |
| Baseline | 54c717a4ed822f46a23893479b8d4398596c041d |
| Compare | 1926f0939a855d6ef41593377c75ab4c31e9fcbb |
| Diff | Diff |
Regressions in System.Buffers.Tests.NonStandardArrayPoolTests<Object>
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| RentNoReturn - Duration of single invocation | 51.08 ns | 54.81 ns | 1.07 | 0.01 | False |
_1.png)
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Buffers.Tests.NonStandardArrayPoolTests<Object>*'Details
Payloads
Histogram
System.Buffers.Tests.NonStandardArrayPoolTests<Object>.RentNoReturn(RentalSize: 64, UseSharedPool: True)
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 54.809619047619044 > 53.66642539062501.
IsChangePoint: Marked as a change because one of 5/7/2022 9:43:21 AM, 8/2/2022 9:32:02 PM, 8/4/2022 5:17:15 AM falls between 7/26/2022 2:47:26 PM and 8/4/2022 5:17:15 AM.
IsRegressionStdDev: Marked as regression because -18.036614790113084 (T) = (0 -54.494937921626985) / Math.Sqrt((0.35170233208069396 / (41)) + (0.10175683495165423 / (4))) is less than -2.016692199226234 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (41) + (4) - 2, .025) and -0.06501357108349988 = (51.168303767421605 - 54.494937921626985) / 51.168303767421605 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Architecture | arm64 |
|---|---|
| OS | Windows 10.0.19041 |
| Baseline | 54c717a4ed822f46a23893479b8d4398596c041d |
| Compare | 1926f0939a855d6ef41593377c75ab4c31e9fcbb |
| Diff | Diff |
Regressions in System.IO.Compression.ZLib
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| Compress - Duration of single invocation | 2.80 ms | 2.99 ms | 1.07 | 0.00 | False | |||||
| Compress - Duration of single invocation | 440.94 μs | 469.89 μs | 1.07 | 0.00 | False |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.IO.Compression.ZLib*'Details
Payloads
Histogram
System.IO.Compression.ZLib.Compress(level: Fastest, file: "TestDocument.pdf")
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.989092638888889 > 2.9355226614583327.
IsChangePoint: Marked as a change because one of 5/9/2022 6:32:22 AM, 8/2/2022 9:32:02 PM, 8/4/2022 5:17:15 AM falls between 7/26/2022 2:47:26 PM and 8/4/2022 5:17:15 AM.
IsRegressionStdDev: Marked as regression because -77.61854139659978 (T) = (0 -2990808.179372711) / Math.Sqrt((47550707.749999985 / (41)) + (19984158.008067783 / (4))) is less than -2.016692199226234 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (41) + (4) - 2, .025) and -0.06882161333651177 = (2798229.510008116 - 2990808.179372711) / 2798229.510008116 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.IO.Compression.ZLib.Compress(level: Fastest, file: "sum")
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 469.8932116596639 > 461.38697743055553.
IsChangePoint: Marked as a change because one of 6/15/2022 7:24:26 PM, 7/27/2022 2:14:42 PM, 8/2/2022 9:32:02 PM, 8/4/2022 5:17:15 AM falls between 7/26/2022 2:47:26 PM and 8/4/2022 5:17:15 AM.
IsRegressionStdDev: Marked as regression because -15.066854202865196 (T) = (0 -469237.5212257106) / Math.Sqrt((34829965.80911427 / (41)) + (5512974.956863488 / (4))) is less than -2.016692199226234 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (41) + (4) - 2, .025) and -0.050337598102365544 = (446749.2376484259 - 469237.5212257106) / 446749.2376484259 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Architecture | arm64 |
|---|---|
| OS | Windows 10.0.19041 |
| Baseline | 54c717a4ed822f46a23893479b8d4398596c041d |
| Compare | 1926f0939a855d6ef41593377c75ab4c31e9fcbb |
| Diff | Diff |
Regressions in System.IO.Compression.Deflate
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| Compress - Duration of single invocation | 2.75 ms | 2.94 ms | 1.07 | 0.00 | False | |||||
| Compress - Duration of single invocation | 424.17 μs | 458.61 μs | 1.08 | 0.00 | False |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.IO.Compression.Deflate*'Details
Payloads
Histogram
System.IO.Compression.Deflate.Compress(level: Fastest, file: "TestDocument.pdf")
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.944779722222223 > 2.8891870000000006.
IsChangePoint: Marked as a change because one of 5/9/2022 6:32:22 AM, 8/2/2022 9:32:02 PM, 8/4/2022 5:17:15 AM falls between 7/26/2022 2:47:26 PM and 8/4/2022 5:17:15 AM.
IsRegressionStdDev: Marked as regression because -91.72982329289043 (T) = (0 -2947826.1991567463) / Math.Sqrt((71882321.59658116 / (41)) + (11089664.504111752 / (4))) is less than -2.016692199226234 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (41) + (4) - 2, .025) and -0.07089152752181736 = (2752684.210676688 - 2947826.1991567463) / 2752684.210676688 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.IO.Compression.Deflate.Compress(level: Fastest, file: "sum")
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 458.6060171568628 > 445.7794805743244.
IsChangePoint: Marked as a change because one of 6/15/2022 7:24:26 PM, 7/27/2022 2:14:42 PM, 8/2/2022 9:32:02 PM, 8/4/2022 5:17:15 AM falls between 7/26/2022 2:47:26 PM and 8/4/2022 5:17:15 AM.
IsRegressionStdDev: Marked as regression because -14.913874638375276 (T) = (0 -458959.1882002802) / Math.Sqrt((28626618.235419188 / (40)) + (10684463.222928915 / (4))) is less than -2.0180817028167235 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (40) + (4) - 2, .025) and -0.06360486089072237 = (431512.8720039151 - 458959.1882002802) / 431512.8720039151 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Architecture | arm64 |
|---|---|
| OS | Windows 10.0.19041 |
| Baseline | 54c717a4ed822f46a23893479b8d4398596c041d |
| Compare | 1926f0939a855d6ef41593377c75ab4c31e9fcbb |
| Diff | Diff |
Regressions in System.Collections.CtorFromCollection<String>
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| ImmutableArray - Duration of single invocation | 468.98 ns | 495.24 ns | 1.06 | 0.00 | False |
_1.png)
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.CtorFromCollection<String>*'Details
Payloads
Histogram
System.Collections.CtorFromCollection<String>.ImmutableArray(Size: 512)
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 495.243094848904 > 487.6974805387518.
IsChangePoint: Marked as a change because one of 8/2/2022 9:32:02 PM, 8/4/2022 5:17:15 AM falls between 7/26/2022 2:47:26 PM and 8/4/2022 5:17:15 AM.
IsRegressionStdDev: Marked as regression because -18.059649380719744 (T) = (0 -498.1264953871379) / Math.Sqrt((27.368457010300922 / (41)) + (7.571560273277371 / (4))) is less than -2.016692199226234 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (41) + (4) - 2, .025) and -0.06158568436984263 = (469.22872333458963 - 498.1264953871379) / 469.22872333458963 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Activity
[-][Perf] Changes at 8/3/2022 1:34:55 AM[/-][+]Regressions in System.IO.Compression.Gzip[/+]10 remaining items
jeffhandley commentedon Aug 9, 2022
The regression here is beyond our control at the moment. However, I'm going to put this into the 8.0.0 milestone as we are going to be considering zlib options next year.
DrewScoggins commentedon Aug 9, 2022
Also regressed on
win-x64: dotnet/perf-autofiling-issues#7165
win-x86: dotnet/perf-autofiling-issues#7180
win-amd64: dotnet/perf-autofiling-issues#7203
carlossanlop commentedon Jul 20, 2023
We're not changing our zlib implementation at the moment, only taking updates on the native version. Moving to future as we are still exploring our options.
karelz commentedon Oct 13, 2025
Triage: We should resolve this in .NET 11, let's not carry it forward anymore.
We should compare current code base perf to the one from 2022/2. We can then decide if we can do anything about it - since the native implementation changed at least once since then.
rzikm commentedon Oct 15, 2025
Todays values have not changed much since the regression was reported, in some cases the gap was closed a bit, but not to the level of the values before the regression.
rzikm commentedon Oct 15, 2025
The regression seems to be caused by zlib update at the time (67f0ee5) we moved to zlib-ng since then, I don't think this issue is actionable