Repository navigation
Arm64: Regressions in System.IO.Compression.Brotli #69563
Description
Run Information
| Architecture | arm64 |
|---|---|
| OS | Windows 10.0.19041 |
| Baseline | 5ecaae992806a2d422157a095a63efb467365f54 |
| Compare | f63fdeefa535a1d2fd1855249f673f12e021774d |
| Diff | Diff |
Regressions in System.IO.Compression.Brotli
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| Compress_WithoutState - Duration of single invocation | 527.24 ms | 708.90 ms | 1.34 | 0.00 | False | |||||
| Compress_WithState - Duration of single invocation | 527.39 ms | 709.39 ms | 1.35 | 0.00 | False |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.IO.Compression.Brotli*'Details
Payloads
Histogram
System.IO.Compression.Brotli.Compress_WithoutState(level: Optimal, file: "TestDocument.pdf")
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 708.9047 > 553.57772625.
IsChangePoint: Marked as a change because one of 5/9/2022 6:32:22 AM, 5/17/2022 8:30:53 PM falls between 5/9/2022 6:32:22 AM and 5/17/2022 8:30:53 PM.
IsRegressionStdDev: Marked as regression because -61.40464591869179 (T) = (0 -705992985.7142857) / Math.Sqrt((29143437314.790493 / (10)) + (16956160163265.107 / (2))) is less than -2.2281388519818126 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (10) + (2) - 2, .025) and -0.33921446726888654 = (527169473.57509166 - 705992985.7142857) / 527169473.57509166 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.IO.Compression.Brotli.Compress_WithState(level: Optimal, file: "TestDocument.pdf")
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 709.3912933333334 > 553.6597018749999.
IsChangePoint: Marked as a change because one of 5/9/2022 6:32:22 AM, 5/17/2022 8:30:53 PM falls between 5/9/2022 6:32:22 AM and 5/17/2022 8:30:53 PM.
IsRegressionStdDev: Marked as regression because -60.62789474015853 (T) = (0 -706462682.3809524) / Math.Sqrt((1092207047380.6239 / (10)) + (17153524220811.371 / (2))) is less than -2.2281388519818126 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (10) + (2) - 2, .025) and -0.338554721803709 = (527780202.6868132 - 706462682.3809524) / 527780202.6868132 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
| Architecture | arm64 |
|---|---|
| OS | Windows 10.0.19041 |
| Baseline | 5ecaae992806a2d422157a095a63efb467365f54 |
| Compare | f63fdeefa535a1d2fd1855249f673f12e021774d |
| Diff | Diff |
Regressions in System.Tests.Perf_Single
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| IsNaN - Duration of single invocation | 742.22 μs | 795.28 μs | 1.07 | 0.00 | False | |||||
| IsNaN - Duration of single invocation | 742.14 μs | 795.23 μs | 1.07 | 0.00 | False | |||||
| ToStringWithFormat - Duration of single invocation | 202.15 ns | 216.19 ns | 1.07 | 0.01 | False |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_Single*'Details
Payloads
Histogram
System.Tests.Perf_Single.IsNaN(value: 0)
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 795.2829294217688 > 779.3603883928571.
IsChangePoint: Marked as a change because one of 5/9/2022 6:32:22 AM, 5/17/2022 8:30:53 PM falls between 5/9/2022 6:32:22 AM and 5/17/2022 8:30:53 PM.
IsRegressionStdDev: Marked as regression because -472.8005197662142 (T) = (0 -795187.1598639457) / Math.Sqrt((33485.02551849763 / (10)) + (18343.616411255676 / (2))) is less than -2.2281388519818126 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (10) + (2) - 2, .025) and -0.07127146355055547 = (742283.5265567766 - 795187.1598639457) / 742283.5265567766 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Tests.Perf_Single.IsNaN(value: NaN)
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 795.2336111111111 > 779.2831799450549.
IsChangePoint: Marked as a change because one of 5/9/2022 6:32:22 AM, 5/17/2022 8:30:53 PM falls between 5/9/2022 6:32:22 AM and 5/17/2022 8:30:53 PM.
IsRegressionStdDev: Marked as regression because -794.8996752949271 (T) = (0 -795290.4875283446) / Math.Sqrt((12236.421899764118 / (10)) + (6469.85367464983 / (2))) is less than -2.2281388519818126 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (10) + (2) - 2, .025) and -0.07151242238020486 = (742213.0354417409 - 795290.4875283446) / 742213.0354417409 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Tests.Perf_Single.ToStringWithFormat(value: -3.4028235E+38, format: "G17")
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 216.19012128700993 > 211.75857937478648.
IsChangePoint: Marked as a change because one of 1/27/2022 1:01:32 PM, 3/18/2022 3:36:38 PM, 5/9/2022 6:32:22 AM, 5/17/2022 8:30:53 PM falls between 5/9/2022 6:32:22 AM and 5/17/2022 8:30:53 PM.
IsRegressionStdDev: Marked as regression because -62.82174010256999 (T) = (0 -216.3522947489078) / Math.Sqrt((0.2727707275827177 / (10)) + (0.05260046348789496 / (2))) is less than -2.2281388519818126 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (10) + (2) - 2, .025) and -0.07205354686611726 = (201.81108992303612 - 216.3522947489078) / 201.81108992303612 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Architecture | arm64 |
|---|---|
| OS | Windows 10.0.19041 |
| Baseline | 5ecaae992806a2d422157a095a63efb467365f54 |
| Compare | f63fdeefa535a1d2fd1855249f673f12e021774d |
| Diff | Diff |
Regressions in System.Buffers.Text.Tests.Utf8ParserTests
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| TryParseByte - Duration of single invocation | 2.14 ns | 3.52 ns | 1.65 | 0.61 | False |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Buffers.Text.Tests.Utf8ParserTests*'Details
Payloads
Histogram
System.Buffers.Text.Tests.Utf8ParserTests.TryParseByte(value: 0)
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.5241708728008043 > 2.2094135851604833.
IsChangePoint: Marked as a change because one of 1/27/2022 1:01:32 PM, 1/28/2022 4:34:00 PM, 5/9/2022 6:32:22 AM, 5/17/2022 8:30:53 PM falls between 5/9/2022 6:32:22 AM and 5/17/2022 8:30:53 PM.
IsRegressionStdDev: Marked as regression because -20.208488163818785 (T) = (0 -3.478737380470318) / Math.Sqrt((0.024332820845502482 / (10)) + (0.00412840445068874 / (2))) is less than -2.2281388519818126 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (10) + (2) - 2, .025) and -0.6382171662459164 = (2.123489762008828 - 3.478737380470318) / 2.123489762008828 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Architecture | arm64 |
|---|---|
| OS | Windows 10.0.19041 |
| Baseline | 5ecaae992806a2d422157a095a63efb467365f54 |
| Compare | f63fdeefa535a1d2fd1855249f673f12e021774d |
| Diff | Diff |
Regressions in System.Threading.Tests.Perf_Monitor
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| EnterExit - Duration of single invocation | 15.40 ns | 20.03 ns | 1.30 | 0.14 | False | |||||
| TryEnterExit - Duration of single invocation | 18.07 ns | 20.60 ns | 1.14 | 0.19 | False |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Threading.Tests.Perf_Monitor*'Details
Payloads
Histogram
System.Threading.Tests.Perf_Monitor.EnterExit
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 20.03218778848863 > 16.549471216011902.
IsChangePoint: Marked as a change because one of 12/29/2021 4:01:07 PM, 2/28/2022 3:30:29 AM, 3/1/2022 7:31:17 PM, 5/9/2022 6:32:22 AM, 5/17/2022 8:30:53 PM falls between 5/9/2022 6:32:22 AM and 5/17/2022 8:30:53 PM.
IsRegressionStdDev: Marked as regression because -52.85668703306936 (T) = (0 -20.006069340974264) / Math.Sqrt((0.05839641698272925 / (10)) + (0.0013643466011216943 / (2))) is less than -2.2281388519818126 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (10) + (2) - 2, .025) and -0.2712369993563619 = (15.737481957419039 - 20.006069340974264) / 15.737481957419039 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Threading.Tests.Perf_Monitor.TryEnterExit
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 20.600065774291973 > 18.96711211126011.
IsChangePoint: Marked as a change because one of 5/9/2022 6:32:22 AM, 5/17/2022 8:30:53 PM falls between 5/9/2022 6:32:22 AM and 5/17/2022 8:30:53 PM.
IsRegressionStdDev: Marked as regression because -8.13391565310743 (T) = (0 -20.93326902435892) / Math.Sqrt((0.05387068934867149 / (10)) + (0.22204881171035618 / (2))) is less than -2.2281388519818126 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (10) + (2) - 2, .025) and -0.15283696247535825 = (18.15804810717662 - 20.93326902435892) / 18.15804810717662 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Architecture | arm64 |
|---|---|
| OS | Windows 10.0.19041 |
| Baseline | 5ecaae992806a2d422157a095a63efb467365f54 |
| Compare | f63fdeefa535a1d2fd1855249f673f12e021774d |
| Diff | Diff |
Regressions in System.Text.Encodings.Web.Tests.Perf_Encoders
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| EncodeUtf8 - Duration of single invocation | 48.98 ns | 52.25 ns | 1.07 | 0.06 | False |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Encodings.Web.Tests.Perf_Encoders*'Details
Payloads
Histogram
System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf8(arguments: Url,©2020,16)
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 52.24841658561437 > 51.36087699793563.
IsChangePoint: Marked as a change because one of 3/31/2022 1:23:38 AM, 5/9/2022 6:32:22 AM, 5/17/2022 8:30:53 PM falls between 5/9/2022 6:32:22 AM and 5/17/2022 8:30:53 PM.
IsRegressionStdDev: Marked as regression because -26.578904586123684 (T) = (0 -52.21156747774739) / Math.Sqrt((0.1528501608413665 / (10)) + (0.0027157135011853634 / (2))) is less than -2.2281388519818126 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (10) + (2) - 2, .025) and -0.07028868666923811 = (48.782695853985835 - 52.21156747774739) / 48.782695853985835 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Architecture | arm64 |
|---|---|
| OS | Windows 10.0.19041 |
| Baseline | 5ecaae992806a2d422157a095a63efb467365f54 |
| Compare | f63fdeefa535a1d2fd1855249f673f12e021774d |
| Diff | Diff |
Regressions in Benchstone.BenchI.IniArray
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| Test - Duration of single invocation | 90.94 ms | 107.57 ms | 1.18 | 0.00 | False |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'Benchstone.BenchI.IniArray*'Details
Payloads
Histogram
Benchstone.BenchI.IniArray.Test
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 107.57303 > 95.57885137500001.
IsChangePoint: Marked as a change because one of 3/22/2022 12:26:23 PM, 5/9/2022 6:32:22 AM, 5/17/2022 8:30:53 PM falls between 5/9/2022 6:32:22 AM and 5/17/2022 8:30:53 PM.
IsRegressionStdDev: Marked as regression because -654.7580431819277 (T) = (0 -107575472.5) / Math.Sqrt((6338076669.629628 / (10)) + (11931612.5 / (2))) is less than -2.2281388519818126 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (10) + (2) - 2, .025) and -0.18196373670976004 = (91014190.33333334 - 107575472.5) / 91014190.33333334 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Architecture | arm64 |
|---|---|
| OS | Windows 10.0.19041 |
| Baseline | 5ecaae992806a2d422157a095a63efb467365f54 |
| Compare | f63fdeefa535a1d2fd1855249f673f12e021774d |
| Diff | Diff |
Regressions in System.Collections.CtorFromCollectionNonGeneric<Int32>
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| ArrayList - Duration of single invocation | 16.46 μs | 17.98 μs | 1.09 | 0.05 | False |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.CtorFromCollectionNonGeneric<Int32>*'Details
Payloads
Histogram
System.Collections.CtorFromCollectionNonGeneric<Int32>.ArrayList(Size: 512)
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 17.9770497698504 > 17.279166106797845.
IsChangePoint: Marked as a change because one of 1/7/2022 4:24:34 PM, 1/20/2022 4:24:09 PM, 3/22/2022 5:45:59 AM, 5/9/2022 6:32:22 AM, 5/17/2022 8:30:53 PM falls between 5/9/2022 6:32:22 AM and 5/17/2022 8:30:53 PM.
IsRegressionStdDev: Marked as regression because -9.64621003685597 (T) = (0 -17837.213972903963) / Math.Sqrt((2752.1729363807885 / (10)) + (39108.10021528973 / (2))) is less than -2.2281388519818126 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (10) + (2) - 2, .025) and -0.08242949607674517 = (16478.86909729896 - 17837.213972903963) / 16478.86909729896 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Architecture | arm64 |
|---|---|
| OS | Windows 10.0.19041 |
| Baseline | 5ecaae992806a2d422157a095a63efb467365f54 |
| Compare | f63fdeefa535a1d2fd1855249f673f12e021774d |
| Diff | Diff |
Regressions in System.Tests.Perf_Double
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| IsNaN - Duration of single invocation | 742.71 μs | 794.95 μs | 1.07 | 0.00 | False | |||||
| IsNaN - Duration of single invocation | 742.23 μs | 795.23 μs | 1.07 | 0.00 | False |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_Double*'Details
Payloads
Histogram
System.Tests.Perf_Double.IsNaN(value: NaN)
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 794.954693223443 > 779.4533255208333.
IsChangePoint: Marked as a change because one of 5/9/2022 6:32:22 AM, 5/17/2022 8:30:53 PM falls between 5/9/2022 6:32:22 AM and 5/17/2022 8:30:53 PM.
IsRegressionStdDev: Marked as regression because -271.23505385931946 (T) = (0 -795135.0421899529) / Math.Sqrt((53571.0886143715 / (10)) + (65051.49944234955 / (2))) is less than -2.2281388519818126 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (10) + (2) - 2, .025) and -0.07111520711372005 = (742343.1549744897 - 795135.0421899529) / 742343.1549744897 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Tests.Perf_Double.IsNaN(value: 0)
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 795.2322172619049 > 779.3736834935899.
IsChangePoint: Marked as a change because one of 5/9/2022 6:32:22 AM, 5/17/2022 8:30:53 PM falls between 5/9/2022 6:32:22 AM and 5/17/2022 8:30:53 PM.
IsRegressionStdDev: Marked as regression because -1182.3355422442517 (T) = (0 -795191.9642857143) / Math.Sqrt((3863.9117428603486 / (10)) + (3240.604184391012 / (2))) is less than -2.2281388519818126 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (10) + (2) - 2, .025) and -0.07135821242903778 = (742227.9075854701 - 795191.9642857143) / 742227.9075854701 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Architecture | arm64 |
|---|---|
| OS | Windows 10.0.19041 |
| Baseline | 5ecaae992806a2d422157a095a63efb467365f54 |
| Compare | f63fdeefa535a1d2fd1855249f673f12e021774d |
| Diff | Diff |
Regressions in System.Globalization.Tests.StringSearch
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| IsSuffix_SecondHalf - Duration of single invocation | 11.95 μs | 13.06 μs | 1.09 | 0.05 | False | |||||
| IsSuffix_SecondHalf - Duration of single invocation | 12.12 μs | 13.13 μs | 1.08 | 0.04 | False | |||||
| IsSuffix_SecondHalf - Duration of single invocation | 12.12 μs | 13.06 μs | 1.08 | 0.05 | False | |||||
| IsSuffix_SecondHalf - Duration of single invocation | 12.09 μs | 13.07 μs | 1.08 | 0.04 | False | |||||
| IsSuffix_SecondHalf - Duration of single invocation | 13.41 μs | 14.51 μs | 1.08 | 0.04 | False |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Globalization.Tests.StringSearch*'Details
Payloads
Histogram
System.Globalization.Tests.StringSearch.IsSuffix_SecondHalf(Options: (en-US, None, True))
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 13.061121446488292 > 12.707290198686099.
IsChangePoint: Marked as a change because one of 1/28/2022 8:15:35 PM, 2/7/2022 4:59:24 AM, 5/9/2022 6:32:22 AM, 5/17/2022 8:30:53 PM falls between 5/9/2022 6:32:22 AM and 5/17/2022 8:30:53 PM.
IsRegressionStdDev: Marked as regression because -7.2225287652392325 (T) = (0 -12947.520896315827) / Math.Sqrt((11065.322537310944 / (10)) + (25810.169998973415 / (2))) is less than -2.2281388519818126 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (10) + (2) - 2, .025) and -0.07069916007497908 = (12092.585274288565 - 12947.520896315827) / 12092.585274288565 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Globalization.Tests.StringSearch.IsSuffix_SecondHalf(Options: (, IgnoreCase, True))
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 13.13423675390999 > 12.717306678795344.
IsChangePoint: Marked as a change because one of 2/2/2022 1:56:18 PM, 2/7/2022 4:59:24 AM, 5/9/2022 6:32:22 AM, 5/17/2022 8:30:53 PM falls between 5/9/2022 6:32:22 AM and 5/17/2022 8:30:53 PM.
IsRegressionStdDev: Marked as regression because -14.942246818466865 (T) = (0 -13076.128484390676) / Math.Sqrt((11267.905379460586 / (10)) + (6753.141973058799 / (2))) is less than -2.2281388519818126 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (10) + (2) - 2, .025) and -0.08305290024141689 = (12073.397782763846 - 13076.128484390676) / 12073.397782763846 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Globalization.Tests.StringSearch.IsSuffix_SecondHalf(Options: (en-US, IgnoreCase, True))
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 13.057671404682274 > 12.707167318511598.
IsChangePoint: Marked as a change because one of 2/1/2022 9:56:54 PM, 2/7/2022 4:59:24 AM, 5/9/2022 6:32:22 AM, 5/17/2022 8:30:53 PM falls between 5/9/2022 6:32:22 AM and 5/17/2022 8:30:53 PM.
IsRegressionStdDev: Marked as regression because -5.655853765723982 (T) = (0 -12913.983522244962) / Math.Sqrt((11539.343217617832 / (10)) + (41292.41511863788 / (2))) is less than -2.2281388519818126 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (10) + (2) - 2, .025) and -0.06913537174072121 = (12078.904003726822 - 12913.983522244962) / 12078.904003726822 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Globalization.Tests.StringSearch.IsSuffix_SecondHalf(Options: (, None, True))
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 13.073867057724252 > 12.708649096441212.
IsChangePoint: Marked as a change because one of 1/28/2022 8:15:35 PM, 2/7/2022 4:59:24 AM, 5/9/2022 6:32:22 AM, 5/17/2022 8:30:53 PM falls between 5/9/2022 6:32:22 AM and 5/17/2022 8:30:53 PM.
IsRegressionStdDev: Marked as regression because -65.0992318671322 (T) = (0 -13063.48794545888) / Math.Sqrt((1080.898521710222 / (10)) + (215.45194283431493 / (2))) is less than -2.2281388519818126 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (10) + (2) - 2, .025) and -0.07899071889005133 = (12107.136527455195 - 13063.48794545888) / 12107.136527455195 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Globalization.Tests.StringSearch.IsSuffix_SecondHalf(Options: (pl-PL, None, False))
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 14.510119458695078 > 14.061413924441476.
IsChangePoint: Marked as a change because one of 2/2/2022 1:56:18 PM, 2/7/2022 4:59:24 AM, 2/17/2022 1:06:33 PM, 2/18/2022 3:29:36 AM, 5/9/2022 6:32:22 AM, 5/17/2022 8:30:53 PM falls between 5/9/2022 6:32:22 AM and 5/17/2022 8:30:53 PM.
IsRegressionStdDev: Marked as regression because -10.556975694180354 (T) = (0 -14425.861774524585) / Math.Sqrt((17574.650586370746 / (10)) + (14198.71468354944 / (2))) is less than -2.2281388519818126 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (10) + (2) - 2, .025) and -0.07396510786878999 = (13432.337483618734 - 14425.861774524585) / 13432.337483618734 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Architecture | arm64 |
|---|---|
| OS | Windows 10.0.19041 |
| Baseline | 5ecaae992806a2d422157a095a63efb467365f54 |
| Compare | f63fdeefa535a1d2fd1855249f673f12e021774d |
| Diff | Diff |
Regressions in System.Collections.ContainsKeyFalse<String, String>
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| ConcurrentDictionary - Duration of single invocation | 20.71 μs | 21.87 μs | 1.06 | 0.01 | False |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.ContainsKeyFalse<String, String>*'Details
Payloads
Histogram
System.Collections.ContainsKeyFalse<String, String>.ConcurrentDictionary(Size: 512)
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 21.86824113386613 > 21.772439326199553.
IsChangePoint: Marked as a change because one of 3/7/2022 10:45:01 PM, 4/12/2022 3:59:36 PM, 5/9/2022 6:32:22 AM, 5/17/2022 8:30:53 PM falls between 5/9/2022 6:32:22 AM and 5/17/2022 8:30:53 PM.
IsRegressionStdDev: Marked as regression because -8.246592231914601 (T) = (0 -21979.834388934956) / Math.Sqrt((55821.57276758178 / (10)) + (24906.109153710804 / (2))) is less than -2.2281388519818126 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (10) + (2) - 2, .025) and -0.053059562542777834 = (20872.35629470112 - 21979.834388934956) / 20872.35629470112 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository










_1.png)







_1.png)
Activity
performanceautofiler commentedon May 19, 2022
Run Information
Regressions in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Mariomkas
Test Report
Repro
Details
Payloads
Baseline
Compare
Histogram
System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Mariomkas.Count(Pattern: "(?:(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]).){3}(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9])", Options: None)
Description of detection logic
Description of detection logic
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Regressions in Benchstone.BenchI.XposMatrix
Test Report
Repro
Details
Payloads
Baseline
Compare
Histogram
Benchstone.BenchI.XposMatrix.Test
Description of detection logic
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Regressions in System.Threading.Channels.Tests.SpscUnboundedChannelPerfTests
Test Report
Repro
Details
Payloads
Baseline
Compare
Histogram
System.Threading.Channels.Tests.SpscUnboundedChannelPerfTests.ReadAsyncThenWriteAsync
Description of detection logic
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Regressions in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig
Test Report
Repro
Details
Payloads
Baseline
Compare
Histogram
System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern: "\p{Sm}", Options: None)
Description of detection logic
Description of detection logic
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Regressions in System.Memory.Span<Int32>
Test Report
Repro
Details
Payloads
Baseline
Compare
Histogram
System.Memory.Span<Int32>.IndexOfAnyFourValues(Size: 512)
Description of detection logic
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Regressions in System.Collections.Concurrent.Count<Int32>
Test Report
Repro
Details
Payloads
Baseline
Compare
Histogram
System.Collections.Concurrent.Count<Int32>.Dictionary(Size: 512)
Description of detection logic
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Regressions in System.Memory.Span<Byte>
Test Report
Repro
Details
Payloads
Baseline
Compare
Histogram
System.Memory.Span<Byte>.IndexOfAnyFourValues(Size: 512)
Description of detection logic
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Regressions in System.Threading.Channels.Tests.BoundedChannelPerfTests
Test Report
Repro
Details
Payloads
Baseline
Compare
Histogram
System.Threading.Channels.Tests.BoundedChannelPerfTests.TryWriteThenTryRead
Description of detection logic
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
kunalspathak commentedon May 19, 2022
There was an outage between 5/11 and 5/17 and we don't have enough data to track down. Next action would be to pick one of this benchmark and manually verify if the regression repros and then track down the commit.
19 remaining items
kunalspathak commentedon Jun 30, 2022
@jeffhandley, I was not able to locate a commit 3e5517b...6506705 that would affect
System.IO.Compression.Brotli. Could you please double check?jeffhandley commentedon Aug 10, 2022
We updated the default compression level a while back. @adamsitnik is going to take a look.
adamsitnik commentedon Aug 12, 2022
The regression is gone:
In the commits from the "first drop" I can see two JIT changes: #69865 and #70792
In the second drop there were two JIT/PGO changes: #72842 and #72773