Skip to content

Arm64: Regressions in System.IO.Compression.Brotli #69563

Closed

Description

@performanceautofiler

Run Information

Architecture arm64
OS Windows 10.0.19041
Baseline 5ecaae992806a2d422157a095a63efb467365f54
Compare f63fdeefa535a1d2fd1855249f673f12e021774d
Diff Diff

Regressions in System.IO.Compression.Brotli

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Compress_WithoutState - Duration of single invocation 527.24 ms 708.90 ms 1.34 0.00 False
Compress_WithState - Duration of single invocation 527.39 ms 709.39 ms 1.35 0.00 False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.IO.Compression.Brotli*'
Details

Payloads

Baseline
Compare

Histogram

System.IO.Compression.Brotli.Compress_WithoutState(level: Optimal, file: "TestDocument.pdf")


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 708.9047 > 553.57772625.
IsChangePoint: Marked as a change because one of 5/9/2022 6:32:22 AM, 5/17/2022 8:30:53 PM falls between 5/9/2022 6:32:22 AM and 5/17/2022 8:30:53 PM.
IsRegressionStdDev: Marked as regression because -61.40464591869179 (T) = (0 -705992985.7142857) / Math.Sqrt((29143437314.790493 / (10)) + (16956160163265.107 / (2))) is less than -2.2281388519818126 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (10) + (2) - 2, .025) and -0.33921446726888654 = (527169473.57509166 - 705992985.7142857) / 527169473.57509166 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Compression.Brotli.Compress_WithState(level: Optimal, file: "TestDocument.pdf")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 709.3912933333334 > 553.6597018749999.
IsChangePoint: Marked as a change because one of 5/9/2022 6:32:22 AM, 5/17/2022 8:30:53 PM falls between 5/9/2022 6:32:22 AM and 5/17/2022 8:30:53 PM.
IsRegressionStdDev: Marked as regression because -60.62789474015853 (T) = (0 -706462682.3809524) / Math.Sqrt((1092207047380.6239 / (10)) + (17153524220811.371 / (2))) is less than -2.2281388519818126 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (10) + (2) - 2, .025) and -0.338554721803709 = (527780202.6868132 - 706462682.3809524) / 527780202.6868132 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture arm64
OS Windows 10.0.19041
Baseline 5ecaae992806a2d422157a095a63efb467365f54
Compare f63fdeefa535a1d2fd1855249f673f12e021774d
Diff Diff

Regressions in System.Tests.Perf_Single

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
IsNaN - Duration of single invocation 742.22 μs 795.28 μs 1.07 0.00 False
IsNaN - Duration of single invocation 742.14 μs 795.23 μs 1.07 0.00 False
ToStringWithFormat - Duration of single invocation 202.15 ns 216.19 ns 1.07 0.01 False

graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_Single*'
Details

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Single.IsNaN(value: 0)


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 795.2829294217688 > 779.3603883928571.
IsChangePoint: Marked as a change because one of 5/9/2022 6:32:22 AM, 5/17/2022 8:30:53 PM falls between 5/9/2022 6:32:22 AM and 5/17/2022 8:30:53 PM.
IsRegressionStdDev: Marked as regression because -472.8005197662142 (T) = (0 -795187.1598639457) / Math.Sqrt((33485.02551849763 / (10)) + (18343.616411255676 / (2))) is less than -2.2281388519818126 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (10) + (2) - 2, .025) and -0.07127146355055547 = (742283.5265567766 - 795187.1598639457) / 742283.5265567766 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Single.IsNaN(value: NaN)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 795.2336111111111 > 779.2831799450549.
IsChangePoint: Marked as a change because one of 5/9/2022 6:32:22 AM, 5/17/2022 8:30:53 PM falls between 5/9/2022 6:32:22 AM and 5/17/2022 8:30:53 PM.
IsRegressionStdDev: Marked as regression because -794.8996752949271 (T) = (0 -795290.4875283446) / Math.Sqrt((12236.421899764118 / (10)) + (6469.85367464983 / (2))) is less than -2.2281388519818126 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (10) + (2) - 2, .025) and -0.07151242238020486 = (742213.0354417409 - 795290.4875283446) / 742213.0354417409 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Single.ToStringWithFormat(value: -3.4028235E+38, format: "G17")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 216.19012128700993 > 211.75857937478648.
IsChangePoint: Marked as a change because one of 1/27/2022 1:01:32 PM, 3/18/2022 3:36:38 PM, 5/9/2022 6:32:22 AM, 5/17/2022 8:30:53 PM falls between 5/9/2022 6:32:22 AM and 5/17/2022 8:30:53 PM.
IsRegressionStdDev: Marked as regression because -62.82174010256999 (T) = (0 -216.3522947489078) / Math.Sqrt((0.2727707275827177 / (10)) + (0.05260046348789496 / (2))) is less than -2.2281388519818126 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (10) + (2) - 2, .025) and -0.07205354686611726 = (201.81108992303612 - 216.3522947489078) / 201.81108992303612 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS Windows 10.0.19041
Baseline 5ecaae992806a2d422157a095a63efb467365f54
Compare f63fdeefa535a1d2fd1855249f673f12e021774d
Diff Diff

Regressions in System.Buffers.Text.Tests.Utf8ParserTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
TryParseByte - Duration of single invocation 2.14 ns 3.52 ns 1.65 0.61 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Buffers.Text.Tests.Utf8ParserTests*'
Details

Payloads

Baseline
Compare

Histogram

System.Buffers.Text.Tests.Utf8ParserTests.TryParseByte(value: 0)


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.5241708728008043 > 2.2094135851604833.
IsChangePoint: Marked as a change because one of 1/27/2022 1:01:32 PM, 1/28/2022 4:34:00 PM, 5/9/2022 6:32:22 AM, 5/17/2022 8:30:53 PM falls between 5/9/2022 6:32:22 AM and 5/17/2022 8:30:53 PM.
IsRegressionStdDev: Marked as regression because -20.208488163818785 (T) = (0 -3.478737380470318) / Math.Sqrt((0.024332820845502482 / (10)) + (0.00412840445068874 / (2))) is less than -2.2281388519818126 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (10) + (2) - 2, .025) and -0.6382171662459164 = (2.123489762008828 - 3.478737380470318) / 2.123489762008828 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS Windows 10.0.19041
Baseline 5ecaae992806a2d422157a095a63efb467365f54
Compare f63fdeefa535a1d2fd1855249f673f12e021774d
Diff Diff

Regressions in System.Threading.Tests.Perf_Monitor

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
EnterExit - Duration of single invocation 15.40 ns 20.03 ns 1.30 0.14 False
TryEnterExit - Duration of single invocation 18.07 ns 20.60 ns 1.14 0.19 False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Threading.Tests.Perf_Monitor*'
Details

Payloads

Baseline
Compare

Histogram

System.Threading.Tests.Perf_Monitor.EnterExit


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 20.03218778848863 > 16.549471216011902.
IsChangePoint: Marked as a change because one of 12/29/2021 4:01:07 PM, 2/28/2022 3:30:29 AM, 3/1/2022 7:31:17 PM, 5/9/2022 6:32:22 AM, 5/17/2022 8:30:53 PM falls between 5/9/2022 6:32:22 AM and 5/17/2022 8:30:53 PM.
IsRegressionStdDev: Marked as regression because -52.85668703306936 (T) = (0 -20.006069340974264) / Math.Sqrt((0.05839641698272925 / (10)) + (0.0013643466011216943 / (2))) is less than -2.2281388519818126 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (10) + (2) - 2, .025) and -0.2712369993563619 = (15.737481957419039 - 20.006069340974264) / 15.737481957419039 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Threading.Tests.Perf_Monitor.TryEnterExit

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 20.600065774291973 > 18.96711211126011.
IsChangePoint: Marked as a change because one of 5/9/2022 6:32:22 AM, 5/17/2022 8:30:53 PM falls between 5/9/2022 6:32:22 AM and 5/17/2022 8:30:53 PM.
IsRegressionStdDev: Marked as regression because -8.13391565310743 (T) = (0 -20.93326902435892) / Math.Sqrt((0.05387068934867149 / (10)) + (0.22204881171035618 / (2))) is less than -2.2281388519818126 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (10) + (2) - 2, .025) and -0.15283696247535825 = (18.15804810717662 - 20.93326902435892) / 18.15804810717662 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS Windows 10.0.19041
Baseline 5ecaae992806a2d422157a095a63efb467365f54
Compare f63fdeefa535a1d2fd1855249f673f12e021774d
Diff Diff

Regressions in System.Text.Encodings.Web.Tests.Perf_Encoders

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
EncodeUtf8 - Duration of single invocation 48.98 ns 52.25 ns 1.07 0.06 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Encodings.Web.Tests.Perf_Encoders*'
Details

Payloads

Baseline
Compare

Histogram

System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf8(arguments: Url,©2020,16)


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 52.24841658561437 > 51.36087699793563.
IsChangePoint: Marked as a change because one of 3/31/2022 1:23:38 AM, 5/9/2022 6:32:22 AM, 5/17/2022 8:30:53 PM falls between 5/9/2022 6:32:22 AM and 5/17/2022 8:30:53 PM.
IsRegressionStdDev: Marked as regression because -26.578904586123684 (T) = (0 -52.21156747774739) / Math.Sqrt((0.1528501608413665 / (10)) + (0.0027157135011853634 / (2))) is less than -2.2281388519818126 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (10) + (2) - 2, .025) and -0.07028868666923811 = (48.782695853985835 - 52.21156747774739) / 48.782695853985835 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS Windows 10.0.19041
Baseline 5ecaae992806a2d422157a095a63efb467365f54
Compare f63fdeefa535a1d2fd1855249f673f12e021774d
Diff Diff

Regressions in Benchstone.BenchI.IniArray

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Test - Duration of single invocation 90.94 ms 107.57 ms 1.18 0.00 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'Benchstone.BenchI.IniArray*'
Details

Payloads

Baseline
Compare

Histogram

Benchstone.BenchI.IniArray.Test


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 107.57303 > 95.57885137500001.
IsChangePoint: Marked as a change because one of 3/22/2022 12:26:23 PM, 5/9/2022 6:32:22 AM, 5/17/2022 8:30:53 PM falls between 5/9/2022 6:32:22 AM and 5/17/2022 8:30:53 PM.
IsRegressionStdDev: Marked as regression because -654.7580431819277 (T) = (0 -107575472.5) / Math.Sqrt((6338076669.629628 / (10)) + (11931612.5 / (2))) is less than -2.2281388519818126 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (10) + (2) - 2, .025) and -0.18196373670976004 = (91014190.33333334 - 107575472.5) / 91014190.33333334 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS Windows 10.0.19041
Baseline 5ecaae992806a2d422157a095a63efb467365f54
Compare f63fdeefa535a1d2fd1855249f673f12e021774d
Diff Diff

Regressions in System.Collections.CtorFromCollectionNonGeneric<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ArrayList - Duration of single invocation 16.46 μs 17.98 μs 1.09 0.05 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.CtorFromCollectionNonGeneric&lt;Int32&gt;*'
Details

Payloads

Baseline
Compare

Histogram

System.Collections.CtorFromCollectionNonGeneric<Int32>.ArrayList(Size: 512)


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 17.9770497698504 > 17.279166106797845.
IsChangePoint: Marked as a change because one of 1/7/2022 4:24:34 PM, 1/20/2022 4:24:09 PM, 3/22/2022 5:45:59 AM, 5/9/2022 6:32:22 AM, 5/17/2022 8:30:53 PM falls between 5/9/2022 6:32:22 AM and 5/17/2022 8:30:53 PM.
IsRegressionStdDev: Marked as regression because -9.64621003685597 (T) = (0 -17837.213972903963) / Math.Sqrt((2752.1729363807885 / (10)) + (39108.10021528973 / (2))) is less than -2.2281388519818126 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (10) + (2) - 2, .025) and -0.08242949607674517 = (16478.86909729896 - 17837.213972903963) / 16478.86909729896 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS Windows 10.0.19041
Baseline 5ecaae992806a2d422157a095a63efb467365f54
Compare f63fdeefa535a1d2fd1855249f673f12e021774d
Diff Diff

Regressions in System.Tests.Perf_Double

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
IsNaN - Duration of single invocation 742.71 μs 794.95 μs 1.07 0.00 False
IsNaN - Duration of single invocation 742.23 μs 795.23 μs 1.07 0.00 False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_Double*'
Details

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Double.IsNaN(value: NaN)


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 794.954693223443 > 779.4533255208333.
IsChangePoint: Marked as a change because one of 5/9/2022 6:32:22 AM, 5/17/2022 8:30:53 PM falls between 5/9/2022 6:32:22 AM and 5/17/2022 8:30:53 PM.
IsRegressionStdDev: Marked as regression because -271.23505385931946 (T) = (0 -795135.0421899529) / Math.Sqrt((53571.0886143715 / (10)) + (65051.49944234955 / (2))) is less than -2.2281388519818126 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (10) + (2) - 2, .025) and -0.07111520711372005 = (742343.1549744897 - 795135.0421899529) / 742343.1549744897 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Double.IsNaN(value: 0)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 795.2322172619049 > 779.3736834935899.
IsChangePoint: Marked as a change because one of 5/9/2022 6:32:22 AM, 5/17/2022 8:30:53 PM falls between 5/9/2022 6:32:22 AM and 5/17/2022 8:30:53 PM.
IsRegressionStdDev: Marked as regression because -1182.3355422442517 (T) = (0 -795191.9642857143) / Math.Sqrt((3863.9117428603486 / (10)) + (3240.604184391012 / (2))) is less than -2.2281388519818126 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (10) + (2) - 2, .025) and -0.07135821242903778 = (742227.9075854701 - 795191.9642857143) / 742227.9075854701 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS Windows 10.0.19041
Baseline 5ecaae992806a2d422157a095a63efb467365f54
Compare f63fdeefa535a1d2fd1855249f673f12e021774d
Diff Diff

Regressions in System.Globalization.Tests.StringSearch

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
IsSuffix_SecondHalf - Duration of single invocation 11.95 μs 13.06 μs 1.09 0.05 False
IsSuffix_SecondHalf - Duration of single invocation 12.12 μs 13.13 μs 1.08 0.04 False
IsSuffix_SecondHalf - Duration of single invocation 12.12 μs 13.06 μs 1.08 0.05 False
IsSuffix_SecondHalf - Duration of single invocation 12.09 μs 13.07 μs 1.08 0.04 False
IsSuffix_SecondHalf - Duration of single invocation 13.41 μs 14.51 μs 1.08 0.04 False

graph
graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Globalization.Tests.StringSearch*'
Details

Payloads

Baseline
Compare

Histogram

System.Globalization.Tests.StringSearch.IsSuffix_SecondHalf(Options: (en-US, None, True))


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 13.061121446488292 > 12.707290198686099.
IsChangePoint: Marked as a change because one of 1/28/2022 8:15:35 PM, 2/7/2022 4:59:24 AM, 5/9/2022 6:32:22 AM, 5/17/2022 8:30:53 PM falls between 5/9/2022 6:32:22 AM and 5/17/2022 8:30:53 PM.
IsRegressionStdDev: Marked as regression because -7.2225287652392325 (T) = (0 -12947.520896315827) / Math.Sqrt((11065.322537310944 / (10)) + (25810.169998973415 / (2))) is less than -2.2281388519818126 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (10) + (2) - 2, .025) and -0.07069916007497908 = (12092.585274288565 - 12947.520896315827) / 12092.585274288565 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Globalization.Tests.StringSearch.IsSuffix_SecondHalf(Options: (, IgnoreCase, True))

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 13.13423675390999 > 12.717306678795344.
IsChangePoint: Marked as a change because one of 2/2/2022 1:56:18 PM, 2/7/2022 4:59:24 AM, 5/9/2022 6:32:22 AM, 5/17/2022 8:30:53 PM falls between 5/9/2022 6:32:22 AM and 5/17/2022 8:30:53 PM.
IsRegressionStdDev: Marked as regression because -14.942246818466865 (T) = (0 -13076.128484390676) / Math.Sqrt((11267.905379460586 / (10)) + (6753.141973058799 / (2))) is less than -2.2281388519818126 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (10) + (2) - 2, .025) and -0.08305290024141689 = (12073.397782763846 - 13076.128484390676) / 12073.397782763846 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Globalization.Tests.StringSearch.IsSuffix_SecondHalf(Options: (en-US, IgnoreCase, True))

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 13.057671404682274 > 12.707167318511598.
IsChangePoint: Marked as a change because one of 2/1/2022 9:56:54 PM, 2/7/2022 4:59:24 AM, 5/9/2022 6:32:22 AM, 5/17/2022 8:30:53 PM falls between 5/9/2022 6:32:22 AM and 5/17/2022 8:30:53 PM.
IsRegressionStdDev: Marked as regression because -5.655853765723982 (T) = (0 -12913.983522244962) / Math.Sqrt((11539.343217617832 / (10)) + (41292.41511863788 / (2))) is less than -2.2281388519818126 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (10) + (2) - 2, .025) and -0.06913537174072121 = (12078.904003726822 - 12913.983522244962) / 12078.904003726822 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Globalization.Tests.StringSearch.IsSuffix_SecondHalf(Options: (, None, True))

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 13.073867057724252 > 12.708649096441212.
IsChangePoint: Marked as a change because one of 1/28/2022 8:15:35 PM, 2/7/2022 4:59:24 AM, 5/9/2022 6:32:22 AM, 5/17/2022 8:30:53 PM falls between 5/9/2022 6:32:22 AM and 5/17/2022 8:30:53 PM.
IsRegressionStdDev: Marked as regression because -65.0992318671322 (T) = (0 -13063.48794545888) / Math.Sqrt((1080.898521710222 / (10)) + (215.45194283431493 / (2))) is less than -2.2281388519818126 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (10) + (2) - 2, .025) and -0.07899071889005133 = (12107.136527455195 - 13063.48794545888) / 12107.136527455195 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Globalization.Tests.StringSearch.IsSuffix_SecondHalf(Options: (pl-PL, None, False))

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 14.510119458695078 > 14.061413924441476.
IsChangePoint: Marked as a change because one of 2/2/2022 1:56:18 PM, 2/7/2022 4:59:24 AM, 2/17/2022 1:06:33 PM, 2/18/2022 3:29:36 AM, 5/9/2022 6:32:22 AM, 5/17/2022 8:30:53 PM falls between 5/9/2022 6:32:22 AM and 5/17/2022 8:30:53 PM.
IsRegressionStdDev: Marked as regression because -10.556975694180354 (T) = (0 -14425.861774524585) / Math.Sqrt((17574.650586370746 / (10)) + (14198.71468354944 / (2))) is less than -2.2281388519818126 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (10) + (2) - 2, .025) and -0.07396510786878999 = (13432.337483618734 - 14425.861774524585) / 13432.337483618734 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS Windows 10.0.19041
Baseline 5ecaae992806a2d422157a095a63efb467365f54
Compare f63fdeefa535a1d2fd1855249f673f12e021774d
Diff Diff

Regressions in System.Collections.ContainsKeyFalse<String, String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ConcurrentDictionary - Duration of single invocation 20.71 μs 21.87 μs 1.06 0.01 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.ContainsKeyFalse&lt;String, String&gt;*'
Details

Payloads

Baseline
Compare

Histogram

System.Collections.ContainsKeyFalse<String, String>.ConcurrentDictionary(Size: 512)


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 21.86824113386613 > 21.772439326199553.
IsChangePoint: Marked as a change because one of 3/7/2022 10:45:01 PM, 4/12/2022 3:59:36 PM, 5/9/2022 6:32:22 AM, 5/17/2022 8:30:53 PM falls between 5/9/2022 6:32:22 AM and 5/17/2022 8:30:53 PM.
IsRegressionStdDev: Marked as regression because -8.246592231914601 (T) = (0 -21979.834388934956) / Math.Sqrt((55821.57276758178 / (10)) + (24906.109153710804 / (2))) is less than -2.2281388519818126 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (10) + (2) - 2, .025) and -0.053059562542777834 = (20872.35629470112 - 21979.834388934956) / 20872.35629470112 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Activity

  1. performanceautofiler commented on May 19, 2022

    @performanceautofiler
    Author

    Run Information

    Architecture arm64
    OS Windows 10.0.19041
    Baseline 5ecaae992806a2d422157a095a63efb467365f54
    Compare f63fdeefa535a1d2fd1855249f673f12e021774d
    Diff Diff

    Regressions in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Mariomkas

    Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
    Count - Duration of single invocation 30.93 ms 33.78 ms 1.09 0.00 False
    Count - Duration of single invocation 20.17 ms 23.39 ms 1.16 0.01 False

    graph
    graph
    Test Report

    Repro

    git clone https://github.com/dotnet/performance.git
    py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Mariomkas*'
    Details

    Payloads

    Baseline
    Compare

    Histogram

    System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Mariomkas.Count(Pattern: "(?:(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]).){3}(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9])", Options: None)

    
    

    Description of detection logic

    IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
    IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
    IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
    IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
    IsRegressionWindowed: Marked as regression because 33.77565 > 32.21087275.
    IsChangePoint: Marked as a change because one of 1/11/2022 6:41:22 PM, 2/3/2022 8:57:20 PM, 4/6/2022 12:51:22 AM, 4/16/2022 8:46:33 PM, 5/9/2022 6:32:22 AM, 5/17/2022 8:30:53 PM falls between 5/9/2022 6:32:22 AM and 5/17/2022 8:30:53 PM.
    IsRegressionStdDev: Marked as regression because -106.86540606115163 (T) = (0 -33771555.83333333) / Math.Sqrt((8127602526.697591 / (10)) + (33524401.388909224 / (2))) is less than -2.2281388519818126 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (10) + (2) - 2, .025) and -0.10027721505635707 = (30693679.166666668 - 33771555.83333333) / 30693679.166666668 is less than -0.05.
    IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
    IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
    
    ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Mariomkas.Count(Pattern: "(?:(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9])\.){3}(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9])", Options: NonBacktracking)
    
    ```log
    
    

    Description of detection logic

    IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
    IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
    IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
    IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
    IsRegressionWindowed: Marked as regression because 23.387071666666667 > 21.174688500000002.
    IsChangePoint: Marked as a change because one of 1/11/2022 6:41:22 PM, 2/3/2022 8:57:20 PM, 3/4/2022 12:09:12 AM, 4/16/2022 8:46:33 PM, 5/9/2022 6:32:22 AM, 5/17/2022 8:30:53 PM falls between 5/9/2022 6:32:22 AM and 5/17/2022 8:30:53 PM.
    IsRegressionStdDev: Marked as regression because -28.57498300042821 (T) = (0 -23282056.666666668) / Math.Sqrt((6528929665.698002 / (10)) + (22056300450 / (2))) is less than -2.2281388519818126 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (10) + (2) - 2, .025) and -0.1529363761976432 = (20193704.654761903 - 23282056.666666668) / 20193704.654761903 is less than -0.05.
    IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
    IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
    
    

    Docs

    Profiling workflow for dotnet/runtime repository
    Benchmarking workflow for dotnet/runtime repository

    ### Run Information
    Architecture arm64
    OS Windows 10.0.19041
    Baseline 5ecaae992806a2d422157a095a63efb467365f54
    Compare f63fdeefa535a1d2fd1855249f673f12e021774d
    Diff Diff

    Regressions in Benchstone.BenchI.XposMatrix

    Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
    Test - Duration of single invocation 24.12 μs 25.78 μs 1.07 0.00 False

    graph
    Test Report

    Repro

    git clone https://github.com/dotnet/performance.git
    py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'Benchstone.BenchI.XposMatrix*'
    Details

    Payloads

    Baseline
    Compare

    Histogram

    Benchstone.BenchI.XposMatrix.Test

    
    

    Description of detection logic

    IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
    IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
    IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
    IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
    IsRegressionWindowed: Marked as regression because 25.784004324546952 > 25.325630787037035.
    IsChangePoint: Marked as a change because one of 5/9/2022 6:32:22 AM, 5/17/2022 8:30:53 PM falls between 5/9/2022 6:32:22 AM and 5/17/2022 8:30:53 PM.
    IsRegressionStdDev: Marked as regression because -327.143938557808 (T) = (0 -25781.323789748727) / Math.Sqrt((185.20340154617458 / (10)) + (14.37053360902362 / (2))) is less than -2.2281388519818126 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (10) + (2) - 2, .025) and -0.06875863267882398 = (24122.68121299597 - 25781.323789748727) / 24122.68121299597 is less than -0.05.
    IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
    IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
    
    

    Docs

    Profiling workflow for dotnet/runtime repository
    Benchmarking workflow for dotnet/runtime repository

    Run Information

    Architecture arm64
    OS Windows 10.0.19041
    Baseline 5ecaae992806a2d422157a095a63efb467365f54
    Compare f63fdeefa535a1d2fd1855249f673f12e021774d
    Diff Diff

    Regressions in System.Threading.Channels.Tests.SpscUnboundedChannelPerfTests

    Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
    ReadAsyncThenWriteAsync - Duration of single invocation 168.29 ns 194.46 ns 1.16 0.06 False

    graph
    Test Report

    Repro

    git clone https://github.com/dotnet/performance.git
    py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Threading.Channels.Tests.SpscUnboundedChannelPerfTests*'
    Details

    Payloads

    Baseline
    Compare

    Histogram

    System.Threading.Channels.Tests.SpscUnboundedChannelPerfTests.ReadAsyncThenWriteAsync

    
    

    Description of detection logic

    IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
    IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
    IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
    IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
    IsRegressionWindowed: Marked as regression because 194.4608147099597 > 175.76909812485712.
    IsChangePoint: Marked as a change because one of 1/5/2022 6:46:20 PM, 3/2/2022 12:52:37 PM, 3/3/2022 4:26:18 PM, 5/9/2022 6:32:22 AM, 5/17/2022 8:30:53 PM falls between 5/9/2022 6:32:22 AM and 5/17/2022 8:30:53 PM.
    IsRegressionStdDev: Marked as regression because -30.641167500126812 (T) = (0 -193.80642707396535) / Math.Sqrt((2.9871470104624986 / (10)) + (0.8564463562845208 / (2))) is less than -2.2281388519818126 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (10) + (2) - 2, .025) and -0.1558004230676812 = (167.6815678606275 - 193.80642707396535) / 167.6815678606275 is less than -0.05.
    IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
    IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
    
    

    Docs

    Profiling workflow for dotnet/runtime repository
    Benchmarking workflow for dotnet/runtime repository

    Run Information

    Architecture arm64
    OS Windows 10.0.19041
    Baseline 5ecaae992806a2d422157a095a63efb467365f54
    Compare f63fdeefa535a1d2fd1855249f673f12e021774d
    Diff Diff

    Regressions in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig

    Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
    Count - Duration of single invocation 32.73 ms 41.54 ms 1.27 0.01 False
    Count - Duration of single invocation 32.70 ms 41.50 ms 1.27 0.01 False

    Test Report

    Repro

    git clone https://github.com/dotnet/performance.git
    py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig*'
    Details

    Payloads

    Baseline
    Compare

    Histogram

    System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern: "\p{Sm}", Options: None)

    
    

    Description of detection logic

    IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
    IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
    IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
    IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
    IsRegressionWindowed: Marked as regression because 41.54026632653062 > 34.367013.
    IsChangePoint: Marked as a change because one of 1/11/2022 6:41:22 PM, 2/1/2022 10:36:16 AM, 4/16/2022 8:46:33 PM, 5/9/2022 6:32:22 AM, 5/17/2022 8:30:53 PM falls between 5/9/2022 6:32:22 AM and 5/17/2022 8:30:53 PM.
    IsRegressionStdDev: Marked as regression because -1286.0234074281723 (T) = (0 -41546363.63945578) / Math.Sqrt((97648184.06650366 / (10)) + (74354449.81473556 / (2))) is less than -2.2281388519818126 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (10) + (2) - 2, .025) and -0.26916212907528175 = (32735268.952380955 - 41546363.63945578) / 32735268.952380955 is less than -0.05.
    IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
    IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
    
    ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern: "\p{Sm}", Options: NonBacktracking)
    
    ```log
    
    

    Description of detection logic

    IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
    IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
    IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
    IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
    IsRegressionWindowed: Marked as regression because 41.497014285714286 > 34.3561525.
    IsChangePoint: Marked as a change because one of 1/11/2022 6:41:22 PM, 2/3/2022 8:57:20 PM, 4/16/2022 8:46:33 PM, 5/9/2022 6:32:22 AM, 5/17/2022 8:30:53 PM falls between 5/9/2022 6:32:22 AM and 5/17/2022 8:30:53 PM.
    IsRegressionStdDev: Marked as regression because -1007.8843652505626 (T) = (0 -41505112.85714286) / Math.Sqrt((103437346.19365954 / (10)) + (131173718.36745037 / (2))) is less than -2.2281388519818126 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (10) + (2) - 2, .025) and -0.2683931247440609 = (32722593.687598117 - 41505112.85714286) / 32722593.687598117 is less than -0.05.
    IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
    IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
    
    

    Docs

    Profiling workflow for dotnet/runtime repository
    Benchmarking workflow for dotnet/runtime repository

    Run Information

    Architecture arm64
    OS Windows 10.0.19041
    Baseline 5ecaae992806a2d422157a095a63efb467365f54
    Compare f63fdeefa535a1d2fd1855249f673f12e021774d
    Diff Diff

    Regressions in System.Memory.Span<Int32>

    Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
    IndexOfAnyFourValues - Duration of single invocation 821.69 ns 982.93 ns 1.20 0.00 False

    graph
    Test Report

    Repro

    git clone https://github.com/dotnet/performance.git
    py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Memory.Span&lt;Int32&gt;*'
    Details

    Payloads

    Baseline
    Compare

    Histogram

    System.Memory.Span<Int32>.IndexOfAnyFourValues(Size: 512)

    
    

    Description of detection logic

    IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
    IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
    IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
    IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
    IsRegressionWindowed: Marked as regression because 982.9297130326609 > 863.026433773239.
    IsChangePoint: Marked as a change because one of 5/9/2022 6:32:22 AM, 5/17/2022 8:30:53 PM falls between 5/9/2022 6:32:22 AM and 5/17/2022 8:30:53 PM.
    IsRegressionStdDev: Marked as regression because -277.0450230416103 (T) = (0 -982.3550880685057) / Math.Sqrt((0.053334644081946915 / (10)) + (0.6603876988607097 / (2))) is less than -2.2281388519818126 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (10) + (2) - 2, .025) and -0.19525725646658348 = (821.8775353621708 - 982.3550880685057) / 821.8775353621708 is less than -0.05.
    IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
    IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
    
    

    Docs

    Profiling workflow for dotnet/runtime repository
    Benchmarking workflow for dotnet/runtime repository

    Run Information

    Architecture arm64
    OS Windows 10.0.19041
    Baseline 5ecaae992806a2d422157a095a63efb467365f54
    Compare f63fdeefa535a1d2fd1855249f673f12e021774d
    Diff Diff

    Regressions in System.Collections.Concurrent.Count<Int32>

    Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
    Dictionary - Duration of single invocation 5.80 μs 6.30 μs 1.09 0.07 False

    graph
    Test Report

    Repro

    git clone https://github.com/dotnet/performance.git
    py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.Concurrent.Count&lt;Int32&gt;*'
    Details

    Payloads

    Baseline
    Compare

    Histogram

    System.Collections.Concurrent.Count<Int32>.Dictionary(Size: 512)

    
    

    Description of detection logic

    IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
    IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
    IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
    IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
    IsRegressionWindowed: Marked as regression because 6.296471492572908 > 6.082854025152663.
    IsChangePoint: Marked as a change because one of 5/9/2022 6:32:22 AM, 5/17/2022 8:30:53 PM falls between 5/9/2022 6:32:22 AM and 5/17/2022 8:30:53 PM.
    IsRegressionStdDev: Marked as regression because -2.840582161165397 (T) = (0 -6513.196321659288) / Math.Sqrt((52004.27870976801 / (10)) + (93939.3030850404 / (2))) is less than -2.2281388519818126 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (10) + (2) - 2, .025) and -0.11063577823854155 = (5864.385471166037 - 6513.196321659288) / 5864.385471166037 is less than -0.05.
    IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
    IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
    
    

    Docs

    Profiling workflow for dotnet/runtime repository
    Benchmarking workflow for dotnet/runtime repository

    Run Information

    Architecture arm64
    OS Windows 10.0.19041
    Baseline 5ecaae992806a2d422157a095a63efb467365f54
    Compare f63fdeefa535a1d2fd1855249f673f12e021774d
    Diff Diff

    Regressions in System.Memory.Span<Byte>

    Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
    IndexOfAnyFourValues - Duration of single invocation 822.53 ns 981.32 ns 1.19 0.00 False

    graph
    Test Report

    Repro

    git clone https://github.com/dotnet/performance.git
    py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Memory.Span&lt;Byte&gt;*'
    Details

    Payloads

    Baseline
    Compare

    Histogram

    System.Memory.Span<Byte>.IndexOfAnyFourValues(Size: 512)

    
    

    Description of detection logic

    IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
    IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
    IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
    IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
    IsRegressionWindowed: Marked as regression because 981.3217244412144 > 863.6489362880656.
    IsChangePoint: Marked as a change because one of 5/9/2022 6:32:22 AM, 5/17/2022 8:30:53 PM falls between 5/9/2022 6:32:22 AM and 5/17/2022 8:30:53 PM.
    IsRegressionStdDev: Marked as regression because -316.1429559098024 (T) = (0 -981.8188312206914) / Math.Sqrt((0.06389853733114643 / (10)) + (0.4942303004041181 / (2))) is less than -2.2281388519818126 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (10) + (2) - 2, .025) and -0.19349302028553053 = (822.6431278046366 - 981.8188312206914) / 822.6431278046366 is less than -0.05.
    IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
    IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
    
    

    Docs

    Profiling workflow for dotnet/runtime repository
    Benchmarking workflow for dotnet/runtime repository

    Run Information

    Architecture arm64
    OS Windows 10.0.19041
    Baseline 5ecaae992806a2d422157a095a63efb467365f54
    Compare f63fdeefa535a1d2fd1855249f673f12e021774d
    Diff Diff

    Regressions in System.Threading.Channels.Tests.BoundedChannelPerfTests

    Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
    TryWriteThenTryRead - Duration of single invocation 56.70 ns 60.59 ns 1.07 0.02 False

    graph
    Test Report

    Repro

    git clone https://github.com/dotnet/performance.git
    py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Threading.Channels.Tests.BoundedChannelPerfTests*'
    Details

    Payloads

    Baseline
    Compare

    Histogram

    System.Threading.Channels.Tests.BoundedChannelPerfTests.TryWriteThenTryRead

    
    

    Description of detection logic

    IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
    IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
    IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
    IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
    IsRegressionWindowed: Marked as regression because 60.59148048986128 > 59.378575043414315.
    IsChangePoint: Marked as a change because one of 4/21/2022 2:39:24 PM, 5/9/2022 6:32:22 AM, 5/17/2022 8:30:53 PM falls between 5/9/2022 6:32:22 AM and 5/17/2022 8:30:53 PM.
    IsRegressionStdDev: Marked as regression because -16.222875469155635 (T) = (0 -60.39177998702054) / Math.Sqrt((0.2319089572385833 / (10)) + (0.07976058166968716 / (2))) is less than -2.2281388519818126 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (10) + (2) - 2, .025) and -0.07234348435389731 = (56.317570692759396 - 60.39177998702054) / 56.317570692759396 is less than -0.05.
    IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
    IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
    
    

    Docs

    Profiling workflow for dotnet/runtime repository
    Benchmarking workflow for dotnet/runtime repository

  2. kunalspathak commented on May 19, 2022

    @kunalspathak
    Contributor

    There was an outage between 5/11 and 5/17 and we don't have enough data to track down. Next action would be to pick one of this benchmark and manually verify if the regression repros and then track down the commit.

  3. 19 remaining items

  4. kunalspathak commented on Jun 30, 2022

    @kunalspathak
    Contributor

    @jeffhandley, I was not able to locate a commit 3e5517b...6506705 that would affect System.IO.Compression.Brotli. Could you please double check?

  5. removed their assignment
    on Jul 28, 2022
  6. jeffhandley commented on Aug 10, 2022

    @jeffhandley
    Member

    We updated the default compression level a while back. @adamsitnik is going to take a look.

  7. adamsitnik commented on Aug 12, 2022

    @adamsitnik
    Member

    The regression is gone:

    image

    In the commits from the "first drop" I can see two JIT changes: #69865 and #70792

    In the second drop there were two JIT/PGO changes: #72842 and #72773

  8. ghost locked as resolved and limited conversation to collaborators on Sep 11, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions