Repository navigation
[Perf] Windows/arm64: Regressions in IfStatements #77984
Description
Run Information
| Architecture | arm64 |
|---|---|
| OS | Windows 10.0.19041 |
| Baseline | 7d5efbb9e10b6d8beb91c90cbdefd7360869cece |
| Compare | 0e24ea7c2a0436a8f2bf83e8f5981ec035518b99 |
| Diff | Diff |
Regressions in IfStatements.IfStatements
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| AndAnd - Duration of single invocation | 50.85 μs | 60.02 μs | 1.18 | 0.00 | False | |||||
| AndAndAnd - Duration of single invocation | 48.76 μs | 51.73 μs | 1.06 | 0.00 | False |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'IfStatements.IfStatements*'Related Issues
Regressions
Improvements
Details
Payloads
Histogram
Edge Detector Info
IfStatements.IfStatements.AndAnd
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 60.016950830140495 > 53.430863129058444.
IsChangePoint: Marked as a change because one of 10/11/2022 1:45:10 PM, 11/1/2022 8:41:52 AM, 11/3/2022 4:18:41 AM falls between 10/25/2022 1:30:53 PM and 11/3/2022 4:18:41 AM.
IsRegressionStdDev: Marked as regression because -701.6727127857075 (T) = (0 -60020.57172055355) / Math.Sqrt((3457.9591975832996 / (39)) + (568.9918365919992 / (7))) is less than -2.0153675744421933 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (39) + (7) - 2, .025) and -0.17980682074412616 = (50873.21980618611 - 60020.57172055355) / 50873.21980618611 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### IfStatements.IfStatements.AndAndAnd
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 51.733831401766004 > 51.209566442757016.
IsChangePoint: Marked as a change because one of 11/1/2022 8:41:52 AM, 11/3/2022 4:18:41 AM falls between 10/25/2022 1:30:53 PM and 11/3/2022 4:18:41 AM.
IsRegressionStdDev: Marked as regression because -578.8484547464244 (T) = (0 -51739.81218834986) / Math.Sqrt((497.56047658656183 / (39)) + (91.36335693339602 / (7))) is less than -2.0153675744421933 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (39) + (7) - 2, .025) and -0.06026238051797967 = (48799.06440052408 - 51739.81218834986) / 48799.06440052408 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
| Architecture | arm64 |
|---|---|
| OS | Windows 10.0.19041 |
| Baseline | 7d5efbb9e10b6d8beb91c90cbdefd7360869cece |
| Compare | 0e24ea7c2a0436a8f2bf83e8f5981ec035518b99 |
| Diff | Diff |
Activity
9 remaining items
kunalspathak commentedon Nov 7, 2022
#73472 (comment)
@a74nh
EgorBo commentedon Nov 10, 2022
Same regression on Ampere arm64 Ubuntu - dotnet/perf-autofiling-issues#9668
a74nh commentedon Nov 11, 2022
I'm seeing the same slowdown.
Without if conversion. =~50us
With if conversion. =~60us
.... But with #77728 =~43us
I think this is because with the new patch, the if conversion happens after optimise bools. I could pull that part out of 77728 and submit it separately, or we could just wait for 77728 to get merged?
a74nh commentedon Nov 11, 2022
... Yes, it is.
The IR is a chain of JTRUEs.
Without If conversion: Optimize bools turns the sequence into a single JTRUE statement.
With If conversion: If conversion converts the final JTRUE, and ignores the others. Then optimize bools takes the remaining JTRUEs and combines them
With the If conversion moved later: Optimize bools turns the sequence into a single JTRUE statement. Then If conversion converts this single block.
a74nh commentedon Nov 30, 2022
Hoping this should be fixed now that #77728 is merged
a74nh commentedon Dec 5, 2022
Not sure how to link the latest graphs here. But they drop quite nicely with the latest HEAD:
And And
50.85 μs -> 60.02 μs -> 41.39 μs
And And And
48.76 μs -> 51.73 μs -> 46.22 μs
@kunalspathak - should ok to close this now.
jakobbotsch commentedon Feb 1, 2023
Closing since this was fixed by #77728.