Repository navigation
[Perf] Regressions in System.Numerics.Tests.Perf_BigInteger #57293
Description
Run Information
| Architecture | arm64 |
|---|---|
| OS | ubuntu 18.04 |
| Baseline | 749fa1381f122f7a0bb23374a9e220f834ccbbb3 |
| Compare | f83a9d9689433ce522b91e74a9631c83847e3b64 |
| Diff | Diff |
Regressions in System.Numerics.Tests.Perf_BigInteger
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| Ctor_ByteArray - Duration of single invocation | 24.62 ns | 32.53 ns | 1.32 | 0.43 | False | |||||
| Ctor_ByteArray - Duration of single invocation | 12.83 ns | 26.17 ns | 2.04 | 0.41 | False |
Historical Data in Reporting System
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f netcoreapp5.0 --filter 'System.Numerics.Tests.Perf_BigInteger*'Details
Payloads
Histogram
System.Numerics.Tests.Perf_BigInteger.Ctor_ByteArray(numberString: -2147483648)
System.Numerics.Tests.Perf_BigInteger.Ctor_ByteArray(numberString: 123)
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Activity
[-][Perf] Changes at 8/2/2021 5:40:25 PM[/-][+][Perf] Regressions in System.Numerics.Tests.Perf_BigInteger[/+]DrewScoggins commentedon Aug 12, 2021
Seems related to https://github.com/dotnet/runtime/pull/53984/files
stephentoub commentedon Aug 12, 2021
Assuming #53984, as something intended to be a minor cleanup but regressing some benchmarks by 30%-100%, at this point for .NET 6 it should likely be reverted and revisted for .NET 7.
cc: @tannergooding, @huoyaoyuan
tannergooding commentedon Aug 12, 2021
Agreed. Additionally, we should likely look into why the regression is so bad as a follow up.
I don't believe its expected that using indexers should be this impactful.
huoyaoyuan commentedon Aug 12, 2021
https://sharplab.io/#v2:EYLgxg9gTgpgtADwGwBYA0AXEBDAzgWwB8ABAJgEYBYAKGIGYACMhgYQYG8aHuGBLAOwwBtALoNgvDLgDcXHvSbkkfQQwCy5ABRsAbtgA2AVxgBKOd07Ue1pgHYGeozAB0EqUIB65EbKs8AvuZMjMRKKhjqpNoOBsZmfhZB1sT2jsaukrhCaS5uuM4AMjD8AOYYABYMcAzevtaB1P5AA
JIT looks happy with index expression and generates slightly better code for it.
stephentoub commentedon Aug 12, 2021
FYI, that's on .NET 5 and amd64, rather than .NET 6 and arm64.
tannergooding commentedon Aug 12, 2021
Notably, that's a relatively simple example. Looking through other scenarios and directly copying the full expression/context they are used in shows worse codegen in several cases.
I'm investigating more so I can log some issues on the runtime and/or compiler.
AndyAyersMS commentedon Aug 12, 2021
Possibly relevant: #11870.