Skip to content

[Perf] Windows/x64: 5 Regressions on 2/3/2024 12:19:35 AM #98044

New issue
New issue
Open
Open
[Perf] Windows/x64: 5 Regressions on 2/3/2024 12:19:35 AM#98044
Assignees
AndyAyersMS
Labels
arch-x64area-CodeGen-coreclrCLR JIT compiler in src/coreclr/src/jit and related components such as SuperPMIos-windowsruntime-coreclrspecific to the CoreCLR runtimetenet-performance-benchmarksIssue from performance benchmark
Milestone
Future

Description

@performanceautofiler
performanceautofiler
bot
opened on Feb 6, 2024

Run Information

Name Value
Architecture x64
OS Windows 10.0.18362
Queue TigerWindows
Baseline 1a2f095fb212dcbf394f01122b9f317b7cc70fdb
Compare 2361c00717a54a5dd9b0cf727102d64f783855b9
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Globalization.Tests.StringEquality

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
761.52 ns 973.83 ns 1.28 0.00 True
1.19 μs 1.28 μs 1.08 0.01 False

graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Globalization.Tests.StringEquality*'
Details

Payloads

Baseline
Compare

System.Globalization.Tests.StringEquality.Compare_Same(Count: 1024, Options: (en-US, OrdinalIgnoreCase))

ETL Files

Histogram

JIT Disasms

System.Globalization.Tests.StringEquality.Compare_Same_Upper(Count: 1024, Options: (en-US, OrdinalIgnoreCase))

ETL Files

Histogram

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.18362
Queue TigerWindows
Baseline 1a2f095fb212dcbf394f01122b9f317b7cc70fdb
Compare 2361c00717a54a5dd9b0cf727102d64f783855b9
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in Benchmark.GetChildKeysTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
14.99 ms 16.20 ms 1.08 0.02 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Benchmark.GetChildKeysTests*'
Details

Payloads

Baseline
Compare

Benchmark.GetChildKeysTests.AddChainedConfigurationEmpty

ETL Files

Histogram

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.18362
Queue TigerWindows
Baseline 1a2f095fb212dcbf394f01122b9f317b7cc70fdb
Compare 2361c00717a54a5dd9b0cf727102d64f783855b9
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in Span.Sorting

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
8.54 μs 16.21 μs 1.90 0.45 True

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Span.Sorting*'
Details

Payloads

Baseline
Compare

Span.Sorting.QuickSortArray(Size: 512)

ETL Files

Histogram

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.18362
Queue TigerWindows
Baseline 1a2f095fb212dcbf394f01122b9f317b7cc70fdb
Compare 2361c00717a54a5dd9b0cf727102d64f783855b9
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in Benchstone.BenchI.EightQueens

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
1.80 μs 2.04 μs 1.13 0.03 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Benchstone.BenchI.EightQueens*'
Details

Payloads

Baseline
Compare

Benchstone.BenchI.EightQueens.Test

ETL Files

Histogram

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Activity

  1. performanceautofiler
    added
    arch-x64
    os-windows
    runtime-coreclrspecific to the CoreCLR runtime
    untriagedNew issue has not been triaged by the area owner
    on Feb 6, 2024
  2. DrewScoggins
    removed
    untriagedNew issue has not been triaged by the area owner
    on Feb 6, 2024
  3. DrewScoggins
    transferred this issue fromdotnet/perf-autofiling-issueson Feb 6, 2024
  4. ghost added
    needs-area-labelAn area label is needed to ensure this gets routed to the appropriate area owners
    on Feb 6, 2024
  5. ghost added
    untriagedNew issue has not been triaged by the area owner
    on Feb 6, 2024

  6. DrewScoggins commented on Feb 6, 2024

    @DrewScoggins
    DrewScoggins
    on Feb 6, 2024
    Member

    Diff here: 207e1fb...df0778d

    Nothing is jumping out as the culprit, but there were a few JIT changes.

  7. DrewScoggins commented on Feb 6, 2024

    @DrewScoggins
    DrewScoggins
    on Feb 6, 2024
    Member

    Linux related regressions: dotnet/perf-autofiling-issues#28564

  8. jeffschwMSFT
    added
    area-CodeGen-coreclrCLR JIT compiler in src/coreclr/src/jit and related components such as SuperPMI
    on Feb 7, 2024
  10. vcsjones
    removed
    needs-area-labelAn area label is needed to ensure this gets routed to the appropriate area owners
    on Feb 13, 2024

  11. BruceForstall commented on Feb 13, 2024

    @BruceForstall
    BruceForstall
    on Feb 13, 2024
    Contributor

    Maybe #97722?

  12. BruceForstall
    assigned
    AndyAyersMS
    on Feb 13, 2024

  13. 1 remaining item

  14. ghost removed
    untriagedNew issue has not been triaged by the area owner
    on Feb 13, 2024
  15. AndyAyersMS
    added
    Priority:2Work that is important, but not critical for the release
    on May 8, 2024

  16. AndyAyersMS commented on Jul 22, 2024

    @AndyAyersMS
    AndyAyersMS
    on Jul 22, 2024
    Member

    EightQueens seems to be an intel-only regression, and then only on some cases, and two other regressions since.

    image

    Most all the time is in TryMe.

    Codegen from baseline to latest shows RBO did one jump thread (from #97722), different layout, and an IV widening.

    There are a lot of spilled CSEs here in both baseline and latest codegen, but more spill occurrences in latest. Possibly the one extra jump thread by RBO has created more critical edges and so made life more difficult for LSRA.

    Final flow graphs. You can clearly see the impact of RPO layout at least...

    MAIN BASELINE
  17. AndyAyersMS
    added
    tenet-performance-benchmarksIssue from performance benchmark
    on Jul 27, 2024

  18. AndyAyersMS commented on Aug 6, 2024

    @AndyAyersMS
    AndyAyersMS
    on Aug 6, 2024
    Member

    Span.Sorting.QuickSortArray(Size: 512)

    Regressions here were fixed by RPO layout:

    image

  19. AndyAyersMS commented on Aug 6, 2024

    @AndyAyersMS
    AndyAyersMS
    on Aug 6, 2024
    Member

    System.Globalization.Tests.StringEquality.Compare_Same(Count: 1024, Options: (en-US, OrdinalIgnoreCase))

    Ditto for this benchmark

    image

  20. AndyAyersMS commented on Aug 6, 2024

    @AndyAyersMS
    AndyAyersMS
    on Aug 6, 2024
    Member

    System.Globalization.Tests.StringEquality.Compare_Same_Upper(Count: 1024, Options: (en-US, OrdinalIgnoreCase))

    image

    Same as the two above, recovers with later changes.

  21. AndyAyersMS commented on Aug 6, 2024

    @AndyAyersMS
    AndyAyersMS
    on Aug 6, 2024
    Member

    Benchmark.GetChildKeysTests.AddChainedConfigurationEmpty

    Ditto like the above

    image

  22. AndyAyersMS commented on Aug 6, 2024

    @AndyAyersMS
    AndyAyersMS
    on Aug 6, 2024
    Member

    So the only persisted regression is in 8 queens, and that one seems to be the increase in resolution moves by the allocator.

    Going to move this to .NET 10 as there's no simple fix available now.

  23. AndyAyersMS
    modified the milestones: 9.0.0, 10.0.0 on Aug 6, 2024
  24. AndyAyersMS
    removed
    Priority:2Work that is important, but not critical for the release
    on Aug 6, 2024

  25. AndyAyersMS commented on Feb 25, 2025

    @AndyAyersMS
    AndyAyersMS
    on Feb 25, 2025
    Member

    Likely won't get to this in .NET 10 either

  26. AndyAyersMS
    modified the milestones: 10.0.0, Future on Feb 25, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

arch-x64area-CodeGen-coreclrCLR JIT compiler in src/coreclr/src/jit and related components such as SuperPMIos-windowsruntime-coreclrspecific to the CoreCLR runtimetenet-performance-benchmarksIssue from performance benchmark

Type

No type

Projects

No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions