Skip to content

[Perf] Windows/x64: 33 Regressions on 6/21/2024 6:31:57 PM #103972

Closed
Closed
[Perf] Windows/x64: 33 Regressions on 6/21/2024 6:31:57 PM#103972
Assignees
AndyAyersMS
Labels
Priority:2Work that is important, but not critical for the releasearch-x64area-CodeGen-coreclrCLR JIT compiler in src/coreclr/src/jit and related components such as SuperPMIos-windowsruntime-coreclrspecific to the CoreCLR runtimetenet-performancePerformance related issuetenet-performance-benchmarksIssue from performance benchmark
Milestone
Future

Description

@performanceautofiler
performanceautofiler
bot
opened on Jun 25, 2024

Run Information

Name Value
Architecture x64
OS Windows 10.0.22631
Queue ViperWindows
Baseline 8923c96baa4ca6f5540ca185a3e614a96fe63b34
Compare 0262a63be0f560fa902b1e31e9a10760e466352d
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Text.Json.Tests.Utf8JsonReaderCommentsTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
69.18 ns 81.88 ns 1.18 0.08 False
110.55 ns 125.75 ns 1.14 0.08 False

graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Tests.Utf8JsonReaderCommentsTests*'
Details

System.Text.Json.Tests.Utf8JsonReaderCommentsTests.Utf8JsonReaderCommentParsing(CommentHandling: Skip, SegmentSize: 100, TestCase: ShortMultiLine)

ETL Files

Histogram

JIT Disasms

System.Text.Json.Tests.Utf8JsonReaderCommentsTests.Utf8JsonReaderCommentParsing(CommentHandling: Skip, SegmentSize: 0, TestCase: LongSingleLine)

ETL Files

Histogram

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.22631
Queue ViperWindows
Baseline 8923c96baa4ca6f5540ca185a3e614a96fe63b34
Compare 0262a63be0f560fa902b1e31e9a10760e466352d
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Linq.Tests.Perf_Enumerable

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
104.29 ns 118.45 ns 1.14 0.15 False
29.31 ns 39.29 ns 1.34 0.11 False
38.40 ns 62.93 ns 1.64 0.03 True
108.24 ns 122.41 ns 1.13 0.05 True
102.96 ns 125.29 ns 1.22 0.09 False
101.79 ns 124.12 ns 1.22 0.16 False
102.43 ns 121.89 ns 1.19 0.04 False
102.21 ns 117.97 ns 1.15 0.12 False
334.88 ns 355.25 ns 1.06 0.05 False
119.81 ns 131.68 ns 1.10 0.08 False
129.29 ns 147.50 ns 1.14 0.01 False
103.39 ns 121.52 ns 1.18 0.22 False
63.77 ns 89.59 ns 1.40 0.03 True
120.53 ns 135.08 ns 1.12 0.14 False
122.35 ns 143.10 ns 1.17 0.24 False
130.47 ns 148.53 ns 1.14 0.00 False
82.69 ns 109.31 ns 1.32 0.07 False
132.98 ns 153.08 ns 1.15 0.00 True
102.37 ns 122.38 ns 1.20 0.05 False
103.77 ns 126.73 ns 1.22 0.30 False
101.73 ns 119.91 ns 1.18 0.20 False

graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Linq.Tests.Perf_Enumerable*'
Details

System.Linq.Tests.Perf_Enumerable.FirstWithPredicate_LastElementMatches(input: Array)

ETL Files

Histogram

JIT Disasms

System.Linq.Tests.Perf_Enumerable.ElementAt(input: IEnumerable)

ETL Files

Histogram

JIT Disasms

System.Linq.Tests.Perf_Enumerable.Count(input: IEnumerable)

ETL Files

Histogram

JIT Disasms

System.Linq.Tests.Perf_Enumerable.CastToSameType(input: IEnumerable)

ETL Files

Histogram

JIT Disasms

System.Linq.Tests.Perf_Enumerable.FirstWithPredicate_LastElementMatches(input: IEnumerable)

ETL Files

Histogram

JIT Disasms

System.Linq.Tests.Perf_Enumerable.AnyWithPredicate_LastElementMatches(input: Array)

ETL Files

Histogram

JIT Disasms

System.Linq.Tests.Perf_Enumerable.Sum(input: IEnumerable)

ETL Files

Histogram

JIT Disasms

System.Linq.Tests.Perf_Enumerable.All_AllElementsMatch(input: IEnumerable)

ETL Files

Histogram

JIT Disasms

System.Linq.Tests.Perf_Enumerable.Reverse(input: IEnumerable)

ETL Files

Histogram

JIT Disasms

System.Linq.Tests.Perf_Enumerable.Select(input: Array)

ETL Files

Histogram

JIT Disasms

System.Linq.Tests.Perf_Enumerable.AnyWithPredicate_LastElementMatches(input: List)

ETL Files

Histogram

JIT Disasms

System.Linq.Tests.Perf_Enumerable.SingleWithPredicate_LastElementMatches(input: Array)

ETL Files

Histogram

JIT Disasms

System.Linq.Tests.Perf_Enumerable.Repeat

ETL Files

Histogram

JIT Disasms

System.Linq.Tests.Perf_Enumerable.WhereFirst_LastElementMatches(input: IEnumerable)

ETL Files

Histogram

JIT Disasms

System.Linq.Tests.Perf_Enumerable.WhereAny_LastElementMatches(input: IEnumerable)

ETL Files

Histogram

JIT Disasms

System.Linq.Tests.Perf_Enumerable.FirstWithPredicate_LastElementMatches(input: List)

ETL Files

Histogram

JIT Disasms

System.Linq.Tests.Perf_Enumerable.Range

ETL Files

Histogram

JIT Disasms

System.Linq.Tests.Perf_Enumerable.SingleWithPredicate_LastElementMatches(input: List)

ETL Files

Histogram

JIT Disasms

System.Linq.Tests.Perf_Enumerable.Aggregate_Seed(input: IEnumerable)

ETL Files

Histogram

JIT Disasms

System.Linq.Tests.Perf_Enumerable.SingleWithPredicate_LastElementMatches(input: IEnumerable)

ETL Files

Histogram

JIT Disasms

System.Linq.Tests.Perf_Enumerable.AnyWithPredicate_LastElementMatches(input: IEnumerable)

ETL Files

Histogram

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.22631
Queue ViperWindows
Baseline 8923c96baa4ca6f5540ca185a3e614a96fe63b34
Compare 0262a63be0f560fa902b1e31e9a10760e466352d
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Collections.IterateForEach<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
1.00 μs 1.26 μs 1.25 0.01 True
1.09 μs 1.44 μs 1.32 0.02 True
8.26 μs 14.23 μs 1.72 0.08 True
8.81 μs 12.91 μs 1.47 0.09 True
9.44 μs 13.60 μs 1.44 0.09 False

graph
graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.IterateForEach&lt;String&gt;*'
Details

System.Collections.IterateForEach<String>.Stack(Size: 512)

ETL Files

Histogram

JIT Disasms

System.Collections.IterateForEach<String>.Queue(Size: 512)

ETL Files

Histogram

JIT Disasms

System.Collections.IterateForEach<String>.ImmutableSortedDictionary(Size: 512)

ETL Files

Histogram

JIT Disasms

System.Collections.IterateForEach<String>.ImmutableSortedSet(Size: 512)

ETL Files

Histogram

JIT Disasms

System.Collections.IterateForEach<String>.ImmutableList(Size: 512)

ETL Files

Histogram

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.22631
Queue ViperWindows
Baseline 8923c96baa4ca6f5540ca185a3e614a96fe63b34
Compare 0262a63be0f560fa902b1e31e9a10760e466352d
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in SciMark2.kernel

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
326.62 ms 388.15 ms 1.19 0.02 False
462.87 ms 512.19 ms 1.11 0.02 False

graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'SciMark2.kernel*'
Details

SciMark2.kernel.benchSparseMult

ETL Files

Histogram

JIT Disasms

SciMark2.kernel.benchMonteCarlo

ETL Files

Histogram

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.22631
Queue ViperWindows
Baseline 8923c96baa4ca6f5540ca185a3e614a96fe63b34
Compare 0262a63be0f560fa902b1e31e9a10760e466352d
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Collections.CtorFromCollection<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
3.98 μs 7.22 μs 1.81 0.22 True
3.05 μs 6.16 μs 2.02 0.15 True

graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.CtorFromCollection&lt;Int32&gt;*'
Details

System.Collections.CtorFromCollection<Int32>.ImmutableStack(Size: 512)

ETL Files

Histogram

JIT Disasms

System.Collections.CtorFromCollection<Int32>.ConcurrentStack(Size: 512)

ETL Files

Histogram

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Activity

  1. performanceautofiler
    added
    arch-x64
    os-windows
    runtime-coreclrspecific to the CoreCLR runtime
    untriagedNew issue has not been triaged by the area owner
    on Jun 25, 2024

  2. performanceautofiler commented on Jun 25, 2024

    @performanceautofiler
    performanceautofilerBot
    on Jun 25, 2024 – with PerformanceAutoFiler
    Author

    Run Information

    Name Value
    Architecture x64
    OS Windows 10.0.22631
    Queue ViperWindows
    Baseline 8923c96baa4ca6f5540ca185a3e614a96fe63b34
    Compare 0262a63be0f560fa902b1e31e9a10760e466352d
    Diff Diff
    Configs CompilationMode:tiered, RunKind:micro

    Regressions in LinqBenchmarks

    Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
    369.44 ms 408.80 ms 1.11 0.45 False

    graph
    Test Report

    Repro

    General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

    git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'LinqBenchmarks*'
    Details

    LinqBenchmarks.Count00LinqMethodX

    ETL Files

    Histogram

    JIT Disasms

    Docs

    Profiling workflow for dotnet/runtime repository
    Benchmarking workflow for dotnet/runtime repository

  3. DrewScoggins
    transferred this issue fromdotnet/perf-autofiling-issueson Jun 25, 2024
  4. ghost added
    area-System.Linq
    on Jun 25, 2024
  5. DrewScoggins
    added
    tenet-performancePerformance related issue
    tenet-performance-benchmarksIssue from performance benchmark
    and removed
    area-System.Linq
    on Jun 25, 2024

  6. DrewScoggins commented on Jun 25, 2024

    @DrewScoggins
    DrewScoggins
    on Jun 25, 2024
    Member

    Seems to be related to #103785

  7. AndyAyersMS commented on Jun 25, 2024

    @AndyAyersMS
    AndyAyersMS
    on Jun 25, 2024
    Member

    Regressions:

    Improvements:

  8. DrewScoggins
    assigned
    amanasifkhalid
    on Jun 25, 2024
  9. jeffschwMSFT
    added
    area-CodeGen-coreclrCLR JIT compiler in src/coreclr/src/jit and related components such as SuperPMI
    on Jul 1, 2024
  10. JulieLeeMSFT
    removed
    untriagedNew issue has not been triaged by the area owner
    on Jul 5, 2024
  11. JulieLeeMSFT
    added this to the 9.0.0 milestone on Jul 5, 2024
  12. JulieLeeMSFT
    added
    Priority:2Work that is important, but not critical for the release
    on Jul 5, 2024

  13. AndyAyersMS commented on Jul 19, 2024

    @AndyAyersMS
    AndyAyersMS
    on Jul 19, 2024
    Member

    Collated Regression Table

    Notes Recent Score Orig Score arm64 ubuntu arm64 Windows x64 ubuntu x64 Windows amd64 Windows Benchmark
    1.85 1.81 1.85
    1.81    		 System.Collections.CtorFromCollection(Int32).ImmutableStack(Size: 512)
    1.73 1.72 1.73
    1.72    		 System.Collections.IterateForEach(String).ImmutableSortedDictionary(Size: 512)
    1.59 1.60 1.59
    1.60    		 System.Collections.IterateForEachNonGeneric(String).ArrayList(Size: 512)
    1.59 1.25 1.59
    1.25    		 System.Collections.ContainsTrueComparer(Int32).ImmutableSortedSet(Size: 512)
    1.52 1.47 1.52
    1.47    		 System.Collections.IterateForEach(String).ImmutableSortedSet(Size: 512)
    1.52 1.38 1.34
    1.36    		 1.74
    1.40    		 System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "(?i)Sherlock", Options: NonBacktracking)
    1.50 1.50 1.50
    1.50    		 Benchstone.BenchI.BubbleSort2.Test
    1.46 2.02 1.46
    2.02    		 System.Collections.CtorFromCollection(Int32).ConcurrentStack(Size: 512)
    1.44 1.43 1.40
    1.39    		 1.49
    1.48    		 System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "(?i)Sherlock Holmes", Options: None)
    1.41 1.44 1.41
    1.44    		 System.Collections.IterateForEach(String).ImmutableList(Size: 512)
    1.40 1.40 1.40
    1.40    		 System.Collections.IterateForEachNonGeneric(String).Queue(Size: 512)
    1.38 1.55 1.28
    1.50    		 1.48
    1.60    		 System.Collections.IterateForEachNonGeneric(Int32).ArrayList(Size: 512)
    1.36 1.36 1.36
    1.36    		 System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "(?i)Sherlock", Options: None)
    1.34 1.31 1.44
    1.44    		 1.40
    1.25    		 1.28
    1.28    		 1.29
    1.29    		 1.32
    1.32    		 System.Collections.IterateForEach(String).Queue(Size: 512)
    1.34 1.38 1.39
    1.39    		 1.29
    1.38    		 System.Collections.IterateForEachNonGeneric(Int32).Queue(Size: 512)
    1.32 1.38 1.39
    1.40    		 1.26
    1.37    		 System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "(?i)Sherlock Holmes", Options: NonBacktracking)
    1.31 1.28 1.32
    1.32    		 1.38
    1.39    		 1.23
    1.15    		 System.Linq.Tests.Perf_Enumerable.All_AllElementsMatch(input: IEnumerable)
    1.30 1.26 1.45
    1.45    		 1.35
    1.13    		 1.44
    1.44    		 1.16
    1.16    		 1.14
    1.15    		 System.Linq.Tests.Perf_Enumerable.SingleWithPredicate_LastElementMatches(input: List)
    1.29 1.22 1.29
    1.22    		 System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "\p{Lu}", Options: None)
    1.28 1.28 1.28
    1.28    		 System.Linq.Tests.Perf_Enumerable.WhereAny_LastElementMatches(input: Array)
    1.27 1.24 1.28
    1.23    		 1.27
    1.25    		 System.Linq.Tests.Perf_Enumerable.SelectToArray(input: IEnumerable)
    1.27 1.24 1.36
    1.37    		 1.47
    1.39    		 1.08
    1.08    		 1.19
    1.14    		 System.Linq.Tests.Perf_Enumerable.FirstWithPredicate_LastElementMatches(input: Array)
    1.24 1.24 1.24
    1.24    		 System.Linq.Tests.Perf_Enumerable.SelectToArray(input: List)
    1.24 1.25 1.15
    1.15    		 1.33
    1.37    		 1.26
    1.25    		 System.Collections.IterateForEach(String).Stack(Size: 512)
    1.24 1.26 1.23
    1.23    		 1.41
    1.65    		 1.41
    1.22    		 1.06
    1.06    		 1.15
    1.20    		 System.Linq.Tests.Perf_Enumerable.Aggregate_Seed(input: IEnumerable)
    1.24 1.34 1.21
    1.21    		 1.31
    1.73    		 1.27
    1.28    		 1.19
    1.19    		 SciMark2.kernel.benchSparseMult
    1.24 1.26 1.24
    1.26    		 System.Linq.Tests.Perf_Enumerable.Concat_TenTimes(input: IEnumerable)
    1.23 1.31 1.34
    1.36    		 1.22
    1.37    		 1.15
    1.22    		 System.Linq.Tests.Perf_Enumerable.AnyWithPredicate_LastElementMatches(input: Array)
    1.23 1.23 1.23
    1.23    		 System.Collections.Perf_Frozen(Int16).Contains_True(Count: 4)
    1.23 1.18 1.14
    1.16    		 1.44
    1.18    		 1.21
    1.21    		 1.14
    1.19    		 System.Linq.Tests.Perf_Enumerable.Sum(input: IEnumerable)
    1.23 1.25 1.33
    1.33    		 1.13
    1.17    		 System.Linq.Tests.Perf_Enumerable.WhereAny_LastElementMatches(input: IEnumerable)
    1.22 1.23 1.21
    1.19    		 1.35
    1.38    		 1.13
    1.12    		 System.Linq.Tests.Perf_Enumerable.WhereFirst_LastElementMatches(input: IEnumerable)
    1.21 1.24 1.10
    1.23    		 1.25
    1.12    		 1.28
    1.40    		 System.Linq.Tests.Perf_Enumerable.Repeat
    1.21 1.21 1.21
    1.21    		 System.Linq.Tests.Perf_Enumerable.Take_All(input: IEnumerable)
    1.20 1.22 1.20
    1.22    		 System.Net.Tests.Perf_WebUtility.Decode_NoDecodingRequired
    1.20 1.22 1.20
    1.22    		 System.Collections.Tests.Perf_BitArray.BitArrayLeftShift(Size: 4)
    1.20 1.26 1.20
    1.26    		 Microsoft.Extensions.Primitives.StringSegmentBenchmark.Equals_Object_Valid
    1.19 1.20 1.35
    1.35    		 1.06
    1.06    		 1.19
    1.22    		 System.Linq.Tests.Perf_Enumerable.SingleWithPredicate_LastElementMatches(input: IEnumerable)
    1.19 1.20 1.19
    1.21    		 1.35
    1.35    		 1.10
    1.10    		 1.14
    1.14    		 System.Linq.Tests.Perf_Enumerable.FirstWithPredicate_LastElementMatches(input: List)
    1.19 1.11 1.19
    1.11    		 Benchstone.BenchI.Midpoint.Test
    1.19 1.17 0.84
    0.84    		 1.68
    1.64    		 System.Linq.Tests.Perf_Enumerable.Count(input: IEnumerable)
    1.19 1.19 1.19
    1.19    		 Benchstone.BenchI.IniArray.Test
    1.18 1.20 1.27
    1.27    		 1.13
    1.11    		 1.21
    1.20    		 1.13
    1.22    		 System.Linq.Tests.Perf_Enumerable.FirstWithPredicate_LastElementMatches(input: IEnumerable)
    1.18 1.29 1.18
    1.29    		 System.Collections.Perf_Frozen(Int16).Contains_True(Count: 64)
    1.18 1.15 1.23
    1.16    		 1.16
    1.16    		 1.15
    1.14    		 Benchstone.BenchF.InProd.Test
    1.18 1.15 1.18
    1.15    		 System.Collections.IterateForEach(String).ConcurrentBag(Size: 512)
    1.18 1.28 1.18
    1.28    		 Benchstone.BenchI.NDhrystone.Test
    1.17 1.24 1.34
    1.31    		 1.16
    1.45    		 1.07
    1.07    		 1.15
    1.18    		 System.Linq.Tests.Perf_Enumerable.SingleWithPredicate_LastElementMatches(input: Array)
    1.17 1.19 1.17
    1.19    		 System.Linq.Tests.Perf_Enumerable.SelectToList(input: IList)
    1.17 1.17 1.22
    1.21    		 1.13
    1.13    		 System.Collections.IterateForEachNonGeneric(String).Stack(Size: 512)
    1.17 1.18 1.17
    1.18    		 System.Text.Json.Tests.Utf8JsonReaderCommentsTests.Utf8JsonReaderCommentParsing(CommentHandling: Skip, SegmentSize: 100, TestCase: ShortMultiLine)
    1.17 1.16 1.17
    1.17    		 1.16
    1.16    		 System.Linq.Tests.Perf_Enumerable.Where(input: IEnumerable)
    1.16 1.21 1.15
    1.14    		 1.18
    1.28    		 System.Collections.IterateForEach(Int32).IEnumerable(Size: 512)
    1.16 1.18 1.34
    1.34    		 1.06
    1.14    		 1.07
    1.08    		 1.17
    1.18    		 System.Linq.Tests.Perf_Enumerable.AnyWithPredicate_LastElementMatches(input: IEnumerable)
    1.15 1.16 1.21
    1.22    		 1.10
    1.10    		 System.Collections.IterateForEachNonGeneric(Int32).Stack(Size: 512)
    1.14 1.16 1.14
    1.16    		 System.Linq.Tests.Perf_Enumerable.TakeLastHalf(input: IEnumerable)
    1.14 1.14 1.19
    1.19    		 1.10
    1.10    		 1.14
    1.14    		 System.Linq.Tests.Perf_Enumerable.AnyWithPredicate_LastElementMatches(input: List)
    1.14 1.15 1.14
    1.15    		 System.Linq.Tests.Perf_Enumerable.Select(input: List)
    1.14 1.14 1.14
    1.14    		 System.Net.Primitives.Tests.CredentialCacheTests.ForEach(uriCount: 0, hostPortCount: 10)
    1.14 1.14 1.08
    1.08    		 1.19
    1.20    		 System.Linq.Tests.Perf_Enumerable.WhereSelect(input: Array)
    1.14 1.14 1.14
    1.14    		 System.Text.Json.Tests.Utf8JsonReaderCommentsTests.Utf8JsonReaderCommentParsing(CommentHandling: Skip, SegmentSize: 0, TestCase: LongSingleLine)
    1.13 1.13 1.13
    1.13    		 System.Linq.Tests.Perf_Enumerable.Where(input: Array)
    1.13 1.13 1.13
    1.13    		 System.Net.Primitives.Tests.CredentialCacheTests.ForEach(uriCount: 10, hostPortCount: 0)
    1.13 1.24 1.13
    1.40    		 1.12
    1.10    		 System.Linq.Tests.Perf_Enumerable.Select(input: Array)
    1.12 1.11 1.12
    1.11    		 LinqBenchmarks.Order00ManualX
    1.12 1.12 1.12
    1.12    		 System.Text.Json.Document.Tests.Perf_EnumerateArray.EnumerateUsingIndexer(TestCase: ArrayOfStrings)
    1.12 1.13 1.12
    1.13    		 Benchstone.MDBenchI.MDNDhrystone.Test
    1.12 1.15 1.11
    1.19    		 1.12
    1.12    		 1.12
    1.13    		 System.Linq.Tests.Perf_Enumerable.CastToSameType(input: IEnumerable)
    1.11 1.07 1.11
    1.07    		 System.Collections.IterateForEach(String).ConcurrentQueue(Size: 512)
    1.10 1.15 1.22
    1.24    		 1.00
    1.07    		 System.Linq.Tests.Perf_Enumerable.Concat_Once(input: IEnumerable)
    1.10 1.16 1.10
    1.16    		 System.Text.Tests.Perf_Encoding.GetByteCount(size: 16, encName: "utf-8")
    1.10 1.10 1.10
    1.10    		 System.Linq.Tests.Perf_Enumerable.Where(input: List)
    1.09 1.09 1.09
    1.09    		 System.Collections.IterateForEach(Int32).ConcurrentQueue(Size: 512)
    1.09 1.09 1.09
    1.09    		 System.Collections.IterateForEach(String).IEnumerable(Size: 512)
    1.09 1.06 1.15
    1.06    		 1.07
    1.07    		 1.04
    1.06    		 System.Linq.Tests.Perf_Enumerable.Reverse(input: IEnumerable)
    1.09 1.08 1.11
    1.11    		 1.06
    1.06    		 System.Linq.Tests.Perf_Enumerable.WhereSelect(input: IEnumerable)
    1.08 1.07 1.08
    1.07    		 System.Collections.Perf_Frozen(Int16).Contains_True(Count: 512)
    1.08 1.08 1.08
    1.08    		 System.Linq.Tests.Perf_Enumerable.WhereSingle_LastElementMatches(input: List)
    1.08 1.12 1.08
    1.12    		 System.Collections.IterateForEachNonGeneric(Int32).SortedList(Size: 512)
    1.07 1.09 1.07
    1.09    		 System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Holmes", Options: None)
    1.07 1.09 1.07
    1.07    		 1.07
    1.11    		 SciMark2.kernel.benchMonteCarlo
    1.07 1.09 1.07
    1.09    		 System.Collections.Perf_Frozen(Int16).ToFrozenSet(Count: 4)
    1.07 1.07 1.07
    1.07    		 System.Linq.Tests.Perf_Enumerable.WhereSingleOrDefault_LastElementMatches(input: List)
    1.06 1.07 0.86
    0.85    		 1.31
    1.34    		 System.Linq.Tests.Perf_Enumerable.ElementAt(input: IEnumerable)
    1.06 1.08 1.06
    1.08    		 System.Linq.Tests.Perf_Enumerable.TakeLastHalf(input: List)
    1.06 1.09 1.02
    1.08    		 1.10
    1.11    		 LinqBenchmarks.Count00LinqMethodX
    1.06 1.06 1.06
    1.06    		 BilinearTest.Interpol_Scalar
    1.05 1.09 1.05
    1.09    		 System.Collections.IterateForEachNonGeneric(Int32).Hashtable(Size: 512)
    1.05 1.06 1.05
    1.06    		 System.Linq.Tests.Perf_Enumerable.WhereSelect(input: List)
    1.04 1.10 1.04
    1.10    		 System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "The", Options: None)
    1.04 1.11 1.04
    1.11    		 System.Buffers.Tests.ReadOnlySequenceTests(Char).IterateGetPositionMemory
    1.03 1.14 0.94
    1.07    		 1.13
    1.22    		 System.Linq.Tests.Perf_Enumerable.SelectToList(input: IEnumerable)
    1.03 1.07 1.03
    1.07    		 System.Linq.Tests.Perf_Enumerable.Except(input: IEnumerable)
    1.02 1.01 0.82
    0.77    		 1.28
    1.32    		 System.Linq.Tests.Perf_Enumerable.Range
    1.02 1.19 1.02
    1.19    		 System.Collections.IterateForEach(Int32).ConcurrentDictionary(Size: 512)
    1.02 1.10 1.02
    1.10    		 System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sherlock", Options: None)
    1.00 1.13 1.00
    1.13    		 System.Xml.Linq.Perf_XElement.GetValue
    1.00 1.16 1.00
    1.16    		 System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "zqj", Options: None)
    0.98 1.09 0.98
    1.09    		 System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sherlock Holmes", Options: NonBacktracking)
    0.97 1.24 0.97
    1.24    		 System.Tests.Perf_Version.TryFormatL
    0.96 1.13 0.96
    1.13    		 System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "aqj", Options: None)
    0.94 1.11 0.94
    1.11    		 System.Diagnostics.Perf_Activity.EnumerateActivityTagsLarge
    0.94 1.11 0.94
    1.11    		 System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "aei", Options: None)
    0.90 0.90 0.90
    0.90    		 System.Text.Perf_Utf8Encoding.GetByteCount(Input: Cyrillic)
    0.89 0.90 0.89
    0.90    		 System.Text.Perf_Utf8Encoding.GetBytes(Input: Chinese)
    0.84 0.89 0.84
    0.89    		 System.Diagnostics.Perf_Activity.EnumerateActivityTagObjectsLarge
    0.84 0.83 0.84
    0.83    		 System.Numerics.Tensors.Tests.Perf_NumberTensorPrimitives(Double).IndexOfMax(BufferLength: 3079)
    0.84 2.34 0.84
    2.34    		 GuardedDevirtualization.ThreeClassVirtual.Call(testInput: pB=0.00 pD=0.67)
    0.83 2.29 0.83
    2.29    		 GuardedDevirtualization.ThreeClassVirtual.Call(testInput: pB=0.67 pD=0.33)
    0.83 2.31 0.83
    2.31    		 GuardedDevirtualization.ThreeClassVirtual.Call(testInput: pB=0.67 pD=0.00)
    0.83 1.09 0.83
    1.09    		 System.Collections.Tests.Perf_PriorityQueue(Int32, Int32).HeapSort(Size: 10)
  14. JulieLeeMSFT
    modified the milestones: 9.0.0, 10.0.0 on Aug 1, 2024

  15. JulieLeeMSFT commented on Aug 1, 2024

    @JulieLeeMSFT
    JulieLeeMSFT
    on Aug 1, 2024
    Member

    pushing to .NET 10 becasue any fix on flow graph has a high risk.

  16. amanasifkhalid commented on Apr 28, 2025

    @amanasifkhalid
    amanasifkhalid
    on Apr 28, 2025
    Contributor

    Updated table:

    Notes Recent Score Orig Score arm64 Ubuntu arm64 Windows x64 Ubuntu x64 Windows amd64 Windows Benchmark
    1.65 1.50 1.65
    1.50    		 Benchstone.BenchI.BubbleSort2.Test
    1.56 1.72 1.56
    1.72    		 System.Collections.IterateForEach(String).ImmutableSortedDictionary(Size: 512)
    1.49 1.64 1.49
    1.64    		 System.Linq.Tests.Perf_Enumerable.Count(input: IEnumerable)
    1.46 1.19 1.46
    1.19    		 Benchstone.BenchI.IniArray.Test
    1.33 1.44 1.33
    1.44    		 System.Collections.IterateForEach(String).ImmutableList(Size: 512)
    1.32 1.24 1.32
    1.24    		 System.Tests.Perf_Version.TryFormatL
    1.25 1.34 1.26
    1.21    		 1.31
    1.73    		 1.29
    1.28    		 1.17
    1.19    		 SciMark2.kernel.benchSparseMult
    1.21 1.16 1.22
    1.16    		 1.20
    1.16    		 Benchstone.BenchF.InProd.Test
    1.18 1.18 1.18
    1.18    		 System.Text.Json.Tests.Utf8JsonReaderCommentsTests.Utf8JsonReaderCommentParsing(CommentHandling: Skip, SegmentSize: 100, TestCase: ShortMultiLine)
    1.17 1.09 1.17
    1.09    		 System.Collections.IterateForEach(String).IEnumerable(Size: 512)
    1.16 1.12 1.16
    1.12    		 System.Text.Json.Document.Tests.Perf_EnumerateArray.EnumerateUsingIndexer(TestCase: ArrayOfStrings)
    1.12 1.25 1.12
    1.25    		 System.Collections.IterateForEach(String).Stack(Size: 512)
    1.12 1.09 1.12
    1.09    		 System.Collections.IterateForEach(Int32).ConcurrentQueue(Size: 512)
    1.10 1.09 1.06
    1.07    		 1.13
    1.11    		 SciMark2.kernel.benchMonteCarlo
    1.09 1.43 1.06
    1.39    		 1.12
    1.48    		 System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "(?i)Sherlock Holmes", Options: None)
    1.08 1.17 1.08
    1.17    		 System.Linq.Tests.Perf_Enumerable.Where(input: IEnumerable)
    1.08 1.34 1.08
    1.34    		 System.Linq.Tests.Perf_Enumerable.ElementAt(input: IEnumerable)
    1.07 1.33 1.14
    1.44    		 0.91
    1.25    		 1.18
    1.32    		 System.Collections.IterateForEach(String).Queue(Size: 512)
    1.07 1.14 1.07
    1.14    		 System.Net.Primitives.Tests.CredentialCacheTests.ForEach(uriCount: 0, hostPortCount: 10)
    1.07 1.38 1.05
    1.40    		 1.09
    1.37    		 System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "(?i)Sherlock Holmes", Options: NonBacktracking)
    1.07 1.11 1.07
    1.11    		 System.Linq.Tests.Perf_Enumerable.WhereSelect(input: IEnumerable)
    1.06 1.25 1.06
    1.25    		 System.Collections.ContainsTrueComparer(Int32).ImmutableSortedSet(Size: 512)
    1.06 1.16 1.06
    1.16    		 System.Linq.Tests.Perf_Enumerable.TakeLastHalf(input: IEnumerable)
    1.06 1.38 1.03
    1.36    		 1.08
    1.40    		 System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "(?i)Sherlock", Options: NonBacktracking)
    1.05 1.36 1.05
    1.36    		 System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "(?i)Sherlock", Options: None)
    1.05 1.06 1.05
    1.06    		 System.Linq.Tests.Perf_Enumerable.WhereSelect(input: List)
    1.05 1.22 1.05
    1.22    		 System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "\p{Lu}", Options: None)
    1.05 1.15 1.05
    1.15    		 System.Linq.Tests.Perf_Enumerable.Select(input: List)
    1.04 1.06 1.01
    1.06    		 1.08
    1.06    		 System.Linq.Tests.Perf_Enumerable.Reverse(input: IEnumerable)
    1.04 1.11 1.04
    1.11    		 LinqBenchmarks.Order00ManualX
    1.03 1.13 1.03
    1.13    		 System.Net.Primitives.Tests.CredentialCacheTests.ForEach(uriCount: 10, hostPortCount: 0)
    1.03 1.10 1.03
    1.10    		 System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "The", Options: None)
    1.02 1.19 1.02
    1.19    		 System.Collections.IterateForEach(Int32).ConcurrentDictionary(Size: 512)
    1.02 1.13 1.02
    1.13    		 System.Linq.Tests.Perf_Enumerable.Where(input: Array)
    1.02 1.31 1.01
    1.23    		 1.15
    1.65    		 0.99
    1.22    		 0.93
    1.20    		 System.Linq.Tests.Perf_Enumerable.Aggregate_Seed(input: IEnumerable)
    1.02 1.28 1.06
    1.35    		 0.98
    1.22    		 System.Linq.Tests.Perf_Enumerable.SingleWithPredicate_LastElementMatches(input: IEnumerable)
    1.01 1.47 1.01
    1.47    		 System.Collections.IterateForEach(String).ImmutableSortedSet(Size: 512)
    1.01 1.22 1.01
    1.22    		 System.Net.Tests.Perf_WebUtility.Decode_NoDecodingRequired
    1.01 1.31 1.04
    1.23    		 0.98
    1.40    		 System.Linq.Tests.Perf_Enumerable.Repeat
    1.00 1.21 1.00
    1.21    		 System.Collections.IterateForEachNonGeneric(String).Stack(Size: 512)
    1.00 1.22 1.00
    1.22    		 System.Collections.IterateForEachNonGeneric(Int32).Stack(Size: 512)
    1.00 1.11 1.00
    1.11    		 System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "aei", Options: None)
    1.00 1.23 1.03
    1.32    		 0.97
    1.15    		 System.Linq.Tests.Perf_Enumerable.All_AllElementsMatch(input: IEnumerable)
    1.00 1.09 1.00
    1.09    		 System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Holmes", Options: None)
    1.00 1.22 1.00
    1.22    		 System.Collections.Tests.Perf_BitArray.BitArrayLeftShift(Size: 4)
    0.99 1.16 0.99
    1.16    		 System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "zqj", Options: None)
    0.99 1.13 0.99
    1.13    		 System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "aqj", Options: None)
    0.99 1.10 0.99
    1.10    		 System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sherlock", Options: None)
    0.99 1.11 0.99
    1.11    		 Benchstone.BenchI.Midpoint.Test
    0.99 1.18 1.03
    1.16    		 0.99
    1.18    		 0.95
    1.19    		 System.Linq.Tests.Perf_Enumerable.Sum(input: IEnumerable)
    0.99 1.32 0.99
    1.32    		 System.Linq.Tests.Perf_Enumerable.Range
    0.98 1.16 1.01
    1.19    		 0.96
    1.13    		 System.Linq.Tests.Perf_Enumerable.CastToSameType(input: IEnumerable)
    0.98 1.10 0.98
    1.10    		 System.Linq.Tests.Perf_Enumerable.Where(input: List)
    0.98 1.14 0.98
    1.14    		 System.Text.Json.Tests.Utf8JsonReaderCommentsTests.Utf8JsonReaderCommentParsing(CommentHandling: Skip, SegmentSize: 0, TestCase: LongSingleLine)
    0.97 1.15 0.98
    1.19    		 0.96
    1.12    		 System.Linq.Tests.Perf_Enumerable.WhereFirst_LastElementMatches(input: IEnumerable)
    0.97 1.24 0.93
    1.40    		 1.01
    1.10    		 System.Linq.Tests.Perf_Enumerable.Select(input: Array)
    0.97 1.11 0.97
    1.11    		 System.Diagnostics.Perf_Activity.EnumerateActivityTagsLarge
    0.96 1.17 0.96
    1.17    		 System.Linq.Tests.Perf_Enumerable.WhereAny_LastElementMatches(input: IEnumerable)
    0.96 1.09 0.96
    1.09    		 System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sherlock Holmes", Options: NonBacktracking)
    0.95 1.23 0.95
    1.23    		 System.Linq.Tests.Perf_Enumerable.SelectToArray(input: IEnumerable)
    0.95 1.06 0.95
    1.06    		 BilinearTest.Interpol_Scalar
    0.94 1.22 1.04
    1.34    		 0.79
    1.14    		 0.99
    1.18    		 System.Linq.Tests.Perf_Enumerable.AnyWithPredicate_LastElementMatches(input: IEnumerable)
    0.93 1.20 0.99
    1.27    		 0.84
    1.11    		 0.96
    1.22    		 System.Linq.Tests.Perf_Enumerable.FirstWithPredicate_LastElementMatches(input: IEnumerable)
    0.93 1.08 0.93
    1.08    		 System.Linq.Tests.Perf_Enumerable.WhereSelect(input: Array)
    0.91 1.07 0.91
    1.07    		 System.Linq.Tests.Perf_Enumerable.Except(input: IEnumerable)
    0.91 1.14 0.99
    1.07    		 0.83
    1.22    		 System.Linq.Tests.Perf_Enumerable.SelectToList(input: IEnumerable)
    0.87 1.19 0.87
    1.19    		 System.Linq.Tests.Perf_Enumerable.SelectToList(input: IList)
    0.82 2.02 0.82
    2.02    		 System.Collections.CtorFromCollection(Int32).ConcurrentStack(Size: 512)
    0.81 1.26 0.81
    1.26    		 System.Linq.Tests.Perf_Enumerable.Concat_TenTimes(input: IEnumerable)
    0.70 1.24 0.70
    1.24    		 System.Linq.Tests.Perf_Enumerable.Concat_Once(input: IEnumerable)
    0.62 2.31 0.62
    2.31    		 GuardedDevirtualization.ThreeClassVirtual.Call(testInput: pB=0.67 pD=0.00)
    0.62 2.34 0.62
    2.34    		 GuardedDevirtualization.ThreeClassVirtual.Call(testInput: pB=0.00 pD=0.67)
    0.61 2.29 0.61
    2.29    		 GuardedDevirtualization.ThreeClassVirtual.Call(testInput: pB=0.67 pD=0.33)
    0.59 1.81 0.59
    1.81    		 System.Collections.CtorFromCollection(Int32).ImmutableStack(Size: 512)
    0.38 1.31 0.33
    1.31    		 0.30
    1.45    		 0.54
    1.18    		 System.Linq.Tests.Perf_Enumerable.SingleWithPredicate_LastElementMatches(input: Array)
    0.36 1.21 0.35
    1.14    		 0.37
    1.28    		 System.Collections.IterateForEach(Int32).IEnumerable(Size: 512)
    0.32 1.29 0.33
    1.36    		 0.31
    1.22    		 System.Linq.Tests.Perf_Enumerable.AnyWithPredicate_LastElementMatches(input: Array)
    0.31 1.29 0.34
    1.37    		 0.32
    1.39    		 0.27
    1.14    		 System.Linq.Tests.Perf_Enumerable.FirstWithPredicate_LastElementMatches(input: Array)
    0.25 1.28 0.28
    1.45    		 0.22
    1.13    		 0.24
    1.44    		 0.24
    1.15    		 System.Linq.Tests.Perf_Enumerable.SingleWithPredicate_LastElementMatches(input: List)
    0.22 1.16 0.22
    1.19    		 0.23
    1.14    		 System.Linq.Tests.Perf_Enumerable.AnyWithPredicate_LastElementMatches(input: List)
    0.22 1.23 0.22
    1.21    		 0.18
    1.35    		 0.25
    1.14    		 System.Linq.Tests.Perf_Enumerable.FirstWithPredicate_LastElementMatches(input: List)
    0.06 1.08 0.06
    1.08    		 LinqBenchmarks.Count00LinqMethodX

  17. amanasifkhalid commented on Apr 28, 2025

    @amanasifkhalid
    amanasifkhalid
    on Apr 28, 2025
    Contributor

    The remaining regressions fall into one of the following categories:

    I think it's worth keeping this issue open to track the last category. Moving to future.

  18. amanasifkhalid
    modified the milestones: 10.0.0, Future on Apr 28, 2025
  19. JulieLeeMSFT
    assigned
    AndyAyersMS
    and unassigned
    amanasifkhalid
    on Apr 17, 2026

  20. AndyAyersMS commented on Jun 27, 2026

    @AndyAyersMS
    AndyAyersMS
    on Jun 27, 2026
    Member

    Closing — the work tracked here has shipped.

    Summary of resolution:

    Residual benchmark sensitivity (BubbleSort2, etc.) is now tracked individually by the perf autofiler when it actually regresses (e.g. perf-autofiling-issues#64975, #64446 on Linux/x64; #60813 on Windows/x86). Notably perf-autofiling-issues#72932 (May 2026) shows a batch of BubbleSort2 improvements on Windows/x64. Keeping this umbrella open 14 months past last activity isn''t adding value — individual perf regressions get their own per-event tracking issue.

    Thanks @amanasifkhalid for the systematic triage.

    Note

    Wrap-up by GitHub Copilot CLI.

  21. AndyAyersMS
    closed this as completedon Jun 27, 2026
  22. github-actions
    locked and limited conversation to collaborators on Jul 28, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

Priority:2Work that is important, but not critical for the releasearch-x64area-CodeGen-coreclrCLR JIT compiler in src/coreclr/src/jit and related components such as SuperPMIos-windowsruntime-coreclrspecific to the CoreCLR runtimetenet-performancePerformance related issuetenet-performance-benchmarksIssue from performance benchmark

Type

No type

Projects

No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions