Skip to content

Performance regression: System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf8 #41104

Closed
Closed
Performance regression: System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf8#41104
Assignees
kunalspathak
Labels
area-CodeGen-coreclrCLR JIT compiler in src/coreclr/src/jit and related components such as SuperPMItenet-performancePerformance related issue
Milestone
6.0.0

Description

@adamsitnik
adamsitnik
opened on Aug 20, 2020

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f netcoreapp3.1 netcoreapp5.0 --filter System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf8

System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf8(arguments: UnsafeRelaxed,no required,512)

Conclusion Base Diff Base/Diff Modality Operating System Bit Processor Name Base Runtime Diff Runtime
Slower 1809.34 3014.46 0.60 Windows 10.0.18363.1016 Arm Microsoft SQ1 3.0 GHz .NET Core 3.1.6 5.0.100-rc.1.20413.9
Slower 1076.94 1540.30 0.70 Windows 10.0.18363.959 X64 Intel Xeon CPU E5-1650 v4 3.60GHz .NET Core 3.1.6 5.0.100-rc.1.20418.3
Slower 950.47 1186.75 0.80 Windows 10.0.19042 X64 Intel Core i7-7700 CPU 3.60GHz (Kaby Lake) .NET Core 3.1.6 5.0.100-rc.1.20418.3
Slower 1142.73 1765.04 0.65 several? Windows 10.0.18363.959 X86 Intel Xeon CPU E5-1650 v4 3.60GHz .NET Core 3.1.6 5.0.100-rc.1.20420.2
Slower 1348.91 2244.69 0.60 Windows 10.0.19041.450 X86 Intel Core i7-5557U CPU 3.10GHz (Broadwell) .NET Core 3.1.6 5.0.100-rc.1.20419.5

System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf8(arguments: JavaScript,no escaping required,512)

Conclusion Base Diff Base/Diff Modality Operating System Bit Processor Name Base Runtime Diff Runtime
Slower 1805.25 2974.82 0.61 Windows 10.0.18363.1016 Arm Microsoft SQ1 3.0 GHz .NET Core 3.1.6 5.0.100-rc.1.20413.9
Slower 1074.65 1529.30 0.70 Windows 10.0.18363.959 X64 Intel Xeon CPU E5-1650 v4 3.60GHz .NET Core 3.1.6 5.0.100-rc.1.20418.3
Slower 943.09 1184.95 0.80 Windows 10.0.19042 X64 Intel Core i7-7700 CPU 3.60GHz (Kaby Lake) .NET Core 3.1.6 5.0.100-rc.1.20418.3
Slower 1146.45 1769.32 0.65 Windows 10.0.18363.959 X86 Intel Xeon CPU E5-1650 v4 3.60GHz .NET Core 3.1.6 5.0.100-rc.1.20420.2
Slower 1330.70 2227.21 0.60 Windows 10.0.19041.450 X86 Intel Core i7-5557U CPU 3.10GHz (Broadwell) .NET Core 3.1.6 5.0.100-rc.1.20419.5

System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf8(arguments: UnsafeRelaxed,hello "there",512)

Conclusion Base Diff Base/Diff Modality Operating System Bit Processor Name Base Runtime Diff Runtime
Slower 1783.16 2957.33 0.60 Windows 10.0.18363.1016 Arm Microsoft SQ1 3.0 GHz .NET Core 3.1.6 5.0.100-rc.1.20413.9
Slower 1111.22 1524.46 0.73 Windows 10.0.18363.959 X64 Intel Xeon CPU E5-1650 v4 3.60GHz .NET Core 3.1.6 5.0.100-rc.1.20418.3
Slower 939.13 1176.89 0.80 Windows 10.0.19042 X64 Intel Core i7-7700 CPU 3.60GHz (Kaby Lake) .NET Core 3.1.6 5.0.100-rc.1.20418.3
Slower 1344.53 2224.69 0.60 Windows 10.0.19041.450 X86 Intel Core i7-5557U CPU 3.10GHz (Broadwell) .NET Core 3.1.6 5.0.100-rc.1.20419.5

System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf8(arguments: JavaScript,&Hello+!,512)

Conclusion Base Diff Base/Diff Modality Operating System Bit Processor Name Base Runtime Diff Runtime
Slower 1872.80 3055.42 0.61 Windows 10.0.18363.1016 Arm Microsoft SQ1 3.0 GHz .NET Core 3.1.6 5.0.100-rc.1.20413.9
Slower 1092.13 1545.38 0.71 Windows 10.0.18363.959 X64 Intel Xeon CPU E5-1650 v4 3.60GHz .NET Core 3.1.6 5.0.100-rc.1.20418.3
Slower 955.69 1194.21 0.80 Windows 10.0.19042 X64 Intel Core i7-7700 CPU 3.60GHz (Kaby Lake) .NET Core 3.1.6 5.0.100-rc.1.20418.3
Slower 1385.56 2252.29 0.62 Windows 10.0.19041.450 X86 Intel Core i7-5557U CPU 3.10GHz (Broadwell) .NET Core 3.1.6 5.0.100-rc.1.20419.5

System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf8(arguments: Url,c2020,512)

Conclusion Base Diff Base/Diff Modality Operating System Bit Processor Name Base Runtime Diff Runtime
Slower 1798.51 3055.97 0.59 Windows 10.0.18363.1016 Arm Microsoft SQ1 3.0 GHz .NET Core 3.1.6 5.0.100-rc.1.20413.9
Slower 1130.74 1516.15 0.75 Windows 10.0.18363.959 X64 Intel Xeon CPU E5-1650 v4 3.60GHz .NET Core 3.1.6 5.0.100-rc.1.20418.3
Slower 936.81 1176.26 0.80 Windows 10.0.19042 X64 Intel Core i7-7700 CPU 3.60GHz (Kaby Lake) .NET Core 3.1.6 5.0.100-rc.1.20418.3
Slower 1350.33 2216.17 0.61 Windows 10.0.19041.450 X86 Intel Core i7-5557U CPU 3.10GHz (Broadwell) .NET Core 3.1.6 5.0.100-rc.1.20419.5

System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf8(arguments: Url,&lorem ipsum=dolor sit amet,512)

Conclusion Base Diff Base/Diff Modality Operating System Bit Processor Name Base Runtime Diff Runtime
Slower 2112.16 3117.17 0.68 Windows 10.0.18363.1016 Arm Microsoft SQ1 3.0 GHz .NET Core 3.1.6 5.0.100-rc.1.20413.9
Slower 1162.38 1610.63 0.72 Windows 10.0.18363.959 X64 Intel Xeon CPU E5-1650 v4 3.60GHz .NET Core 3.1.6 5.0.100-rc.1.20418.3
Slower 979.94 1247.30 0.79 Windows 10.0.19042 X64 Intel Core i7-7700 CPU 3.60GHz (Kaby Lake) .NET Core 3.1.6 5.0.100-rc.1.20418.3
Slower 1251.98 1856.24 0.67 Windows 10.0.18363.959 X86 Intel Xeon CPU E5-1650 v4 3.60GHz .NET Core 3.1.6 5.0.100-rc.1.20420.2
Slower 1448.69 2325.45 0.62 Windows 10.0.19041.450 X86 Intel Core i7-5557U CPU 3.10GHz (Broadwell) .NET Core 3.1.6 5.0.100-rc.1.20419.5

System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf8(arguments: UnsafeRelaxed,no required,16)

Conclusion Base Diff Base/Diff Modality Operating System Bit Processor Name Base Runtime Diff Runtime
Slower 177.61 246.68 0.72 Windows 10.0.18363.1016 Arm Microsoft SQ1 3.0 GHz .NET Core 3.1.6 5.0.100-rc.1.20413.9
Slower 104.77 131.53 0.80 Windows 10.0.18363.959 X64 Intel Xeon CPU E5-1650 v4 3.60GHz .NET Core 3.1.6 5.0.100-rc.1.20418.3
Slower 92.71 102.98 0.90 Windows 10.0.19042 X64 Intel Core i7-7700 CPU 3.60GHz (Kaby Lake) .NET Core 3.1.6 5.0.100-rc.1.20418.3
Slower 104.18 142.99 0.73 Windows 10.0.18363.959 X86 Intel Xeon CPU E5-1650 v4 3.60GHz .NET Core 3.1.6 5.0.100-rc.1.20420.2
Slower 122.05 171.79 0.71 Windows 10.0.19041.450 X86 Intel Core i7-5557U CPU 3.10GHz (Broadwell) .NET Core 3.1.6 5.0.100-rc.1.20419.5

System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf8(arguments: JavaScript,no escaping required,16)

Conclusion Base Diff Base/Diff Modality Operating System Bit Processor Name Base Runtime Diff Runtime
Slower 146.39 224.46 0.65 Windows 10.0.18363.1016 Arm Microsoft SQ1 3.0 GHz .NET Core 3.1.6 5.0.100-rc.1.20413.9
Slower 97.42 116.05 0.84 Windows 10.0.18363.959 X64 Intel Xeon CPU E5-1650 v4 3.60GHz .NET Core 3.1.6 5.0.100-rc.1.20418.3
Slower 86.18 93.76 0.92 Windows 10.0.19042 X64 Intel Core i7-7700 CPU 3.60GHz (Kaby Lake) .NET Core 3.1.6 5.0.100-rc.1.20418.3
Slower 95.23 130.77 0.73 Windows 10.0.18363.959 X86 Intel Xeon CPU E5-1650 v4 3.60GHz .NET Core 3.1.6 5.0.100-rc.1.20420.2
Slower 111.51 152.55 0.73 Windows 10.0.19041.450 X86 Intel Core i7-5557U CPU 3.10GHz (Broadwell) .NET Core 3.1.6 5.0.100-rc.1.20419.5

System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf8(arguments: UnsafeRelaxed,hello "there",16)

Conclusion Base Diff Base/Diff Modality Operating System Bit Processor Name Base Runtime Diff Runtime
Slower 151.00 207.48 0.73 Windows 10.0.18363.1016 Arm Microsoft SQ1 3.0 GHz .NET Core 3.1.6 5.0.100-rc.1.20413.9
Slower 101.15 114.62 0.88 Windows 10.0.18363.959 X64 Intel Xeon CPU E5-1650 v4 3.60GHz .NET Core 3.1.6 5.0.100-rc.1.20418.3
Slower 77.07 93.02 0.83 Windows 10.0.19042 X64 Intel Core i7-7700 CPU 3.60GHz (Kaby Lake) .NET Core 3.1.6 5.0.100-rc.1.20418.3
Slower 121.72 152.87 0.80 Windows 10.0.19041.450 X86 Intel Core i7-5557U CPU 3.10GHz (Broadwell) .NET Core 3.1.6 5.0.100-rc.1.20419.5

System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf8(arguments: Url,&lorem ipsum=dolor sit amet,16)

Conclusion Base Diff Base/Diff Modality Operating System Bit Processor Name Base Runtime Diff Runtime
Slower 281.09 347.81 0.81 Windows 10.0.18363.1016 Arm Microsoft SQ1 3.0 GHz .NET Core 3.1.6 5.0.100-rc.1.20413.9
Slower 156.06 189.22 0.82 Windows 10.0.18363.959 X64 Intel Xeon CPU E5-1650 v4 3.60GHz .NET Core 3.1.6 5.0.100-rc.1.20418.3
Slower 118.21 151.30 0.78 Windows 10.0.19042 X64 Intel Core i7-7700 CPU 3.60GHz (Kaby Lake) .NET Core 3.1.6 5.0.100-rc.1.20418.3
Slower 186.27 223.79 0.83 Windows 10.0.18363.959 X86 Intel Xeon CPU E5-1650 v4 3.60GHz .NET Core 3.1.6 5.0.100-rc.1.20420.2
Slower 226.76 262.86 0.86 Windows 10.0.19041.450 X86 Intel Core i7-5557U CPU 3.10GHz (Broadwell) .NET Core 3.1.6 5.0.100-rc.1.20419.5

System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf8(arguments: JavaScript,&Hello+!,16)

Conclusion Base Diff Base/Diff Modality Operating System Bit Processor Name Base Runtime Diff Runtime
Slower 178.36 264.02 0.68 Windows 10.0.18363.1016 Arm Microsoft SQ1 3.0 GHz .NET Core 3.1.6 5.0.100-rc.1.20413.9
Same 120.66 123.62 0.98 Windows 10.0.18363.959 X64 Intel Xeon CPU E5-1650 v4 3.60GHz .NET Core 3.1.6 5.0.100-rc.1.20418.3
Slower 92.38 99.48 0.93 Windows 10.0.19042 X64 Intel Core i7-7700 CPU 3.60GHz (Kaby Lake) .NET Core 3.1.6 5.0.100-rc.1.20418.3
Slower 157.38 178.95 0.88 Windows 10.0.19041.450 X86 Intel Core i7-5557U CPU 3.10GHz (Broadwell) .NET Core 3.1.6 5.0.100-rc.1.20419.5

System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf8(arguments: Url,c2020,16)

Conclusion Base Diff Base/Diff Modality Operating System Bit Processor Name Base Runtime Diff Runtime
Slower 162.02 204.71 0.79 Windows 10.0.18363.1016 Arm Microsoft SQ1 3.0 GHz .NET Core 3.1.6 5.0.100-rc.1.20413.9
Same 105.67 100.29 1.05 Windows 10.0.18363.959 X64 Intel Xeon CPU E5-1650 v4 3.60GHz .NET Core 3.1.6 5.0.100-rc.1.20418.3
Same 81.69 85.48 0.96 Windows 10.0.19042 X64 Intel Core i7-7700 CPU 3.60GHz (Kaby Lake) .NET Core 3.1.6 5.0.100-rc.1.20418.3
Same 129.29 133.32 0.97 Windows 10.0.19041.450 X86 Intel Core i7-5557U CPU 3.10GHz (Broadwell) .NET Core 3.1.6 5.0.100-rc.1.20419.5

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@eiriktsarpalis @GrabYourPitchforks I am setting 5.0 milestone to make sure that it gets tagged triaged within the next few weeks.

@DrewScoggins this regression did not get detected by the bot because benchmark was added just a few days ago, so everything is 👍

category:performance
theme:needs-triage
skill-level:expert
cost:large

Activity

  1. adamsitnik
    added
    area-System.Text.Encodings.Web
    tenet-performancePerformance related issue
    on Aug 20, 2020
  3. Dotnet-GitSync-Bot
    added
    untriagedNew issue has not been triaged by the area owner
    on Aug 20, 2020
  4. adamsitnik
    added this to the 5.0.0 milestone on Aug 20, 2020

  5. tarekgh commented on Aug 20, 2020

    @tarekgh
    tarekgh
    on Aug 20, 2020
    Member

    @eiriktsarpalis @GrabYourPitchforks I assume you'll triage and address this issue. please let me know if anything needed from me. Thanks!

  6. tarekgh commented on Aug 27, 2020

    @tarekgh
    tarekgh
    on Aug 27, 2020
    Member

    @eiriktsarpalis @GrabYourPitchforks could you please triage this?

  7. jeffschwMSFT
    removed
    untriagedNew issue has not been triaged by the area owner
    on Aug 31, 2020
  8. layomia
    self-assigned this
    on Aug 31, 2020

  9. ericstj commented on Aug 31, 2020

    @ericstj
    ericstj
    on Aug 31, 2020
    Member

    @adamsitnik was this the case that was also discovered by UTF8String benchmarks? Do you know if anyone is investigating yet? cc @layomia

  10. adamsitnik commented on Aug 31, 2020

    @adamsitnik
    adamsitnik
    on Aug 31, 2020
    MemberAuthor

    was this the case that was also discovered by UTF8String benchmarks?

    I don't have enough information to be able to tell (I've not profiled them).

    Do you know if anyone is investigating yet?

    afaik not (but I can see @layomia assigned himself a few minutes ago so I might be wrong)

  11. layomia commented on Aug 31, 2020

    @layomia
    layomia
    on Aug 31, 2020
    Contributor

    Yes, I plan to investigate if no one else already is (or wants to do so). Please let me know.

  12. danmoseley commented on Sep 16, 2020

    @danmoseley
    danmoseley
    on Sep 16, 2020
    Contributor

    Should this block release, or is it OK to ship with it? @GrabYourPitchforks

  13. GrabYourPitchforks commented on Sep 16, 2020

    @GrabYourPitchforks
    GrabYourPitchforks
    on Sep 16, 2020
    Member

    IMO it's fine to ship with this, as reverting doesn't look as trivial as the other UTF8 regression we looked at, and we still have good fuzzer & test coverage so we believe the algorithms are correct. Can address in 5.x servicing if needed.

  14. danmoseley
    removed this from the 5.0.0 milestone on Sep 16, 2020

  15. 3 remaining items

  16. adamsitnik
    unassigned
    layomia
    on Sep 22, 2020
  17. adamsitnik
    added
    area-CodeGen-coreclrCLR JIT compiler in src/coreclr/src/jit and related components such as SuperPMI
    and removed
    area-System.Text.Encodings.Web
    on Sep 22, 2020

  18. JulieLeeMSFT commented on Sep 22, 2020

    @JulieLeeMSFT
    JulieLeeMSFT
    on Sep 22, 2020
    Member

    @dotnet/jit-contrib

  19. BruceForstall commented on Oct 10, 2020

    @BruceForstall
    BruceForstall
    on Oct 10, 2020
    Contributor

    @adamsitnik Can you summarize why you think this is a codegen issue? (I'm having trouble wading through all the data here to figure that out)

    E.g., it looks like the no-tiered-compilation mode is faster in 5.0 versus 3.1. I don't understand what the "default" mode is: shouldn't BDN force tiering up, which would generate the same data as no-tiered-compilation?

  20. adamsitnik commented on Oct 15, 2020

    @adamsitnik
    adamsitnik
    on Oct 15, 2020
    MemberAuthor

    Can you summarize why you think this is a codegen issue?

    @BruceForstall the C# implementation has not changed, but the generated assembly code and performance has regressed

  21. BruceForstall
    added
    JitUntriagedCLR JIT issues needing additional triage
    on Oct 28, 2020
  22. BruceForstall
    removed
    JitUntriagedCLR JIT issues needing additional triage
    on Nov 11, 2020
  23. JulieLeeMSFT
    assigned
    kunalspathak
    on Mar 23, 2021
  24. JulieLeeMSFT
    added
    needs-further-triageIssue has been initially triaged, but needs deeper consideration or reconsideration
    on Mar 23, 2021
  25. kunalspathak
    removed
    needs-further-triageIssue has been initially triaged, but needs deeper consideration or reconsideration
    on Apr 1, 2021

  26. kunalspathak commented on Apr 23, 2021

    @kunalspathak
    kunalspathak
    on Apr 23, 2021
    Contributor

    @adamsitnik - I was trying to repro the regression, but I am getting these errors with netcoreapp3.1. Any idea?

    [2021/04/23 15:32:17][INFO] // Build Error: Standard output:
[2021/04/23 15:32:17][INFO]
[2021/04/23 15:32:17][INFO]  Standard error:
[2021/04/23 15:32:17][INFO]    Restored D:\git\performance\src\benchmarks\micro\MicroBenchmarks.csproj (in 229 ms).
[2021/04/23 15:32:17][INFO]   Restored D:\git\performance\src\harness\BenchmarkDotNet.Extensions\BenchmarkDotNet.Extensions.csproj (in 178 ms).
[2021/04/23 15:32:17][INFO]   Restored D:\git\performance\src\tools\Reporting\Reporting\Reporting.csproj (in 104 ms).
[2021/04/23 15:32:17][INFO]   Failed to restore D:\git\performance\artifacts\bin\MicroBenchmarks\Release\netcoreapp3.1\d310bb2d-5afa-488d-b985-4e604e2bbd09\BenchmarkDotNet.Autogenerated.csproj (in 68 ms).
[2021/04/23 15:32:17][INFO] D:\git\performance\artifacts\bin\MicroBenchmarks\Release\netcoreapp3.1\d310bb2d-5afa-488d-b985-4e604e2bbd09\BenchmarkDotNet.Autogenerated.csproj : error NU1201: Project MicroBenchmarks is not compatible with netcoreapp3.1 (.NETCoreApp,Version=v3.1). Project MicroBenchmarks supports: net5.0 (.NETCoreApp,Version=v5.0)

    I also did comparison of .NET 5 vs./ .NET 6 and we do much better in .NET 6 (with and without tiering).

    .NET 5.0 (Tiering)

    Method arguments Mean Error StdDev Median Min Max Gen 0 Gen 1 Gen 2 Allocated
    EncodeUtf8 JavaScript,&Hello+!,16 130.4 ns 1.01 ns 0.89 ns 130.5 ns 128.7 ns 131.9 ns - - - -
    EncodeUtf8 JavaScript,&Hello+!,512 1,543.5 ns 7.71 ns 6.44 ns 1,542.2 ns 1,537.6 ns 1,557.4 ns - - - -
    EncodeUtf8 JavaScript,no escaping required,16 118.2 ns 0.13 ns 0.12 ns 118.2 ns 118.0 ns 118.4 ns - - - -
    EncodeUtf8 JavaScript,no escaping required,512 1,527.4 ns 10.14 ns 8.99 ns 1,524.4 ns 1,519.0 ns 1,550.7 ns - - - -
    EncodeUtf8 UnsafeRelaxed,hello "there",16 117.8 ns 0.43 ns 0.36 ns 117.8 ns 117.2 ns 118.5 ns - - - -
    EncodeUtf8 UnsafeRelaxed,hello "there",512 1,517.5 ns 2.39 ns 2.12 ns 1,517.0 ns 1,514.3 ns 1,522.3 ns - - - -
    EncodeUtf8 UnsafeRelaxed(...)> required,16 [41] 129.9 ns 0.19 ns 0.16 ns 129.9 ns 129.6 ns 130.2 ns - - - -
    EncodeUtf8 UnsafeRelaxed(...) required,512 [42] 1,533.5 ns 1.94 ns 1.72 ns 1,533.5 ns 1,530.2 ns 1,536.9 ns - - - -
    EncodeUtf8 Url,&lorem ipsum=dolor sit amet,16 189.5 ns 1.27 ns 1.13 ns 189.4 ns 188.2 ns 191.9 ns - - - -
    EncodeUtf8 Url,&lorem ipsum=dolor sit amet,512 1,605.8 ns 10.46 ns 9.78 ns 1,601.1 ns 1,596.0 ns 1,627.4 ns - - - -
    EncodeUtf8 Url,c2020,16 101.0 ns 0.43 ns 0.40 ns 101.0 ns 100.3 ns 101.8 ns - - - -
    EncodeUtf8 Url,c2020,512 1,513.6 ns 3.66 ns 3.24 ns 1,512.7 ns 1,509.8 ns 1,519.8 ns - - - -

    .NET 6.0 (Tiering)

    Method arguments Mean Error StdDev Median Min Max Gen 0 Gen 1 Gen 2 Allocated
    EncodeUtf8 JavaScript,&Hello+!,16 41.46 ns 0.200 ns 0.187 ns 41.42 ns 41.24 ns 41.92 ns - - - -
    EncodeUtf8 JavaScript,&Hello+!,512 94.78 ns 0.184 ns 0.163 ns 94.79 ns 94.52 ns 95.11 ns - - - -
    EncodeUtf8 JavaScript,no escaping required,16 14.54 ns 0.028 ns 0.025 ns 14.54 ns 14.49 ns 14.57 ns - - - -
    EncodeUtf8 JavaScript,no escaping required,512 67.88 ns 0.303 ns 0.284 ns 67.82 ns 67.58 ns 68.51 ns - - - -
    EncodeUtf8 UnsafeRelaxed,hello "there",16 28.07 ns 0.051 ns 0.042 ns 28.07 ns 28.00 ns 28.13 ns - - - -
    EncodeUtf8 UnsafeRelaxed,hello "there",512 82.05 ns 0.598 ns 0.560 ns 82.11 ns 81.28 ns 83.29 ns - - - -
    EncodeUtf8 UnsafeRelaxed(...)> required,16 [41] 15.07 ns 0.043 ns 0.036 ns 15.08 ns 15.01 ns 15.12 ns - - - -
    EncodeUtf8 UnsafeRelaxed(...) required,512 [42] 68.81 ns 0.424 ns 0.396 ns 68.92 ns 68.21 ns 69.34 ns - - - -
    EncodeUtf8 Url,&lorem ipsum=dolor sit amet,16 61.81 ns 0.101 ns 0.079 ns 61.81 ns 61.64 ns 61.95 ns - - - -
    EncodeUtf8 Url,&lorem ipsum=dolor sit amet,512 116.93 ns 2.264 ns 2.007 ns 116.89 ns 114.76 ns 121.10 ns - - - -
    EncodeUtf8 Url,c2020,16 37.37 ns 0.182 ns 0.161 ns 37.35 ns 37.14 ns 37.68 ns - - - -
    EncodeUtf8 Url,c2020,512 88.85 ns 0.537 ns 0.449 ns 88.90 ns 87.88 ns 89.44 ns - - - -

    .NET 5.0 (no tiering)

    Method arguments Mean Error StdDev Median Min Max Gen 0 Gen 1 Gen 2 Allocated
    EncodeUtf8 JavaScript,&Hello+!,16 105.58 ns 0.586 ns 0.519 ns 105.73 ns 104.27 ns 106.26 ns - - - -
    EncodeUtf8 JavaScript,&Hello+!,512 1,539.49 ns 4.355 ns 3.637 ns 1,539.20 ns 1,533.81 ns 1,547.55 ns - - - -
    EncodeUtf8 JavaScript,no escaping required,16 96.02 ns 0.487 ns 0.407 ns 96.12 ns 95.03 ns 96.48 ns - - - -
    EncodeUtf8 JavaScript,no escaping required,512 1,531.75 ns 16.633 ns 14.745 ns 1,526.73 ns 1,517.92 ns 1,564.33 ns - - - -
    EncodeUtf8 UnsafeRelaxed,hello "there",16 118.47 ns 0.390 ns 0.365 ns 118.61 ns 117.77 ns 118.98 ns - - - -
    EncodeUtf8 UnsafeRelaxed,hello "there",512 1,513.68 ns 1.910 ns 1.491 ns 1,513.29 ns 1,512.03 ns 1,516.67 ns - - - -
    EncodeUtf8 UnsafeRelaxed(...)> required,16 [41] 131.90 ns 1.278 ns 1.195 ns 131.17 ns 130.59 ns 133.85 ns - - - -
    EncodeUtf8 UnsafeRelaxed(...) required,512 [42] 1,532.92 ns 4.067 ns 3.396 ns 1,532.36 ns 1,528.69 ns 1,539.99 ns - - - -
    EncodeUtf8 Url,&lorem ipsum=dolor sit amet,16 189.52 ns 1.400 ns 1.310 ns 189.53 ns 187.42 ns 191.53 ns - - - -
    EncodeUtf8 Url,&lorem ipsum=dolor sit amet,512 1,602.09 ns 3.674 ns 3.437 ns 1,601.91 ns 1,594.50 ns 1,607.62 ns - - - -
    EncodeUtf8 Url,c2020,16 102.27 ns 0.260 ns 0.244 ns 102.20 ns 101.94 ns 102.76 ns - - - -
    EncodeUtf8 Url,c2020,512 1,511.89 ns 2.273 ns 1.898 ns 1,511.51 ns 1,509.29 ns 1,516.04 ns - - - -

    .NET 6.0 (no tiering)

    Method arguments Mean Error StdDev Median Min Max Gen 0 Gen 1 Gen 2 Allocated
    EncodeUtf8 JavaScript,&Hello+!,16 68.36 ns 0.063 ns 0.059 ns 68.36 ns 68.29 ns 68.48 ns - - - -
    EncodeUtf8 JavaScript,&Hello+!,512 106.93 ns 0.256 ns 0.239 ns 106.88 ns 106.53 ns 107.34 ns - - - -
    EncodeUtf8 JavaScript,no escaping required,16 20.70 ns 0.183 ns 0.171 ns 20.62 ns 20.50 ns 21.00 ns - - - -
    EncodeUtf8 JavaScript,no escaping required,512 63.01 ns 0.303 ns 0.269 ns 62.94 ns 62.70 ns 63.62 ns - - - -
    EncodeUtf8 UnsafeRelaxed,hello "there",16 49.11 ns 0.327 ns 0.306 ns 48.99 ns 48.71 ns 49.61 ns - - - -
    EncodeUtf8 UnsafeRelaxed,hello "there",512 91.20 ns 0.254 ns 0.225 ns 91.20 ns 90.91 ns 91.72 ns - - - -
    EncodeUtf8 UnsafeRelaxed(...)> required,16 [41] 20.85 ns 0.086 ns 0.072 ns 20.82 ns 20.79 ns 21.02 ns - - - -
    EncodeUtf8 UnsafeRelaxed(...) required,512 [42] 65.11 ns 0.269 ns 0.238 ns 65.05 ns 64.85 ns 65.62 ns - - - -
    EncodeUtf8 Url,&lorem ipsum=dolor sit amet,16 115.89 ns 0.216 ns 0.180 ns 115.85 ns 115.69 ns 116.35 ns - - - -
    EncodeUtf8 Url,&lorem ipsum=dolor sit amet,512 161.35 ns 0.425 ns 0.376 ns 161.26 ns 160.86 ns 162.06 ns - - - -
    EncodeUtf8 Url,c2020,16 78.59 ns 0.185 ns 0.173 ns 78.60 ns 78.35 ns 78.93 ns - - - -
    EncodeUtf8 Url,c2020,512 116.46 ns 0.205 ns 0.182 ns 116.42 ns 116.25 ns 116.90 ns - - - -

  27. kunalspathak commented on Apr 28, 2021

    @kunalspathak
    kunalspathak
    on Apr 28, 2021
    Contributor

    @adamsitnik - I was able to get .NET 3.1 numbers and compared them against .NET6 and I don't see any slowness. .NET 6 is fast with and without tiering.

    .NET3.1 (Tier 0)

    Method arguments Mean Error StdDev Median Min Max Gen 0 Gen 1 Gen 2 Allocated
    EncodeUtf8 JavaScript,&Hello+!,16 167.1 ns 1.80 ns 1.68 ns 166.6 ns 164.8 ns 170.3 ns - - - -
    EncodeUtf8 JavaScript,&Hello+!,512 1,678.1 ns 6.35 ns 5.63 ns 1,678.8 ns 1,666.8 ns 1,687.8 ns - - - -
    EncodeUtf8 JavaScript,no escaping required,16 135.8 ns 0.62 ns 0.58 ns 136.0 ns 135.0 ns 136.9 ns - - - -
    EncodeUtf8 JavaScript,no escaping required,512 1,642.3 ns 8.39 ns 7.85 ns 1,642.2 ns 1,627.3 ns 1,654.0 ns - - - -
    EncodeUtf8 UnsafeRelaxed,hello "there",16 136.9 ns 0.92 ns 0.86 ns 137.0 ns 135.8 ns 138.6 ns - - - -
    EncodeUtf8 UnsafeRelaxed,hello "there",512 1,643.8 ns 5.98 ns 4.99 ns 1,645.1 ns 1,634.2 ns 1,649.6 ns - - - -
    EncodeUtf8 UnsafeRelaxed(...)> required,16 [41] 147.6 ns 0.69 ns 0.65 ns 147.6 ns 146.4 ns 148.6 ns - - - -
    EncodeUtf8 UnsafeRelaxed(...) required,512 [42] 1,655.5 ns 9.30 ns 8.70 ns 1,655.2 ns 1,640.3 ns 1,669.5 ns - - - -
    EncodeUtf8 Url,&lorem ipsum=dolor sit amet,16 234.5 ns 1.77 ns 1.65 ns 235.2 ns 231.3 ns 236.5 ns - - - -
    EncodeUtf8 Url,&lorem ipsum=dolor sit amet,512 1,746.3 ns 12.58 ns 11.15 ns 1,747.2 ns 1,724.9 ns 1,765.4 ns - - - -
    EncodeUtf8 Url,c2020,16 125.6 ns 0.91 ns 0.76 ns 125.9 ns 124.3 ns 126.8 ns - - - -
    EncodeUtf8 Url,c2020,512 1,654.6 ns 19.47 ns 16.25 ns 1,657.6 ns 1,629.5 ns 1,693.3 ns - - - -

    .NET 6 (Tier 0)

    Method arguments Mean Error StdDev Median Min Max Gen 0 Gen 1 Gen 2 Allocated
    EncodeUtf8 JavaScript,&Hello+!,16 153.1 ns 0.79 ns 0.70 ns 153.0 ns 151.6 ns 154.1 ns - - - -
    EncodeUtf8 JavaScript,&Hello+!,512 1,561.6 ns 7.63 ns 6.76 ns 1,563.4 ns 1,547.9 ns 1,572.3 ns - - - -
    EncodeUtf8 JavaScript,no escaping required,16 132.4 ns 0.41 ns 0.38 ns 132.4 ns 131.8 ns 133.1 ns - - - -
    EncodeUtf8 JavaScript,no escaping required,512 1,516.4 ns 5.19 ns 4.33 ns 1,517.0 ns 1,506.4 ns 1,523.1 ns - - - -
    EncodeUtf8 UnsafeRelaxed,hello "there",16 128.3 ns 0.49 ns 0.44 ns 128.2 ns 127.6 ns 129.0 ns - - - -
    EncodeUtf8 UnsafeRelaxed,hello "there",512 1,528.4 ns 3.17 ns 2.47 ns 1,529.3 ns 1,522.5 ns 1,531.1 ns - - - -
    EncodeUtf8 UnsafeRelaxed(...)> required,16 [41] 134.8 ns 0.52 ns 0.46 ns 134.9 ns 133.7 ns 135.3 ns - - - -
    EncodeUtf8 UnsafeRelaxed(...) required,512 [42] 1,531.0 ns 6.68 ns 6.25 ns 1,530.7 ns 1,520.8 ns 1,542.4 ns - - - -
    EncodeUtf8 Url,&lorem ipsum=dolor sit amet,16 218.2 ns 1.46 ns 1.37 ns 218.2 ns 216.2 ns 220.9 ns - - - -
    EncodeUtf8 Url,&lorem ipsum=dolor sit amet,512 1,625.8 ns 5.66 ns 5.30 ns 1,624.2 ns 1,616.7 ns 1,634.3 ns - - - -
    EncodeUtf8 Url,c2020,16 118.4 ns 0.55 ns 0.51 ns 118.6 ns 117.1 ns 119.0 ns - - - -
    EncodeUtf8 Url,c2020,512 1,525.8 ns 7.59 ns 7.10 ns 1,526.2 ns 1,513.9 ns 1,539.3 ns - - - -

    .NET 3.1 (Tier)

    Method arguments Mean Error StdDev Median Min Max Gen 0 Gen 1 Gen 2 Allocated
    EncodeUtf8 JavaScript,&Hello+!,16 167.4 ns 0.99 ns 0.88 ns 167.3 ns 165.5 ns 169.0 ns - - - -
    EncodeUtf8 JavaScript,&Hello+!,512 1,691.6 ns 12.00 ns 10.64 ns 1,694.2 ns 1,672.2 ns 1,705.0 ns - - - -
    EncodeUtf8 JavaScript,no escaping required,16 131.9 ns 2.55 ns 2.61 ns 131.2 ns 128.9 ns 136.2 ns - - - -
    EncodeUtf8 JavaScript,no escaping required,512 1,653.2 ns 6.53 ns 5.79 ns 1,653.8 ns 1,641.6 ns 1,664.0 ns - - - -
    EncodeUtf8 UnsafeRelaxed,hello "there",16 149.1 ns 0.81 ns 0.76 ns 149.0 ns 147.6 ns 150.6 ns - - - -
    EncodeUtf8 UnsafeRelaxed,hello "there",512 1,652.8 ns 10.38 ns 9.71 ns 1,649.8 ns 1,635.7 ns 1,671.0 ns - - - -
    EncodeUtf8 UnsafeRelaxed(...)> required,16 [41] 149.6 ns 0.97 ns 0.81 ns 149.7 ns 147.8 ns 150.6 ns - - - -
    EncodeUtf8 UnsafeRelaxed(...) required,512 [42] 1,665.4 ns 9.63 ns 9.01 ns 1,665.6 ns 1,646.2 ns 1,683.1 ns - - - -
    EncodeUtf8 Url,&lorem ipsum=dolor sit amet,16 247.9 ns 2.17 ns 2.03 ns 248.3 ns 243.7 ns 251.5 ns - - - -
    EncodeUtf8 Url,&lorem ipsum=dolor sit amet,512 1,750.2 ns 11.29 ns 10.56 ns 1,750.3 ns 1,737.1 ns 1,769.3 ns - - - -
    EncodeUtf8 Url,c2020,16 128.5 ns 0.97 ns 0.91 ns 128.4 ns 127.4 ns 130.2 ns - - - -
    EncodeUtf8 Url,c2020,512 1,653.7 ns 8.61 ns 8.06 ns 1,652.4 ns 1,639.7 ns 1,664.4 ns - - - -

    .NET 6 (Tier)

    Method arguments Mean Error StdDev Median Min Max Gen 0 Gen 1 Gen 2 Allocated
    EncodeUtf8 JavaScript,&Hello+!,16 153.7 ns 1.41 ns 1.32 ns 153.6 ns 151.7 ns 155.9 ns - - - -
    EncodeUtf8 JavaScript,&Hello+!,512 1,320.7 ns 5.81 ns 5.15 ns 1,320.8 ns 1,312.5 ns 1,330.3 ns - - - -
    EncodeUtf8 JavaScript,no escaping required,16 106.3 ns 0.47 ns 0.44 ns 106.3 ns 105.4 ns 106.9 ns - - - -
    EncodeUtf8 JavaScript,no escaping required,512 1,274.3 ns 7.05 ns 6.25 ns 1,273.4 ns 1,260.7 ns 1,286.2 ns - - - -
    EncodeUtf8 UnsafeRelaxed,hello "there",16 118.9 ns 0.64 ns 0.57 ns 118.7 ns 118.2 ns 120.3 ns - - - -
    EncodeUtf8 UnsafeRelaxed,hello "there",512 1,290.0 ns 5.48 ns 4.57 ns 1,288.9 ns 1,281.2 ns 1,297.0 ns - - - -
    EncodeUtf8 UnsafeRelaxed(...)> required,16 [41] 116.2 ns 0.54 ns 0.48 ns 116.2 ns 115.3 ns 116.8 ns - - - -
    EncodeUtf8 UnsafeRelaxed(...) required,512 [42] 1,288.8 ns 6.11 ns 5.41 ns 1,288.3 ns 1,278.8 ns 1,296.8 ns - - - -
    EncodeUtf8 Url,&lorem ipsum=dolor sit amet,16 202.8 ns 2.71 ns 2.53 ns 202.9 ns 199.3 ns 206.1 ns - - - -
    EncodeUtf8 Url,&lorem ipsum=dolor sit amet,512 1,387.6 ns 11.30 ns 10.57 ns 1,388.3 ns 1,369.0 ns 1,409.9 ns - - - -
    EncodeUtf8 Url,c2020,16 108.2 ns 0.71 ns 0.66 ns 108.2 ns 106.9 ns 109.2 ns - - - -
    EncodeUtf8 Url,c2020,512 1,285.9 ns 6.73 ns 5.97 ns 1,286.1 ns 1,273.9 ns 1,293.3 ns - - - -
  28. kunalspathak
    closed this as completedon Apr 28, 2021
  29. ghost locked as resolved and limited conversation to collaborators on May 28, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

area-CodeGen-coreclrCLR JIT compiler in src/coreclr/src/jit and related components such as SuperPMItenet-performancePerformance related issue

Type

No type

Projects

No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions