Regressions in System.Net.NetworkInformation.Tests.PhysicalAddressTests on 2/19/2023

Run Information

Architecture	x64
OS	Windows 10.0.19042
Baseline	edce5ea1600bf4c61f10bc9de80e4a11ff14c842
Compare	7908e8ef62c7d5ebdcd6459faccd755b9b10d369
Diff	Diff

Regressions in System.Net.NetworkInformation.Tests.PhysicalAddressTests

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
PALong - Duration of single invocation	101.37 ns	112.53 ns	1.11	0.02	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Net.NetworkInformation.Tests.PhysicalAddressTests*'

Details

Payloads

Baseline
Compare

Histogram

System.Net.NetworkInformation.Tests.PhysicalAddressTests.PALong

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 112.52664760231312 > 106.46904325914913.
IsChangePoint: Marked as a change because one of 12/8/2022 9:11:55 AM, 1/11/2023 2:37:52 AM, 2/19/2023 8:14:07 AM, 2/21/2023 5:46:41 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 5:46:41 AM.
IsRegressionStdDev: Marked as regression because -13.027258951308566 (T) = (0 -110.3927897711911) / Math.Sqrt((0.4042938203688266 / (52)) + (5.142474481164312 / (10))) is less than -2.0002978220134566 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (52) + (10) - 2, .025) and -0.09320952255602948 = (100.98045021880353 - 110.3927897711911) / 100.98045021880353 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Likely caused by #81666

Can't reproduce regression locally on Win-x64, codegen diff: https://www.diffchecker.com/6WrxIw4x/
two notes:

Static PGO is invalid -- it needs PGO update in the repo
Loop is not aligned in new codegen:

G_M140_IG01:              ;; offset=0000H
       C5F877               vzeroupper 
						;; size=3 bbWeight=1 PerfScore 1.00
G_M140_IG02:              ;; offset=0003H
       488B01               mov      rax, bword ptr [rcx]
       8B4908               mov      ecx, dword ptr [rcx+08H]
       488B12               mov      rdx, bword ptr [rdx]
       4585C0               test     r8d, r8d
       740A                 je       SHORT G_M140_IG04
						;; size=14 bbWeight=1 PerfScore 7.25
G_M140_IG03:              ;; offset=0011H
       C5F8100557000000     vmovups  xmm0, xmmword ptr [reloc @RWD00]
       EB08                 jmp      SHORT G_M140_IG05
                            align    [0 bytes for IG06]
						;; size=10 bbWeight=0.50 PerfScore 2.50
G_M140_IG04:              ;; offset=001BH
       C5F810055D000000     vmovups  xmm0, xmmword ptr [reloc @RWD16]
; ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (vmovups: 3) 32B boundary ...............................
						;; size=8 bbWeight=0.50 PerfScore 1.50
G_M140_IG05:              ;; offset=0023H
       4533C0               xor      r8d, r8d
       8BC9                 mov      ecx, ecx
       4C8D49FC             lea      r9, [rcx-04H]
       C5F8100D5C000000     vmovups  xmm1, xmmword ptr [reloc @RWD32]
						;; size=17 bbWeight=1 PerfScore 4.00
G_M140_IG06:              ;; offset=0034H
       C4A1796E1400         vmovd    xmm2, xmmword ptr [rax+r8]
       C5E173D204           vpsrlq   xmm3, xmm2, 4
       C5E160D2             vpunpcklbw xmm2, xmm3, xmm2
; ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (vpunpcklbw: 3) 32B boundary ...............................
       C5E9DBD1             vpand    xmm2, xmm2, xmm1
       C4E27900D2           vpshufb  xmm2, xmm0, xmm2
       C4E27930D2           vpmovzxbw xmm2, xmm2
       C4A178111482         vmovups  xmmword ptr [rdx+4*r8], xmm2
       4983C004             add      r8, 4
       4C3BC1               cmp      r8, rcx
       740A                 je       SHORT G_M140_IG09
; ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (je: 0 ; jcc erratum) 32B boundary ...............................
						;; size=44 bbWeight=8 PerfScore 94.67
G_M140_IG07:              ;; offset=0060H
       4D3BC1               cmp      r8, r9
       76CF                 jbe      SHORT G_M140_IG06
						;; size=5 bbWeight=4 PerfScore 5.00
G_M140_IG08:              ;; offset=0065H
       4D8BC1               mov      r8, r9
       EBCA                 jmp      SHORT G_M140_IG06
						;; size=5 bbWeight=0.50 PerfScore 1.12
G_M140_IG09:              ;; offset=006AH
       C3                   ret      
						;; size=1 bbWeight=1 PerfScore 1.00
RWD00  	dq	3736353433323130h, 6665646362613938h
RWD16  	dq	3736353433323130h, 4645444342413938h
RWD32  	dq	0F0F0F0F0F0F0F0Fh, 0F0F0F0F0F0F0F0Fh


; Total bytes of code 107

jcc erratum warning

@EgorBo

@EgorBo, looks like the 10ns regression is staying pretty consistent in the perf lab.

Could you share the before asm, just so we can easily compare?

Also notably that (Vector128<ushort> v0, _) = Vector128.Widen(hexLow); should just be Vector128<ushort> v0 = Vector128.WidenLower(hexLow);

Could you share the before asm, just so we can easily compare?

in the comment above ^, https://www.diffchecker.com/6WrxIw4x/

Misaligned loop + jcc erratum pretty much explain the regression to me. To align that loop, JIT has to emit more than 15 bytes of nops now and it never does it with adaptive (default) loop alignment behavior.

should just be Vector128 v0 = Vector128.WidenLower(hexLow);

Good point, but fortunately, it doesn't affect codegen

I want to wait with this one for a static pgo update + have a fix for loop alignment, e.g. #82635 fixes the issue

the regression is no more

Navigation Menu

Repository navigation

Regressions in System.Net.NetworkInformation.Tests.PhysicalAddressTests on 2/19/2023 #82442

Run Information