You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I would guess that V08 didn't get a register at entry, or was allocated rax and then latter spilled, but was not marked spilled. I'm not sure why there are two RefPositions at location zero (i.e. entry). The entry RefPositions (which are usually eitherRefTypeParamDef or RefTypeZeroInit but not both, get special handling, so perhaps something is missing there.
Still ~34 failures ( a bit less ) with #35585. They all look like unexpected null reference exceptions. I'll look for a simple case and try and see what's up.
A number of the remaining bugs involve WaitForExitCore -- failure is an AV in the callee GetProcessHandle because we call it with null in RCX.
Looks like either there's a missing reload of RCX before the call, or else the xor of RCX at the end of the zeroing done in the jit prolog inadvertently trashes RCX which we expected we could keep live.
Sigh, I had a bug in my script and was comparing new runs to older baseline files.
Results vs proper baselines are more in line with the data Sebastien has gathered. Odd though that Json RPS was 350K yesterday (for both baseline and EHWT) and only 330K today.
How much variability do you usually see in results? I'd be surprised if enabling this made things slower, but I see cases above where it looks like we lose 2-3% on RPS. If that's accurate, we should try and look at codegen for the key methods more closely.
There are definitely places where EH Write through could be slightly worse. In situations where there are multiple definitions in the Try clause, each of those will do a store, and may also define a register - requiring an additional mov if the value could have been directly defined to memory. It is most effective when there are more uses than defs, and when register pressure is not excessive.
I ran PlaintextPlatform 5 times for each, and the min value for no EHWT is greater than the max value with it, so it's definitely slower with on this scenario:
couldn't we screen candidates and only enable write through for locals that have favorable def/use ratios?
I didn't pursue that; we don't readily have that information until we build Intervals, and at that point it is difficult for the register allocator to change its mind about making something a candidate. That said, it could presumably decide that some of the candidates should never get a register, effectively making it the same as if it were not a candidate, though that would require some tweaking to avoid actually allocating a register when not needed (it does better with RegOptional uses than defs).
It would require retaining additional information on each Interval which would in turn probably require some additional tuning to ensure that it doesn't degrade throughput.
I wonder if we could gather this info during one of our ref count traversals. We currently don't distinguish reads and writes, but it seems like we easily could.
Probably best to first dig into the code that's causing slowdowns in the tests above to ensure this is why things end up slower.
This issue is nominally done. Would like to close and capture what remains in a new issue. @CarolEidt is there an issue for enabling EH Write Thru by default? I couldn't find one.
Feels like there is still a moderate amount of work to do before we can turn EH Write Thru on by default, and some risk that we're just not going to see the benefits here we'd hoped to see. I'd like to do a more careful evaluation and try and find or create examples where we clearly do expect to see benefits.
Seems like all this might be a stretch for 5.0 but possibly worthwhile; trying to assess if we should find a way to do that sometime soon, or defer all of this until after 5.0.
Thanks for all your analysis, Andy. I agree that this issue, the investigation, is complete.
There's no issue for enabling EH write thru by default - I'll add one.
@sebastienros thanks for running all those tests for us -- we'll need to dig in deeper to understand what's going on. Hope we can get to it before too long.
Activity
AndyAyersMS commentedon Apr 28, 2020
Tests show the following issues:
JIT\opt\Devirtualization\box2.JIT\Intrinsics\TypeIntrinsics_r\TypeIntrinsics_rThink I have a fix for some of the runtime failures. Looks like we are not initializing the stack home for a EH live parameter.
CarolEidt commentedon Apr 28, 2020
@AndyAyersMS - let me know if you'd like me to track down the LSRA assertions.
And thanks for tracking down the runtime failures!
AndyAyersMS commentedon Apr 28, 2020
@CarolEidt I'll keep looking, but maybe you can give me some pointers.
The assert is in
veryfyFinalAllocation: there is aRefTypeKill(RAX killed by helper call) with an assigned interval:runtime/src/coreclr/src/jit/lsra.cpp
Lines 10710 to 10714 in 2b30fdb
AndyAyersMS commentedon Apr 28, 2020
The associated interval is a write-through local:
CarolEidt commentedon Apr 28, 2020
I would guess that V08 didn't get a register at entry, or was allocated
raxand then latter spilled, but was not marked spilled. I'm not sure why there are twoRefPositions at location zero (i.e. entry). The entryRefPositions (which are usually eitherRefTypeParamDeforRefTypeZeroInitbut not both, get special handling, so perhaps something is missing there.AndyAyersMS commentedon Apr 28, 2020
For the two ref positions:
I think V08's liveness is over-stated but perhaps the use (in a return) is leaking upwards through infeasible EH paths.
First bit of the allocation table:
Method being jitted:
runtime/src/libraries/System.Linq.Expressions/src/System/Linq/Expressions/Compiler/VariableBinder.cs
Lines 145 to 195 in b4f7380
V08 (local6) is defined/used by that inner return.
AndyAyersMS commentedon Apr 28, 2020
I'm also looking for simpler failing test cases or a simple repro based on the above, but no luck so far.
CarolEidt commentedon Apr 28, 2020
I was able to reproduce this using SuperPmi. It is indeed the double zero-init that is causing the problem. I submitted PR #35585 to fix it.
AndyAyersMS commentedon Apr 29, 2020
Still ~34 failures ( a bit less ) with #35585. They all look like unexpected null reference exceptions. I'll look for a simple case and try and see what's up.
AndyAyersMS commentedon May 1, 2020
A number of the remaining bugs involve
WaitForExitCore-- failure is an AV in the calleeGetProcessHandlebecause we call it with null in RCX.Looks like either there's a missing reload of RCX before the call, or else the xor of RCX at the end of the zeroing done in the jit prolog inadvertently trashes RCX which we expected we could keep live.
Will keep digging.
11 remaining items
sebastienros commentedon May 4, 2020
Preliminary results on 12 core machines
AndyAyersMS commentedon May 4, 2020
Sigh, I had a bug in my script and was comparing new runs to older baseline files.
Results vs proper baselines are more in line with the data Sebastien has gathered. Odd though that Json RPS was 350K yesterday (for both baseline and EHWT) and only 330K today.
sebastienros commentedon May 4, 2020
Adding more numbers, this time on the citrine environment. INTEL machines are 14/28(ht) cores, ARM is 32 cores.
AndyAyersMS commentedon May 5, 2020
How much variability do you usually see in results? I'd be surprised if enabling this made things slower, but I see cases above where it looks like we lose 2-3% on RPS. If that's accurate, we should try and look at codegen for the key methods more closely.
sebastienros commentedon May 5, 2020
Each number is the average of two runs. I will redo the ones that regressed just to be sure. It might happen that a run is bad.
CarolEidt commentedon May 5, 2020
There are definitely places where EH Write through could be slightly worse. In situations where there are multiple definitions in the Try clause, each of those will do a store, and may also define a register - requiring an additional mov if the value could have been directly defined to memory. It is most effective when there are more uses than defs, and when register pressure is not excessive.
sebastienros commentedon May 5, 2020
I ran PlaintextPlatform 5 times for each, and the min value for no EHWT is greater than the max value with it, so it's definitely slower with on this scenario:
COMPlus_EnableEHWriteThru=0
COMPlus_EnableEHWriteThru=1
AndyAyersMS commentedon May 5, 2020
I'm guessing you've looked into this, but couldn't we screen candidates and only enable write through for locals that have favorable def/use ratios?
CarolEidt commentedon May 5, 2020
I didn't pursue that; we don't readily have that information until we build Intervals, and at that point it is difficult for the register allocator to change its mind about making something a candidate. That said, it could presumably decide that some of the candidates should never get a register, effectively making it the same as if it were not a candidate, though that would require some tweaking to avoid actually allocating a register when not needed (it does better with RegOptional uses than defs).
It would require retaining additional information on each
Intervalwhich would in turn probably require some additional tuning to ensure that it doesn't degrade throughput.AndyAyersMS commentedon May 5, 2020
I wonder if we could gather this info during one of our ref count traversals. We currently don't distinguish reads and writes, but it seems like we easily could.
Probably best to first dig into the code that's causing slowdowns in the tests above to ensure this is why things end up slower.
AndyAyersMS commentedon May 6, 2020
This issue is nominally done. Would like to close and capture what remains in a new issue. @CarolEidt is there an issue for enabling EH Write Thru by default? I couldn't find one.
Feels like there is still a moderate amount of work to do before we can turn EH Write Thru on by default, and some risk that we're just not going to see the benefits here we'd hoped to see. I'd like to do a more careful evaluation and try and find or create examples where we clearly do expect to see benefits.
Seems like all this might be a stretch for 5.0 but possibly worthwhile; trying to assess if we should find a way to do that sometime soon, or defer all of this until after 5.0.
CarolEidt commentedon May 6, 2020
Thanks for all your analysis, Andy. I agree that this issue, the investigation, is complete.
There's no issue for enabling EH write thru by default - I'll add one.
AndyAyersMS commentedon May 6, 2020
Thanks, I will close this issue.
@sebastienros thanks for running all those tests for us -- we'll need to dig in deeper to understand what's going on. Hope we can get to it before too long.