Repository navigation
AggressiveInlining not respected when Method grow too large #41692
Description
Description
JIT will refuse to inline AggressiveInlining methods when Methods grow too large (IL size / too many branches, I'm unsure which one, but I expect IL size)
If I understand AggressiveInlining correctly, this is not intended.
This happened to me in a method like the below, but I've encountered it before:
public IntPtr Load(int slot) {
if (slot == SLOT_ONE)
{
return _slotOne;
}
// x 1000 or so
ThrowHelper();
return default;
}This does not inline/constant fold, leading to quite the performance hit since now all those comparisons are actual comparisons.
Splitting this into a binary tree of sorts has led to JIT inlining it, and a good performance increase.
Analysis
I think this can be related to #38106
I don't understand JIT very well, but I think this can have multiple causes.
- a deeper call tree, which stops the changes introduced in JIT: allow some aggressive inlines to go over budget #38163 from helping
It might be an idea to look for comparisons with constants that "end" the method and prioritize inlining? Looking at the inlinepolicy, which I think is where this kind of thing is decided, this would be difficult, but possible?
category:cq
theme:inlining
skill-level:intermediate
cost:medium
Activity
AndyAyersMS commentedon Sep 1, 2020
Aggressive inline candidates are assumed to be profitable; they bypass the normal profitability checks. So in particular we won't analyze their IL for patterns that might be optimizable.
Can you provide a repro for the issues you're seeing? We are trying to be as aggressive as we can without getting into situations where the jit will create enormous methods or use huge amounts of memory while jitting.
HurricanKai commentedon Sep 1, 2020
Sample.zipthe
Bstruct contains a "simple" method with 4000 lookups, and while this is extreme, this is a real-world-like case. I'm writing a Generator that binds native methods, and am generating a table to cache the function pointers to those methods. I generate well over 4000 such cache slots.EgorBo commentedon Sep 1, 2020
Your repro doesn't have
AggressiveInlininganywhere. WhileLookuplooks like it should be inlined to a single instruction when the arg is a constant I believe JIT doesn't want to spend time on analyzing big methods (because in 99% cases those won't be profitable to inline), e.g. in your case:if you put
AggressiveInliningon top ofLookupit should work fine.HurricanKai commentedon Sep 1, 2020
Sorry. Not sure what I thought with that sample.
I've just reverted some changes, and it appears what is actually happening (I didn't know this was a thing when I first encountered this issue) is I'm hitting the
m_CallsiteDepth == 1limitation.I think it works something like this: I'm generating a
Load(int slot, string entrypoint)method, which just calls another method, which is the actual lookup table. This simple call is of course inlined, which "consumes" the top-level inline possibility? The actual lookup is way too big to be considered for lookup, and can't be considered for inlining anyways causem_CallsiteDepthexceeds1already.This is unfortunate, but I'm not too sure if JIT can do much about this (besides raising the limit, but I'm sure there are some reasons for the limit being this low). I'll definitely have to keep this in mind, since I often rely heavily on JIT eliminating branches like this at runtime...
Apologies again for the incorrect sample 😅
Also, here is the actual generated code + disassembly.
(Relevant method is
GeneratedVTable:Loadbeing inlined toGeneratedVTable:Load_2575_67097892)JulieLeeMSFT commentedon Sep 1, 2020
@AndyAyersMS Since you are looking into it, assigning this to you. Please help triage this.
AndyAyersMS commentedon Sep 1, 2020
@HurricanKai thanks for the example. The jit should be able to tell after inlining such a method that it wasn't nearly as big as it seemed; I'm trying to think of how it could figure this out ahead of time, or along the way. Ahead of time seems hard; as @EgorBo says, we don't want to spend a lot of time analyzing large methods; and our analysis isn't all that powerful.
But checking along the way seems viable -- I think something like the following can work. When the jit sees an
AggressiveInliningcandidate that is projected to go over budget, it can go ahead and starts inlining anyways, keeping track in real time of how much of the method has been imported and continually checking that versus the budget. If in the middle of inline the jit goes over budget, wil then abandon the inline.That way if the jit is really just touching a fraction of a huge method during importation then the inline can succeed. The only downside is that the jit may spend a bunch of cycles attempting inlines that later are abandoned, but that shouldn't happen too often, and the use of
AggressiveInlininggives the jit some license to spend a bit more time jitting.JulieLeeMSFT commentedon Sep 2, 2020
Moving to .NET 6.
5 remaining items
JIT: revise criteria for inline candidates with intrinsics or folding…
AndyAyersMS commentedon Jun 14, 2021
@EgorBo perhaps you can follow up here?
AndyAyersMS commentedon Jul 12, 2021
Not sure we're going to get to this in 6.0, so moving to future.