Repository navigation
PGO: class profile details we need to get right #48549
Description
- If we can't record a class because it's collectible, we still need to consider putting something into the table. Otherwise we may overstate the likelihood of other entries. There should be some well-known invalid handle we can store there. Fixed via Use unknown placeholder for collectible class typehandle #48707.
- Allow the sampling strategy to vary depending on use case. Dynamic PGO will want to use a small fixed window so the buffer has some recent observation bias; Static PGO should use the count value so it is not overly biased towards recent samples.
- Would be nice to come up with a scheme where table size could vary dynamically. Current default of 8 is likely too large as many sites will either never be hit or be monomorphic, but smaller fixed sizes will prevent understanding the distribution of classes at more complex sites. Ideally perhaps the table size starts at 1 (or even 0) and then grows somehow. Engineering all this seems tricky as part of the current appeal is that the storage needs are fixed at jit time. Also the probing must be concurrency safe without locking and we can't have too much extra overhead.
- Look critically at the class profile merging done by
dotnet-pgo. See notes below.
For the latter we should also consider enabling devirtualization (at least the analysis, if not the transformation) at Tier0 as there's no need to probe the class at a virtual or interface site if we know what the class must be. Need to experiment to see what fraction of class probes fall into that category.
category:cq
theme:profile-feedback
Metadata
Metadata
Assignees
Type
Projects
- Status
Activity
Use unknown placeholder for collectible class typehandle
Use unknown placeholder for collectible class typehandle (#48707)
AndyAyersMS commentedon Jun 14, 2021
Type Histogram Merging
From the 21301.4 optimization data collection, we have 4
.mibcfiles with data forSystem.Collections.Generic.Dictionary`2<System.__Canon,System.__Canon>.FindValue(__Canon). We want to merge these into a combined histogram.Looking at the offset 52 class probe we see (summarizing details below)
What should the merged table look like? Today we simply concatenate all the tables, producing a table of size 64, and sum all the counts, giving a net count of 159290. We then histogram the table and end up determining that the dominant entry is
System.RuntimeType, with a a likelihood of 26/64 = 0.43.However, on a count-weighted basis, we arguably should would estimate the likelihood of
System.RuntimeTypeasSo because
System.RuntimeTypeis common in the lighter-weight scenarios, we appear to overstate its overall likelihood.Also note that since the default table size in the runtime is 8, we can guess that the hello world
.mibcis already the result of a merge, and presumably we'd end up with yet another value, if we'd done a count-weighted merge instead of a concatenate-merge for the Hello World collection.What would a count-weighted merge look like? If we want to keep the
.mibcentries in the current table format, the correct approach seems to be some kind of probabilistic resampling of the individual tables.Since we do N-way merges as a series of pairwise merges, let's look at the two table case first. Suppose we have tables Ti with counts Ci and sizes Si. Then a count-weighted merging algorithm would be something like the following:
We have some flexibility here in choosing the merged table size, but it does not seem unreasonable to make it larger than either input table, to try and better represent the full diversity of the merge results.
This algorithm randomly samples from each table proportional to the count of observations in the table. To keep this semi-deterministic, the random seed would need to be some joint property of the tables, perhaps the hash of the method name augmented by the IL offset of the probe and the total count, or something similar.
However, this won't pass the associativity test; that is merging N entries will be order sensitive. If we have three tables, merge(merge(T0, T1), T2) != merge(T0, merge(T1, T2)). This seems undesirable (though the "errors" here should be random).
To have a fully general associative merge we'd need to form actual histograms and then merge them on a weight-adjusted basis. This requires a new schema type, analogous to
GetLikelyClassbut extended to represent all possible results.Details
;; techempower
;; orchard core
;; hello world
{
"ILOffset": 52,
"InstrumentationKind": "TypeHandleHistogramIntCount",
"Other": -2147483648,
"Data": 72561
},
{
"ILOffset": 52,
"InstrumentationKind": "TypeHandleHistogramTypeHandle",
"Other": -2147483648,
"Data": [
"[S.P.CoreLib]System.RuntimeType",
"[S.P.CoreLib]System.RuntimeType",
"[S.P.CoreLib]System.RuntimeType",
"[S.P.CoreLib]System.RuntimeType",
"[System.Linq.Expressions]System.Linq.Expressions.Expression1
1<System.Func2<object,object>>","[S.P.CoreLib]System.RuntimeType",
"[S.P.CoreLib]System.RuntimeType",
"[S.P.CoreLib]System.RuntimeType",
"[NuGet.Frameworks]NuGet.Frameworks.NuGetFramework",
"[Microsoft.Build]Microsoft.Build.Construction.XmlDocumentWithLocation",
"[Microsoft.Build]Microsoft.Build.Construction.XmlDocumentWithLocation",
"[Microsoft.Build]Microsoft.Build.Construction.XmlDocumentWithLocation",
"[Microsoft.Build]Microsoft.Build.Construction.XmlDocumentWithLocation",
"[Microsoft.Build]Microsoft.Build.Construction.XmlDocumentWithLocation",
"[NuGet.Frameworks]NuGet.Frameworks.NuGetFramework",
"[Microsoft.Build]Microsoft.Build.Construction.XmlDocumentWithLocation",
"UnknownType(0)",
"UnknownType(0)",
"UnknownType(0)",
"UnknownType(0)",
"UnknownType(0)",
"UnknownType(0)",
"UnknownType(0)",
"UnknownType(0)",
"[Microsoft.CodeAnalysis.CSharp]Microsoft.CodeAnalysis.CSharp.Syntax.AttributeSyntax",
"[Microsoft.CodeAnalysis.CSharp]Microsoft.CodeAnalysis.CSharp.CodeGen.DummyLocal",
"[S.P.CoreLib]System.RuntimeType",
"[Microsoft.CodeAnalysis.CSharp]Microsoft.CodeAnalysis.CSharp.CodeGen.DummyLocal",
"[S.P.CoreLib]System.RuntimeType",
"[Microsoft.CodeAnalysis.CSharp]Microsoft.CodeAnalysis.CSharp.CodeGen.DummyLocal",
"[S.P.CoreLib]System.RuntimeType",
"[S.P.CoreLib]System.RuntimeType",
"[Microsoft.Build]Microsoft.Build.Construction.XmlDocumentWithLocation",
"[Microsoft.Build]Microsoft.Build.Construction.XmlDocumentWithLocation",
"[NuGet.Frameworks]NuGet.Frameworks.NuGetFramework",
"[NuGet.Frameworks]NuGet.Frameworks.NuGetFramework",
"[Microsoft.Build]Microsoft.Build.Construction.XmlDocumentWithLocation",
"[Microsoft.Build]Microsoft.Build.Construction.XmlDocumentWithLocation",
"[NuGet.Frameworks]NuGet.Frameworks.NuGetFramework",
"[Microsoft.Build]Microsoft.Build.Construction.XmlDocumentWithLocation"
]
},
;; first time
@jakobbotsch -- FYI in case you want to look into class profile merging.
AndyAyersMS commentedon Jul 9, 2021
I think we are good enough now for .NET 6 -- we can revisit if we get evidence that the table size, merging, or sampling strategies are causing particular problems.