JIT: Improve tier1 OSR code quality for long-running runtime-async methods #133672

Motivation

Long-running async methods frequently contain request-processing or I/O loops. With runtime async, these methods can remain logically active for the lifetime of a connection, suspending and resuming many times without completing the invocation.

This makes Tier1 OSR code quality particularly important. Ordinary tier promotion benefits subsequent invocations, but a long-running invocation needs OSR to transition away from Tier0. In the ASP.NET benchmarks investigated here, the hot versions of these loops were Tier1 OSR bodies, not ordinary Tier1 bodies.

We should consider making this a focus area for .NET 12: OSR is not just a startup transition for these methods. Its code can become the steady-state implementation for a substantial part of the workload.

Examples in ASP.NET Core

Three Kestrel methods illustrate this pattern:

• SocketConnection.DoSend: a while (true) loop that awaits pipe reads and socket sends.
• SocketConnection.DoReceive: a connection-lifetime loop that awaits socket readiness, receives data, and flushes the input pipe.
• HttpProtocol.ProcessRequests<TContext>: a keep-alive request loop containing a nested read/parse loop, followed by application processing and response completion.

These are not necessarily expensive individual invocations by call count. Their importance comes from the amount of work done inside one invocation and across its async resumptions.

Observed problem

In the Linux Arm64 JSON benchmark, the normal DoSend compilation produced Instrumented Tier0 followed by Tier1 OSR. The OSR compilation received Dynamic PGO data.

One dump had an entry count of approximately 1 and approximately 245,000 loop backedges. That entry count is consistent with a single transfer into the long-running OSR invocation; it does not mean the loop executes only once.

The resulting high loop frequencies relative to entry make call sites inside the loop appear extremely hot to the inliner. In this workload, the Tier1 OSR compilation performed extensive inlining. This increased code size, frame size, and the number of suspension paths.

For one matched DoSend comparison, with the same experimental Pipe.ReadAsync source restructuring in both configurations:

Exact numbers vary between Dynamic PGO captures, but the large difference in compilation shape was clear.

AggressiveOptimization is not an isolated test of OSR machinery: it bypasses tiering and Dynamic PGO as well. The comparison therefore implicates the overall OSR/PGO/inlining strategy, rather than proving that OSR entry or continuation resumption itself accounts for the cost.

Performance evidence

In earlier adjacent Arm64 JSON comparisons, applying AggressiveOptimization to these three methods improved throughput (measured by RPS) by approximately 4%. Isolated experiments showed improvements for ProcessRequests and DoSend.

Continuation capture was also contributing unnecessary work. Separate JIT experiments removed falsely live struct temporaries and reduced continuation copying. However, reducing capture alone did not consistently reproduce the gains from AggressiveOptimization.

This suggests that continuation size is only part of the problem. The larger inline graph, frame size, spills, and duplicated suspension/control-flow paths also warrant investigation. We have not isolated their individual contributions.

Suggested .NET 12 investigation

• Add representative long-lived runtime-async loops to steady-state performance and code-quality coverage, including execution across repeated suspension and resumption.
• Examine how OSR entry counts and loop frequencies influence Dynamic PGO inlining profitability.
• Account for the code-size and suspension-state costs of async inlining, not just the apparent hotness of the call site.
• Compare Tier1 OSR, ordinary Tier1 where reachable, and non-tiered FullOpts code using equivalent workloads and carefully controlled profiling inputs.
• Track frame size, spills, continuation capture, and suspension-path duplication alongside throughput.

The goal should be good steady-state code for these common async patterns without requiring applications or frameworks to add AggressiveOptimization annotations that give up Dynamic PGO.