Skip to content

JIT: (bug) AVX10.2 saturating float->int conversions (vcvttss2sis & co.) don't kill the destination GPR's GC liveness (GC hole) #135387

Open

Description

@EgorBo

When an AVX10.2 saturating conversion (vcvtt{ss,sd}2{s,us}is) writes a GPR that held a GC reference, the register stays reported as a live gcref until the next IG boundary.

Minimal Repro

using System;
using System.Runtime.CompilerServices;

public class Program
{
    static object s_o = new object();

    [MethodImpl(MethodImplOptions.NoInlining)]
    static object Get() => s_o;

    [MethodImpl(MethodImplOptions.NoInlining)]
    static int Test(int[] arr, float f)
    {
        object o = Get();
        GC.KeepAlive(o);
        int r = (int)f;
        int sum = 0;
        for (int i = 0; i < arr.Length; i++)
            sum += arr[i];
        return sum + r;
    }

    public static void Main() => Console.WriteLine(Test(new int[1], 123.0f));
}

Requires AVX10.2 codegen: ReadyToRun/NativeAOT with --instruction-set avx10v2 (no AVX10.2 hardware needed to see the codegen), or the JIT with DOTNET_EnableAVX10v2=1 on AVX10.2 hardware.

Expected

rax stops being reported as a gcref once vcvttss2sis overwrites it.

Actual

JitDisasmWithGC output (the method is fully interruptible):

G_M35747_IG02:        ; bbWeight=1, gcrefRegs=0008 {rbx}, byrefRegs=0000 {}, byref, isz
       call     [Program:Get():System.Object]
                            ; gcrRegs +[rax]
                            ; gcr arg pop 0
       vcvttss2sis eax, dword ptr [rsp+0x38]     ; integer in eax, rax still reported as gcref
       xor      ecx, ecx
       mov      edx, dword ptr [rbx+0x08]
       test     edx, edx
       jle      SHORT G_M35747_IG05
G_M35747_IG03:        ; bbWeight=0.50, gcrefRegs=0008 {rbx}, byrefRegs=0000 {}, byref
                            ; gcrRegs -[rax]

A GC anywhere between the conversion and the end of IG02 sees the integer as an object reference. The register form (vcvttss2sis eax, xmm0) and the 64-bit forms (e.g. vcvttsd2usis rax, qword ptr [rsp+0x38]) behave the same.

Regression?

Not applicable to .NET 8.0/9.0 (no avx10v2 support). .NET 10.0 and 11.0-rc2 crossgen2 emit the same code, and the emitter code in release/10.0 is the same.

Notes

  • emitInsCanOnlyWriteSSE2OrAVXReg (emitxarch.cpp) lists the SIMD instructions that write a GPR (cvttss2si*, rorx, pextr*, ...) but not INS_vcvtt{ss,sd}2{s,us}is{32,64} (nor the AVX512-FP16 vcvt[t]sh2[u]si*), so emitOutputRR/AM/SV/CV skip emitGCregDeadUpd for the destination.
  • With AVX10.2, every plain float/double -> int/uint/long/ulong cast is lowered to these instructions, so it's easy to hit.
  • Fix: add them to the "writes a general purpose register" list.

Activity

  1. added this to the 12.0.0 milestone on Oct 8, 2026
  2. added
    area-CodeGen-coreclrCLR JIT compiler in src/coreclr/src/jit and related components such as SuperPMI
    on Oct 8, 2026
  3. dotnet-policy-service commented on Oct 8, 2026

    @dotnet-policy-service
    Contributor

    Tagging subscribers to this area: @JulieLeeMSFT, @jakobbotsch
    See info in area-owners.md if you want to be subscribed.

  4. JulieLeeMSFT commented on Oct 8, 2026

    @JulieLeeMSFT
    Member

    @Ruihan-Yin, please take a look.

  5. Ruihan-Yin commented on Oct 8, 2026

    @Ruihan-Yin
    Member

    Looking into it, thanks for the notice.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

area-CodeGen-coreclrCLR JIT compiler in src/coreclr/src/jit and related components such as SuperPMIavx10Related to the AVX10 architecture

Type

No type

Projects

No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions