Repository navigation
mono gc not set static field back after object moved when enable interp mode #100065
Description
Description
We are embeding mono in UnrealEngine, when we upgrade .ne8.0(v8.0.2), we found sgen-gc will not set static field back after object move when enable hotreloadcomponent.
Reproduction Steps
We have not separate code to reproduct it, but I will describe it as detailed as possible.
we have restore object in static field, when next pass it to internal call the object address after the gc have move it, the value not set back.
I show the code first:
public partial class FTicker : ScriptingWrapperOfType
{
public static FTicker Internal_GetCoreTicker_Cache = null;
[MethodImpl(MethodImplOptions.InternalCall)]
private static extern FTicker Internal_GetCoreTicker();
public static FTicker GetCoreTicker()
{
if(Internal_GetCoreTicker_Cache == null)
{
Internal_GetCoreTicker_Cache = Internal_GetCoreTicker();
}
return Internal_GetCoreTicker_Cache;
}
[MethodImpl(MethodImplOptions.InternalCall)]
public static extern void TearDownCoreTicker();
[MethodImpl(MethodImplOptions.InternalCall)]
public extern FDelegateHandle AddTicker(TDelegate__float_Ret_bool InDelegate, float InDelay = 0.0f);
}the internal call
static FDelegateHandle_Proxy Internal_FTicker_AddTicker(MonoObject* self, MonoDelegate* InDelegate, float InDelay)
{
// these code is use for debug this problem
MonoImage* EngineImage = GetScriptingAssemblyManager().FindImage("UnrealEngine");
if (EngineImage)
{
MonoClass* TickerClass = mono_class_from_name(EngineImage, "UnrealEngine", "FTicker");
if (TickerClass)
{
MonoClassField* Internal_GetCoreTicker_Cache = mono_class_get_field_from_name(TickerClass, "Internal_GetCoreTicker_Cache");
if (Internal_GetCoreTicker_Cache)
{
MonoObject* data = nullptr;
MONO_ENTER_GC_UNSAFE
mono_field_static_get_value(mono_class_vtable(mono_domain_get(), TickerClass), Internal_GetCoreTicker_Cache, &data);
MONO_EXIT_GC_UNSAFE
}
}
}
// below is the real logic
TRACE_INTERALCALL_PROFILER_EVENT_SCOPE
TDelegate<bool(float)> InDelegate__Temp;
BindScriptIfException<bool, float>(InDelegate__Temp, InDelegate, true);
return FDelegateHandle_Proxy::Proxy(ReadonlyScriptingObjectConverter<FTicker>(self)->AddTicker(InDelegate__Temp, InDelay));
}the code logic execute as below :
tickHandle = FTicker.GetCoreTicker().AddTicker(OnTick);
// gc occured
tickHandle1 = FTicker.GetCoreTicker().AddTicker(OnTick1);
// but the second call `Internal_FTicker_AddTicker` the `self` object the vtable have set null and then can't get the object class. I show the behaviors with hotreload component enable or not.
if we not enable hotreload component, just use jit execute mode, it work correctly:
the first call AddTicker, we can checkout the self object:
and we check the Internal_GetCoreTicker_Cache static field:
the second call the self object address have changed:
if we enable hotreload component, the static field will not write back:
the first call:
the second call: the self object address have not been changed but the vtable is null
Expected behavior
work correctly
Actual behavior
crash
Regression?
.net7.0 work correctly
Known Workarounds
no
Configuration
windows
build v8.0.2 tag
set excute mode mono_jit_set_aot_mode(MONO_AOT_MODE_INTERP_ONLY);
Other information
No response
Activity
[-]mono gc not set static field back after object move when enable hotreload[/-][+]mono gc not set static field back after object moved when enable hotreload[/+]srxqds commentedon Mar 21, 2024
@lateralusX @BrzVlad can you help to check these problem?
BrzVlad commentedon Mar 21, 2024
This looks like when hot reload is enabled, the roots from the static fields of that class are not being registered at all to the GC. @lambdageek Do you understand what might be going on here ?
srxqds commentedon Mar 21, 2024
yeah, can you tell me where register static field to roots or scan static field
BrzVlad commentedon Mar 21, 2024
runtime/src/mono/mono/metadata/object.c
Line 2231 in e12e2fa
mono_gc_alloc_fixedwhich registers the memory as root, together with a gc descriptor for which slots are to be scanned. During gc, they are then scanned hereruntime/src/mono/mono/sgen/sgen-gc.c
Line 1288 in e12e2fa
I'm not familiar with the implications of hot reload over these static fields.
srxqds commentedon Mar 21, 2024
thank you @BrzVlad waiting for @lambdageek for help, or anyone can help me to reproduce it?
lambdageek commentedon Mar 21, 2024
All you do is just enable or disable the hot reload component? You're not actually applying any changes?
What happens if you disable hot reload, but set the execution mode to
mono_jit_set_aot_mode(MONO_AOT_MODE_INTERP_ONLY);?srxqds commentedon Mar 21, 2024
yes, I have not apply any change
the problem also exists.
@lambdageek
srxqds commentedon Mar 22, 2024
anyone can take a look at this question
srxqds commentedon Apr 1, 2024
we have comment out this line
runtime/src/mono/mono/metadata/assembly-load-context.c
Line 43 in 0f6df43
but the alc is not unloading.
and the second clue is the FTicker
Internal_GetCoreTicker_Cacheobject have been gc, I have found call the destructor functionit should introduce this problem.
but why it work on jit, but crash when interp enable.
@vargaz can you help me?
srxqds commentedon Apr 1, 2024
we found that call
mono_gc_walk_heapcan get the same FTicker object after the destructor has been called.srxqds commentedon Apr 1, 2024
how can we fixed it, our project is blocking by this problem.
anyone can help me, because the gc is diffcult to debug.
srxqds commentedon Apr 1, 2024
I have set gc_level=9, because the log file is too big for upload, I paste the log about the
FTickerobject:in jit mode:
in interp mode:
[-]mono gc not set static field back after object moved when enable hotreload[/-][+]mono gc not set static field back after object moved when enable interp mode[/+]srxqds commentedon Apr 2, 2024
I have modified
MINT_STSFLD_Oopcode withmono_gc_wbarrier_generic_store_internalseems solve the problem:but I have hit another problem: the generic method out parameter pass also have gc object error.
srxqds commentedon Apr 2, 2024
the second crash show below:
and the source code is :
BrzVlad commentedon Apr 2, 2024
Interesting. So it turns out we actually can allocate static field storage as part of another object when using hot reload. (https://github.com/dotnet/runtime/blob/main/src/mono/mono/component/hot_reload.c#L3092). The interpreter was not aware of this so it doesn't do write barriers when storing to static fields. My understanding is that you still encounter this crash when having hot reload enabled (not just by using mono interpreter) ?
If static field storage is part of another object then you are correct that we need write barriers when we store into it. MINT_STSFLD_O and also MINT_STSFLD_VT would need write barriers. For the latter you would need to either pass the field_klass to the opcode so you can use
mono_value_copy_internalor hack a call tosgen_card_table_mark_rangefor the area that is being written.Also, from the debugger screenshot, the bad object 0x1f4f1dee470 has its vtable 0x1f4f309df01. Because the vtable has its least significant bit 1, it is a sign that the value of the vtable is actually the location of the new object. This means that the object from the crash was promoted to location 0x1f4f309df00. You could investigate the type of object if it is something out of the ordinary.
In this type of crashes, it would be useful to run with
MONO_GC_DEBUG=check-remset-consistency. It should point to locations where there are missing cards.srxqds commentedon Apr 2, 2024
thank you, I will continue to try tomorrow.
yeah, we have enable hot reload component or apply any hot reload delta.
srxqds commentedon Apr 3, 2024
when I set
MONO_GC_DEBUG=check-remset-consistency, there are many object not set in ImmutableArrayso this set may not useful, and the
0x1f4f309df00class is System.Object so it doesn't provide much useful information for me.srxqds commentedon Apr 3, 2024
the
ImmutableArray<T>is struct soBrzVlad commentedon Apr 3, 2024
The logs support the existing theory that the issue is only with storing into static fields. Am I assuming correctly that you didn't tweak
MINT_STSFLD_VTto emit write barriers ? It could look something like this:srxqds commentedon Apr 7, 2024
this change works, thank you @BrzVlad
but but why won't happen when not enable collectiable alc?