[mono][ios] Mono prevents crash logs from being generated in case SIGSEGV is raised from native code

@lambdageek

Description

While investigating the behavior of the crash reported in: #105245 on iOS device we noticed that such error does not:

Produce any useful information in the Console.app
It did not generate a crash report on a device

Repro

Install official .NET 9 preview 6 release
Install MAUI workloads
Create a MAUI template app
Build/run it on a physical device in Debug mode:

dotnet build -f net9.0-ios -r ios-arm64 -t:Run -p:_DeviceName=XXXXXX

Console.app shows only that the program exited:

default  14:14:51.131793+0200  SpringBoard  [app<com.companyname.myapp(80F7F2DD-E9BB-43D8-AB25-7A0DE1A64558)>:6714] Process exited: <RBSProcessExitContext| voluntary>.

No crash logs are generated on a device

Investigation

As initially assumed, the crash was not caused by asserting from the runtime (where we abort() properly), but instead the code was trying to read from an invalid memory address (something like 0xa8) so SIGSEGV was raised.
The mono's signal handler catches this, but does not remove it self as a SIGSEGV handler when it starts handling the signal.
After the handling is done, the handler returns, but the same signal is caught again and on second handling the program exits out with -1 in case of double faulting:

runtime/src/mono/mono/mini/mini-posix.c

Lines 789 to 792 in fdb7415

    
           } else { 
        
           	g_async_safe_printf ("\nAn error has occurred in the native fault reporting. Some diagnostic information will be unavailable.\n"); 
        
           	g_async_safe_printf ("\nExiting early due to double fault.\n"); 
        
           	_exit (-1);

This causes the app to silently exit not including any information about the crash.
Additionally, the messages from the signal handler (like information about the native and managed stack traces) that are using g_async_safe_printf are not shown in the system log.

FWIW In the signal handler we do unregister the runtime for some signals, but not for SIGSEGV:

runtime/src/mono/mono/mini/mini-exceptions.c

Lines 2932 to 2942 in d606c60

    
           	g_assert (sigaction (SIGABRT, &sa, NULL) != -1); 
        
           	/* On some systems we get a SIGILL when calling abort (), because it might 
        
           	 * fail to raise SIGABRT */ 
        
           	g_assert (sigaction (SIGILL, &sa, NULL) != -1); 
        
           	/* Remove SIGCHLD, it uses the finalizer thread */ 
        
           	g_assert (sigaction (SIGCHLD, &sa, NULL) != -1); 
        
           	/* Remove SIGQUIT, we are already dumping threads */ 
        
           	g_assert (sigaction (SIGQUIT, &sa, NULL) != -1);

Proposal

Investigate if we can unregister the SIGSEGV handler if we've detected that the signal it's not coming from managed code (to distinguish NullReferenceException)
Try using a platform specific logging mechanism so that messages from the signal handler end up in the system log when a fatal error occurs

PS Thanks @lambdageek for assistance

@rolfbjarne

/cc: @rolfbjarne

The mono's signal handler catches this, but does not remove it self as a SIGSEGV handler when it starts handling the signal.
After the handling is done, the handler returns

This sounds a bit weird.

If the SIGSEGV comes from managed code, the signal handler should raise a NullReferenceException.

This didn't happen, so the SIGSEGV didn't come from managed code, and in that case the signal handler should chain to the previous signal handler.

This didn't happen (presumably because there was no previous signal handler), and in that case Mono will print native crash info and call abort:

runtime/src/mono/mono/mini/mini-posix.c

Lines 225 to 226 in af1b3b0

    
           mono_handle_native_crash (mono_get_signame (info->si_signo), &mctx, info); 
        
           abort ();

But that didn't happen?

Instead we got a double fault... it sounds like the second SIGSEGV didn't come from returning from the first signal handler, but instead occurred during the first signal handler.

This didn't happen (presumably because there was no previous signal handler), and in that case Mono will print native crash info and call abort:

runtime/src/mono/mono/mini/mini-posix.c

Lines 225 to 226 in af1b3b0

mono_handle_native_crash (mono_get_signame (info->si_signo), &mctx, info);

abort ();

But that didn't happen?

If I am not mistaken the code you linked is the SIGABRT handler.

We are instead reaching this code:

runtime/src/mono/mono/mini/mini-runtime.c

Line 3808 in af1b3b0

MONO_SIG_HANDLER_FUNC (, mono_sigsegv_signal_handler)

which further handles the signal and returns since signal chaining is enabled:

runtime/src/mono/mono/mini/mini-runtime.c

Lines 3908 to 3912 in af1b3b0

    
           mono_handle_native_crash (mono_get_signame (SIGSEGV), &mctx, (MONO_SIG_HANDLER_INFO_TYPE*)info); 
        
           if (mono_do_crash_chaining) { 
        
           	mono_chain_signal (MONO_SIG_HANDLER_PARAMS); 
        
           	return;

If I am not mistaken the code you linked is the SIGABRT handler.

Ah my bad, sorry about the confusion.

I'm just really surprised this isn't a regression, because I'm certain I've seen crash reports for SIGSEGVs originating in native code before (but then again I might be misremembering - I just tried in .NET 8, and I got the double fault with no crash report).

FWIW I've seen customers run into this now.

FWIW I've seen customers run into this now.

Do we have a specific issue to link to this one + raise a priority on fixing this?

FWIW I've seen customers run into this now.

Do we have a specific issue to link to this one + raise a priority on fixing this?

The customer in question is the one who reported this: #107641

Although much of the interaction happened via email, so reading the issue isn't all that useful.

@ivanpovazan

@ivanpovazan @steveisok @vitek-karas this one is happening more frequently with customers now - could we get this prioritized? I think @StephaneDelcroix can provide a repro...

@vitek-karas

I think @StephaneDelcroix can provide a repro...

If you have a repro, great and I'll work with @vitek-karas to get this prioritized.

@vitek-karas

I just synced with @vitek-karas I will look into this as soon as possible.

Navigation Menu

Repository navigation

[mono][ios] Mono prevents crash logs from being generated in case `SIGSEGV` is raised from native code #106064

Description

Repro

Investigation

Proposal

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

	} else {
	g_async_safe_printf ("\nAn error has occurred in the native fault reporting. Some diagnostic information will be unavailable.\n");
	g_async_safe_printf ("\nExiting early due to double fault.\n");
	_exit (-1);

	g_assert (sigaction (SIGABRT, &sa, NULL) != -1);

	/* On some systems we get a SIGILL when calling abort (), because it might
	* fail to raise SIGABRT */
	g_assert (sigaction (SIGILL, &sa, NULL) != -1);

	/* Remove SIGCHLD, it uses the finalizer thread */
	g_assert (sigaction (SIGCHLD, &sa, NULL) != -1);

	/* Remove SIGQUIT, we are already dumping threads */
	g_assert (sigaction (SIGQUIT, &sa, NULL) != -1);

	mono_handle_native_crash (mono_get_signame (info->si_signo), &mctx, info);
	abort ();

	mono_handle_native_crash (mono_get_signame (SIGSEGV), &mctx, (MONO_SIG_HANDLER_INFO_TYPE*)info);

	if (mono_do_crash_chaining) {
	mono_chain_signal (MONO_SIG_HANDLER_PARAMS);
	return;

[mono][ios] Mono prevents crash logs from being generated in case SIGSEGV is raised from native code #106064

Description

Description

Repro

Investigation

Proposal

Activity

ivanpovazan commented on Aug 7, 2024

rolfbjarne commented on Aug 7, 2024

ivanpovazan commented on Aug 7, 2024

rolfbjarne commented on Aug 7, 2024

rolfbjarne commented on Oct 4, 2024

ivanpovazan commented on Oct 7, 2024

rolfbjarne commented on Oct 7, 2024

Redth commented on Dec 2, 2024

steveisok commented on Dec 2, 2024

ivanpovazan commented on Dec 2, 2024

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions

[mono][ios] Mono prevents crash logs from being generated in case `SIGSEGV` is raised from native code #106064