[Android] Application crashes when an exception is thrown after running a System.Diagnostics.Process

[simonrozsival]

Description

Moved from an original issue in xamarin-android: dotnet/android#8645

Application does not catch exception after running a System.Diagnostics.Process in release mode, in debug mode it runs fine.

Note that it does not cause a crash on every exception for example if you throw new Exception(), this won't result in a crash in release mode.

I can reproduce the crash in the runtime directly using the Android sample app. The original report mentions that there isn't a crash in Debug mode. I can confirm that the sample app doesn't crash when I enable interpreter.

Reproduction Steps

Change the code in the Android sample project (src/mono/sample/Android/Program.cs):

#nullable enable

using System;
using System.Xml;
using System.Diagnostics;

Test();

void Test()
{
    CatchException("Before ShellCommand");

    using var _ = Process.Start(new ProcessStartInfo("/system/bin/hostname"));

    CatchException("After ShellCommand"); // this will crash the application in release mode
}

void CatchException(string name)
{
    Console.WriteLine(name);
    try
    {
        XmlAttributeCollection? attributes = null;
        _ = attributes!["foo"];
    }
    catch (Exception e)
    {
        Console.WriteLine($"EXCEPTION '{name}' {e}");
    }
}

Run the code in Release configuration on Android emulator:

make -C src/mono/sample/Android MONO_CONFIG=Release MONO_ARCH=arm64 runtimepack run

Read the app output using adb:

adb logcat -s DOTNET
adb logcat --buffer=crash

Expected behavior

The app should run without crashing and with the following output:

> adb logcat -s DOTNET
DOTNET  : Before ShellCommand
DOTNET  : EXCEPTION 'Before ShellCommand' System.NullReferenceException: Object reference not set to an instance of an object
DOTNET  :    at Program.<<Main>$>g__CatchException|0_1(String )
DOTNET  : After ShellCommand
DOTNET  : EXCEPTION 'After ShellCommand' System.NullReferenceException: Object reference not set to an instance of an object
DOTNET  :    at Program.<<Main>$>g__CatchException|0_1(String )

Actual behavior

The app crashes with the following outputs:

> adb logcat -s DOTNET
...
DOTNET  : Before ShellCommand
DOTNET  : EXCEPTION 'Before ShellCommand' System.NullReferenceException: Object reference not set to an instance of an object
DOTNET  :    at Program.<<Main>$>g__CatchException|0_1(String )
DOTNET  : After ShellCommand

> adb logcat --buffer=crash
--------- beginning of crash
libc    : exiting due to SIG_DFL handler for signal 11, ucontext 0x7394f9de20
libc    : Fatal signal 6 (SIGABRT), code -1 (SI_QUEUE) in tid 9369 (.dot.MonoRunner), pid 9354 (ot.HelloAndroid)
DEBUG   : *** *** *** *** *** *** *** *** *** *** *** *** *** *** *** ***
DEBUG   : Build fingerprint: 'google/sdk_gphone64_arm64/emu64a:13/TE1A.220922.012/9302419:user/release-keys'
DEBUG   : Revision: '0'
DEBUG   : ABI: 'arm64'
DEBUG   : Timestamp: 2024-01-19 11:36:40.147858138+0100
DEBUG   : Process uptime: 2s
DEBUG   : Cmdline: net.dot.HelloAndroid
DEBUG   : pid: 9354, tid: 9369, name: .dot.MonoRunner  >>> net.dot.HelloAndroid <<<
DEBUG   : uid: 10188
DEBUG   : tagged_addr_ctrl: 0000000000000001 (PR_TAGGED_ADDR_ENABLE)
DEBUG   : pac_enabled_keys: 000000000000000f (PR_PAC_APIAKEY, PR_PAC_APIBKEY, PR_PAC_APDAKEY, PR_PAC_APDBKEY)
DEBUG   : signal 6 (SIGABRT), code -1 (SI_QUEUE), fault addr --------
DEBUG   : Abort message: 'exiting due to SIG_DFL handler for signal 11, ucontext 0x7394f9de20'
DEBUG   :     x0  0000000000000000  x1  0000000000002499  x2  0000000000000006  x3  0000007394f9dc60
DEBUG   :     x4  0000008080808080  x5  0000008080808080  x6  0000008080808080  x7  8080808080000000
DEBUG   :     x8  00000000000000f0  x9  0000007380c2ea00  x10 0000000000000001  x11 0000007380c6cde4
DEBUG   :     x12 0101010101010101  x13 000000007fffffff  x14 000000000035c8e0  x15 0000000000000060
DEBUG   :     x16 0000007380cd1d58  x17 0000007380caec70  x18 0000007077334000  x19 000000000000248a
DEBUG   :     x20 0000000000002499  x21 00000000ffffffff  x22 0000007394f9de48  x23 0000007394ba06f8
DEBUG   :     x24 0000000000000040  x25 00000070d95be000  x26 000000000000000b  x27 0000007394ba0318
DEBUG   :     x28 0000000000000000  x29 0000007394f9dce0
DEBUG   :     lr  0000007380c5e968  sp  0000007394f9dc40  pc  0000007380c5e994  pst 0000000000001000
DEBUG   : backtrace:
DEBUG   :       #00 pc 0000000000051994  /apex/com.android.runtime/lib64/bionic/libc.so (abort+164) (BuildId: 01331f74b0bb2cb958bdc15282b8ec7b)
DEBUG   :       #01 pc 0000000000002520  /apex/com.android.art/lib64/libsigchain.so (art::SignalChain::Handler(int, siginfo*, void*)+1172) (BuildId: d5ff66d01eb05cf124e6fe95ee64abfd)
DEBUG   :       #02 pc 000000000000084c  [vdso]
DEBUG   :       #03 pc 000000000000b03c  <anonymous:7394e14000>

Regression?

I can reproduce the crash on release/7.0.

Known Workarounds

No response

Configuration

• replicated with dotnet/runtime main (f6824bc9e8c68d3edb3d0a4c91e9bacd73d7767f)
• OS: Android
• arch: arm64
• the issue seems to be specific to Android (I wasn't able to reproduce the same issue with mono on macOS), I haven't tested on Android x64 yet

Other information

No response

Tagging subscribers to 'arch-android': @steveisok, @akoeplinger
See info in area-owners.md if you want to be subscribed.

Issue Details

---

Description

Moved from an original issue in xamarin-android: dotnet/android#8645

Application does not catch exception after running a System.Diagnostics.Process in release mode, in debug mode it runs fine.

Note that it does not cause a crash on every exception for example if you throw new Exception(), this won't result in a crash in release mode.

I can reproduce the crash in the runtime directly using the Android sample app. The original report mentions that there isn't a crash in Debug mode. I can confirm that the sample app doesn't crash when I enable interpreter.

Reproduction Steps

Change the code in the Android sample project (src/mono/sample/Android/Program.cs):

#nullable enable

using System;
using System.Xml;
using System.Diagnostics;

Test();

void Test()
{
    CatchException("Before ShellCommand");

    using var _ = Process.Start(new ProcessStartInfo("/system/bin/hostname"));

    CatchException("After ShellCommand"); // this will crash the application in release mode
}

void CatchException(string name)
{
    Console.WriteLine(name);
    try
    {
        XmlAttributeCollection? attributes = null;
        _ = attributes!["foo"];
    }
    catch (Exception e)
    {
        Console.WriteLine($"EXCEPTION '{name}' {e}");
    }
}

Run the code in Release configuration on Android emulator:

make -C src/mono/sample/Android MONO_CONFIG=Release MONO_ARCH=arm64 runtimepack run

Read the app output using adb:

adb logcat -s DOTNET
adb logcat --buffer=crash

Expected behavior

The app should run without crashing and with the following output:

> adb logcat -s DOTNET
DOTNET  : Before ShellCommand
DOTNET  : EXCEPTION 'Before ShellCommand' System.NullReferenceException: Object reference not set to an instance of an object
DOTNET  :    at Program.<<Main>$>g__CatchException|0_1(String )
DOTNET  : After ShellCommand
DOTNET  : EXCEPTION 'After ShellCommand' System.NullReferenceException: Object reference not set to an instance of an object
DOTNET  :    at Program.<<Main>$>g__CatchException|0_1(String )

Actual behavior

The app crashes with the following outputs:

> adb logcat -s DOTNET
...
DOTNET  : Before ShellCommand
DOTNET  : EXCEPTION 'Before ShellCommand' System.NullReferenceException: Object reference not set to an instance of an object
DOTNET  :    at Program.<<Main>$>g__CatchException|0_1(String )
DOTNET  : After ShellCommand

> adb logcat --buffer=crash
--------- beginning of crash
libc    : exiting due to SIG_DFL handler for signal 11, ucontext 0x7394f9de20
libc    : Fatal signal 6 (SIGABRT), code -1 (SI_QUEUE) in tid 9369 (.dot.MonoRunner), pid 9354 (ot.HelloAndroid)
DEBUG   : *** *** *** *** *** *** *** *** *** *** *** *** *** *** *** ***
DEBUG   : Build fingerprint: 'google/sdk_gphone64_arm64/emu64a:13/TE1A.220922.012/9302419:user/release-keys'
DEBUG   : Revision: '0'
DEBUG   : ABI: 'arm64'
DEBUG   : Timestamp: 2024-01-19 11:36:40.147858138+0100
DEBUG   : Process uptime: 2s
DEBUG   : Cmdline: net.dot.HelloAndroid
DEBUG   : pid: 9354, tid: 9369, name: .dot.MonoRunner  >>> net.dot.HelloAndroid <<<
DEBUG   : uid: 10188
DEBUG   : tagged_addr_ctrl: 0000000000000001 (PR_TAGGED_ADDR_ENABLE)
DEBUG   : pac_enabled_keys: 000000000000000f (PR_PAC_APIAKEY, PR_PAC_APIBKEY, PR_PAC_APDAKEY, PR_PAC_APDBKEY)
DEBUG   : signal 6 (SIGABRT), code -1 (SI_QUEUE), fault addr --------
DEBUG   : Abort message: 'exiting due to SIG_DFL handler for signal 11, ucontext 0x7394f9de20'
DEBUG   :     x0  0000000000000000  x1  0000000000002499  x2  0000000000000006  x3  0000007394f9dc60
DEBUG   :     x4  0000008080808080  x5  0000008080808080  x6  0000008080808080  x7  8080808080000000
DEBUG   :     x8  00000000000000f0  x9  0000007380c2ea00  x10 0000000000000001  x11 0000007380c6cde4
DEBUG   :     x12 0101010101010101  x13 000000007fffffff  x14 000000000035c8e0  x15 0000000000000060
DEBUG   :     x16 0000007380cd1d58  x17 0000007380caec70  x18 0000007077334000  x19 000000000000248a
DEBUG   :     x20 0000000000002499  x21 00000000ffffffff  x22 0000007394f9de48  x23 0000007394ba06f8
DEBUG   :     x24 0000000000000040  x25 00000070d95be000  x26 000000000000000b  x27 0000007394ba0318
DEBUG   :     x28 0000000000000000  x29 0000007394f9dce0
DEBUG   :     lr  0000007380c5e968  sp  0000007394f9dc40  pc  0000007380c5e994  pst 0000000000001000
DEBUG   : backtrace:
DEBUG   :       #00 pc 0000000000051994  /apex/com.android.runtime/lib64/bionic/libc.so (abort+164) (BuildId: 01331f74b0bb2cb958bdc15282b8ec7b)
DEBUG   :       #01 pc 0000000000002520  /apex/com.android.art/lib64/libsigchain.so (art::SignalChain::Handler(int, siginfo*, void*)+1172) (BuildId: d5ff66d01eb05cf124e6fe95ee64abfd)
DEBUG   :       #02 pc 000000000000084c  [vdso]
DEBUG   :       #03 pc 000000000000b03c  <anonymous:7394e14000>

Regression?

I can reproduce the crash on release/7.0.

Known Workarounds

No response

Configuration

• replicated with dotnet/runtime main (f6824bc9e8c68d3edb3d0a4c91e9bacd73d7767f)
• OS: Android
• arch: arm64
• the issue seems to be specific to Android (I wasn't able to reproduce the same issue with mono on macOS), I haven't tested on Android x64 yet

Other information

No response

Author:	simonrozsival
Assignees:	-
Labels:	os-android, area-Diagnostics-mono
Milestone:	-

[akoeplinger]

Process.Start() eventually calls into SystemNative_ForkAndExecProcess and that does something with signals, maybe that confuses the runtime somehow.

/cc @lambdageek @vargaz

[lambdageek]

@simonrozsival do you still have the sample app on your machine. can you try setting MONO_DEBUG to explicit-null-checks and runnign it again with the JIT.

I think you can do this in the sample app csproj:

<ItemGroup>
  <SampleAppEnvironmentVariables Include="MONO_DEBUG">
    <Value>explicit-null-checks</Value>
  </SampleAppEnvironmentVariables>
</ItemGroup>
...
<AndroidAppBuilderTask AppDir="..." ... 
    EnvironmentVariables="@(SampleAppEnvironmentVariables)"
    ...
/>

[grendello]

Process.Start() eventually calls into SystemNative_ForkAndExecProcess and that does something with signals, maybe that confuses the runtime somehow.

/cc @lambdageek @vargaz

In the original issue's log excerpt it's shown that the default signal 11 (segfault) handler is used, as if indeed all the chaining MonoVM does for signal handlers were gone. I think we've seen this error in the past, but can't recall the details. One thing that could cause it is a thread that's not attached to neither JavaVM nor MonoVM, perhaps?

[vargaz]

Its possible that either the sys.native code or the android system code resets these signal handlers that the runtime depend on when forking/exec-ing.

[simonrozsival]

@lambdageek with those changes it works as expected.

[lambdageek]

Its possible that either the sys.native code or the android system code resets these signal handlers that the runtime depend on when forking/exec-ing.

System.Native does change the signal handlers, but only in the child after the fork, before exec. Any chance that on Android this somehow affects the parent process too?

runtime/src/native/libs/System.Native/pal_process.c

Lines 342 to 391 in 843316f

	#if defined (__GLIBC__)
	if ((processId = vfork()) == 0) // processId == 0 if this is child process
	#else
	// musl libc has an undocumented failure mode around setuid(); we must exclude it.
	if (setCredentials)
	{
	processId = fork();
	}
	else
	{
	processId = vfork();
	}
	if (processId == 0)
	#endif
	#else
	if ((processId = fork()) == 0) // processId == 0 if this is child process
	#endif
	{
	// It turns out that child processes depend on their sigmask being set to something sane rather than mask all.
	// On the other hand, we have to mask all to avoid our own signal handlers running in the child process, writing
	// to the pipe, and waking up the handling thread in the parent process. This also avoids third-party code getting
	// equally confused.
	// Remove all signals, then restore signal mask.
	// Since we are in a vfork() child, the only safe signal values are SIG_DFL and SIG_IGN. See man 3 libthr on BSD.
	// "The implementation interposes the user-installed signal(3) handlers....to postpone signal delivery to threads
	// which entered (libthr-internal) critical sections..." We want to pass SIG_DFL anyway.
	sigset_t junk_signal_set;
	struct sigaction sa_default;
	struct sigaction sa_old;
	memset(&sa_default, 0, sizeof(sa_default)); // On some architectures, sa_mask is a struct so assigning zero to it doesn't compile
	sa_default.sa_handler = SIG_DFL;
	for (int sig = 1; sig < NSIG; ++sig)
	{
	if (sig == SIGKILL || sig == SIGSTOP)
	{
	continue;
	}
	if (!sigaction(sig, NULL, &sa_old))
	{
	void (*oldhandler)(int) = handler_from_sigaction (&sa_old);
	if (oldhandler != SIG_IGN && oldhandler != SIG_DFL)
	{
	// It has a custom handler, put the default handler back.
	// We check first to preserve flags on default handlers.
	sigaction(sig, &sa_default, NULL);
	}
	}
	}
	pthread_sigmask(SIG_SETMASK, &old_signal_set, &junk_signal_set); // Not all architectures allow NULL here

[mohachouch]

After hours of not understanding why my Android application (net7.0-android) crashes with each Exception, this bug allowed me to understand.

I have exactly the same symptoms, my application crashes after an Exception (null ref ect).

[vargaz]

I remember a similar issue as well years ago, can't find it any more.