Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update readme #11

Merged
merged 2 commits into from
Apr 28, 2016
Merged

Update readme #11

merged 2 commits into from
Apr 28, 2016

Conversation

radical
Copy link
Member

@radical radical commented Apr 28, 2016

No description provided.

@jonpryor jonpryor merged commit b7ae30a into dotnet:master Apr 28, 2016
radical pushed a commit that referenced this pull request May 8, 2018
…State

We want to retrofit Xamarin.Android to use Java.Interop. This in turn
requires that Java.Interop be able to do everything that
Xamarin.Android does, with the added complication that Java.Interop
will be used in Xamarin.Android before major parts are complete (e.g.
Activation [8c83f64] and Issue #11 [0]).

Fortunately we can look at how Xamarin.Android is architected to see
what we might need to provide in advance before it's actually used,
which leads us to Xamarin.Android's internal
Java.Interop.IJavaObjectEx interface:

	interface IJavaObjectEx {
		IntPtr      KeyHandle         {get; set;}
		bool        IsProxy           {get; set;}
		bool        NeedsActivation   {get; set;}
		IntPtr      ToLocalJniHandle ();
	}

Of those members, IJavaObjectEx.KeyHandle is already exposed as
IJavaPeerable.JniIdentityHashCode, and
IJavaObjectEx.ToLocalJniHandle() shouldn't be needed (it was added to
incorrectly address a multithreading-related bug).

That leaves IJavaObjectEx.IsProxy and IJavaObjectEx.NeedsActivation,
which are both involved in Java-side activation.

In the interest of avoiding API breaks in the future, we need to
support those constructs in Java.Interop *now*, even if they won't be
fully utilized until later.

Additionally, those names suck for "public" names -- what do they
*mean*, sans context? -- and, while things have been reasonably stable
here for the past several years, I'm not entirely certain that more
such states won't need to be added in the future, so we need to
support IJavaObjectEx.IsProxy and IJavaObjectEx.NeedsActivation
semantics in an "extensible" manner?

The solution? Yet another [Flags] enum!

	[Flags]
	public enum JniManagedPeerStates {
		None,
		Activatable         = (1 << 0),
		Replaceable         = (1 << 1),
	}

The use of a [Flags] enum allows us to add additional states in the
future, should we need to do so.

JniManagedPeerStates.Activatable is IJavaObjectEx.NeedsActivation, and
means that IJavaPeerable.PeerReference was set *before* the
constructor was invoked. (Setting IJavaPeerable.PeerReference before
the constructor executes is not yet done in Java.Interop.)
It means that a future "proper" constructor invocation is assumed to
be forthcoming, as during Java activation, IF the Java constructor
virtually invokes a method which is overridden in managed code, a
managed peer will need to be constructed so that the method override
can be invoked, and "later" the "real" constructor will be invoked.

	https://developer.xamarin.com/guides/android/under_the_hood/architecture/#Java_Activation

 1. `new Peer(...)` is invoked from Java.

 2. A super class constructor of NativePeer virtually invokes a method
    overridden by Peer in managed code.

 3. The Marshal Method is executed, which needs to delegate the method
    to *something*, and thus creates a new managed Peer instance
    through the activation constructor.
    This created managed peer instance will have the
    JniManagedPeerStates.Activatable state set.

 4. The managed override is executed and returns back to Java.

 5. Once all super constructors have finished, the Peer constructor
    executes com.xamarin.java_interop.ManagedPeer.runConstructor(),

 6. ManagedPeer.runConstructor() invokes the appropriate corresponding
    constructor on the instance created in (3). If the
    JniManagedPeerStates.Activatable state *isn't* set, then the
    ManagedPeer.runConstructor() call would be *ignored*.
    This also means that *two constructors* will be invoked on *one
    instrance*.

The JniManagedPeerStates.Activatable state needs to be set to sanely
prevent invoking constructors more than is intended on a given
peer instance.

JniManagedPeerStates.Replaceable is IJavaObjectEx.IsProxy, and means
that the Peer instance was created through the activation constructor.
It additionally means that if two managed instances are created around
the same Java instance, the non-Replaceable instance will be the one
returned by JniRuntime.JniValueManager.PeekObject().

Normally, JniManagedPeerStates.Replaceable shouldn't be needed, but
there is one environment where it is:

Android devices which are pre-Honeycomb (API-11). On those devices,
JNIEnv.AllocObject() cannot be used (8c4248b), so something very
similar but not quite like the above Activation case can happen when
constructing instances *from managed code*.

In "normal" use -- JNIEnv::AllocObject() works! -- managed
construction is:

 1. `new Peer()` invokes managed constructor.

 2. Managed constructor calls
    JniPeerMembers.InstanceMethods.StartCreateInstance(), which uses
    JNIEnv::AllocObject() to create a Java instance *without executing
    the Java instance constructor*.

 3. JniRuntime.JniValueManager.Construct() adds the Java instance from
    (2) to the mapping table.

 4. The constructor calls
    JniPeerMembers.InstanceMethods.FinishCreateInstance(), which
    invokes the Java instance constructor, and if a Java instance
    constructor virtually invokes a method overridden in managed code,
    the marshal method will find the instance created in (1).

When JNIEnv::AllocObject() doesn't work, the above falls down:

 1. `new Peer()` invokes managed constructor.

 2. Managed constructor calls
    JniPeerMembers.InstanceMethods.StartCreateInstance(), which uses
    JNIEnv::NewObject() to create a Java instance *and executes
    the Java instance constructor*.

 3. If a Java constructor virtually invokes a method overridden in
    managed code, the marshal method will be invoked and won't find an
    already-existing peer for the Java instance, and thus will create
    one via JniRuntime.JniValueManager.CreatePeer().

 4. JniRuntime.JniValueManager.CreatePeer() will set the
    JniManagedPeerStates.Replaceable state on the instance created in
    (3).

    Note: at this point there are *two* managed peers which want to
    "own" the Java instance: the instance created in (1) which is
    still being constructed (!), and the instance created in (3).

 5. The overriding managed method returns control to the Java
    constructor, which eventually completes execution and returns
    execution to the Peer managed constructor in (1).

 6. The Peer constructor invokes
    JniRuntime.JniValueManager.Construct() to add the instance from
    (1) to the instance mapping table, a mapping which *already
    exists* because of (3).

There needs to be a way for (6) to replace the mapping created in (3)
with the Peer instance in (1), and JniManagedPeerStates.Replaceable is
how that is tracked.

(Aside: supporting platforms that have broken JNIEnv::AllocObject()
implementations is a GIANT PAIN IN THE ASS.)

To use JniManagedPeerStates, update IJavaPeerable to add the following
members:

  partial interface IJavaPeerable {
    JniManagedPeerStates     JniManagedPeerState {get;}
    void                     SetJniManagedPeerState (JniManagedPeerStates value);
  }

IJavaPeerable.JniManagedPeerState is the current state of the managed
peer. IJavaPeerable.SetJniManagedPeerState() allows updating the
current state of the managed peer, permitting the state to be tracked
cross activation constructor calls.

[0]: dotnet/java-interop#11
radical pushed a commit that referenced this pull request May 8, 2018
When `JniRuntime.CreationOptions.DestroyRuntimeOnDispose` is true,
`JavaVM::DestroyJavaVM()` will be invoked when the `JniRuntime`
instance is disposed *or* finalized.

`JreRuntime.CreateJreVM()` would *always* set
`DestroyRuntimeOnDispose` to true, because it called
`JNI_CreateJavaVM()`, so *of course* you'd want to destroy the
Java VM, right?

Which brings us to unit tests. I don't know of any "before all test
fixtures run" and "after all test fixtures run" extension points,
which means:

1. The JVM needs to be created implicitly, "on demand."
2. There's no good way to destroy the JVM created in (1) after all
    tests have finished executing.

Which *really* means that the `JreRuntime` instance is *finalized*,
which sets us up for the unholy trifecta of AppDomain unloads,
finalizers, and JVM shutdown:

For unknown reasons, ~randomly, when running the unit tests (e.g.
`make run-tests`), the test runner will *hang*, indefinitely.

Attaching `lldb` and triggering a backtrace shows the unholy trifecta:

Finalization:

	thread #4: tid = 0x403831, 0x00007fff9656bdb6 libsystem_kernel.dylib`__psynch_cvwait + 10, name = 'tid_1403'
	  ...
	  frame #10: 0x00000001001ccb4a mono64`mono_gc_run_finalize(obj=<unavailable>, data=<unavailable>) + 938 at gc.c:256 [opt]
	  frame #11: 0x00000001001cdd4a mono64`finalizer_thread [inlined] finalize_domain_objects + 51 at gc.c:681 [opt]
	  frame #12: 0x00000001001cdd17 mono64`finalizer_thread(unused=<unavailable>) + 295 at gc.c:730 [opt]

JVM destruction:

	thread #4: tid = 0x403831, 0x00007fff9656bdb6 libsystem_kernel.dylib`__psynch_cvwait + 10, name = 'tid_1403'
	  frame #0: 0x00007fff9656bdb6 libsystem_kernel.dylib`__psynch_cvwait + 10
	  frame #1: 0x00007fffa04d4728 libsystem_pthread.dylib`_pthread_cond_wait + 767
	  frame #2: 0x000000010ba5bc76 libjvm.dylib`os::PlatformEvent::park() + 192
	  frame #3: 0x000000010ba38e32 libjvm.dylib`ParkCommon(ParkEvent*, long) + 42
	  frame #4: 0x000000010ba39708 libjvm.dylib`Monitor::IWait(Thread*, long) + 168
	  frame #5: 0x000000010ba398f0 libjvm.dylib`Monitor::wait(bool, long, bool) + 246
	  frame #6: 0x000000010bb3dca2 libjvm.dylib`Threads::destroy_vm() + 80
	  frame #7: 0x000000010b8fd665 libjvm.dylib`jni_DestroyJavaVM + 254

AppDomain unload:

	thread #37: tid = 0x4038fb, 0x00007fff9656bdb6 libsystem_kernel.dylib`__psynch_cvwait + 10
	  frame #0: 0x00007fff9656bdb6 libsystem_kernel.dylib`__psynch_cvwait + 10
	  frame #1: 0x00007fffa04d4728 libsystem_pthread.dylib`_pthread_cond_wait + 767
	  frame #2: 0x0000000100234a7f mono64`mono_os_cond_timedwait [inlined] mono_os_cond_wait(cond=0x0000000102016e50, mutex=0x0000000102016e10) + 11 at mono-os-mutex.h:105 [opt]
	  frame #3: 0x0000000100234a74 mono64`mono_os_cond_timedwait(cond=0x0000000102016e50, mutex=0x0000000102016e10, timeout_ms=<unavailable>) + 164 at mono-os-mutex.h:120 [opt]
	  frame #4: 0x0000000100234828 mono64`_wapi_handle_timedwait_signal_handle(handle=0x0000000000000440, timeout=4294967295, alertable=1, poll=<unavailable>, alerted=0x0000700000a286f4) + 536 at handles.c:1554 [opt]
	  frame #5: 0x0000000100246370 mono64`wapi_WaitForSingleObjectEx(handle=<unavailable>, timeout=<unavailable>, alertable=<unavailable>) + 592 at wait.c:189 [opt]
	  frame #6: 0x00000001001c832e mono64`mono_domain_try_unload [inlined] guarded_wait(timeout=4294967295, alertable=1) + 30 at appdomain.c:2390 [opt]
	  frame #7: 0x00000001001c8310 mono64`mono_domain_try_unload(domain=0x000000010127ccb0, exc=0x0000700000a287a0) + 416 at appdomain.c:2482 [opt]
	  frame #8: 0x00000001001c7db2 mono64`ves_icall_System_AppDomain_InternalUnload [inlined] mono_domain_unload(domain=<unavailable>) + 20 at appdomain.c:2379 [opt]
	  frame #9: 0x00000001001c7d9e mono64`ves_icall_System_AppDomain_InternalUnload(domain_id=<unavailable>) + 46 at appdomain.c:2039 [opt]

This randomly results in deadlock, and hung Jenkins bots.

Fix this behavior by altering `JreRuntime.CreateJreVM()` to *not*
override the value of
`JniRuntime.CreationOptions.DestroyRuntimeOnDispose`. This allows the
constructor of the `JreRuntime` instance to decide whether or not the
JVM is destroyed.

In the case of TestJVM, we *don't* want to destroy the JVM.

This prevents the JVM from being destroyed, which in turn prevents the
hang during process shutdown.
grendello added a commit to grendello/xamarin-android that referenced this pull request Jul 18, 2018
Xamarin.Android uses the ILRepack
task (https://www.nuget.org/packages/ILRepack.Lib.MSBuild.Task/) to repack/merge
the `Xamarin.Android.Build.Tasks` assembly, a step that is performed after the
`Xamarin.Android.Build.Tasks` project build completes. The step takes the
project's output assembly and several other dependencies and merges them into
one, replacing the original output assembly. For the majority of time it works
flawlessly, however the scenario described below causes a Mono runtime
crash (similar to this one mono/mono#9613, although
not identical).

The crash causes aren't yet know in every detail, but the high level overview of
the issue is as follows. Mono runtime uses the `mmap(2)` system call on Unix to
map the on-disk image of the assembly into memory. The mmapped area is used
whenever the same assembly is loaded (e.g. via `Assembly.LoadFrom`) into the
application to save time and memory. The memory is mapped as private which means
that anything written to the mapped area by the Mono process is effectively
discarded, that is not reflected in the file on disk. The protection, however,
doesn't work in the other direction - when an external process (or even the same
Mono process) writes to the file that was mmapped. Unfortunately, the OS kernel
behavior in such instance is left undefined by the POSIX standard - the kernel
can choose to reflect the on-disk change in the mapping or ignore it. On macOS
the write is ignored, on Linux, however, the write leads either to the SIGBUS
signal being sent to the process or corruption to the mmapped memory
area (details of how this happens are, as of yet, unknown - it's just a theory
at this point). The problem occurs in our case when the `ILRepacker` task
overwrites the `Xamarin.Android.Build.Tasks.dll` assembly with its
merged/repackaged form and that, somehow, corrupts Mono runtime's memory mapping
of that assembly, eventually leading to a segfault:

  #0  0x00007f974fd1b23a in __waitpid (pid=pid@entry=120561, stat_loc=stat_loc@entry=0x7f9712162b9c, options=options@entry=0) at ../sysdeps/unix/sysv/linux/waitpid.c:30
  #1  0x000055b471f36c96 in dump_native_stacktrace (ctx=0x7f97121637c0, signal=0x55b472147cd3 "SIGSEGV") at mini-posix.c:719
  #2  0x000055b471f36df2 in mono_dump_native_crash_info (signal=signal@entry=0x55b472147cd3 "SIGSEGV", ctx=ctx@entry=0x7f97121637c0, info=info@entry=0x7f97121638f0) at mini-posix.c:742
  #3  0x000055b471ecc620 in mono_handle_native_crash (signal=signal@entry=0x55b472147cd3 "SIGSEGV", ctx=ctx@entry=0x7f97121637c0, info=info@entry=0x7f97121638f0) at mini-exceptions.c:2938
  dotnet#4  0x000055b471e4c82c in mono_sigsegv_signal_handler (_dummy=7, _info=0x7f97121638f0, context=0x7f97121637c0) at mini-runtime.c:3441
  dotnet#5  <signal handler called>
  dotnet#6  __strcasecmp_l_avx () at ../sysdeps/x86_64/multiarch/strcmp-sse42.S:199
  dotnet#7  0x000055b471fb2105 in mono_assembly_names_equal_flags (l=l@entry=0x7f96e0004370, r=r@entry=0x7f97307edb60, flags=flags@entry=MONO_ANAME_EQ_IGNORE_CASE) at assembly.c:663
  dotnet#8  0x000055b471fa7baf in mono_domain_assembly_search (aname=0x7f96e0004370, user_data=0x0) at appdomain.c:2242
  dotnet#9  0x000055b471fb10f7 in mono_assembly_invoke_search_hook_internal (aname=aname@entry=0x7f96e0004370, requesting=requesting@entry=0x0, refonly=0, postload=postload@entry=0) at assembly.c:1755
  dotnet#10 0x000055b471fb4526 in mono_assembly_load_from_predicate (image=image@entry=0x7f96e001aae0, fname=fname@entry=0x7f96e000bc50 "/home/grendel/vc/xamarin/monodroid/monodroid/external/xamarin-android/src/Xamarin.Android.Build.Tasks/../../packages/ILRepack.Lib.MSBuild.Task.2.0.15.4/build/ILRepack.Lib.MSBuild.Task.dll", asmctx=asmctx@entry=MONO_ASMCTX_LOADFROM, predicate=predicate@entry=0x0, user_data=user_data@entry=0x0, status=status@entry=0x7f9712163fd4) at assembly.c:2638
  dotnet#11 0x000055b471fb635c in mono_assembly_open_predicate (filename=<optimized out>, filename@entry=0x7f96e000ae40 "/home/grendel/vc/xamarin/monodroid/monodroid/external/xamarin-android/src/Xamarin.Android.Build.Tasks/../../packages/ILRepack.Lib.MSBuild.Task.2.0.15.4/build/ILRepack.Lib.MSBuild.Task.dll", asmctx=MONO_ASMCTX_LOADFROM, predicate=predicate@entry=0x0, user_data=user_data@entry=0x0, status=status@entry=0x7f9712163fd4) at assembly.c:2206
  dotnet#12 0x000055b471fabd42 in ves_icall_System_Reflection_Assembly_LoadFrom (fname=..., refOnly=<optimized out>, error=0x7f9712164050) at appdomain.c:2272
  dotnet#13 0x00000000412f4007 in ?? ()
  dotnet#14 0x00007f97440e1918 in ?? ()
  dotnet#15 0x00007f9749bd1800 in ?? ()
  dotnet#16 0x00000000e36e75af in ?? ()
  dotnet#17 0x00007f974434fda8 in ?? ()
  dotnet#18 0x00007f9749bd18e0 in ?? ()
  dotnet#19 0x00007f96e0002820 in ?? ()
  dotnet#20 0x00007f9712164160 in ?? ()
  dotnet#21 0x00007f9712164010 in ?? ()
  dotnet#22 0x0000000000000000 in ?? ()

The crash occurs in frame 7, because the `r` parameter (an instance of the
`MonoAssembly` structure) contains pointers that used to point to valid places
in the `Xamarin.Android.Build.Tasks` assembbly's mmaped area, but now point to
garbage data. One of those corrupted pointers is the ASCII assembly name - which
is used by the function in frame 7.

The situation can be detected by the runtime by using advisory locks and
reporting the attempt to overwrite the mmapped area, however a fix would require
creating a copy of all the assembly's in-memory data before the write is allowed
and that might be too expensive. None of those measures are currently
implemented by Mono (and it isn't sure if any of them will ever be implemented).

The crash this commit fixes occurs in a very specific scenario - when building
the `OpenTK.csproj` project *directly* (that is passing it as parameter to
`msbuild`), which causes a the `Xamarin.Android.Build.Tasks` assembly to be
loaded for use in the `<UsingTask/>` msbuild statement, but at the same time
`OpenTK.csproj` references the `Xamarin.Android.Build.Tasks` *project* which
causes the ILRepacker task to run unconditionally, attempting to repack the
assemblies again and leading to the crash described above.

The fix, courtesy of @jonpryor, is to make the ILRepacker target not run every
time but only whenver the assembly has to be re-packed because it has just been
rebuilt.
grendello added a commit to grendello/xamarin-android that referenced this pull request Jul 19, 2018
Xamarin.Android uses the ILRepack
task (https://www.nuget.org/packages/ILRepack.Lib.MSBuild.Task/) to repack/merge
the `Xamarin.Android.Build.Tasks` assembly, a step that is performed after the
`Xamarin.Android.Build.Tasks` project build completes. The step takes the
project's output assembly and several other dependencies and merges them into
one, replacing the original output assembly. For the majority of time it works
flawlessly, however the scenario described below causes a Mono runtime
crash (similar to this one mono/mono#9613, although
not identical).

The crash causes aren't yet know in every detail, but the high level overview of
the issue is as follows. Mono runtime uses the `mmap(2)` system call on Unix to
map the on-disk image of the assembly into memory. The mmapped area is used
whenever the same assembly is loaded (e.g. via `Assembly.LoadFrom`) into the
application to save time and memory. The memory is mapped as private which means
that anything written to the mapped area by the Mono process is effectively
discarded, that is not reflected in the file on disk. The protection, however,
doesn't work in the other direction - when an external process (or even the same
Mono process) writes to the file that was mmapped. Unfortunately, the OS kernel
behavior in such instance is left undefined by the POSIX standard - the kernel
can choose to reflect the on-disk change in the mapping or ignore it. On macOS
the write is ignored, on Linux, however, the write leads either to the SIGBUS
signal being sent to the process or corruption to the mmapped memory
area (details of how this happens are, as of yet, unknown - it's just a theory
at this point). The problem occurs in our case when the `ILRepacker` task
overwrites the `Xamarin.Android.Build.Tasks.dll` assembly with its
merged/repackaged form and that, somehow, corrupts Mono runtime's memory mapping
of that assembly, eventually leading to a segfault:

  #0  0x00007f974fd1b23a in __waitpid (pid=pid@entry=120561, stat_loc=stat_loc@entry=0x7f9712162b9c, options=options@entry=0) at ../sysdeps/unix/sysv/linux/waitpid.c:30
  #1  0x000055b471f36c96 in dump_native_stacktrace (ctx=0x7f97121637c0, signal=0x55b472147cd3 "SIGSEGV") at mini-posix.c:719
  #2  0x000055b471f36df2 in mono_dump_native_crash_info (signal=signal@entry=0x55b472147cd3 "SIGSEGV", ctx=ctx@entry=0x7f97121637c0, info=info@entry=0x7f97121638f0) at mini-posix.c:742
  #3  0x000055b471ecc620 in mono_handle_native_crash (signal=signal@entry=0x55b472147cd3 "SIGSEGV", ctx=ctx@entry=0x7f97121637c0, info=info@entry=0x7f97121638f0) at mini-exceptions.c:2938
  dotnet#4  0x000055b471e4c82c in mono_sigsegv_signal_handler (_dummy=7, _info=0x7f97121638f0, context=0x7f97121637c0) at mini-runtime.c:3441
  dotnet#5  <signal handler called>
  dotnet#6  __strcasecmp_l_avx () at ../sysdeps/x86_64/multiarch/strcmp-sse42.S:199
  dotnet#7  0x000055b471fb2105 in mono_assembly_names_equal_flags (l=l@entry=0x7f96e0004370, r=r@entry=0x7f97307edb60, flags=flags@entry=MONO_ANAME_EQ_IGNORE_CASE) at assembly.c:663
  dotnet#8  0x000055b471fa7baf in mono_domain_assembly_search (aname=0x7f96e0004370, user_data=0x0) at appdomain.c:2242
  dotnet#9  0x000055b471fb10f7 in mono_assembly_invoke_search_hook_internal (aname=aname@entry=0x7f96e0004370, requesting=requesting@entry=0x0, refonly=0, postload=postload@entry=0) at assembly.c:1755
  dotnet#10 0x000055b471fb4526 in mono_assembly_load_from_predicate (image=image@entry=0x7f96e001aae0, fname=fname@entry=0x7f96e000bc50 "/home/grendel/vc/xamarin/monodroid/monodroid/external/xamarin-android/src/Xamarin.Android.Build.Tasks/../../packages/ILRepack.Lib.MSBuild.Task.2.0.15.4/build/ILRepack.Lib.MSBuild.Task.dll", asmctx=asmctx@entry=MONO_ASMCTX_LOADFROM, predicate=predicate@entry=0x0, user_data=user_data@entry=0x0, status=status@entry=0x7f9712163fd4) at assembly.c:2638
  dotnet#11 0x000055b471fb635c in mono_assembly_open_predicate (filename=<optimized out>, filename@entry=0x7f96e000ae40 "/home/grendel/vc/xamarin/monodroid/monodroid/external/xamarin-android/src/Xamarin.Android.Build.Tasks/../../packages/ILRepack.Lib.MSBuild.Task.2.0.15.4/build/ILRepack.Lib.MSBuild.Task.dll", asmctx=MONO_ASMCTX_LOADFROM, predicate=predicate@entry=0x0, user_data=user_data@entry=0x0, status=status@entry=0x7f9712163fd4) at assembly.c:2206
  dotnet#12 0x000055b471fabd42 in ves_icall_System_Reflection_Assembly_LoadFrom (fname=..., refOnly=<optimized out>, error=0x7f9712164050) at appdomain.c:2272
  dotnet#13 0x00000000412f4007 in ?? ()
  dotnet#14 0x00007f97440e1918 in ?? ()
  dotnet#15 0x00007f9749bd1800 in ?? ()
  dotnet#16 0x00000000e36e75af in ?? ()
  dotnet#17 0x00007f974434fda8 in ?? ()
  dotnet#18 0x00007f9749bd18e0 in ?? ()
  dotnet#19 0x00007f96e0002820 in ?? ()
  dotnet#20 0x00007f9712164160 in ?? ()
  dotnet#21 0x00007f9712164010 in ?? ()
  dotnet#22 0x0000000000000000 in ?? ()

The accompanying managed stacktrace looks as follows:

  Stacktrace:

  at <unknown> <0xffffffff>
  at (wrapper managed-to-native) System.Reflection.Assembly.LoadFrom (string,bool) [0x0001b] in <79002cfb0e6e431d8465d6dcea08995e>:0
  at System.Reflection.Assembly.LoadFrom (string) [0x00000] in /home/grendel/vc/mono/mono/mcs/class/corlib/System.Reflection/Assembly.cs:539
  at System.Reflection.Assembly.UnsafeLoadFrom (string) [0x00000] in /home/grendel/vc/mono/mono/mcs/class/corlib/System.Reflection/Assembly.cs:571
  at Microsoft.Build.Shared.TypeLoader.LoadAssembly (Microsoft.Build.Shared.AssemblyLoadInfo) [0x0001e] in <bdc5207a22bb42ae9fe3f3d07e82871e>:0
  at Microsoft.Build.Shared.TypeLoader/AssemblyInfoToLoadedTypes.ScanAssemblyForPublicTypes () [0x00007] in <bdc5207a22bb42ae9fe3f3d07e82871e>:0
  at Microsoft.Build.Shared.TypeLoader/AssemblyInfoToLoadedTypes/<>c__DisplayClass8_0.<GetLoadedTypeByTypeName>b__0 (string) [0x000b4] in <bdc5207a22bb42ae9fe3f3d07e82871e>:0
  at System.Collections.Concurrent.ConcurrentDictionary`2<TKey_REF, TValue_REF>.GetOrAdd (TKey_REF,System.Func`2<TKey_REF, TValue_REF>) [0x00034] in <79002cfb0e6e431d8465d6dcea08995e>:0
  at Microsoft.Build.Shared.TypeLoader/AssemblyInfoToLoadedTypes.GetLoadedTypeByTypeName (string) [0x0003c] in <bdc5207a22bb42ae9fe3f3d07e82871e>:0
  at Microsoft.Build.Shared.TypeLoader.GetLoadedType (System.Collections.Concurrent.ConcurrentDictionary`2<System.Func`3<System.Type, object, bool>, System.Collections.Concurrent.ConcurrentDictionary`2<Microsoft.Build.Shared.AssemblyLoadInfo, Microsoft.Build.Shared.TypeLoader/AssemblyInfoToLoadedTypes>>,string,Microsoft.Build.Shared.AssemblyLoadInfo) [0x00042] in <bdc5207a22bb42ae9fe3f3d07e82871e>:0
  at Microsoft.Build.Shared.TypeLoader.Load (string,Microsoft.Build.Shared.AssemblyLoadInfo) [0x00008] in <bdc5207a22bb42ae9fe3f3d07e82871e>:0
  at Microsoft.Build.BackEnd.AssemblyTaskFactory.InitializeFactory (Microsoft.Build.Shared.AssemblyLoadInfo,string,System.Collections.Generic.IDictionary`2<string, Microsoft.Build.Framework.TaskPropertyInfo>,string,System.Collections.Generic.IDictionary`2<string, string>,bool,Microsoft.Build.BackEnd.Logging.TargetLoggingContext,Microsoft.Build.Construction.ElementLocation,string) [0x00058] in <bdc5207a22bb42ae9fe3f3d07e82871e>:0
  at Microsoft.Build.Execution.TaskRegistry/RegisteredTaskRecord.GetTaskFactory (Microsoft.Build.BackEnd.Logging.TargetLoggingContext,Microsoft.Build.Construction.ElementLocation,string) [0x000b8] in <bdc5207a22bb42ae9fe3f3d07e82871e>:0
  at Microsoft.Build.Execution.TaskRegistry/RegisteredTaskRecord.CanTaskBeCreatedByFactory (string,string,System.Collections.Generic.IDictionary`2<string, string>,Microsoft.Build.BackEnd.Logging.TargetLoggingContext,Microsoft.Build.Construction.ElementLocation) [0x0003b] in <bdc5207a22bb42ae9fe3f3d07e82871e>:0
  at Microsoft.Build.Execution.TaskRegistry.GetMatchingRegistration (string,System.Collections.Generic.List`1<Microsoft.Build.Execution.TaskRegistry/RegisteredTaskRecord>,string,System.Collections.Generic.IDictionary`2<string, string>,Microsoft.Build.BackEnd.Logging.TargetLoggingContext,Microsoft.Build.Construction.ElementLocation) [0x0001a] in <bdc5207a22bb42ae9fe3f3d07e82871e>:0
  at Microsoft.Build.Execution.TaskRegistry.GetTaskRegistrationRecord (string,string,System.Collections.Generic.IDictionary`2<string, string>,bool,Microsoft.Build.BackEnd.Logging.TargetLoggingContext,Microsoft.Build.Construction.ElementLocation,bool&) [0x00169] in <bdc5207a22bb42ae9fe3f3d07e82871e>:0
  at Microsoft.Build.Execution.TaskRegistry.GetRegisteredTask (string,string,System.Collections.Generic.IDictionary`2<string, string>,bool,Microsoft.Build.BackEnd.Logging.TargetLoggingContext,Microsoft.Build.Construction.ElementLocation) [0x00010] in <bdc5207a22bb42ae9fe3f3d07e82871e>:0
  at Microsoft.Build.BackEnd.TaskExecutionHost.FindTaskInRegistry (System.Collections.Generic.IDictionary`2<string, string>) [0x00038] in <bdc5207a22bb42ae9fe3f3d07e82871e>:0
  at Microsoft.Build.BackEnd.TaskExecutionHost.Microsoft.Build.BackEnd.ITaskExecutionHost.FindTask (System.Collections.Generic.IDictionary`2<string, string>) [0x0000b] in <bdc5207a22bb42ae9fe3f3d07e82871e>:0
  at Microsoft.Build.BackEnd.TaskBuilder/<ExecuteBucket>d__19.MoveNext () [0x0011e] in <bdc5207a22bb42ae9fe3f3d07e82871e>:0
  at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1<Microsoft.Build.BackEnd.WorkUnitResult>.Start<Microsoft.Build.BackEnd.TaskBuilder/<ExecuteBucket>d__19> (Microsoft.Build.BackEnd.TaskBuilder/<ExecuteBucket>d__19&) [0x0002c] in /home/grendel/vc/mono/mono/mcs/class/referencesource/mscorlib/system/runtime/compilerservices/AsyncMethodBuilder.cs:471
  at Microsoft.Build.BackEnd.TaskBuilder.ExecuteBucket (Microsoft.Build.BackEnd.TaskHost,Microsoft.Build.BackEnd.ItemBucket,Microsoft.Build.BackEnd.TaskExecutionMode,System.Collections.Generic.Dictionary`2<string, string>) [0x00048] in <bdc5207a22bb42ae9fe3f3d07e82871e>:0
  at Microsoft.Build.BackEnd.TaskBuilder/<ExecuteTask>d__18.MoveNext () [0x0016a] in <bdc5207a22bb42ae9fe3f3d07e82871e>:0
  at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1<Microsoft.Build.BackEnd.WorkUnitResult>.Start<Microsoft.Build.BackEnd.TaskBuilder/<ExecuteTask>d__18> (Microsoft.Build.BackEnd.TaskBuilder/<ExecuteTask>d__18&) [0x0002c] in /home/grendel/vc/mono/mono/mcs/class/referencesource/mscorlib/system/runtime/compilerservices/AsyncMethodBuilder.cs:471
  at Microsoft.Build.BackEnd.TaskBuilder.ExecuteTask (Microsoft.Build.BackEnd.TaskExecutionMode,Microsoft.Build.BackEnd.Lookup) [0x00037] in <bdc5207a22bb42ae9fe3f3d07e82871e>:0
  at Microsoft.Build.BackEnd.TaskBuilder/<ExecuteTask>d__13.MoveNext () [0x00193] in <bdc5207a22bb42ae9fe3f3d07e82871e>:0
  at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1<Microsoft.Build.BackEnd.WorkUnitResult>.Start<Microsoft.Build.BackEnd.TaskBuilder/<ExecuteTask>d__13> (Microsoft.Build.BackEnd.TaskBuilder/<ExecuteTask>d__13&) [0x0002c] in /home/grendel/vc/mono/mono/mcs/class/referencesource/mscorlib/system/runtime/compilerservices/AsyncMethodBuilder.cs:471
  at Microsoft.Build.BackEnd.TaskBuilder.ExecuteTask (Microsoft.Build.BackEnd.Logging.TargetLoggingContext,Microsoft.Build.BackEnd.BuildRequestEntry,Microsoft.Build.BackEnd.ITargetBuilderCallback,Microsoft.Build.Execution.ProjectTargetInstanceChild,Microsoft.Build.BackEnd.TaskExecutionMode,Microsoft.Build.BackEnd.Lookup,Microsoft.Build.BackEnd.Lookup,System.Threading.CancellationToken) [0x0006c] in <bdc5207a22bb42ae9fe3f3d07e82871e>:0
  at Microsoft.Build.BackEnd.TargetEntry/<ProcessBucket>d__52.MoveNext () [0x00091] in <bdc5207a22bb42ae9fe3f3d07e82871e>:0
  at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1<Microsoft.Build.BackEnd.WorkUnitResult>.Start<Microsoft.Build.BackEnd.TargetEntry/<ProcessBucket>d__52> (Microsoft.Build.BackEnd.TargetEntry/<ProcessBucket>d__52&) [0x0002c] in /home/grendel/vc/mono/mono/mcs/class/referencesource/mscorlib/system/runtime/compilerservices/AsyncMethodBuilder.cs:471
  at Microsoft.Build.BackEnd.TargetEntry.ProcessBucket (Microsoft.Build.BackEnd.ITaskBuilder,Microsoft.Build.BackEnd.Logging.TargetLoggingContext,Microsoft.Build.BackEnd.TaskExecutionMode,Microsoft.Build.BackEnd.Lookup,Microsoft.Build.BackEnd.Lookup) [0x00051] in <bdc5207a22bb42ae9fe3f3d07e82871e>:0
  at Microsoft.Build.BackEnd.TargetEntry/<ExecuteTarget>d__45.MoveNext () [0x002bb] in <bdc5207a22bb42ae9fe3f3d07e82871e>:0
  at System.Runtime.CompilerServices.AsyncTaskMethodBuilder.Start<Microsoft.Build.BackEnd.TargetEntry/<ExecuteTarget>d__45> (Microsoft.Build.BackEnd.TargetEntry/<ExecuteTarget>d__45&) [0x0002c] in /home/grendel/vc/mono/mono/mcs/class/referencesource/mscorlib/system/runtime/compilerservices/AsyncMethodBuilder.cs:316
  at Microsoft.Build.BackEnd.TargetEntry.ExecuteTarget (Microsoft.Build.BackEnd.ITaskBuilder,Microsoft.Build.BackEnd.BuildRequestEntry,Microsoft.Build.BackEnd.Logging.ProjectLoggingContext,System.Threading.CancellationToken) [0x00048] in <bdc5207a22bb42ae9fe3f3d07e82871e>:0
  at Microsoft.Build.BackEnd.TargetBuilder/<ProcessTargetStack>d__21.MoveNext () [0x003a5] in <bdc5207a22bb42ae9fe3f3d07e82871e>:0
  at System.Runtime.CompilerServices.AsyncMethodBuilderCore/MoveNextRunner.InvokeMoveNext (object) [0x00000] in /home/grendel/vc/mono/mono/mcs/class/referencesource/mscorlib/system/runtime/compilerservices/AsyncMethodBuilder.cs:1094
  at System.Threading.ExecutionContext.RunInternal (System.Threading.ExecutionContext,System.Threading.ContextCallback,object,bool) [0x00071] in /home/grendel/vc/mono/mono/mcs/class/referencesource/mscorlib/system/threading/executioncontext.cs:961
  at System.Threading.ExecutionContext.Run (System.Threading.ExecutionContext,System.Threading.ContextCallback,object,bool) [0x00000] in /home/grendel/vc/mono/mono/mcs/class/referencesource/mscorlib/system/threading/executioncontext.cs:908
  at System.Runtime.CompilerServices.AsyncMethodBuilderCore/MoveNextRunner.Run () [0x00024] in /home/grendel/vc/mono/mono/mcs/class/referencesource/mscorlib/system/runtime/compilerservices/AsyncMethodBuilder.cs:1075
  at System.Threading.Tasks.AwaitTaskContinuation.RunOrScheduleAction (System.Action,bool,System.Threading.Tasks.Task&) [0x0001a] in /home/grendel/vc/mono/mono/external/corert/src/System.Private.CoreLib/src/System/Threading/Tasks/TaskContinuation.cs:680
  at System.Threading.Tasks.Task.FinishContinuations () [0x00052] in /home/grendel/vc/mono/mono/external/corert/src/System.Private.CoreLib/src/System/Threading/Tasks/Task.cs:3198
  at System.Threading.Tasks.Task.FinishStageThree () [0x0003c] in /home/grendel/vc/mono/mono/external/corert/src/System.Private.CoreLib/src/System/Threading/Tasks/Task.cs:2247
  at System.Threading.Tasks.Task`1<System.Threading.Tasks.VoidTaskResult>.TrySetResult (System.Threading.Tasks.VoidTaskResult) [0x0004f] in /home/grendel/vc/mono/mono/external/corert/src/System.Private.CoreLib/src/System/Threading/Tasks/Future.cs:433
  at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1<System.Threading.Tasks.VoidTaskResult>.SetResult (System.Threading.Tasks.VoidTaskResult) [0x0003d] in /home/grendel/vc/mono/mono/mcs/class/referencesource/mscorlib/system/runtime/compilerservices/AsyncMethodBuilder.cs:608
  at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1<System.Threading.Tasks.VoidTaskResult>.SetResult (System.Threading.Tasks.Task`1<System.Threading.Tasks.VoidTaskResult>) [0x00010] in /home/grendel/vc/mono/mono/mcs/class/referencesource/mscorlib/system/runtime/compilerservices/AsyncMethodBuilder.cs:636
  at System.Runtime.CompilerServices.AsyncTaskMethodBuilder.SetResult () [0x00000] in /home/grendel/vc/mono/mono/mcs/class/referencesource/mscorlib/system/runtime/compilerservices/AsyncMethodBuilder.cs:378
  at Microsoft.Build.BackEnd.TargetEntry/<ExecuteTarget>d__45.MoveNext () [0x00720] in <bdc5207a22bb42ae9fe3f3d07e82871e>:0
  at System.Runtime.CompilerServices.AsyncMethodBuilderCore/MoveNextRunner.InvokeMoveNext (object) [0x00000] in /home/grendel/vc/mono/mono/mcs/class/referencesource/mscorlib/system/runtime/compilerservices/AsyncMethodBuilder.cs:1094
  at System.Threading.ExecutionContext.RunInternal (System.Threading.ExecutionContext,System.Threading.ContextCallback,object,bool) [0x00071] in /home/grendel/vc/mono/mono/mcs/class/referencesource/mscorlib/system/threading/executioncontext.cs:961
  at System.Threading.ExecutionContext.Run (System.Threading.ExecutionContext,System.Threading.ContextCallback,object,bool) [0x00000] in /home/grendel/vc/mono/mono/mcs/class/referencesource/mscorlib/system/threading/executioncontext.cs:908
  at System.Runtime.CompilerServices.AsyncMethodBuilderCore/MoveNextRunner.Run () [0x00024] in /home/grendel/vc/mono/mono/mcs/class/referencesource/mscorlib/system/runtime/compilerservices/AsyncMethodBuilder.cs:1075
  at System.Threading.Tasks.AwaitTaskContinuation.RunOrScheduleAction (System.Action,bool,System.Threading.Tasks.Task&) [0x0001a] in /home/grendel/vc/mono/mono/external/corert/src/System.Private.CoreLib/src/System/Threading/Tasks/TaskContinuation.cs:680
  at System.Threading.Tasks.Task.FinishContinuations () [0x00052] in /home/grendel/vc/mono/mono/external/corert/src/System.Private.CoreLib/src/System/Threading/Tasks/Task.cs:3198
  at System.Threading.Tasks.Task.FinishStageThree () [0x0003c] in /home/grendel/vc/mono/mono/external/corert/src/System.Private.CoreLib/src/System/Threading/Tasks/Task.cs:2247
  at System.Threading.Tasks.Task`1<TResult_REF>.TrySetResult (TResult_REF) [0x0004f] in <79002cfb0e6e431d8465d6dcea08995e>:0
  at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1<TResult_REF>.SetResult (TResult_REF) [0x0003d] in <79002cfb0e6e431d8465d6dcea08995e>:0
  at Microsoft.Build.BackEnd.TargetEntry/<ProcessBucket>d__52.MoveNext () [0x001d5] in <bdc5207a22bb42ae9fe3f3d07e82871e>:0
  at System.Runtime.CompilerServices.AsyncMethodBuilderCore/MoveNextRunner.InvokeMoveNext (object) [0x00000] in /home/grendel/vc/mono/mono/mcs/class/referencesource/mscorlib/system/runtime/compilerservices/AsyncMethodBuilder.cs:1094
  at System.Threading.ExecutionContext.RunInternal (System.Threading.ExecutionContext,System.Threading.ContextCallback,object,bool) [0x00071] in /home/grendel/vc/mono/mono/mcs/class/referencesource/mscorlib/system/threading/executioncontext.cs:961
  at System.Threading.ExecutionContext.Run (System.Threading.ExecutionContext,System.Threading.ContextCallback,object,bool) [0x00000] in /home/grendel/vc/mono/mono/mcs/class/referencesource/mscorlib/system/threading/executioncontext.cs:908
  at System.Runtime.CompilerServices.AsyncMethodBuilderCore/MoveNextRunner.Run () [0x00024] in /home/grendel/vc/mono/mono/mcs/class/referencesource/mscorlib/system/runtime/compilerservices/AsyncMethodBuilder.cs:1075
  at System.Threading.Tasks.AwaitTaskContinuation.RunOrScheduleAction (System.Action,bool,System.Threading.Tasks.Task&) [0x0001a] in /home/grendel/vc/mono/mono/external/corert/src/System.Private.CoreLib/src/System/Threading/Tasks/TaskContinuation.cs:680
  at System.Threading.Tasks.Task.FinishContinuations () [0x00052] in /home/grendel/vc/mono/mono/external/corert/src/System.Private.CoreLib/src/System/Threading/Tasks/Task.cs:3198
  at System.Threading.Tasks.Task.FinishStageThree () [0x0003c] in /home/grendel/vc/mono/mono/external/corert/src/System.Private.CoreLib/src/System/Threading/Tasks/Task.cs:2247
  at System.Threading.Tasks.Task`1<TResult_REF>.TrySetResult (TResult_REF) [0x0004f] in <79002cfb0e6e431d8465d6dcea08995e>:0
  at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1<TResult_REF>.SetResult (TResult_REF) [0x0003d] in <79002cfb0e6e431d8465d6dcea08995e>:0
  at Microsoft.Build.BackEnd.TaskBuilder/<ExecuteTask>d__13.MoveNext () [0x0021e] in <bdc5207a22bb42ae9fe3f3d07e82871e>:0
  at System.Runtime.CompilerServices.AsyncMethodBuilderCore/MoveNextRunner.InvokeMoveNext (object) [0x00000] in /home/grendel/vc/mono/mono/mcs/class/referencesource/mscorlib/system/runtime/compilerservices/AsyncMethodBuilder.cs:1094
  at System.Threading.ExecutionContext.RunInternal (System.Threading.ExecutionContext,System.Threading.ContextCallback,object,bool) [0x00071] in /home/grendel/vc/mono/mono/mcs/class/referencesource/mscorlib/system/threading/executioncontext.cs:961
  at System.Threading.ExecutionContext.Run (System.Threading.ExecutionContext,System.Threading.ContextCallback,object,bool) [0x00000] in /home/grendel/vc/mono/mono/mcs/class/referencesource/mscorlib/system/threading/executioncontext.cs:908
  at System.Runtime.CompilerServices.AsyncMethodBuilderCore/MoveNextRunner.Run () [0x00024] in /home/grendel/vc/mono/mono/mcs/class/referencesource/mscorlib/system/runtime/compilerservices/AsyncMethodBuilder.cs:1075
  at System.Threading.Tasks.AwaitTaskContinuation.RunOrScheduleAction (System.Action,bool,System.Threading.Tasks.Task&) [0x0001a] in /home/grendel/vc/mono/mono/external/corert/src/System.Private.CoreLib/src/System/Threading/Tasks/TaskContinuation.cs:680
  at System.Threading.Tasks.Task.FinishContinuations () [0x00052] in /home/grendel/vc/mono/mono/external/corert/src/System.Private.CoreLib/src/System/Threading/Tasks/Task.cs:3198
  at System.Threading.Tasks.Task.FinishStageThree () [0x0003c] in /home/grendel/vc/mono/mono/external/corert/src/System.Private.CoreLib/src/System/Threading/Tasks/Task.cs:2247
  at System.Threading.Tasks.Task`1<TResult_REF>.TrySetResult (TResult_REF) [0x0004f] in <79002cfb0e6e431d8465d6dcea08995e>:0
  at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1<TResult_REF>.SetResult (TResult_REF) [0x0003d] in <79002cfb0e6e431d8465d6dcea08995e>:0
  at Microsoft.Build.BackEnd.TaskBuilder/<ExecuteTask>d__18.MoveNext () [0x002be] in <bdc5207a22bb42ae9fe3f3d07e82871e>:0
  at System.Runtime.CompilerServices.AsyncMethodBuilderCore/MoveNextRunner.InvokeMoveNext (object) [0x00000] in /home/grendel/vc/mono/mono/mcs/class/referencesource/mscorlib/system/runtime/compilerservices/AsyncMethodBuilder.cs:1094
  at System.Threading.ExecutionContext.RunInternal (System.Threading.ExecutionContext,System.Threading.ContextCallback,object,bool) [0x00071] in /home/grendel/vc/mono/mono/mcs/class/referencesource/mscorlib/system/threading/executioncontext.cs:961
  at System.Threading.ExecutionContext.Run (System.Threading.ExecutionContext,System.Threading.ContextCallback,object,bool) [0x00000] in /home/grendel/vc/mono/mono/mcs/class/referencesource/mscorlib/system/threading/executioncontext.cs:908
  at System.Runtime.CompilerServices.AsyncMethodBuilderCore/MoveNextRunner.Run () [0x00024] in /home/grendel/vc/mono/mono/mcs/class/referencesource/mscorlib/system/runtime/compilerservices/AsyncMethodBuilder.cs:1075
  at System.Threading.Tasks.AwaitTaskContinuation.RunOrScheduleAction (System.Action,bool,System.Threading.Tasks.Task&) [0x0001a] in /home/grendel/vc/mono/mono/external/corert/src/System.Private.CoreLib/src/System/Threading/Tasks/TaskContinuation.cs:680
  at System.Threading.Tasks.Task.FinishContinuations () [0x00052] in /home/grendel/vc/mono/mono/external/corert/src/System.Private.CoreLib/src/System/Threading/Tasks/Task.cs:3198
  at System.Threading.Tasks.Task.FinishStageThree () [0x0003c] in /home/grendel/vc/mono/mono/external/corert/src/System.Private.CoreLib/src/System/Threading/Tasks/Task.cs:2247
  at System.Threading.Tasks.Task`1<TResult_REF>.TrySetResult (TResult_REF) [0x0004f] in <79002cfb0e6e431d8465d6dcea08995e>:0
  at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1<TResult_REF>.SetResult (TResult_REF) [0x0003d] in <79002cfb0e6e431d8465d6dcea08995e>:0
  at Microsoft.Build.BackEnd.TaskBuilder/<ExecuteBucket>d__19.MoveNext () [0x003e5] in <bdc5207a22bb42ae9fe3f3d07e82871e>:0
  at System.Runtime.CompilerServices.AsyncMethodBuilderCore/MoveNextRunner.InvokeMoveNext (object) [0x00000] in /home/grendel/vc/mono/mono/mcs/class/referencesource/mscorlib/system/runtime/compilerservices/AsyncMethodBuilder.cs:1094
  at System.Threading.ExecutionContext.RunInternal (System.Threading.ExecutionContext,System.Threading.ContextCallback,object,bool) [0x00071] in /home/grendel/vc/mono/mono/mcs/class/referencesource/mscorlib/system/threading/executioncontext.cs:961
  at System.Threading.ExecutionContext.Run (System.Threading.ExecutionContext,System.Threading.ContextCallback,object,bool) [0x00000] in /home/grendel/vc/mono/mono/mcs/class/referencesource/mscorlib/system/threading/executioncontext.cs:908
  at System.Runtime.CompilerServices.AsyncMethodBuilderCore/MoveNextRunner.Run () [0x00024] in /home/grendel/vc/mono/mono/mcs/class/referencesource/mscorlib/system/runtime/compilerservices/AsyncMethodBuilder.cs:1075
  at System.Threading.Tasks.AwaitTaskContinuation.RunOrScheduleAction (System.Action,bool,System.Threading.Tasks.Task&) [0x0001a] in /home/grendel/vc/mono/mono/external/corert/src/System.Private.CoreLib/src/System/Threading/Tasks/TaskContinuation.cs:680
  at System.Threading.Tasks.Task.FinishContinuations () [0x00052] in /home/grendel/vc/mono/mono/external/corert/src/System.Private.CoreLib/src/System/Threading/Tasks/Task.cs:3198
  at System.Threading.Tasks.Task.FinishStageThree () [0x0003c] in /home/grendel/vc/mono/mono/external/corert/src/System.Private.CoreLib/src/System/Threading/Tasks/Task.cs:2247
  at System.Threading.Tasks.Task`1<TResult_REF>.TrySetResult (TResult_REF) [0x0004f] in <79002cfb0e6e431d8465d6dcea08995e>:0
  at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1<TResult_REF>.SetResult (TResult_REF) [0x0003d] in <79002cfb0e6e431d8465d6dcea08995e>:0
  at Microsoft.Build.BackEnd.TaskBuilder/<InitializeAndExecuteTask>d__24.MoveNext () [0x0012d] in <bdc5207a22bb42ae9fe3f3d07e82871e>:0
  at System.Runtime.CompilerServices.AsyncMethodBuilderCore/MoveNextRunner.InvokeMoveNext (object) [0x00000] in /home/grendel/vc/mono/mono/mcs/class/referencesource/mscorlib/system/runtime/compilerservices/AsyncMethodBuilder.cs:1094
  at System.Threading.ExecutionContext.RunInternal (System.Threading.ExecutionContext,System.Threading.ContextCallback,object,bool) [0x00071] in /home/grendel/vc/mono/mono/mcs/class/referencesource/mscorlib/system/threading/executioncontext.cs:961
  at System.Threading.ExecutionContext.Run (System.Threading.ExecutionContext,System.Threading.ContextCallback,object,bool) [0x00000] in /home/grendel/vc/mono/mono/mcs/class/referencesource/mscorlib/system/threading/executioncontext.cs:908
  at System.Runtime.CompilerServices.AsyncMethodBuilderCore/MoveNextRunner.Run () [0x00024] in /home/grendel/vc/mono/mono/mcs/class/referencesource/mscorlib/system/runtime/compilerservices/AsyncMethodBuilder.cs:1075
  at System.Threading.Tasks.AwaitTaskContinuation.RunOrScheduleAction (System.Action,bool,System.Threading.Tasks.Task&) [0x0001a] in /home/grendel/vc/mono/mono/external/corert/src/System.Private.CoreLib/src/System/Threading/Tasks/TaskContinuation.cs:680
  at System.Threading.Tasks.Task.FinishContinuations () [0x00052] in /home/grendel/vc/mono/mono/external/corert/src/System.Private.CoreLib/src/System/Threading/Tasks/Task.cs:3198
  at System.Threading.Tasks.Task.FinishStageThree () [0x0003c] in /home/grendel/vc/mono/mono/external/corert/src/System.Private.CoreLib/src/System/Threading/Tasks/Task.cs:2247
  at System.Threading.Tasks.Task`1<TResult_REF>.TrySetResult (TResult_REF) [0x0004f] in <79002cfb0e6e431d8465d6dcea08995e>:0
  at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1<TResult_REF>.SetResult (TResult_REF) [0x0003d] in <79002cfb0e6e431d8465d6dcea08995e>:0
  at Microsoft.Build.BackEnd.TaskBuilder/<ExecuteInstantiatedTask>d__26.MoveNext () [0x0065e] in <bdc5207a22bb42ae9fe3f3d07e82871e>:0
  at System.Runtime.CompilerServices.AsyncMethodBuilderCore/MoveNextRunner.InvokeMoveNext (object) [0x00000] in /home/grendel/vc/mono/mono/mcs/class/referencesource/mscorlib/system/runtime/compilerservices/AsyncMethodBuilder.cs:1094
  at System.Threading.ExecutionContext.RunInternal (System.Threading.ExecutionContext,System.Threading.ContextCallback,object,bool) [0x00071] in /home/grendel/vc/mono/mono/mcs/class/referencesource/mscorlib/system/threading/executioncontext.cs:961
  at System.Threading.ExecutionContext.Run (System.Threading.ExecutionContext,System.Threading.ContextCallback,object,bool) [0x00000] in /home/grendel/vc/mono/mono/mcs/class/referencesource/mscorlib/system/threading/executioncontext.cs:908
  at System.Runtime.CompilerServices.AsyncMethodBuilderCore/MoveNextRunner.Run () [0x00024] in /home/grendel/vc/mono/mono/mcs/class/referencesource/mscorlib/system/runtime/compilerservices/AsyncMethodBuilder.cs:1075
  at System.Threading.Tasks.AwaitTaskContinuation.RunOrScheduleAction (System.Action,bool,System.Threading.Tasks.Task&) [0x0001a] in /home/grendel/vc/mono/mono/external/corert/src/System.Private.CoreLib/src/System/Threading/Tasks/TaskContinuation.cs:680
  at System.Threading.Tasks.Task.FinishContinuations () [0x00052] in /home/grendel/vc/mono/mono/external/corert/src/System.Private.CoreLib/src/System/Threading/Tasks/Task.cs:3198
  at System.Threading.Tasks.Task.FinishStageThree () [0x0003c] in /home/grendel/vc/mono/mono/external/corert/src/System.Private.CoreLib/src/System/Threading/Tasks/Task.cs:2247
  at System.Threading.Tasks.Task`1<bool>.TrySetResult (bool) [0x0004f] in /home/grendel/vc/mono/mono/external/corert/src/System.Private.CoreLib/src/System/Threading/Tasks/Future.cs:433
  at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1<bool>.SetResult (bool) [0x0003d] in /home/grendel/vc/mono/mono/mcs/class/referencesource/mscorlib/system/runtime/compilerservices/AsyncMethodBuilder.cs:608
  at Microsoft.Build.BackEnd.MSBuild/<ExecuteInternal>d__76.MoveNext () [0x004dd] in <bdc5207a22bb42ae9fe3f3d07e82871e>:0
  at System.Runtime.CompilerServices.AsyncMethodBuilderCore/MoveNextRunner.InvokeMoveNext (object) [0x00000] in /home/grendel/vc/mono/mono/mcs/class/referencesource/mscorlib/system/runtime/compilerservices/AsyncMethodBuilder.cs:1094
  at System.Threading.ExecutionContext.RunInternal (System.Threading.ExecutionContext,System.Threading.ContextCallback,object,bool) [0x00071] in /home/grendel/vc/mono/mono/mcs/class/referencesource/mscorlib/system/threading/executioncontext.cs:961
  at System.Threading.ExecutionContext.Run (System.Threading.ExecutionContext,System.Threading.ContextCallback,object,bool) [0x00000] in /home/grendel/vc/mono/mono/mcs/class/referencesource/mscorlib/system/threading/executioncontext.cs:908
  at System.Runtime.CompilerServices.AsyncMethodBuilderCore/MoveNextRunner.Run () [0x00024] in /home/grendel/vc/mono/mono/mcs/class/referencesource/mscorlib/system/runtime/compilerservices/AsyncMethodBuilder.cs:1075
  at System.Threading.Tasks.AwaitTaskContinuation.RunOrScheduleAction (System.Action,bool,System.Threading.Tasks.Task&) [0x0001a] in /home/grendel/vc/mono/mono/external/corert/src/System.Private.CoreLib/src/System/Threading/Tasks/TaskContinuation.cs:680
  at System.Threading.Tasks.Task.FinishContinuations () [0x00052] in /home/grendel/vc/mono/mono/external/corert/src/System.Private.CoreLib/src/System/Threading/Tasks/Task.cs:3198
  at System.Threading.Tasks.Task.FinishStageThree () [0x0003c] in /home/grendel/vc/mono/mono/external/corert/src/System.Private.CoreLib/src/System/Threading/Tasks/Task.cs:2247
  at System.Threading.Tasks.Task`1<bool>.TrySetResult (bool) [0x0004f] in /home/grendel/vc/mono/mono/external/corert/src/System.Private.CoreLib/src/System/Threading/Tasks/Future.cs:433
  at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1<bool>.SetResult (bool) [0x0003d] in /home/grendel/vc/mono/mono/mcs/class/referencesource/mscorlib/system/runtime/compilerservices/AsyncMethodBuilder.cs:608
  at Microsoft.Build.BackEnd.MSBuild/<BuildProjectsInParallel>d__77.MoveNext () [0x0014a] in <bdc5207a22bb42ae9fe3f3d07e82871e>:0
  at System.Runtime.CompilerServices.AsyncMethodBuilderCore/MoveNextRunner.InvokeMoveNext (object) [0x00000] in /home/grendel/vc/mono/mono/mcs/class/referencesource/mscorlib/system/runtime/compilerservices/AsyncMethodBuilder.cs:1094
  at System.Threading.ExecutionContext.RunInternal (System.Threading.ExecutionContext,System.Threading.ContextCallback,object,bool) [0x00071] in /home/grendel/vc/mono/mono/mcs/class/referencesource/mscorlib/system/threading/executioncontext.cs:961
  at System.Threading.ExecutionContext.Run (System.Threading.ExecutionContext,System.Threading.ContextCallback,object,bool) [0x00000] in /home/grendel/vc/mono/mono/mcs/class/referencesource/mscorlib/system/threading/executioncontext.cs:908
  at System.Runtime.CompilerServices.AsyncMethodBuilderCore/MoveNextRunner.Run () [0x00024] in /home/grendel/vc/mono/mono/mcs/class/referencesource/mscorlib/system/runtime/compilerservices/AsyncMethodBuilder.cs:1075
  at System.Threading.Tasks.AwaitTaskContinuation.RunOrScheduleAction (System.Action,bool,System.Threading.Tasks.Task&) [0x0001a] in /home/grendel/vc/mono/mono/external/corert/src/System.Private.CoreLib/src/System/Threading/Tasks/TaskContinuation.cs:680
  at System.Threading.Tasks.Task.FinishContinuations () [0x00052] in /home/grendel/vc/mono/mono/external/corert/src/System.Private.CoreLib/src/System/Threading/Tasks/Task.cs:3198
  at System.Threading.Tasks.Task.FinishStageThree () [0x0003c] in /home/grendel/vc/mono/mono/external/corert/src/System.Private.CoreLib/src/System/Threading/Tasks/Task.cs:2247
  at System.Threading.Tasks.Task`1<bool>.TrySetResult (bool) [0x0004f] in /home/grendel/vc/mono/mono/external/corert/src/System.Private.CoreLib/src/System/Threading/Tasks/Future.cs:433
  at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1<bool>.SetResult (bool) [0x0003d] in /home/grendel/vc/mono/mono/mcs/class/referencesource/mscorlib/system/runtime/compilerservices/AsyncMethodBuilder.cs:608
  at Microsoft.Build.BackEnd.MSBuild/<ExecuteTargets>d__80.MoveNext () [0x00713] in <bdc5207a22bb42ae9fe3f3d07e82871e>:0
  at System.Runtime.CompilerServices.AsyncMethodBuilderCore/MoveNextRunner.InvokeMoveNext (object) [0x00000] in /home/grendel/vc/mono/mono/mcs/class/referencesource/mscorlib/system/runtime/compilerservices/AsyncMethodBuilder.cs:1094
  at System.Threading.ExecutionContext.RunInternal (System.Threading.ExecutionContext,System.Threading.ContextCallback,object,bool) [0x00071] in /home/grendel/vc/mono/mono/mcs/class/referencesource/mscorlib/system/threading/executioncontext.cs:961
  at System.Threading.ExecutionContext.Run (System.Threading.ExecutionContext,System.Threading.ContextCallback,object,bool) [0x00000] in /home/grendel/vc/mono/mono/mcs/class/referencesource/mscorlib/system/threading/executioncontext.cs:908
  at System.Runtime.CompilerServices.AsyncMethodBuilderCore/MoveNextRunner.Run () [0x00024] in /home/grendel/vc/mono/mono/mcs/class/referencesource/mscorlib/system/runtime/compilerservices/AsyncMethodBuilder.cs:1075
  at System.Threading.Tasks.AwaitTaskContinuation.RunOrScheduleAction (System.Action,bool,System.Threading.Tasks.Task&) [0x0001a] in /home/grendel/vc/mono/mono/external/corert/src/System.Private.CoreLib/src/System/Threading/Tasks/TaskContinuation.cs:680
  at System.Threading.Tasks.Task.FinishContinuations () [0x00052] in /home/grendel/vc/mono/mono/external/corert/src/System.Private.CoreLib/src/System/Threading/Tasks/Task.cs:3198
  at System.Threading.Tasks.Task.FinishStageThree () [0x0003c] in /home/grendel/vc/mono/mono/external/corert/src/System.Private.CoreLib/src/System/Threading/Tasks/Task.cs:2247
  at System.Threading.Tasks.Task`1<Microsoft.Build.Framework.BuildEngineResult>.TrySetResult (Microsoft.Build.Framework.BuildEngineResult) [0x0004f] in /home/grendel/vc/mono/mono/external/corert/src/System.Private.CoreLib/src/System/Threading/Tasks/Future.cs:433
  at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1<Microsoft.Build.Framework.BuildEngineResult>.SetResult (Microsoft.Build.Framework.BuildEngineResult) [0x0003d] in /home/grendel/vc/mono/mono/mcs/class/referencesource/mscorlib/system/runtime/compilerservices/AsyncMethodBuilder.cs:608
  at Microsoft.Build.BackEnd.TaskHost/<InternalBuildProjects>d__49.MoveNext () [0x0025c] in <bdc5207a22bb42ae9fe3f3d07e82871e>:0
  at System.Runtime.CompilerServices.AsyncMethodBuilderCore/MoveNextRunner.InvokeMoveNext (object) [0x00000] in /home/grendel/vc/mono/mono/mcs/class/referencesource/mscorlib/system/runtime/compilerservices/AsyncMethodBuilder.cs:1094
  at System.Threading.ExecutionContext.RunInternal (System.Threading.ExecutionContext,System.Threading.ContextCallback,object,bool) [0x00071] in /home/grendel/vc/mono/mono/mcs/class/referencesource/mscorlib/system/threading/executioncontext.cs:961
  at System.Threading.ExecutionContext.Run (System.Threading.ExecutionContext,System.Threading.ContextCallback,object,bool) [0x00000] in /home/grendel/vc/mono/mono/mcs/class/referencesource/mscorlib/system/threading/executioncontext.cs:908
  at System.Runtime.CompilerServices.AsyncMethodBuilderCore/MoveNextRunner.Run () [0x00024] in /home/grendel/vc/mono/mono/mcs/class/referencesource/mscorlib/system/runtime/compilerservices/AsyncMethodBuilder.cs:1075
  at System.Threading.Tasks.AwaitTaskContinuation.RunOrScheduleAction (System.Action,bool,System.Threading.Tasks.Task&) [0x0001a] in /home/grendel/vc/mono/mono/external/corert/src/System.Private.CoreLib/src/System/Threading/Tasks/TaskContinuation.cs:680
  at System.Threading.Tasks.Task.FinishContinuations () [0x00052] in /home/grendel/vc/mono/mono/external/corert/src/System.Private.CoreLib/src/System/Threading/Tasks/Task.cs:3198
  at System.Threading.Tasks.Task.FinishStageThree () [0x0003c] in /home/grendel/vc/mono/mono/external/corert/src/System.Private.CoreLib/src/System/Threading/Tasks/Task.cs:2247
  at System.Threading.Tasks.Task`1<Microsoft.Build.Framework.BuildEngineResult>.TrySetResult (Microsoft.Build.Framework.BuildEngineResult) [0x0004f] in /home/grendel/vc/mono/mono/external/corert/src/System.Private.CoreLib/src/System/Threading/Tasks/Future.cs:433
  at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1<Microsoft.Build.Framework.BuildEngineResult>.SetResult (Microsoft.Build.Framework.BuildEngineResult) [0x0003d] in /home/grendel/vc/mono/mono/mcs/class/referencesource/mscorlib/system/runtime/compilerservices/AsyncMethodBuilder.cs:608
  at Microsoft.Build.BackEnd.TaskHost/<BuildProjectFilesInParallelAsync>d__53.MoveNext () [0x004fb] in <bdc5207a22bb42ae9fe3f3d07e82871e>:0
  at System.Runtime.CompilerServices.AsyncMethodBuilderCore/MoveNextRunner.InvokeMoveNext (object) [0x00000] in /home/grendel/vc/mono/mono/mcs/class/referencesource/mscorlib/system/runtime/compilerservices/AsyncMethodBuilder.cs:1094
  at System.Threading.ExecutionContext.RunInternal (System.Threading.ExecutionContext,System.Threading.ContextCallback,object,bool) [0x00071] in /home/grendel/vc/mono/mono/mcs/class/referencesource/mscorlib/system/threading/executioncontext.cs:961
  at System.Threading.ExecutionContext.Run (System.Threading.ExecutionContext,System.Threading.ContextCallback,object,bool) [0x00000] in /home/grendel/vc/mono/mono/mcs/class/referencesource/mscorlib/system/threading/executioncontext.cs:908
  at System.Runtime.CompilerServices.AsyncMethodBuilderCore/MoveNextRunner.Run () [0x00024] in /home/grendel/vc/mono/mono/mcs/class/referencesource/mscorlib/system/runtime/compilerservices/AsyncMethodBuilder.cs:1075
  at System.Threading.Tasks.AwaitTaskContinuation.RunOrScheduleAction (System.Action,bool,System.Threading.Tasks.Task&) [0x0001a] in /home/grendel/vc/mono/mono/external/corert/src/System.Private.CoreLib/src/System/Threading/Tasks/TaskContinuation.cs:680
  at System.Threading.Tasks.Task.FinishContinuations () [0x00052] in /home/grendel/vc/mono/mono/external/corert/src/System.Private.CoreLib/src/System/Threading/Tasks/Task.cs:3198
  at System.Threading.Tasks.Task.FinishStageThree () [0x0003c] in /home/grendel/vc/mono/mono/external/corert/src/System.Private.CoreLib/src/System/Threading/Tasks/Task.cs:2247
  at System.Threading.Tasks.Task`1<TResult_REF>.TrySetResult (TResult_REF) [0x0004f] in <79002cfb0e6e431d8465d6dcea08995e>:0
  at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1<TResult_REF>.SetResult (TResult_REF) [0x0003d] in <79002cfb0e6e431d8465d6dcea08995e>:0
  at Microsoft.Build.BackEnd.RequestBuilder/<BuildProjects>d__33.MoveNext () [0x00263] in <bdc5207a22bb42ae9fe3f3d07e82871e>:0
  at System.Runtime.CompilerServices.AsyncMethodBuilderCore/MoveNextRunner.InvokeMoveNext (object) [0x00000] in /home/grendel/vc/mono/mono/mcs/class/referencesource/mscorlib/system/runtime/compilerservices/AsyncMethodBuilder.cs:1094
  at System.Threading.ExecutionContext.RunInternal (System.Threading.ExecutionContext,System.Threading.ContextCallback,object,bool) [0x00071] in /home/grendel/vc/mono/mono/mcs/class/referencesource/mscorlib/system/threading/executioncontext.cs:961
  at System.Threading.ExecutionContext.Run (System.Threading.ExecutionContext,System.Threading.ContextCallback,object,bool) [0x00000] in /home/grendel/vc/mono/mono/mcs/class/referencesource/mscorlib/system/threading/executioncontext.cs:908
  at System.Runtime.CompilerServices.AsyncMethodBuilderCore/MoveNextRunner.Run () [0x00024] in /home/grendel/vc/mono/mono/mcs/class/referencesource/mscorlib/system/runtime/compilerservices/AsyncMethodBuilder.cs:1075
  at System.Threading.Tasks.AwaitTaskContinuation.RunOrScheduleAction (System.Action,bool,System.Threading.Tasks.Task&) [0x0001a] in /home/grendel/vc/mono/mono/external/corert/src/System.Private.CoreLib/src/System/Threading/Tasks/TaskContinuation.cs:680
  at System.Threading.Tasks.Task.FinishContinuations () [0x00052] in /home/grendel/vc/mono/mono/external/corert/src/System.Private.CoreLib/src/System/Threading/Tasks/Task.cs:3198
  at System.Threading.Tasks.Task.FinishStageThree () [0x0003c] in /home/grendel/vc/mono/mono/external/corert/src/System.Private.CoreLib/src/System/Threading/Tasks/Task.cs:2247
  at System.Threading.Tasks.Task`1<TResult_REF>.TrySetResult (TResult_REF) [0x0004f] in <79002cfb0e6e431d8465d6dcea08995e>:0
  at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1<TResult_REF>.SetResult (TResult_REF) [0x0003d] in <79002cfb0e6e431d8465d6dcea08995e>:0
  at Microsoft.Build.BackEnd.RequestBuilder/<StartNewBuildRequests>d__51.MoveNext () [0x0020e] in <bdc5207a22bb42ae9fe3f3d07e82871e>:0
  at System.Runtime.CompilerServices.AsyncMethodBuilderCore/MoveNextRunner.InvokeMoveNext (object) [0x00000] in /home/grendel/vc/mono/mono/mcs/class/referencesource/mscorlib/system/runtime/compilerservices/AsyncMethodBuilder.cs:1094
  at System.Threading.ExecutionContext.RunInternal (System.Threading.ExecutionContext,System.Threading.ContextCallback,object,bool) [0x00071] in /home/grendel/vc/mono/mono/mcs/class/referencesource/mscorlib/system/threading/executioncontext.cs:961
  at System.Threading.ExecutionContext.Run (System.Threading.ExecutionContext,System.Threading.ContextCallback,object,bool) [0x00000] in /home/grendel/vc/mono/mono/mcs/class/referencesource/mscorlib/system/threading/executioncontext.cs:908
  at System.Runtime.CompilerServices.AsyncMethodBuilderCore/MoveNextRunner.Run () [0x00024] in /home/grendel/vc/mono/mono/mcs/class/referencesource/mscorlib/system/runtime/compilerservices/AsyncMethodBuilder.cs:1075
  at System.Threading.Tasks.AwaitTaskContinuation.RunOrScheduleAction (System.Action,bool,System.Threading.Tasks.Task&) [0x0001a] in /home/grendel/vc/mono/mono/external/corert/src/System.Private.CoreLib/src/System/Threading/Tasks/TaskContinuation.cs:680
  at System.Threading.Tasks.Task.FinishContinuations () [0x00052] in /home/grendel/vc/mono/mono/external/corert/src/System.Private.CoreLib/src/System/Threading/Tasks/Task.cs:3198
  at System.Threading.Tasks.Task.FinishStageThree () [0x0003c] in /home/grendel/vc/mono/mono/external/corert/src/System.Private.CoreLib/src/System/Threading/Tasks/Task.cs:2247
  at System.Threading.Tasks.Task`1<int>.TrySetResult (int) [0x0004f] in /home/grendel/vc/mono/mono/external/corert/src/System.Private.CoreLib/src/System/Threading/Tasks/Future.cs:433
  at System.Threading.Tasks.TaskCompletionSource`1<int>.TrySetResult (int) [0x00000] in /home/grendel/vc/mono/mono/external/corefx/src/Common/src/CoreLib/System/Threading/Tasks/TaskCompletionSource.cs:274
  at Microsoft.Build.Shared.AwaitExtensions/<>c__DisplayClass7_2.<ToTask>b__0 (object,bool) [0x00038] in <bdc5207a22bb42ae9fe3f3d07e82871e>:0
  at System.Threading.RegisteredWaitHandle.DoCallBack (object) [0x00008] in /home/grendel/vc/mono/mono/mcs/class/corlib/System.Threading/RegisteredWaitHandle.cs:103
  at System.Threading.QueueUserWorkItemCallback.WaitCallback_Context (object) [0x0000d] in /home/grendel/vc/mono/mono/mcs/class/referencesource/mscorlib/system/threading/threadpool.cs:1308
  at System.Threading.ExecutionContext.RunInternal (System.Threading.ExecutionContext,System.Threading.ContextCallback,object,bool) [0x00071] in /home/grendel/vc/mono/mono/mcs/class/referencesource/mscorlib/system/threading/executioncontext.cs:961
  at System.Threading.ExecutionContext.Run (System.Threading.ExecutionContext,System.Threading.ContextCallback,object,bool) [0x00000] in /home/grendel/vc/mono/mono/mcs/class/referencesource/mscorlib/system/threading/executioncontext.cs:908
  at System.Threading.QueueUserWorkItemCallback.System.Threading.IThreadPoolWorkItem.ExecuteWorkItem () [0x00021] in /home/grendel/vc/mono/mono/mcs/class/referencesource/mscorlib/system/threading/threadpool.cs:1285
  at System.Threading.ThreadPoolWorkQueue.Dispatch () [0x00074] in /home/grendel/vc/mono/mono/mcs/class/referencesource/mscorlib/system/threading/threadpool.cs:858
  at System.Threading._ThreadPoolWaitCallback.PerformWaitCallback () [0x00000] in /home/grendel/vc/mono/mono/mcs/class/referencesource/mscorlib/system/threading/threadpool.cs:1213
  at (wrapper runtime-invoke) <Module>.runtime_invoke_bool (object,intptr,intptr,intptr) [0x0001f] in <79002cfb0e6e431d8465d6dcea08995e>:0

The crash occurs in frame 7, because the `r` parameter (an instance of the
`MonoAssembly` structure) contains pointers that used to point to valid places
in the `Xamarin.Android.Build.Tasks` assembbly's mmaped area, but now point to
garbage data. One of those corrupted pointers is the ASCII assembly name - which
is used by the function in frame 7.

The situation can be detected by the runtime by using advisory locks and
reporting the attempt to overwrite the mmapped area, however a fix would require
creating a copy of all the assembly's in-memory data before the write is allowed
and that might be too expensive. None of those measures are currently
implemented by Mono (and it isn't sure if any of them will ever be implemented).

The crash this commit fixes occurs in a very specific scenario - when building
the `OpenTK.csproj` project *directly* (that is passing it as parameter to
`msbuild`), which causes a the `Xamarin.Android.Build.Tasks` assembly to be
loaded for use in the `<UsingTask/>` msbuild statement, but at the same time
`OpenTK.csproj` references the `Xamarin.Android.Build.Tasks` *project* which
causes the ILRepacker task to run unconditionally, attempting to repack the
assemblies again and leading to the crash described above.

The fix, courtesy of @jonpryor, is to make the ILRepacker target not run every
time but only whenver the assembly has to be re-packed because it has just been
rebuilt.
jonpryor pushed a commit that referenced this pull request Jul 19, 2018
Xamarin.Android uses the [ILRepack task][0] to repack/merge the
`Xamarin.Android.Build.Tasks.dll` assembly (610ade7), a step that is
performed after the `Xamarin.Android.Build.Tasks.csproj` project's
`Build` target completes.  The step takes the project's output
assembly -- `Xamarin.Android.Build.Tasks.dll` -- and several other
dependencies, then merges them into one, *replacing* the original
output assembly.

The majority of time it works flawlessly, however when building the
`monodroid` repo we observe a Mono runtime crash, similar-but-not-
identical to [mono issue 9613][m9613].  Not all details are
understood, but the high level overview of the issue is as follows:
Mono runtime uses the **mmap**(2) system call on Unix to map the
on-disk image of the assembly into memory.  The `mmapp`ed area is
used whenever the same assembly is loaded (e.g.
`Assembly.LoadFrom()`) into the application to save time and memory.
The memory is mapped as private which means that anything written to
the mapped area by the Mono process is effectively discarded, that is
not reflected in the file on disk.  The protection, however, doesn't
work in the other direction: when an external process (or even the
same Mono process!) writes to the file that was `mmapp`ed.
Unfortunately, the OS kernel behavior in such instance is left
undefined by the POSIX standard: the kernel can choose to reflect the
on-disk change in the mapping or ignore it.  On macOS the write is
ignored; on Linux, the write leads either to the `SIGBUS` signal
being sent to the process or corruption to the `mmap`ped memory area
(details of how this happens are, as of yet, unknown; it's just a
theory at this point).  The problem occurs in our case when the
`<ILRepacker/>` task overwrites the `Xamarin.Android.Build.Tasks.dll`
assembly with its merged/repackaged form and that, somehow, corrupts
Mono runtime's memory mapping of that assembly, eventually crashing:

        ## native crash
        #0  0x00007f974fd1b23a in __waitpid (pid=pid@entry=120561, stat_loc=stat_loc@entry=0x7f9712162b9c, options=options@entry=0) at ../sysdeps/unix/sysv/linux/waitpid.c:30
        #1  0x000055b471f36c96 in dump_native_stacktrace (ctx=0x7f97121637c0, signal=0x55b472147cd3 "SIGSEGV") at mini-posix.c:719
        #2  0x000055b471f36df2 in mono_dump_native_crash_info (signal=signal@entry=0x55b472147cd3 "SIGSEGV", ctx=ctx@entry=0x7f97121637c0, info=info@entry=0x7f97121638f0) at mini-posix.c:742
        #3  0x000055b471ecc620 in mono_handle_native_crash (signal=signal@entry=0x55b472147cd3 "SIGSEGV", ctx=ctx@entry=0x7f97121637c0, info=info@entry=0x7f97121638f0) at mini-exceptions.c:2938
        #4  0x000055b471e4c82c in mono_sigsegv_signal_handler (_dummy=7, _info=0x7f97121638f0, context=0x7f97121637c0) at mini-runtime.c:3441
        #5  <signal handler called>
        #6  __strcasecmp_l_avx () at ../sysdeps/x86_64/multiarch/strcmp-sse42.S:199
        #7  0x000055b471fb2105 in mono_assembly_names_equal_flags (l=l@entry=0x7f96e0004370, r=r@entry=0x7f97307edb60, flags=flags@entry=MONO_ANAME_EQ_IGNORE_CASE) at assembly.c:663
        #8  0x000055b471fa7baf in mono_domain_assembly_search (aname=0x7f96e0004370, user_data=0x0) at appdomain.c:2242
        #9  0x000055b471fb10f7 in mono_assembly_invoke_search_hook_internal (aname=aname@entry=0x7f96e0004370, requesting=requesting@entry=0x0, refonly=0, postload=postload@entry=0) at assembly.c:1755
        #10 0x000055b471fb4526 in mono_assembly_load_from_predicate (image=image@entry=0x7f96e001aae0, fname=fname@entry=0x7f96e000bc50 "/home/grendel/vc/xamarin/monodroid/monodroid/external/xamarin-android/src/Xamarin.Android.Build.Tasks/../../packages/ILRepack.Lib.MSBuild.Task.2.0.15.4/build/ILRepack.Lib.MSBuild.Task.dll", asmctx=asmctx@entry=MONO_ASMCTX_LOADFROM, predicate=predicate@entry=0x0, user_data=user_data@entry=0x0, status=status@entry=0x7f9712163fd4) at assembly.c:2638
        #11 0x000055b471fb635c in mono_assembly_open_predicate (filename=<optimized out>, filename@entry=0x7f96e000ae40 "/home/grendel/vc/xamarin/monodroid/monodroid/external/xamarin-android/src/Xamarin.Android.Build.Tasks/../../packages/ILRepack.Lib.MSBuild.Task.2.0.15.4/build/ILRepack.Lib.MSBuild.Task.dll", asmctx=MONO_ASMCTX_LOADFROM, predicate=predicate@entry=0x0, user_data=user_data@entry=0x0, status=status@entry=0x7f9712163fd4) at assembly.c:2206
        #12 0x000055b471fabd42 in ves_icall_System_Reflection_Assembly_LoadFrom (fname=..., refOnly=<optimized out>, error=0x7f9712164050) at appdomain.c:2272
        ...

        ## managed stacktrace
        at <unknown> <0xffffffff>
        at (wrapper managed-to-native) System.Reflection.Assembly.LoadFrom (string,bool) [0x0001b] in <79002cfb0e6e431d8465d6dcea08995e>:0
        at System.Reflection.Assembly.LoadFrom (string) [0x00000] in /home/grendel/vc/mono/mono/mcs/class/corlib/System.Reflection/Assembly.cs:539
        at System.Reflection.Assembly.UnsafeLoadFrom (string) [0x00000] in /home/grendel/vc/mono/mono/mcs/class/corlib/System.Reflection/Assembly.cs:571
        at Microsoft.Build.Shared.TypeLoader.LoadAssembly (Microsoft.Build.Shared.AssemblyLoadInfo) [0x0001e] in <bdc5207a22bb42ae9fe3f3d07e82871e>:0
        ...

The crash occurs in frame 7, because the `r` parameter (an instance
of the `MonoAssembly` structure) contains pointers that used to point
to valid places in the `Xamarin.Android.Build.Tasks.dll`s `mmap`ed
area, but now point to garbage data.  One of those corrupted pointers
is the ASCII assembly name - which is used by the function in frame 7.

The crash this commit fixes occurs in a very specific scenario: when
building the `OpenTK.csproj` project *directly*
(e.g. `msbuild OpenTK.csproj`), which causes the
`Xamarin.Android.Build.Tasks.dll` assembly to be loaded for use in
the `<UsingTask/>` MSBuild statement, but at the same time 
`OpenTK.csproj` has a `@(ProjectReference)` to
`Xamarin.Android.Build.Tasks.csproj`, which caused the
`<ILRepacker/>` task to run unconditionally, attempting to repack the
assemblies again and leading to the crash described above.

The fix, courtesy of @jonpryor, is to make the `ILRepacker` target
not run every time but only whenever the assembly has to be re-packed
because it has just been rebuilt.

This also improves rebuild times.

[0]: https://www.nuget.org/packages/ILRepack.Lib.MSBuild.Task
[m9613]: mono/mono#9613
jonpryor pushed a commit that referenced this pull request Aug 3, 2018
Xamarin.Android uses the [ILRepack task][0] to repack/merge the
`Xamarin.Android.Build.Tasks.dll` assembly (610ade7), a step that is
performed after the `Xamarin.Android.Build.Tasks.csproj` project's
`Build` target completes.  The step takes the project's output
assembly -- `Xamarin.Android.Build.Tasks.dll` -- and several other
dependencies, then merges them into one, *replacing* the original
output assembly.

The majority of time it works flawlessly, however when building the
`monodroid` repo we observe a Mono runtime crash, similar-but-not-
identical to [mono issue 9613][m9613].  Not all details are
understood, but the high level overview of the issue is as follows:
Mono runtime uses the **mmap**(2) system call on Unix to map the
on-disk image of the assembly into memory.  The `mmapp`ed area is
used whenever the same assembly is loaded (e.g.
`Assembly.LoadFrom()`) into the application to save time and memory.
The memory is mapped as private which means that anything written to
the mapped area by the Mono process is effectively discarded, that is
not reflected in the file on disk.  The protection, however, doesn't
work in the other direction: when an external process (or even the
same Mono process!) writes to the file that was `mmapp`ed.
Unfortunately, the OS kernel behavior in such instance is left
undefined by the POSIX standard: the kernel can choose to reflect the
on-disk change in the mapping or ignore it.  On macOS the write is
ignored; on Linux, the write leads either to the `SIGBUS` signal
being sent to the process or corruption to the `mmap`ped memory area
(details of how this happens are, as of yet, unknown; it's just a
theory at this point).  The problem occurs in our case when the
`<ILRepacker/>` task overwrites the `Xamarin.Android.Build.Tasks.dll`
assembly with its merged/repackaged form and that, somehow, corrupts
Mono runtime's memory mapping of that assembly, eventually crashing:

        ## native crash
        #0  0x00007f974fd1b23a in __waitpid (pid=pid@entry=120561, stat_loc=stat_loc@entry=0x7f9712162b9c, options=options@entry=0) at ../sysdeps/unix/sysv/linux/waitpid.c:30
        #1  0x000055b471f36c96 in dump_native_stacktrace (ctx=0x7f97121637c0, signal=0x55b472147cd3 "SIGSEGV") at mini-posix.c:719
        #2  0x000055b471f36df2 in mono_dump_native_crash_info (signal=signal@entry=0x55b472147cd3 "SIGSEGV", ctx=ctx@entry=0x7f97121637c0, info=info@entry=0x7f97121638f0) at mini-posix.c:742
        #3  0x000055b471ecc620 in mono_handle_native_crash (signal=signal@entry=0x55b472147cd3 "SIGSEGV", ctx=ctx@entry=0x7f97121637c0, info=info@entry=0x7f97121638f0) at mini-exceptions.c:2938
        #4  0x000055b471e4c82c in mono_sigsegv_signal_handler (_dummy=7, _info=0x7f97121638f0, context=0x7f97121637c0) at mini-runtime.c:3441
        #5  <signal handler called>
        #6  __strcasecmp_l_avx () at ../sysdeps/x86_64/multiarch/strcmp-sse42.S:199
        #7  0x000055b471fb2105 in mono_assembly_names_equal_flags (l=l@entry=0x7f96e0004370, r=r@entry=0x7f97307edb60, flags=flags@entry=MONO_ANAME_EQ_IGNORE_CASE) at assembly.c:663
        #8  0x000055b471fa7baf in mono_domain_assembly_search (aname=0x7f96e0004370, user_data=0x0) at appdomain.c:2242
        #9  0x000055b471fb10f7 in mono_assembly_invoke_search_hook_internal (aname=aname@entry=0x7f96e0004370, requesting=requesting@entry=0x0, refonly=0, postload=postload@entry=0) at assembly.c:1755
        #10 0x000055b471fb4526 in mono_assembly_load_from_predicate (image=image@entry=0x7f96e001aae0, fname=fname@entry=0x7f96e000bc50 "/home/grendel/vc/xamarin/monodroid/monodroid/external/xamarin-android/src/Xamarin.Android.Build.Tasks/../../packages/ILRepack.Lib.MSBuild.Task.2.0.15.4/build/ILRepack.Lib.MSBuild.Task.dll", asmctx=asmctx@entry=MONO_ASMCTX_LOADFROM, predicate=predicate@entry=0x0, user_data=user_data@entry=0x0, status=status@entry=0x7f9712163fd4) at assembly.c:2638
        #11 0x000055b471fb635c in mono_assembly_open_predicate (filename=<optimized out>, filename@entry=0x7f96e000ae40 "/home/grendel/vc/xamarin/monodroid/monodroid/external/xamarin-android/src/Xamarin.Android.Build.Tasks/../../packages/ILRepack.Lib.MSBuild.Task.2.0.15.4/build/ILRepack.Lib.MSBuild.Task.dll", asmctx=MONO_ASMCTX_LOADFROM, predicate=predicate@entry=0x0, user_data=user_data@entry=0x0, status=status@entry=0x7f9712163fd4) at assembly.c:2206
        #12 0x000055b471fabd42 in ves_icall_System_Reflection_Assembly_LoadFrom (fname=..., refOnly=<optimized out>, error=0x7f9712164050) at appdomain.c:2272
        ...

        ## managed stacktrace
        at <unknown> <0xffffffff>
        at (wrapper managed-to-native) System.Reflection.Assembly.LoadFrom (string,bool) [0x0001b] in <79002cfb0e6e431d8465d6dcea08995e>:0
        at System.Reflection.Assembly.LoadFrom (string) [0x00000] in /home/grendel/vc/mono/mono/mcs/class/corlib/System.Reflection/Assembly.cs:539
        at System.Reflection.Assembly.UnsafeLoadFrom (string) [0x00000] in /home/grendel/vc/mono/mono/mcs/class/corlib/System.Reflection/Assembly.cs:571
        at Microsoft.Build.Shared.TypeLoader.LoadAssembly (Microsoft.Build.Shared.AssemblyLoadInfo) [0x0001e] in <bdc5207a22bb42ae9fe3f3d07e82871e>:0
        ...

The crash occurs in frame 7, because the `r` parameter (an instance
of the `MonoAssembly` structure) contains pointers that used to point
to valid places in the `Xamarin.Android.Build.Tasks.dll`s `mmap`ed
area, but now point to garbage data.  One of those corrupted pointers
is the ASCII assembly name - which is used by the function in frame 7.

The crash this commit fixes occurs in a very specific scenario: when
building the `OpenTK.csproj` project *directly*
(e.g. `msbuild OpenTK.csproj`), which causes the
`Xamarin.Android.Build.Tasks.dll` assembly to be loaded for use in
the `<UsingTask/>` MSBuild statement, but at the same time 
`OpenTK.csproj` has a `@(ProjectReference)` to
`Xamarin.Android.Build.Tasks.csproj`, which caused the
`<ILRepacker/>` task to run unconditionally, attempting to repack the
assemblies again and leading to the crash described above.

The fix, courtesy of @jonpryor, is to make the `ILRepacker` target
not run every time but only whenever the assembly has to be re-packed
because it has just been rebuilt.

This also improves rebuild times.

[0]: https://www.nuget.org/packages/ILRepack.Lib.MSBuild.Task
[m9613]: mono/mono#9613
grendello added a commit to grendello/xamarin-android that referenced this pull request Nov 22, 2022
It won't fix the failure, but extra logging might be useful at some
point. The current failure is:

    droid.NET_Test: java_vm_ext.cc:570] JNI DETECTED ERROR IN APPLICATION: use of invalid jobject 0x7100941bb830
    backtrace:
           #00 pc 00000000000943f8  /apex/com.android.runtime/lib64/bionic/libc.so (syscall+24) (BuildId: a08a19770d6696739c847e29c3f5f650)
           #1 pc 0000000000097146  /apex/com.android.runtime/lib64/bionic/libc.so (abort+182) (BuildId: a08a19770d6696739c847e29c3f5f650)
           #2 pc 000000000055321f  /apex/com.android.runtime/lib64/libart.so (art::Runtime::Abort(char const*)+2399) (BuildId: 8bb3225e7c408f2ca23abac3db0417f2)
           #3 pc 000000000000c873  /system/lib64/libbase.so (android::base::LogMessage::~LogMessage()+611) (BuildId: 40d2b536dbf0730fdc31abd2b469f94f)
           dotnet#4 pc 00000000003ede64  /apex/com.android.runtime/lib64/libart.so (art::JavaVMExt::JniAbort(char const*, char const*)+1604) (BuildId: 8bb3225e7c408f2ca23abac3db0417f2)
           dotnet#5 pc 00000000003ee16a  /apex/com.android.runtime/lib64/libart.so (art::JavaVMExt::JniAbortF(char const*, char const*, ...)+234) (BuildId: 8bb3225e7c408f2ca23abac3db0417f2)
           dotnet#6 pc 00000000005adf7b  /apex/com.android.runtime/lib64/libart.so (art::Thread::DecodeJObject(_jobject*) const+875) (BuildId: 8bb3225e7c408f2ca23abac3db0417f2)
           dotnet#7 pc 00000000003def7b  /apex/com.android.runtime/lib64/libart.so (art::(anonymous namespace)::ScopedCheck::CheckInstance(art::ScopedObjectAccess&, art::(anonymous namespace)::ScopedCheck::InstanceKind, _jobject*, bool)+91) (BuildId: 8bb3225e7c408f2ca23abac3db0417f2)
           dotnet#8 pc 00000000003de1e5  /apex/com.android.runtime/lib64/libart.so (art::(anonymous namespace)::ScopedCheck::CheckPossibleHeapValue(art::ScopedObjectAccess&, char, art::(anonymous namespace)::JniValueType)+485) (BuildId: 8bb3225e7c408f2ca23abac3db0417f2)
           dotnet#9 pc 00000000003de2d4  /apex/com.android.runtime/lib64/libart.so (art::(anonymous namespace)::ScopedCheck::CheckPossibleHeapValue(art::ScopedObjectAccess&, char, art::(anonymous namespace)::JniValueType)+724) (BuildId: 8bb3225e7c408f2ca23abac3db0417f2)
           dotnet#10 pc 00000000003dd712  /apex/com.android.runtime/lib64/libart.so (art::(anonymous namespace)::ScopedCheck::Check(art::ScopedObjectAccess&, bool, char const*, art::(anonymous namespace)::JniValueType*)+690) (BuildId: 8bb3225e7c408f2ca23abac3db0417f2)
           dotnet#11 pc 00000000003e28c0  /apex/com.android.runtime/lib64/libart.so (art::(anonymous namespace)::CheckJNI::CheckCallArgs(art::ScopedObjectAccess&, art::(anonymous namespace)::ScopedCheck&, _JNIEnv*, _jobject*, _jclass*, _jmethodID*, art::InvokeType, art::(anonymous namespace)::VarArgs const*)+160) (BuildId: 8bb3225e7c408f2ca23abac3db0417f2)
           dotnet#12 pc 00000000003e1a9e  /apex/com.android.runtime/lib64/libart.so (art::(anonymous namespace)::CheckJNI::CallMethodV(char const*, _JNIEnv*, _jobject*, _jclass*, _jmethodID*, __va_list_tag*, art::Primitive::Type, art::InvokeType)+910) (BuildId: 8bb3225e7c408f2ca23abac3db0417f2)
           dotnet#13 pc 00000000003cf551  /apex/com.android.runtime/lib64/libart.so (art::(anonymous namespace)::CheckJNI::CallObjectMethod(_JNIEnv*, _jobject*, _jmethodID*, ...)+177) (BuildId: 8bb3225e7c408f2ca23abac3db0417f2)
           dotnet#14 pc 00000000000013ec  /data/app/Mono.Android.NET_Tests--O9vgexkYeCx3nX-AuvLTQ==/split_config.x86_64.apk!libreuse-threads.so (offset 0xa8d000) (BuildId: 562d86d81ebdd3bb6b7528e2a9235ff84827294e)
           dotnet#15 pc 0000000000100fce  /apex/com.android.runtime/lib64/bionic/libc.so (__pthread_start(void*)+30) (BuildId: a08a19770d6696739c847e29c3f5f650)
           dotnet#16 pc 0000000000098fe7  /apex/com.android.runtime/lib64/bionic/libc.so (__start_thread+55) (BuildId: a08a19770d6696739c847e29c3f5f650)
jonpryor added a commit that referenced this pull request Jan 26, 2024
Context: dotnet/java-interop@6b3637d

The story so far is that some of our unit tests are crashing, and
have been in one form or another since 4332ea0
(the bump to dotnet/java-interop@def5bc0).

The current crash, from the PR build for 7b46391:

	D monodroid-assembly: typemap: assembly 'Java.Interop-Tests' hasn't been loaded yet, attempting a full load
	W monodroid-assembly: typemap: failed to load managed assembly 'Java.Interop-Tests.dll'. No such file or directory
	E monodroid-assembly: typemap: unable to load assembly 'Java.Interop-Tests' when looking up managed type corresponding to Java type 'java/lang/Object'
	I monodroid-timing: [1/5] Typemap.java_to_managed: end, total time; elapsed: 0:0::260000
	W monodroid-assembly: typemap: called from
	W monodroid-assembly: at Java.Interop.TypeManager.GetJavaToManagedType(String )
	W monodroid-assembly:    at Java.Interop.TypeManager.CreateInstance(IntPtr , JniHandleOwnership , Type )
	W monodroid-assembly:    at Java.Lang.Object.GetObject(IntPtr , JniHandleOwnership , Type )
	W monodroid-assembly:    at Java.Lang.Object._GetObject[IIterator](IntPtr , JniHandleOwnership )
	W monodroid-assembly:    at Java.Lang.Object.GetObject[IIterator](IntPtr handle, JniHandleOwnership transfer)
	W monodroid-assembly:    at Android.Runtime.JavaSet.Iterator()
	W monodroid-assembly:    at Android.Runtime.JavaSet`1[[System.String, System.Private.CoreLib, Version=9.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].GetEnumerator()
	W monodroid-assembly:    at Xamarin.Android.UnitTests.TestInstrumentation`1[[Xamarin.Android.UnitTests.NUnit.NUnitTestRunner, TestRunner.NUnit.NET, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null]].ProcessArguments()
	W monodroid-assembly:    at Xamarin.Android.UnitTests.TestInstrumentation`1[[Xamarin.Android.UnitTests.NUnit.NUnitTestRunner, TestRunner.NUnit.NET, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null]].OnCreate(Bundle arguments)
	W monodroid-assembly:    at Android.App.Instrumentation.n_OnCreate_Landroid_os_Bundle_(IntPtr jnienv, IntPtr native__this, IntPtr native_arguments)
	W monodroid-assembly:    at Android.Runtime.JNINativeWrapper.Wrap_JniMarshal_PPL_V(_JniMarshal_PPL_V callback, IntPtr jnienv, IntPtr klazz, IntPtr p0)
	E droid.NET_Test: JNI ERROR (app bug): accessed stale Local 0x75  (index 7 in a table of size 7)
	F droid.NET_Test: java_vm_ext.cc:570] JNI DETECTED ERROR IN APPLICATION: use of deleted local reference 0x75

There are *three* "concerning" items here:

 1. typemaps are trying to load `Java.Interop-Tests`, and failing:

        typemap: failed to load managed assembly 'Java.Interop-Tests.dll'. No such file or directory

    @grendello is looking into this.

 2. The binding for `java/lang/Object` is coming from
    Java.Interop-Tests, not Mono.Android (?!)

        typemap: unable to load assembly 'Java.Interop-Tests' when looking up managed type corresponding to Java type 'java/lang/Object'

    dotnet/java-interop#1181 has a fix for this, and we're not
    applying the fix yet because we believe that it will hide (1).

 3. The JNI error, which crashes the process:

        F droid.NET_Test: java_vm_ext.cc:570] JNI DETECTED ERROR IN APPLICATION: use of deleted local reference 0x75
        F droid.NET_Test: java_vm_ext.cc:570]     from void crc643df67da7b13bb6b1.TestInstrumentation_1.n_onCreate(android.os.Bundle)
        F droid.NET_Test: runtime.cc:630] Runtime aborting...
        F droid.NET_Test: runtime.cc:630] Dumping all threads without mutator lock held
        F droid.NET_Test: runtime.cc:630] All threads:
        F droid.NET_Test: runtime.cc:630] DALVIK THREADS (14):
        F droid.NET_Test: runtime.cc:630] "main" prio=5 tid=1 Runnable
        F droid.NET_Test: runtime.cc:630]   | group="" sCount=0 dsCount=0 flags=0 obj=0x729e9d98 self=0x7567e0f51000
        F droid.NET_Test: runtime.cc:630]   | sysTid=9143 nice=0 cgrp=default sched=0/0 handle=0x7567e14daed8
        F droid.NET_Test: runtime.cc:630]   | state=R schedstat=( 1270418000 334229000 139 ) utm=16 stm=110 core=0 HZ=100
        F droid.NET_Test: runtime.cc:630]   | stack=0x7ffcbb3e4000-0x7ffcbb3e6000 stackSize=8192KB
        F droid.NET_Test: runtime.cc:630]   | held mutexes= "abort lock" "mutator lock"(shared held)
        F droid.NET_Test: runtime.cc:630]   native: #00 pc 000000000048df4e  /apex/com.android.runtime/lib64/libart.so (art::DumpNativeStack(std::__1::basic_ostream<char, std::__1::char_traits<char>>&, int, BacktraceMap*, char const*, art::ArtMethod*, void*, bool)+126)
        F droid.NET_Test: runtime.cc:630]   native: #1 pc 00000000005a77c3  /apex/com.android.runtime/lib64/libart.so (art::Thread::DumpStack(std::__1::basic_ostream<char, std::__1::char_traits<char>>&, bool, BacktraceMap*, bool) const+675)
        F droid.NET_Test: runtime.cc:630]   native: #2 pc 00000000005c49cb  /apex/com.android.runtime/lib64/libart.so (art::DumpCheckpoint::Run(art::Thread*)+859)
        F droid.NET_Test: runtime.cc:630]   native: #3 pc 00000000005bcf28  /apex/com.android.runtime/lib64/libart.so (art::ThreadList::RunCheckpoint(art::Closure*, art::Closure*)+456)
        F droid.NET_Test: runtime.cc:630]   native: #4 pc 00000000005bc2e1  /apex/com.android.runtime/lib64/libart.so (art::ThreadList::Dump(std::__1::basic_ostream<char, std::__1::char_traits<char>>&, bool)+1601)
        F droid.NET_Test: runtime.cc:630]   native: #5 pc 0000000000552eb9  /apex/com.android.runtime/lib64/libart.so (art::Runtime::Abort(char const*)+1529)
        F droid.NET_Test: runtime.cc:630]   native: #6 pc 000000000000c873  /system/lib64/libbase.so (android::base::LogMessage::~LogMessage()+611)
        F droid.NET_Test: runtime.cc:630]   native: #7 pc 00000000003ede84  /apex/com.android.runtime/lib64/libart.so (art::JavaVMExt::JniAbort(char const*, char const*)+1604)
        F droid.NET_Test: runtime.cc:630]   native: #8 pc 00000000003ee18a  /apex/com.android.runtime/lib64/libart.so (art::JavaVMExt::JniAbortF(char const*, char const*, ...)+234)
        F droid.NET_Test: runtime.cc:630]   native: #9 pc 00000000005adf31  /apex/com.android.runtime/lib64/libart.so (art::Thread::DecodeJObject(_jobject*) const+785)
        F droid.NET_Test: runtime.cc:630]   native: #10 pc 00000000003def9b  /apex/com.android.runtime/lib64/libart.so (art::(anonymous namespace)::ScopedCheck::CheckInstance(art::ScopedObjectAccess&, art::(anonymous namespace)::ScopedCheck::InstanceKind, _jobject*, bool)+91)
        F droid.NET_Test: runtime.cc:630]   native: #11 pc 00000000003de205  /apex/com.android.runtime/lib64/libart.so (art::(anonymous namespace)::ScopedCheck::CheckPossibleHeapValue(art::ScopedObjectAccess&, char, art::(anonymous namespace)::JniValueType)+485)
        F droid.NET_Test: runtime.cc:630]   native: #12 pc 00000000003dd732  /apex/com.android.runtime/lib64/libart.so (art::(anonymous namespace)::ScopedCheck::Check(art::ScopedObjectAccess&, bool, char const*, art::(anonymous namespace)::JniValueType*)+690)
        F droid.NET_Test: runtime.cc:630]   native: #13 pc 00000000003ce865  /apex/com.android.runtime/lib64/libart.so (art::(anonymous namespace)::CheckJNI::GetObjectClass(_JNIEnv*, _jobject*)+837)
        F droid.NET_Test: runtime.cc:630]   native: #14 pc 0000000000017196  /data/app/Mono.Android.NET_Tests-LUUW792fOvX745oAS4jeRQ==/split_config.x86_64.apk (offset 331000) (???)
        F droid.NET_Test: runtime.cc:630]   at crc643df67da7b13bb6b1.TestInstrumentation_1.n_onCreate(Native method)

As per `native #13`, we're (somehow) passing an invalid JNI reference
to `JNIEnv::GetObjectClass()`.

***HOW*** are we passing an invalid JNI reference to
`JNIEnv::GetObjectClass()`?

Attempt to investigate (3) further, by:

 1. Reviewing all calls to `JNIEnv::GetObjectClass()` within this
    repo to see if we could potentially be passing an invalid value.
    The "most obvious" candidate is `TypeManager.CreateInstance()`,
    which calls `JNIEnv::GetObjectClass()` in a loop.

    I'm still not sure if that could possibly be the cause, but
    Just In Case™…

    "Cleanup" some C++ code so that calls to
    `JNIEnv::DeleteLocalReference()` are closer to the
    `JNIEnv::GetObjectClass()` calls.

 2. Update
    `tests/Mono.Android-Tests/Runtime-Microsoft.Android.Sdk/Mono.Android.NET-Tests.csproj`
    to add an `@(AndroidEnvironment)` item, which sets
    `debug.mono.log=gref+,lref+`.  This will enable LREF and GREF
    logging within `adb logcat`, which *may* allow us to track down
    where "local reference 0x75" came from.
@github-actions github-actions bot locked and limited conversation to collaborators Feb 2, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants