Skip to content

Commit

Permalink
[generator] Use GC.KeepAlive for reference type method parameters. (#725
Browse files Browse the repository at this point in the history
)

Fixes: #719

@brendanzagaeski has been investigating a Xamarin.Android app crash:

	JNI DETECTED ERROR IN APPLICATION: use of deleted global reference 0x3d86
	    from android.view.View crc64720bb2db43a66fe9.FragmentContainer.n_onCreateView(android.view.LayoutInflater, android.view.ViewGroup, android.os.Bundle)
	…
	  at crc64720bb2db43a66fe9.FragmentContainer.n_onCreateView(Native method)
	  at crc64720bb2db43a66fe9.FragmentContainer.onCreateView(FragmentContainer.java:41)

This had been a head-scratcher, but we had a GREF log, so all should
be clear, right?

	09-10 17:56:48.280 10123 10123 I monodroid-gref: +g+ grefc 1141 gwrefc 0 obj-handle 0x79/I -> new-handle 0x3d86/G from thread '(null)'(1)
	09-10 17:56:48.294 10123 10123 I monodroid-gref: +w+ grefc 1140 gwrefc 2 obj-handle 0x3d86/G -> new-handle 0x1e3/W from thread 'finalizer'(10123)
	09-10 17:56:48.294 10123 10123 I monodroid-gref: -g- grefc 1139 gwrefc 2 handle 0x3d86/G from thread 'finalizer'(10123)

The GREF log *wasn't* immediately clear: sure, the GREF was turned
into a Weak-GREF, and the Weak-GREF was then collected, but none of
this explained *how* were were using this deleted GREF.

(We were at this state of affairs for months: we "know" we're using
a deleted GREF, but we don't know *how* or *why*.  It was a very hard
to hit bug.)

Eventually we had a ["that's funny"][0] event: *sure*, the GREF is
deleted, but:

 1. Why is it being deleted by the finalizer?
 2. …14ms *after construction*?

A [Garbage Collection][1] refresher may be in order, but in short:

 1. All `Java.Lang.Object` subclasses are "bridged" objects.

 2. During a GC, all "collectable" bridged objects are gathered.
    A collectable object is one in which nothing in the managed
    GC references the object.

 3. Once the GC is complete, all gathered collectable bridged objects
    are passed to a "cross references" callback.

    The callback is called *outside* the "scope" of a GC; "the world"
    is *not* frozen, other threads may be executing.

 4. The "cross references" callback is the
    `MonoGCBridgeCallbacks::cross_references` field provided provided
    to [`mono_gc_register_bridge_callbacks()`][2].
    
    In a Xamarin.Android app, the "cross references" callback will
    "toggle" a JNI Global Reference to a JNI Weak Global Reference,
    invoke a Java-side GC, and then try to obtain a
    JNI Global Reference from the JNI Weak Global Reference.
    If a non-`NULL` pointer is returned, the bridged object is kept
    alive.  If a `NULL` pointer is returned, the bridged object will
    be considered dead, and will be added to the Finalization Queue
    (as `Java.Lang.Object` has a finalizer).

Thus, it seemed "funny" that within 14ms an instance was created,
GC'd, and determined to be garbage, *especially* when we *knew* that
this instance was being passed to Java, which we expected to retain
the instance.  (Yet wasn't…?)

After much banging of heads, and the yeoman's work of creating a
simplified and consistently reproducible test case, we *think* we
know the cause of the crash.

Consider our normal Managed-to-Java marshaling code, e.g.

	partial class MaterialButton  {
	    public unsafe MaterialButton (global::Android.Content.Context context)
	    	: base (IntPtr.Zero, JniHandleOwnership.DoNotTransfer)
	    {
	        const string __id = "(Landroid/content/Context;)V";

	        if (((global::Java.Lang.Object) this).Handle != IntPtr.Zero)
	            return;

	        try {
	            JniArgumentValue* __args = stackalloc JniArgumentValue [1];
	/* 1 */     __args [0] = new JniArgumentValue ((context == null) ? IntPtr.Zero : ((global::Java.Lang.Object) context).Handle);
	/* 2 */     var __r = _members.InstanceMethods.StartCreateInstance (__id, ((object) this).GetType (), __args);
	            SetHandle (__r.Handle, JniHandleOwnership.TransferLocalRef);
	/* 3 */     _members.InstanceMethods.FinishCreateInstance (__id, this, __args);
	        } finally {
	        }
	    }
	}

At (1), `context.Handle` is -- a JNI GREF -- is stored into a
`JniArgumentValue* __args` value, and at (3) `__args` is passed into
JNI, which will presumably "Do Something" with that handle.

However, nothing ties `context.Handle` to `context`, so from
(2) onward, `context` *may* be eligible for garbage collection.

See also Chris Brumme's [Lifetime, GC.KeepAlive, handle recycling][3]
blog article.  It's about .NET Framework, but the same fundamental
multithreading concepts apply.

The fix is to *ensure* that `context` is kept alive
*for as long as* `context.Handle` will be used, i.e. across the
JNI `_members.InstanceMethods.FinishCreateInstance()` call:

	partial class MaterialButton  {
	    public unsafe MaterialButton (global::Android.Content.Context context)
	    	: base (IntPtr.Zero, JniHandleOwnership.DoNotTransfer)
	    {
	        const string __id = "(Landroid/content/Context;)V";

	        if (((global::Java.Lang.Object) this).Handle != IntPtr.Zero)
	            return;

	        try {
	            JniArgumentValue* __args = stackalloc JniArgumentValue [1];
	/* 1 */     __args [0] = new JniArgumentValue ((context == null) ? IntPtr.Zero : ((global::Java.Lang.Object) context).Handle);
	/* 2 */     var __r = _members.InstanceMethods.StartCreateInstance (__id, ((object) this).GetType (), __args);
	            SetHandle (__r.Handle, JniHandleOwnership.TransferLocalRef);
	/* 3 */     _members.InstanceMethods.FinishCreateInstance (__id, this, __args);
	        } finally {
	/* 4 */     global::System.GC.KeepAlive (context);
	        }
	    }
	}

This should prevent e.g. `context` from being prematurely GC'd,
which in turn should prevent the `JNI DETECTED ERROR` message.

Update `tools/generator` to emit `GC.KeepAlive()` statements for
every parameter type which isn't a value type (`enum`, `int`, `string`,
etc.).  `string` is considered a value type because we always send a
"deep copy" of the string contents, so it won't matter if the `string`
instance is GC'd immediately.

[0]: https://quoteinvestigator.com/2015/03/02/eureka-funny/
[1]: https://docs.microsoft.com/xamarin/android/internals/garbage-collection
[2]: http://docs.go-mono.com/?link=xhtml%3adeploy%2fmono-api-gc.html
[3]: https://docs.microsoft.com/archive/blogs/cbrumme/lifetime-gc-keepalive-handle-recycling
  • Loading branch information
jpobst authored Sep 24, 2020
1 parent 5136ef9 commit 1f21f38
Show file tree
Hide file tree
Showing 24 changed files with 74 additions and 0 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,7 @@ public partial class MyClass {
JNIEnv.CopyArray (native_value, value);
JNIEnv.DeleteLocalRef (native_value);
}
global::System.GC.KeepAlive (value);
}
}

Expand All @@ -46,6 +47,7 @@ public partial class MyClass {
JNIEnv.CopyArray (native_value, value);
JNIEnv.DeleteLocalRef (native_value);
}
global::System.GC.KeepAlive (value);
}
}

Expand All @@ -65,6 +67,7 @@ public partial class MyClass {
JNIEnv.CopyArray (native_value, value);
JNIEnv.DeleteLocalRef (native_value);
}
global::System.GC.KeepAlive (value);
}
}

Expand All @@ -84,6 +87,7 @@ public partial class MyClass {
JNIEnv.CopyArray (native_value, value);
JNIEnv.DeleteLocalRef (native_value);
}
global::System.GC.KeepAlive (value);
}
}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,7 @@ public partial class MyClass {
JNIEnv.CopyArray (native_value, value);
JNIEnv.DeleteLocalRef (native_value);
}
global::System.GC.KeepAlive (value);
}
}
}
Expand Down Expand Up @@ -65,6 +66,7 @@ public partial class MyClass {
JNIEnv.CopyArray (native_value, value);
JNIEnv.DeleteLocalRef (native_value);
}
global::System.GC.KeepAlive (value);
}
}
}
Expand Down Expand Up @@ -94,6 +96,7 @@ public partial class MyClass {
JNIEnv.CopyArray (native_value, value);
JNIEnv.DeleteLocalRef (native_value);
}
global::System.GC.KeepAlive (value);
}
}
}
Expand Down Expand Up @@ -123,6 +126,7 @@ public partial class MyClass {
JNIEnv.CopyArray (native_value, value);
JNIEnv.DeleteLocalRef (native_value);
}
global::System.GC.KeepAlive (value);
}
}
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -89,6 +89,7 @@ static void n_SetAdapter_Lxamarin_test_SpinnerAdapter_ (IntPtr jnienv, IntPtr na
__args [0] = new JniArgumentValue ((value == null) ? IntPtr.Zero : ((global::Java.Lang.Object) value).Handle);
_members.InstanceMethods.InvokeVirtualVoidMethod (__id, this, __args);
} finally {
global::System.GC.KeepAlive (value);
}
}
}
Expand Down Expand Up @@ -137,6 +138,7 @@ public AbsSpinnerInvoker (IntPtr handle, JniHandleOwnership transfer) : base (ha
_members.InstanceMethods.InvokeAbstractVoidMethod (__id, this, __args);
} finally {
JNIEnv.DeleteLocalRef (native_value);
global::System.GC.KeepAlive (value);
}
}
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -124,6 +124,7 @@ public AdapterViewInvoker (IntPtr handle, JniHandleOwnership transfer) : base (h
_members.InstanceMethods.InvokeAbstractVoidMethod (__id, this, __args);
} finally {
JNIEnv.DeleteLocalRef (native_value);
global::System.GC.KeepAlive (value);
}
}
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,7 @@ public unsafe SpannableString (global::Java.Lang.ICharSequence source) : base (I
_members.InstanceMethods.FinishCreateInstance (__id, this, __args);
} finally {
JNIEnv.DeleteLocalRef (native_source);
global::System.GC.KeepAlive (source);
}
}

Expand All @@ -74,6 +75,7 @@ public unsafe SpannableString (string source) : base (IntPtr.Zero, JniHandleOwne
_members.InstanceMethods.FinishCreateInstance (__id, this, __args);
} finally {
JNIEnv.DeleteLocalRef (native_source);
global::System.GC.KeepAlive (source);
}
}

Expand Down Expand Up @@ -106,6 +108,7 @@ static int n_GetSpanFlags_Ljava_lang_Object_ (IntPtr jnienv, IntPtr native__this
var __rm = _members.InstanceMethods.InvokeVirtualInt32Method (__id, this, __args);
return (global::Android.Text.SpanTypes) __rm;
} finally {
global::System.GC.KeepAlive (what);
}
}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -66,6 +66,7 @@ static int n_GetSpanFlags_Ljava_lang_Object_ (IntPtr jnienv, IntPtr native__this
var __rm = _members.InstanceMethods.InvokeVirtualInt32Method (__id, this, __args);
return (global::Android.Text.SpanTypes) __rm;
} finally {
global::System.GC.KeepAlive (p0);
}
}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -181,6 +181,7 @@ public virtual unsafe void SetOnClickListener (global::Android.Views.View.IOnCli
__args [0] = new JniArgumentValue ((l == null) ? IntPtr.Zero : ((global::Java.Lang.Object) l).Handle);
_members.InstanceMethods.InvokeVirtualVoidMethod (__id, this, __args);
} finally {
global::System.GC.KeepAlive (l);
}
}

Expand Down Expand Up @@ -211,6 +212,7 @@ public virtual unsafe void SetOn123Listener (global::Android.Views.View.IOnClick
__args [0] = new JniArgumentValue ((l == null) ? IntPtr.Zero : ((global::Java.Lang.Object) l).Handle);
_members.InstanceMethods.InvokeVirtualVoidMethod (__id, this, __args);
} finally {
global::System.GC.KeepAlive (l);
}
}

Expand Down Expand Up @@ -243,6 +245,7 @@ public virtual unsafe void AddTouchables (global::System.Collections.Generic.ILi
_members.InstanceMethods.InvokeVirtualVoidMethod (__id, this, __args);
} finally {
JNIEnv.DeleteLocalRef (native_views);
global::System.GC.KeepAlive (views);
}
}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -63,6 +63,7 @@ public unsafe void SetOnEventListener (global::Com.Google.Android.Exoplayer.Drm.
__args [0] = new JniArgumentValue ((p0 == null) ? IntPtr.Zero : ((global::Java.Lang.Object) p0).Handle);
_members.InstanceMethods.InvokeAbstractVoidMethod (__id, this, __args);
} finally {
global::System.GC.KeepAlive (p0);
}
}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -202,6 +202,7 @@ public unsafe InstanceInner (global::Xamarin.Test.NotificationCompatBase __self)
SetHandle (__r.Handle, JniHandleOwnership.TransferLocalRef);
_members.InstanceMethods.FinishCreateInstance (__id, this, __args);
} finally {
global::System.GC.KeepAlive (__self);
}
}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,7 @@ public unsafe SomeObject (global::Java.Lang.Class c) : base (IntPtr.Zero, JniHan
SetHandle (__r.Handle, JniHandleOwnership.TransferLocalRef);
_members.InstanceMethods.FinishCreateInstance (__id, this, __args);
} finally {
global::System.GC.KeepAlive (c);
}
}

Expand Down Expand Up @@ -114,6 +115,8 @@ static int n_Handle_Ljava_lang_Object_Ljava_lang_Throwable_ (IntPtr jnienv, IntP
var __rm = _members.InstanceMethods.InvokeVirtualInt32Method (__id, this, __args);
return __rm;
} finally {
global::System.GC.KeepAlive (o);
global::System.GC.KeepAlive (t);
}
}

Expand Down Expand Up @@ -260,6 +263,7 @@ public virtual unsafe void VoidMethodWithParams (string astring, int anint, glob
_members.InstanceMethods.InvokeVirtualVoidMethod (__id, this, __args);
} finally {
JNIEnv.DeleteLocalRef (native_astring);
global::System.GC.KeepAlive (anObject);
}
}

Expand Down Expand Up @@ -323,6 +327,7 @@ public virtual unsafe void ArrayListTest (global::System.Collections.Generic.ILi
_members.InstanceMethods.InvokeVirtualVoidMethod (__id, this, __args);
} finally {
JNIEnv.DeleteLocalRef (native_p0);
global::System.GC.KeepAlive (p0);
}
}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -230,6 +230,7 @@ public override unsafe int SomeInteger {
__args [0] = new JniArgumentValue ((value == null) ? IntPtr.Zero : ((global::Java.Lang.Object) value).Handle);
_members.InstanceMethods.InvokeAbstractVoidMethod (__id, this, __args);
} finally {
global::System.GC.KeepAlive (value);
}
}
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -66,6 +66,7 @@ public virtual unsafe void SetA (global::Java.Lang.Object adapter)
_members.InstanceMethods.InvokeVirtualVoidMethod (__id, this, __args);
} finally {
JNIEnv.DeleteLocalRef (native_adapter);
global::System.GC.KeepAlive (adapter);
}
}

Expand Down Expand Up @@ -98,6 +99,7 @@ public virtual unsafe void ListTest (global::System.Collections.Generic.IList<gl
_members.InstanceMethods.InvokeVirtualVoidMethod (__id, this, __args);
} finally {
JNIEnv.DeleteLocalRef (native_p0);
global::System.GC.KeepAlive (p0);
}
}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -108,6 +108,7 @@ public static unsafe void SetSomeObject (global::Java.Lang.Object newvalue)
__args [0] = new JniArgumentValue ((newvalue == null) ? IntPtr.Zero : ((global::Java.Lang.Object) newvalue).Handle);
_members.StaticMethods.InvokeVoidMethod (__id, __args);
} finally {
global::System.GC.KeepAlive (newvalue);
}
}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,7 @@ public unsafe FilterOutputStream (global::System.IO.Stream @out) : base (IntPtr.
_members.InstanceMethods.FinishCreateInstance (__id, this, __args);
} finally {
JNIEnv.DeleteLocalRef (native__out);
global::System.GC.KeepAlive (@out);
}
}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -221,6 +221,7 @@ public virtual unsafe int Read (byte[] buffer)
JNIEnv.CopyArray (native_buffer, buffer);
JNIEnv.DeleteLocalRef (native_buffer);
}
global::System.GC.KeepAlive (buffer);
}
}

Expand Down Expand Up @@ -262,6 +263,7 @@ public virtual unsafe int Read (byte[] buffer, int byteOffset, int byteCount)
JNIEnv.CopyArray (native_buffer, buffer);
JNIEnv.DeleteLocalRef (native_buffer);
}
global::System.GC.KeepAlive (buffer);
}
}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -141,6 +141,7 @@ public virtual unsafe void Write (byte[] buffer)
JNIEnv.CopyArray (native_buffer, buffer);
JNIEnv.DeleteLocalRef (native_buffer);
}
global::System.GC.KeepAlive (buffer);
}
}

Expand Down Expand Up @@ -180,6 +181,7 @@ public virtual unsafe void Write (byte[] buffer, int offset, int count)
JNIEnv.CopyArray (native_buffer, buffer);
JNIEnv.DeleteLocalRef (native_buffer);
}
global::System.GC.KeepAlive (buffer);
}
}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -87,6 +87,7 @@ public virtual unsafe void SetObject (byte[] value)
JNIEnv.CopyArray (native_value, value);
JNIEnv.DeleteLocalRef (native_value);
}
global::System.GC.KeepAlive (value);
}
}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -106,6 +106,7 @@ static void n_SetObject_Ljava_lang_Object_ (IntPtr jnienv, IntPtr native__this,
__args [0] = new JniArgumentValue ((value == null) ? IntPtr.Zero : ((global::Java.Lang.Object) value).Handle);
_members.InstanceMethods.InvokeVirtualVoidMethod (__id, this, __args);
} finally {
global::System.GC.KeepAlive (value);
}
}
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -87,6 +87,7 @@ public virtual unsafe void SetObject (string[] value)
JNIEnv.CopyArray (native_value, value);
JNIEnv.DeleteLocalRef (native_value);
}
global::System.GC.KeepAlive (value);
}
}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -188,6 +188,7 @@ public override unsafe int GetSpanFlags (global::Java.Lang.Object tag)
var __rm = _members.InstanceMethods.InvokeAbstractInt32Method (__id, this, __args);
return __rm;
} finally {
global::System.GC.KeepAlive (tag);
}
}

Expand All @@ -203,6 +204,7 @@ public override unsafe void Append (global::Java.Lang.ICharSequence value)
_members.InstanceMethods.InvokeAbstractVoidMethod (__id, this, __args);
} finally {
JNIEnv.DeleteLocalRef (native_value);
global::System.GC.KeepAlive (value);
}
}

Expand All @@ -219,6 +221,7 @@ public override unsafe void Append (global::Java.Lang.ICharSequence value)
return global::Java.Lang.Object.GetObject<Java.Lang.ICharSequence> (__rm.Handle, JniHandleOwnership.TransferLocalRef);
} finally {
JNIEnv.DeleteLocalRef (native_value);
global::System.GC.KeepAlive (value);
}
}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,7 @@ public unsafe int CompareTo (global::Java.Lang.Object o)
return __rm;
} finally {
JNIEnv.DeleteLocalRef (native_o);
global::System.GC.KeepAlive (o);
}
}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -276,6 +276,32 @@ public bool Validate (CodeGenerationOptions opt, GenericParameterDefinitionList
return true;
}

public bool ShouldGenerateKeepAlive ()
{
if (Symbol.IsEnum)
return false;

return Type switch {
"bool" => false,
"sbyte" => false,
"char" => false,
"double" => false,
"float" => false,
"int" => false,
"long" => false,
"short" => false,
"uint" => false,
"ushort" => false,
"ulong" => false,
"byte" => false,
"ubyte" => false, // Not a C# type, but we will see it from Kotlin unsigned types support
"string" => false,
"java.lang.String" => false,
"Android.Graphics.Color" => false,
_ => true
};
}

public ISymbol Symbol => sym;
}
}
4 changes: 4 additions & 0 deletions tools/generator/SourceWriters/BoundConstructor.cs
Original file line number Diff line number Diff line change
Expand Up @@ -76,6 +76,10 @@ protected override void WriteBody (CodeWriter writer)
var call_cleanup = constructor.Parameters.GetCallCleanup (opt);
foreach (string cleanup in call_cleanup)
writer.WriteLine (cleanup);

foreach (var p in constructor.Parameters.Where (para => para.ShouldGenerateKeepAlive ()))
writer.WriteLine ($"global::System.GC.KeepAlive ({opt.GetSafeIdentifier (p.Name)});");

writer.Unindent ();

writer.WriteLine ("}");
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -264,6 +264,9 @@ public static void AddMethodBody (List<string> body, Method method, CodeGenerati
foreach (string cleanup in method.Parameters.GetCallCleanup (opt))
body.Add ("\t" + cleanup);

foreach (var p in method.Parameters.Where (para => para.ShouldGenerateKeepAlive ()))
body.Add ($"\tglobal::System.GC.KeepAlive ({opt.GetSafeIdentifier (p.Name)});");

body.Add ("}");
}

Expand Down

0 comments on commit 1f21f38

Please sign in to comment.