-
Notifications
You must be signed in to change notification settings - Fork 53
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Avoid non-blittable types in native callback methods #1027
Labels
enhancement
Proposed change to current functionality
generator
Issues binding a Java library (generator, class-parse, etc.)
Milestone
Comments
grendello
added a commit
to dotnet/android
that referenced
this issue
Sep 7, 2022
Context: 903ba37 Context: e1af958 Context: dotnet/java-interop#1027 Commit 903ba37 mentioned a TODO: > Update/rewrite infrastructure to focus on implementing the runtime > side of marshal methods, making it possible to actually run > applications which use marshal methods. Implement the necessary runtime elements to enable running of applications with marshal methods. It is now possible, if LLVM marshal methods are enabled/`ENABLE_MARSHAL_METHODS` is defined, to run both plain .NET SDK for Android and MAUI apps. Update `src/Microsoft.Android.Sdk.ILLink/PreserveLists/System.Runtime.InteropServices.xml` so that `System.Runtime.InteropServices.UnmanagedCallersOnlyAttribute` is always preserved, as it is required by LLVM Marshal Methods. The `[UnmanagedCallersOnly]` attribute used by marshal methods requires that the invoked method have [blittable types][0] for the method return type and all parameter types. Unfortunately, `bool` is *not* a blittable type. Implement generation of wrapper methods which replace `bool` with `byte` and convert the values appropriately before calling the actual target method. In a "hello world" MAUI test app there are 133 such methods (out of around 180 total). Implement code that enables us to show error messages with the proper assembly, class and method names on failure to look up or obtain pointer to native callback methods. TODO: * Process *all* assemblies, including `Mono.Android.dll`, for Java Callable Wrapper generation. This is necessary so that we can find and emit LLVM marshal methods for types defined within `Mono.Android.dll`. * Remove the `ENABLE_MARSHAL_METHODS` define, and enable LLVM marshal methods for everyone. * Update `<GenerateJavaStubs/>` to rewrite all assemblies for all Supported ABIs. Currently, we don't support `Java.Lang.Object` & `Java.Lang.Throwable` subclasses being located in per-ABI assemblies. * How do `Java_…` native functions interact with `JNIEnv::RegisterNatives()`? #7285 (comment) * *Can* JNI `native` methods contain "non-printable" characters such as `\n`, or "non-representable in ELF symbols" characters such as `-` (e.g. Kotlin mangled methods)? #7285 (comment) * Cleanup, cleanup, cleanup [0]: https://docs.microsoft.com/en-us/dotnet/framework/interop/blittable-and-non-blittable-types
Example from [global::System.Diagnostics.DebuggerDisableUserUnhandledExceptions]
static bool n_IsEmpty (IntPtr jnienv, IntPtr native__this)
{
if (!global::Java.Interop.JniEnvironment.BeginMarshalMethod (jnienv, out var __envp, out var __r))
return default;
try {
var __this = global::Java.Lang.Object.GetObject<Java.Util.BitSet> (jnienv, native__this, JniHandleOwnership.DoNotTransfer)!;
return __this.IsEmpty;
} catch (global::System.Exception __e) {
__r.OnUserUnhandledException (ref __envp, __e);
return default;
} finally {
global::Java.Interop.JniEnvironment.EndMarshalMethod (ref __envp);
}
} [global::System.Diagnostics.DebuggerDisableUserUnhandledExceptions]
static byte n_IsEmpty (IntPtr jnienv, IntPtr native__this) // changed
{
if (!global::Java.Interop.JniEnvironment.BeginMarshalMethod (jnienv, out var __envp, out var __r))
return default;
try {
var __this = global::Java.Lang.Object.GetObject<Java.Util.BitSet> (jnienv, native__this, JniHandleOwnership.DoNotTransfer)!;
return __this.IsEmpty ? 1 : 0 // changed
} catch (global::System.Exception __e) {
__r.OnUserUnhandledException (ref __envp, __e);
return default;
} finally {
global::Java.Interop.JniEnvironment.EndMarshalMethod (ref __envp);
}
} |
This was referenced Jan 29, 2025
jonpryor
pushed a commit
to dotnet/android
that referenced
this issue
Feb 12, 2025
Changes: dotnet/java-interop@6bc87e8...57f7bc8 * dotnet/java-interop@57f7bc84: [generator] Avoid non-blittable types in native callback methods (dotnet/java-interop#1296) * dotnet/java-interop@9369d4fd: [tests] fix `NU1510` warning as error (dotnet/java-interop#1306) Context: dotnet/java-interop@57f7bc8 Context: dotnet/java-interop#1027 dotnet/java-interop@57f7bc84 updated `generator --codegen-target=XAJavaInterop1` marshal methods to contain only blittable types in parameter and return types. Instead of `bool`, use `sbyte`; instead of `char`, use `ushort`. This change causes warnings when `MarshalMethodsClassifier` verifies that the marshal types match the native types: …/Xamarin.Android.Common.targets(1502,3): Method 'System.SByte Java.Lang.Object::n_Equals_Ljava_lang_Object_(System.IntPtr,System.IntPtr,System.IntPtr)' doesn't match native callback signature (invalid return type: expected 'System.Boolean', found 'System.SByte') …/Xamarin.Android.Common.targets(1502,3): Unable to find native callback method 'n_Equals_Ljava_lang_Object_' in type 'Java.Lang.Object', matching the 'System.Boolean Java.Lang.Object::Equals(Java.Lang.Object)' signature (jniName: 'equals') [Arch: X86_64; Assembly: obj/Release/android-x64/linked/Mono.Android.dll] …/Xamarin.Android.Common.targets(1502,3): Method 'System.Boolean Java.Lang.Object::Equals(Java.Lang.Object)' will be registered dynamically [Arch: X86_64; Assembly: obj/Release/android-x64/linked/Mono.Android.dll] …/Xamarin.Android.Common.targets(1502,3): Method 'System.SByte Java.Lang.Object::n_Equals_Ljava_lang_Object_(System.IntPtr,System.IntPtr,System.IntPtr)' doesn't match native callback signature (invalid return type: expected 'System.Boolean', found 'System.SByte') …/Xamarin.Android.Common.targets(1502,3): Unable to find native callback method 'n_Equals_Ljava_lang_Object_' in type 'Java.Lang.Object', matching the 'System.Boolean Java.Lang.Object::Equals(Java.Lang.Object)' signature (jniName: 'equals') [Arch: Arm64; Assembly: obj/Release/android-arm64/linked/Mono.Android.dll] …/Xamarin.Android.Common.targets(1502,3): Method 'System.Boolean Java.Lang.Object::Equals(Java.Lang.Object)' will be registered dynamically [Arch: Arm64; Assembly: obj/Release/android-arm64/linked/Mono.Android.dll] …/Xamarin.Android.Common.targets(1502,3): Type 'Android.Runtime.JavaObject, Mono.Android, Version=0.0.0.0, Culture=neutral, PublicKeyToken=84e04ff9cfb79065' will register some of its Java override methods dynamically. This may adversely affect runtime performance. See preceding warnings for names of dynamically registered methods. …/Xamarin.Android.Common.targets(1502,3): [Arm64] Number of methods in the project that will be registered dynamically: 1 …/Xamarin.Android.Common.targets(1502,3): [X86_64] Number of methods in the project that will be registered dynamically: 1 These warnings occur because the marshal method signature now differs from the public API signature. Consider: namespace Java.Lang; partial class Object { static sbyte n_Equals_Ljava_lang_Object_ (IntPtr jnienv, IntPtr native__this, IntPtr native_obj) => … [Register ("equals", "(Ljava/lang/Object;)Z", "GetEquals_Ljava_lang_Object_Handler")] public virtual unsafe bool Equals (Java.Lang.Object? obj) => … } `MarshalMethodsClassifier` uses the 3rd `connector` parameter to `RegisterAttribute` to determine the name of the marshal method -- string processing `GetEquals_Ljava_lang_Object_Handler` to obtain `n_Equals_Ljava_lang_Object_` -- and then checks that the parameter and return types match between the two methods. Now that we're emitting blittable types, they *don't* match, thus the warning. Fix this by treating our blittable types as equivalent to their native types in `MarshalMethodsClassifier`. Additionally, `Mono.Android` has "built-in delegates" that marshal `bool` types, e.g. `_JniMarshal_PP_Z`. Because `generator` no longer creates these delegates -- it will now create `_JniMarshal_PP_B` instead -- we need to add them manually.
jonathanpeppers
pushed a commit
to dotnet/android
that referenced
this issue
Feb 12, 2025
Changes: dotnet/java-interop@6bc87e8...57f7bc8 * dotnet/java-interop@57f7bc84: [generator] Avoid non-blittable types in native callback methods (dotnet/java-interop#1296) * dotnet/java-interop@9369d4fd: [tests] fix `NU1510` warning as error (dotnet/java-interop#1306) Context: dotnet/java-interop@57f7bc8 Context: dotnet/java-interop#1027 dotnet/java-interop@57f7bc84 updated `generator --codegen-target=XAJavaInterop1` marshal methods to contain only blittable types in parameter and return types. Instead of `bool`, use `sbyte`; instead of `char`, use `ushort`. This change causes warnings when `MarshalMethodsClassifier` verifies that the marshal types match the native types: …/Xamarin.Android.Common.targets(1502,3): Method 'System.SByte Java.Lang.Object::n_Equals_Ljava_lang_Object_(System.IntPtr,System.IntPtr,System.IntPtr)' doesn't match native callback signature (invalid return type: expected 'System.Boolean', found 'System.SByte') …/Xamarin.Android.Common.targets(1502,3): Unable to find native callback method 'n_Equals_Ljava_lang_Object_' in type 'Java.Lang.Object', matching the 'System.Boolean Java.Lang.Object::Equals(Java.Lang.Object)' signature (jniName: 'equals') [Arch: X86_64; Assembly: obj/Release/android-x64/linked/Mono.Android.dll] …/Xamarin.Android.Common.targets(1502,3): Method 'System.Boolean Java.Lang.Object::Equals(Java.Lang.Object)' will be registered dynamically [Arch: X86_64; Assembly: obj/Release/android-x64/linked/Mono.Android.dll] …/Xamarin.Android.Common.targets(1502,3): Method 'System.SByte Java.Lang.Object::n_Equals_Ljava_lang_Object_(System.IntPtr,System.IntPtr,System.IntPtr)' doesn't match native callback signature (invalid return type: expected 'System.Boolean', found 'System.SByte') …/Xamarin.Android.Common.targets(1502,3): Unable to find native callback method 'n_Equals_Ljava_lang_Object_' in type 'Java.Lang.Object', matching the 'System.Boolean Java.Lang.Object::Equals(Java.Lang.Object)' signature (jniName: 'equals') [Arch: Arm64; Assembly: obj/Release/android-arm64/linked/Mono.Android.dll] …/Xamarin.Android.Common.targets(1502,3): Method 'System.Boolean Java.Lang.Object::Equals(Java.Lang.Object)' will be registered dynamically [Arch: Arm64; Assembly: obj/Release/android-arm64/linked/Mono.Android.dll] …/Xamarin.Android.Common.targets(1502,3): Type 'Android.Runtime.JavaObject, Mono.Android, Version=0.0.0.0, Culture=neutral, PublicKeyToken=84e04ff9cfb79065' will register some of its Java override methods dynamically. This may adversely affect runtime performance. See preceding warnings for names of dynamically registered methods. …/Xamarin.Android.Common.targets(1502,3): [Arm64] Number of methods in the project that will be registered dynamically: 1 …/Xamarin.Android.Common.targets(1502,3): [X86_64] Number of methods in the project that will be registered dynamically: 1 These warnings occur because the marshal method signature now differs from the public API signature. Consider: namespace Java.Lang; partial class Object { static sbyte n_Equals_Ljava_lang_Object_ (IntPtr jnienv, IntPtr native__this, IntPtr native_obj) => … [Register ("equals", "(Ljava/lang/Object;)Z", "GetEquals_Ljava_lang_Object_Handler")] public virtual unsafe bool Equals (Java.Lang.Object? obj) => … } `MarshalMethodsClassifier` uses the 3rd `connector` parameter to `RegisterAttribute` to determine the name of the marshal method -- string processing `GetEquals_Ljava_lang_Object_Handler` to obtain `n_Equals_Ljava_lang_Object_` -- and then checks that the parameter and return types match between the two methods. Now that we're emitting blittable types, they *don't* match, thus the warning. Fix this by treating our blittable types as equivalent to their native types in `MarshalMethodsClassifier`. Additionally, `Mono.Android` has "built-in delegates" that marshal `bool` types, e.g. `_JniMarshal_PP_Z`. Because `generator` no longer creates these delegates -- it will now create `_JniMarshal_PP_B` instead -- we need to add them manually.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Labels
enhancement
Proposed change to current functionality
generator
Issues binding a Java library (generator, class-parse, etc.)
While working on implementation of marshal methods (a way to replace the current native JNI method registration with
generated code) which take advantage of the
[UnmanagedCallersOnly]
attribute, I came across a problem that some ofour registered methods either return a
bool
or take a parameter which is abool
. The problem here is thatbool
is a non-blittable type, while all
[UnmanagedCallersOnly]
methods must use only blittable types (as they arecalled directly by the native code, bypassing marshaling).
Marshal methods convert the code output by the generator from:
to
And at run time, a pointer to
n_SetChildrenDrawingOrderEnabled_Z
is obtained using the Mono embedding APImono_method_get_unmanaged_callers_only_ftnptr
.However, when a non-blittable type is encountered, the function will return an error:
The problem is quite common in MAUI apps, since in a simple Hello World app, out of 183 marshal method candidates, 133 have a
bool
in their parameter list or as a return value.I have implemented a workaround for this issue (which we will keep in the future, to deal with 3rd party libraries that weren't regenerated using the new generator) but the real fix is to make the generator instead output code equivalent to:
or, for a method which returns a
bool
:The reason to choose
byte
is that JNI'sjboolean
type is defined as an unsigned 8-bit value and the reason to explicitly return1
or0
instead ofjust casting the managed
bool
value is that the boolean type in dotnet always has value of0
forfalse
, but can have-1
,1
or!= 0
fortrue
,depending on the VM or context and so it's safer to "downcast" that set to the
0
/1
values common in other languages (including Java, C and C++ aboutwhich we care)
In
Mono.Android
generated MCW code we currently have4824
native callback methods either returningbool
or with at least onebool
parameter.The generator should take into account other
System
namespace non-blittable types, not justbool
(within reason - only those that can potentially happen in bindings). The list can be found in this table,for reference copied below:
The text was updated successfully, but these errors were encountered: