Fix Android mono export with 2 or more cpu architectures fails

[TCROC]

Fixes: #98064

When 2 or more architectures are configured for support in an android export, jars per architecture are attempted to be exported. This is due to mono / dotnet including a jar per cpu architecture. However, this causes problems because jars are cpu agnostic and result in conflicts during the export process. Here are some images illustrating the issue:

And then when exporting we get:

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':mergeExtDexMonoDebug'.
> A failure occurred while executing com.android.build.gradle.internal.tasks.DexMergingTaskDelegate
   > There was a failure while executing work items
      > A failure occurred while executing com.android.build.gradle.internal.tasks.DexMergingWorkAction
         > com.android.builder.dexing.DexArchiveMergerException: Error while merging dex archives: 
           Learn how to resolve the issue at https://developer.android.com/studio/build/dependencies#duplicate_classes.
           Type net.dot.android.crypto.DotnetProxyTrustManager is defined multiple times: /home/tcroc/dev/BlockyBallOT/blockyball-godot/android/app/build/build/intermediates/external_file_lib_dex_archives/monoDebug/0_jetified-libSystem.Security.Cryptography.Native.Android.jar:classes.dex, /home/tcroc/dev/BlockyBallOT/blockyball-godot/android/app/build/build/intermediates/external_file_lib_dex_archives/monoDebug/2_jetified-libSystem.Security.Cryptography.Native.Android.jar:classes.dex

To fix it, I added a HashSet to ExportPlugin.cs that makes sure multiple jars of the same name are not included in the export.

[raulsntos]

It's fine for the exported APK to include the same .jar multiple times. As I understand it, .jar files can only be added to the APK when building the APK so the pre-built export template APK won't care about these .jar files that are added later, we already take care of adding the .jar file to the export template APK.

The problem you are having is likely exclusive to Gradle builds. The Gradle build likely includes these .jar files multiple times because ExportPlugin::add_shared_object adds the files to the android/build/libs directory and the Gradle build will take all the .jar files in there without de-duplicating them when building the APK.

[raulsntos]

The comment is incorrect, there's no issue with having the same .jar file multiple times in the .godot directory. And returning false will still include the .jar file multiple times in the exported APK (just under a different directory).

Also, is the android/build/libs directory cleared for each export? Otherwise, I imagine this will still fail if you export with a different set of architectures at different times. For example, exporting only for the x86_64 architecture, then exporting only for the arm64 architecture.

[TCROC]

The comment is incorrect, there's no issue with having the same .jar file multiple times in the .godot directory.

Ah. I didn't mean to insinuate the issue presides in the .godot directory. I was trying to say that the issue resides with exporting multiple jars of the same name from the .godot directory. I will reword this.

And returning false will still include the .jar file multiple times in the exported APK (just under a different directory).

Ok so I tested this theory and learned something interesting. When returning false, it does indeed only include it once. Here are some screenshots showing it to be so:

x86_64 missing:

arm64 present:

Which now brings about the question: Why the conflict? They should be under different folders just as you said (when returning true). And this is what I found:

The error is occurring at the intermediates stage under external_file_lib_dex_archives/monoDebug.

Where is the logic that puts jars in the intermediates folder? I bet this is where we need to check for duplicates. NOT in the ExportPlugins.cs.

Also, is the android/build/libs directory cleared for each export? Otherwise, I imagine this will still fail if you export with a different set of architectures at different times. For example, exporting only for the x86_64 architecture, then exporting only for the arm64 architecture.

I do this manually in my build script / pipeline.

[TCROC]

And here is the error confirming that it is at the intermediates folder:

Type net.dot.android.crypto.DotnetProxyTrustManager is defined multiple times: 
/home/tcroc/dev/BlockyBallOT/blockyball-godot/android/app/build/build/intermediates/external_file_lib_dex_archives/monoDebug/0_jetified-libSystem.Security.Cryptography.Native.Android.jar:classes.dex, 
/home/tcroc/dev/BlockyBallOT/blockyball-godot/android/app/build/build/intermediates/external_file_lib_dex_archives/monoDebug/2_jetified-libSystem.Security.Cryptography.Native.Android.jar:classes.dex

[TCROC]

@raulsntos I did a lot of research and tried many different things. Ultimately I almost had it right the first time. I had to move where it checked that the jar had already been exported so that the jar wasn't embedded inside the apk. I have since pushed the new code to this branch and updated the comment per your suggestion. Here is a summary of my findings:

1. gradle does not distinguish between abis for .jar files the same way it does for .so files. It simply tries to compile any and all jars into the apk. So if there are 2 jars with the same classes for 2 different abis, only 1 can actually be included. Otherwise we will get conflicts.
2. Microsoft distributes the jar in each nuget package for each abi. However, only one should be chosen. It does not matter which one we choose since they are identical. I verified this with android studio comparing the byte code:
   
   As can be seen, the only difference is in the MSFTSIG.RSA. And this does not seem to have an effect on the built apk. I tested the universal apk on both an x86_64 emulator and an arm64 physical device. It ran fine on both and the https requests were successful.
3. I don't think we need to distribute the https://github.com/godotengine/godot/blob/92e51fca7247c932f95a1662aefc28aca96e8de6/modules/mono/thirdparty/libSystem.Security.Cryptography.Native.Android.jar in the godot repo. This file is automatically pulled down in a nuget package. You can verify this by looking at your nuget cache:
   
   • I believe the msbuild logs will also confirm that it gets pulled from there, but I'm struggling to find those logs on my system currently.
4. Building on point 3, I don't believe we need these lines in our build.gradle for the same reason as point 3: https://github.com/godotengine/godot/blob/92e51fca7247c932f95a1662aefc28aca96e8de6/platform/android/java/app/build.gradle#L72C1-L76C6
5. If you agree with points 3 and 4, I can put in a separate PR to clean that up after this PR goes through. I recommenced keeping this PR simple as it is a simple fix for a major issue. The more changes we make, the more discussion it might provoke and delay the PR from going through. Unless you disagree of course :)

Thanks again for the feedback! Do you have any other questions or anything else you would like me to change?

[raulsntos]

Gradle does not distinguish between abis for .jar files the same way it does for .so files. It simply tries to compile any and all jars into the apk. So if there are 2 jars with the same classes for 2 different abis, only 1 can actually be included. Otherwise we will get conflicts.

I'm not very familiar with the Android platform, but from what I understand, since the .jar is included multiples times under the res://android/build/libs directory (even if they are on different subdirectories), they all get included in the Gradle build because of this:

godot/platform/android/java/app/build.gradle

Lines 47 to 49 in 92e51fc

	debugImplementation fileTree(dir: 'libs/debug', include: ['**/*.jar', '*.aar'])
	devImplementation fileTree(dir: 'libs/dev', include: ['**/*.jar', '*.aar'])
	releaseImplementation fileTree(dir: 'libs/release', include: ['**/*.jar', '*.aar'])

I don't know if there's a way to tell Gradle to de-duplicate or otherwise only include them once. If we have to do this ourselves in ExportPlugin.cs, then we should avoid placing the .jar under one of the architecture's subdirectories but I don't think EditorExportPlugin::add_shared_object allows this. Otherwise, we run into the issue I mentioned earlier about not clearing the res://android/build/libs directory after each build¹.

I do this manually in my build script / pipeline.

I don't think it's acceptable for Godot to relay on users clearing this directory themselves, because most users won't know they may have to, and will be very confused.

Microsoft distributes the jar in each nuget package for each abi. However, only one should be chosen. It does not matter which one we choose since they are identical.

Is this documented somewhere? I'd like to have a guarantee that this won't change in a future version of .NET.

I don't think we need to distribute the libSystem.Security.Cryptography.Native.Android.jar in the godot repo. This file is automatically pulled down in a nuget package.

The .jar file is in the Godot repo for non-Gradle exports, so that it's included in the APK we build for the export templates. As far as I know, there's no way to add a .jar to a pre-built APK, but I'd love to be wrong, I don't like having the .jar in the Godot repo either.

Footnotes

1. https://github.com/godotengine/godot/pull/98066#discussion_r1795854435 ↩

[TCROC]

I'm not very familiar with the Android platform, but from what I understand, since the .jar is included multiples times under the res://android/build/libs directory (even if they are on different subdirectories), they all get included in the Gradle build because of this:

godot/platform/android/java/app/build.gradle

Lines 47 to 49 in 92e51fc
debugImplementation fileTree(dir: 'libs/debug', include: ['/.jar', '.aar'])
devImplementation fileTree(dir: 'libs/dev', include: ['/.jar', '.aar'])
releaseImplementation fileTree(dir: 'libs/release', include: ['**/.jar', '.aar'])

Correct. That is what I have found to be the case after trying several different things over the last few days.

I don't know if there's a way to tell Gradle to de-duplicate or otherwise only include them once. If we have to do this ourselves in ExportPlugin.cs

I tried many different ways. Nothing seemed to work. Gradle always just includes the jar. I haven't been able to find any documentation from Google / Android on how to do this. Nor does anyone else seem to be trying to do this either. And when asking ChatGPT (yes I went this route) it tried its best but made note that "this is an unusual request" 😅. Ultimately to no avail.

then we should avoid placing the .jar under one of the architecture's subdirectories but I don't think EditorExportPlugin::add_shared_object allows this. Otherwise, we run into the issue I mentioned earlier about not clearing the res://android/build/libs directory after each build1.

Agreed. I can open up a separate issue for this if you want? That way this PR can go through with a simple fix and then an additional PR to clean up / improve. With this being one of those improvements.

I don't think it's acceptable for Godot to relay on users clearing this directory themselves, because most users won't know they may have to, and will be very confused.

100% agree

Is this documented somewhere? I'd like to have a guarantee that this won't change in a future version of .NET.

You can confirm yourself here at nuget: https://www.nuget.org/packages/Microsoft.NETCore.App.Runtime.Mono.android-arm64

Select "Open in Nuget Package Explorer"

Then runtime -> native -> libSystem.Security.Cryptography.Native.Android.jar

and then you can see where they compile it for the android runtime here: https://github.com/dotnet/runtime/blob/49006967613007f58daab31abab5316999aa7897/src/native/libs/System.Security.Cryptography.Native.Android/net/dot/android/crypto/DotnetProxyTrustManager.java

But to answer your question more explicitly:

Are there official Microsoft / dotnet / mono docs that say the libSystem.Security.Cryptography.Native.Android.jar will always be shipped with dotnet? Not that I have found.

BUT

All of the sources I linked above strongly suggest (at least to me) that this is how microsoft imlements cryptography for the Android runtime. I expect they will continue doing it this way until these dotnet runtime issues gets resolved:

dotnet/runtime#45741
dotnet/runtime#45737

Which looking at those, it appears they still have a ways to go.

The .jar file is in the Godot repo for non-Gradle exports, so that it's included in the APK we build for the export templates. As far as I know, there's no way to add a .jar to a pre-built APK, but I'd love to be wrong, I don't like having the .jar in the Godot repo either.

Since its included in the nuget package, we could just add some logic to fetch the nuget package and extract it from there. Then it wouldn't have to be included in the repo. This I would recommend be best fit for a separate PR so as to keep the complexity of this PR low and fix the more immediate issue of universal gradle builds failing.

Also to note: Fetching the jar from the nuget package rather than storing in the git repo should increase stability. Looking at the nuget versions: https://www.nuget.org/packages/Microsoft.NETCore.App.Runtime.Mono.android-arm64#versions-body-tab

It appears a new package gets shipped with each version of dotnet. Including patch releases.

[raulsntos]

Is this documented somewhere? I'd like to have a guarantee that this won't change in a future version of .NET.

You can confirm yourself here at nuget

But to answer your question more explicitly:

Are there official Microsoft / dotnet / mono docs that say the libSystem.Security.Cryptography.Native.Android.jar will always be shipped with dotnet? Not that I have found.

Cryptography is always done through OS functions in .NET because managed cryptography is not acceptable security. Therefore, there will always be a need to go through the JNI on Android to call those OS functions.

My question was more whether there's a guarantee that the .jar files for different architectures will always be the same or if they could be different at some point. I'm not really convinced we can just pick one at random and it will be fine.

Since its included in the nuget package, we could just add some logic to fetch the nuget package and extract it from there. Then it wouldn't have to be included in the repo.

I don't want to overcomplicate the process of building the templates, fetching from NuGet.org is not much different from doing it ourselves and including it in the repo like we do now.

Besides, at the moment of building the Android templates, we can't know what .NET version the user C# project will target. We already have to build two different flavors (standard and mono), I wouldn't want to add different mono templates for each .NET version.

For now, the .jar we have in the Godot repo is the one from .NET 8.0 and I hope that works well enough. But if the pre-built APK doesn't work, using Gradle builds should use the .jar from the dotnet publish output.

But I agree, we're getting a bit sidetracked with this, which is unrelated to the issue this PR is trying to fix. So we can continue this part of the conversation on the #dotnet channel if you want.

[TCROC]

My question was more whether there's a guarantee that the .jar files for different architectures will always be the same or if they could be different at some point. I'm not really convinced we can just pick one at random and it will be fine.

Ah I see what you are asking. Well I have not been able to find them commenting on this anywhere in an official capacity. BUT, in combination of deductive reasoning and inspecting their open source repos, I think we can come to a reasonable conclusion. Lets start on the deductive route:

In this screenshot, I have pulled 4 different nuget packages for 4 different Android abis / cpu architectures. As we can see, the only thing that differs between each jar is the MSFTSIG.RSA. If I were to guess, date / time is likely involved in this signature and that is what is different. I suspect the jar bytecode is identical. In fact, going back and enabling the "show equal files" setting in Android Studio confirms this:

Here we can see the compiled DotnetProxyTrustManager.class is identical between each jar. So at the very least dotnet version 8.0.10 has the same jar for each nuget package in regards to the java bytecode. Grabbing one at random should be fine for this version of dotnet. Lets go inspect their source code and see if they have any pipelines to confirm this for future dotnet versions.

Searching libSystem.Security.Cryptography.Native.Android within the dotnetruntime repo here: https://github.com/dotnet/runtime

We get 4 results. One of which appears to be a builder task! This is a great sign that there is only one! :)

Now lets go inspect this task and see what we find. I suspect we are going to find that it is the same for each abi, but lets confirm :)

It appears to be so at first glance! :) Here is where the java code itself is compiled:

https://github.com/dotnet/runtime/blob/main/src/tasks/AndroidAppBuilder/AndroidLibBuilderTask.cs

Here is where the Android sdk and toolsets are configured:

https://github.com/dotnet/runtime/blob/main/src/tasks/AndroidAppBuilder/ApkBuilder.cs

Which if you notice are mostly from environment variables. So if we search for those environment variables, we should see the environment variables being set to the same value for compiling or different values:

https://github.com/search?q=repo%3Adotnet%2Fruntime%20ANDROID_SDK_ROOT&type=code

Looking at the search results, there does not seem to me any discernable differences that would cause any issues! :) So I'm reasonably confident from all of these inspections that the jars are the same in each nuget package and will likely remain the same for each nuget package per dotnet version going forward for the foreseeable future.

To be entirely fair, I have not found where the nuget package is compiled for the specific android abis. BUT, searching for the key terms that should indicate differences seems to indicate the jar is compiled the same way throughout the dotnet runtime repo.

I hope this was helpful! :)

But I agree, we're getting a bit sidetracked with this, which is unrelated to the issue this PR is trying to fix. So we can continue this part of the conversation on the #dotnet channel if you want.

Sounds good! I'll ping you over there to discuss the additional topics.