Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
[Xamarin.Android.Build.Tasks] Make all assemblies RID-specific (#8478)
Fixes: #8168 Context: #4337 Context: #8155 Context: 55e5c34 Context: 6836818 Context: 929e701 Context: c927026 Context: 2f19238 Issue #8155 noted a *fundamental* mismatch in expectations between the Classic Xamarin.Android packaging worldview and the .NET worldview: In Classic Xamarin.Android, all assemblies are presumed to be architecture agnostic ("AnyCPU"), while in .NET: 1. `System.Private.CoreLib.dll` was *always* an architecture-specific assembly (see #4337), and 2. The .NET Trimmer is extensible and can apply ABI-specific changes to IL which *effectively* results in an architecture-specific assembly (#8155). Meanwhile, there is no way of knowing that this is happening, and the trimmer doesn't mark the resulting assembly as architecture-specific. We long tried to "paper over" this difference, by trying to find -- and preserve the "nature" of -- architecture-agnostic assemblies (55e5c34, …). Unfortunately, all attempts at trying to preserve the concept of architecture-agnostic assemblies have failed; we're fighting against .NET tooling in attempting to do so. In commit 6836818 this came to a head: a long worked-on feature LLVM Marshal Methods (8bc7a3e) had to be disabled because of hangs within MAUI+Blazor Hybrid+.NET Android apps, and we suspect that treating an assembly as architecture-agnostic when it was "actually" architecture-specific is a plausible culprit. Bite the bullet: there is no longer such a thing as an architecture- agnostic assembly. Treat *all* assemblies as if they were architecture-specific. Additionally, alter assembly packaging so that instead of using `assemblies/assemblies*.blob` files (c927026), we instead store the assemblies within `lib/ABI` of the `.apk`/`.aab`. The Runtime config blob `rc.bin` is stored as `lib/ABI/libarc.bin.so`. When `$(AndroidUseAssemblyStore)`=true, assemblies will be stored within `lib/ABI/libassemblies.ABI.blob.so`, e.g. `lib/arm64-v8a/libassemblies.arm64-v8a.blob.so`. When `$(AndroidUseAssemblyStore)`=false and Fast Deployment is *not* used, then assemblies are stored individually within `lib/ABI` as compressed assembly data, with the following "name mangling" convention: * Regular assemblies: `lib_` + Assembly File Name + `.so` * Satellite assemblies: `lib-` + culture + `-` + Assembly File Name + `.so` For example, consider this selected `unzip -l` output: % unzip -l bin/Release/net9.0-android/*-Signed.apk | grep lib/arm64-v8a 723560 01-01-1981 01:01 lib/arm64-v8a/libSystem.IO.Compression.Native.so 70843 01-01-1981 01:01 lib/arm64-v8a/lib_Java.Interop.dll.so 157256 01-01-1981 01:01 lib/arm64-v8a/libaot-Java.Interop.dll.so 1512 01-01-1981 01:01 lib/arm64-v8a/libarc.bin.so * `libSystem.IO.Compression.Native.so` is a native shared library from .NET * `lib_Java.Interop.dll.so` is compressed assembly data for `Java.Interop.dll` * `libaot-Java.Interop.dll.so` contains Profiled AOT output for `Java.Interop.dll` * `libarc.bin.so` is the `rc.bin` file used by .NET runtime startup Additionally, note that Android limits the characters that can be used in native library filenames to the regex set `[-._A-Za-z0-9]`. TODO: No error checking is done to ensure that "Assembly File Name" stays within the limits of `[-.A-Za-z0-9]`, e.g. if you set `$(AssemblyName)=Emoji😅` *and `$(AndroidUseAssemblyStore)`=false, then we'll try to add `lib/arm64-v8a/lib_Emoji😅.dll.so`, which will fail at runtime. This works when `$(AndroidUseAssemblyStore)`=true, which is the default. Pros: * We're no longer fighting against .NET tooling features such as ILLink Substitutions. * While `.aab` files will get larger, we expect that the actual `.apk` files sent to Android devices from the Google Play Store will be *smaller*, as the Google Play Store would always preserve/transmit *all* `assemblies/assemblies*.blob` files, while now it will be able to remove `lib/ABI/*` for unsupported ABIs. Cons: * `.apk` files containing more than one ABI ***will get larger***, as there will no longer be "de-duping" of architecture-agnostic assembly data. We don't consider this a significant concern, as we believe `.aab` is the predominant packaging format. ~~ All assemblies are architecture-specific ~~ Assembly pre-processing changes so that every assembly ends up in every target architecture batch, regardless of whether its MVID differs from its brethren or not. This is done very early in the build process on our side, where we make sure that each assembly either has the `%(Abi)` metadata or is given one, and is placed in the corresponding batch. Further processing of those batches is "parallel", in that no code attempts to de-duplicate the batches. ~~ Impact on Fast Deployment, `$(IntermediateOutputPath)` ~~ The changes also required us to place all the assemblies in new locations on disk within `$(IntermediateOutputPath)` when building the application. (Related: 2f19238.) Assemblies are now placed in subdirectories named after either the target architecture/ABI or the .NET `$(RuntimeIdentifier)`, e.g. `obj/Release/netX.Y-android/android-arm64`. This, in turn, affects e.g. Fast Deployment as now the synchronized content is in the `…/.__override__/ABI` directory on device, instead of just in `…/.__override__`. ~~ File Formats ~~ The assembly store format (c927026) is updated to use the following structures: struct AssemblyStoreHeader { uint32_t magic; uint32_t version; uint32_t entry_count; // Number of assemblies in the store uint32_2 index_entry_count; uint32_t index_size; }; struct AssemblyStoreIndexEntry { intptr_t name_hash; // xxhash of assembly filename uint32_t descriptor_index; // index into `descriptors` array }; struct AssemblyStoreEntryDescriptor { uint32_t mapping_index; // index into an internal runtime array uint32_t data_offset; // index into `data` for assembly `.dll` uint32_t data_size; // size of assembly, in bytes uint32_t debug_data_offset; // index into `data` for assembly `.pdb`; 0 if not present uint32_t debug_data_size; // size of `.pdb`, in bytes; 0 if not present uint32_t config_data_offset; // index into `data` for assembly `.config`; 0 if not present uint32_t config_data_size; // size of `.config`, in bytes; 0 if not present }; struct AssemblyStoreAssemblyInfo { uint32_t length; // bytes uint8_t name[length]; }; `libassemblies.ABI.blob.so` has the following format, and is *not* a valid ELF file: AssemblyStoreHeader header {…}; AssemblyStoreIndexEntry index [header.index_entry_count]; AssemblyStoreAssemblyDescriptor descriptors [header.entry_count]; AssemblyStoreAssemblyInfo names [header.entry_count]; uint8_t data[];
- Loading branch information
