Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for network extension on Mac Catalyst #17408

Open
enclave-alistair opened this issue Jan 31, 2023 · 26 comments
Open

Add support for network extension on Mac Catalyst #17408

enclave-alistair opened this issue Jan 31, 2023 · 26 comments
Labels
app-extensions feature A feature to be implemented
Milestone

Comments

@enclave-alistair
Copy link

enclave-alistair commented Jan 31, 2023

Steps to Reproduce

  1. Create a new minimal MacCatalyst App Extension.
  2. Build dotnet build /bl
  3. Build fails; "_CollectBundleResources" does not exist.

Expected Behavior

That I can create an App Extension targeting net7.0-maccatalyst.

Actual Behavior

I cannot build the App Extension.

Additional Investigation

I've gone looking for why the _CollectBundleResources task does not exist, and I've boiled it down to (partly) what appears to be a typo in Xamarin.Shared.Sdk.targets, where MacCatalystAppExtensionProject is spelt MacCatalystSAppExtensionProject:

<Import Project="$(_TargetsDirectory)Xamarin.MacCatalyst.AppExtension.$(_ProjectLanguage).targets" Condition="'$(_ProjectType)' == 'MacCatalystSAppExtensionProject' " />

However, fixing the typo isn't enough, because MacCatalyst is missing the Xamarin.MacCatalyst.AppExtension.CSharp.targets file that the iOS build does have, so fails to import it. I suspect, although haven't tried it all the way through, that creating the correct AppExtension targets and props file for MacCatalyst that duplicate the iOS ones may do the job, but can't state that's definitely the right approach.

Environment

Version information
Visual Studio Community 2022 for Mac
Version 17.4.3 (build 21)
Installation UUID: 12e90ff0-6e67-433e-943f-5a004fe08364

Runtime
.NET 6.0.12 (64-bit)
Architecture: X64

Roslyn (Language Service)
4.4.0-6.22578.12+3c6ab8e1715e5b080fb7bb77070810ab71e09387

NuGet
Version: 6.3.1.1

.NET SDK (x64)
SDK: /usr/local/share/dotnet/sdk/7.0.102/Sdks
SDK Versions:
	7.0.102
	6.0.405
MSBuild SDKs: /Applications/Visual Studio.app/Contents/MonoBundle/MSBuild/Current/bin/Sdks

.NET Runtime (x64)
Runtime: /usr/local/share/dotnet/dotnet
Runtime Versions:
	7.0.2
	6.0.13

Xamarin.Profiler
Version: 1.8.0.19
Location: /Applications/Xamarin Profiler.app/Contents/MacOS/Xamarin Profiler

Updater
Version: 11

Apple Developer Tools
Xcode 14.2 (21534)
Build 14C18

Xamarin.Mac
Version: 9.0.0.27 (Visual Studio Community)
Hash: 933c6c2c9
Branch: xcode14.1
Build date: 2022-11-22 02:00:36-0500

Xamarin.iOS
Version: 16.1.1.27 (Visual Studio Community)
Hash: 933c6c2c9
Branch: xcode14.1
Build date: 2022-11-22 02:00:37-0500

Xamarin Designer
Version: 17.4.0.136
Hash: d49c9ff6d3
Branch: remotes/origin/d17-4
Build date: 2023-01-12 19:21:10 UTC

Xamarin.Android
Version: 13.1.0.1 (Visual Studio Community)
Commit: xamarin-android/d17-4/13ba222
Android SDK: /Users/alistairevans/Library/Developer/Xamarin/android-sdk-macosx
	Supported Android versions:
		13.0 (API level 33)

SDK Command-line Tools Version: 7.0
SDK Platform Tools Version: 33.0.2
SDK Build Tools Version: 32.0.0

Build Information: 
Mono: a96bde9
Java.Interop: xamarin/java.interop/d17-4@fcc33ce2
SQLite: xamarin/sqlite/3.39.3@23e1ae7
Xamarin.Android Tools: xamarin/xamarin-android-tools/main@0be567a

Microsoft Build of OpenJDK
Java SDK: /Library/Java/JavaVirtualMachines/microsoft-11.jdk
11.0.12
Android Designer EPL code available here:
https://github.com/xamarin/AndroidDesigner.EPL

Eclipse Temurin JDK
Java SDK: /Library/Java/JavaVirtualMachines/temurin-8.jdk
1.8.0.302
Android Designer EPL code available here:
https://github.com/xamarin/AndroidDesigner.EPL

Android SDK Manager
Version: 17.4.0.54
Hash: 6eabb9e
Branch: remotes/origin/d17-4
Build date: 2023-01-12 19:21:16 UTC

Android Device Manager
Version: 0.0.0.1206
Hash: 886af39
Branch: 886af39
Build date: 2023-01-12 19:21:16 UTC

Build Information
Release ID: 1704030021
Git revision: 26eac6764d85f12fe50ee0d45a7cd0b266be7b23
Build date: 2023-01-12 19:19:02+00
Build branch: release-17.4
Build lane: release-17.4

Operating System
Mac OS X 13.1.0
Darwin 22.2.0 Darwin Kernel Version 22.2.0
    Fri Nov 11 02:08:47 PST 2022
    root:xnu-8792.61.2~4/RELEASE_X86_64 x86_64

Build Logs

msbuild.zip

Example Project

ReproProject.zip

@rolfbjarne
Copy link
Member

What kind of app extension are you trying to create? As far as I know Apple doesn't support any app extensions on Mac Catalyst (but it was a while I checked, so this may have changed).

@rolfbjarne rolfbjarne added the need-info Waiting for more information before the bug can be investigated label Jan 31, 2023
@ghost
Copy link

ghost commented Jan 31, 2023

Hi @enclave-alistair. We have added the "need-info" label to this issue, which indicates that we have an open question for you before we can take further action. This issue will be closed automatically in 7 days if we do not hear back from you by then - please feel free to re-open it if you come back to this issue after that time.

@rolfbjarne rolfbjarne added this to the Future milestone Jan 31, 2023
@enclave-alistair
Copy link
Author

We're creating a network extension; the apple docs indicate that Mac Catalyst is a supported platform for a PacketTunnelProvider:

https://developer.apple.com/documentation/technotes/tn3134-network-extension-provider-deployment?language=objc

It's possible that not all app extensions are supported for Mac Catalyst, but certainly Network Extensions should be.

@ghost ghost added need-attention An issue requires our attention/response and removed need-info Waiting for more information before the bug can be investigated labels Jan 31, 2023
@rolfbjarne
Copy link
Member

Yes, from that documentation it certainly looks like Network extensions are supposed on Mac Catalyst.

This needs some work on our side, so it may unfortunately take some time to get it done (the earliest would likely be .NET 8 this fall).

@rolfbjarne rolfbjarne added feature A feature to be implemented and removed need-attention An issue requires our attention/response labels Jan 31, 2023
@rolfbjarne rolfbjarne modified the milestones: Future, .NET 8 Jan 31, 2023
@enclave-alistair
Copy link
Author

enclave-alistair commented Jan 31, 2023 via email

@rolfbjarne
Copy link
Member

Do you think it's worth my experimenting with creating functional MacCatalyst versions of the targets files in the meantime? It is after all intended to be pretty close to iOS...right? Or are you expecting there to be way more to do than just setting up the build system for it?

Extensions have been somewhat of a mixed bag in the past, some of them have been really easy, some have required a lot more work. IIRC network extensions haven't been problematic on iOS, so it might be that just creating the missing targets files would work on Mac Catalyst (it's the right approach though, so even if you're not able to get all the way there, it's a step in the right direction).

@enclave-alistair
Copy link
Author

Thanks @rolfbjarne , will investigate and report back.

@enclave-alistair
Copy link
Author

enclave-alistair commented Feb 14, 2023

Wanted to give an update; I've made some progress getting maccatalyst extensions building correctly.

I've got a branch in my fork with my changes so far: https://github.com/enclave-alistair/xamarin-macios/tree/feature/maccatalyst-ext-7.0.1xx

In short, a maccatalyst appex does now build successfully, and I get a built .appex file out that looks about right to my eye:

- MacCatalyst.Extension.appex
  - Contents
    - MacOS
      - MacCatalyst.Extension (binary executable)
    - MonoBundle
      - .xamarin
        - maccatalyst-arm64
          - MacCatalyst.Extension.dll
          - System.Private.CoreLib.dll
          - System.Runtime.dll
        - maccatalyst-x64      
          - MacCatalyst.Extension.dll
          - System.Private.CoreLib.dll
          - System.Runtime.dll
      - MacCatalyst.Extension.aotdata.arm64
      - System.Private.CoreLib.aotdata.arm64
      - System.Runtime.aotdata.arm64
      - icudt.dat
      - libSystem.IO.Compression.Native.dylib
      - libSystem.Native.dylib
      - libSystem.Net.Security.Native.dylib
      - libSystem.Security.Cryptography.Native.Apple.dylib
      - libmonosgen-2.0.dylib
      - libxamarin-dotnet.dylib
    - Info.plist

The MacCatalyst.Extension binary file has the following output from file MacCatalyst.Extension:

MacCatalyst.Extension: Mach-O universal binary with 2 architectures: [x86_64:Mach-O 64-bit executable x86_64] [arm64:Mach-O 64-bit executable arm64]
MacCatalyst.Extension (for architecture x86_64):        Mach-O 64-bit executable x86_64
MacCatalyst.Extension (for architecture arm64): Mach-O 64-bit executable arm64

That all looks right.

The problem I'm having now is when I try to reference an app extension in a regular UI project.

I'm getting an error in the MergeAppBundles task when the build tries to create a 'universal' maccatalyst app for the UI project from the x64 and arm64 builds.

/usr/local/share/dotnet/packs/Microsoft.MacCatalyst.Sdk/16.2.1007/targets/Xamarin.Shared.Sdk.targets(371,3): Unable to merge the file 'PlugIns/Enclave.Fabric.MacCatalyst.Extension.appex/Contents/Info.plist', it's different between the input app bundles. [/Users/alistairevans/git/fabric-clone/src/tray/Enclave.Maui.Tray/Enclave.Maui.Tray.csproj]
/usr/local/share/dotnet/packs/Microsoft.MacCatalyst.Sdk/16.2.1007/targets/Xamarin.Shared.Sdk.targets(371,3): App bundle file #1: /Users/alistairevans/git/fabric-clone/src/tray/Enclave.Maui.Tray/bin/Release/net7.0-maccatalyst/maccatalyst-x64/Enclave.Maui.Tray.app/PlugIns/Enclave.Fabric.MacCatalyst.Extension.appex/Contents/Info.plist [/Users/alistairevans/git/fabric-clone/src/tray/Enclave.Maui.Tray/Enclave.Maui.Tray.csproj]
/usr/local/share/dotnet/packs/Microsoft.MacCatalyst.Sdk/16.2.1007/targets/Xamarin.Shared.Sdk.targets(371,3): App bundle file #2: /Users/alistairevans/git/fabric-clone/src/tray/Enclave.Maui.Tray/bin/Release/net7.0-maccatalyst/maccatalyst-arm64/Enclave.Maui.Tray.app/PlugIns/Enclave.Fabric.MacCatalyst.Extension.appex/Contents/Info.plist [/Users/alistairevans/git/fabric-clone/src/tray/Enclave.Maui.Tray/Enclave.Maui.Tray.csproj]

I think the problem is that the appex created in the extension project is already universal, but MergeAppBundles task thinks it needs to copy over the unmerged appex folders, then merge them locally. In reality, both architectures should use the already-merged app extension build.

I can see that the Info.plist files of the outer app are explicitly excluded from the merge, but not sure if I want to mark all appex folders as excluded.

Here is the latest binlog with the MergeAppBundles error if anyone wants it:
macbinlog.zip

I'm continuing to investigate, but if anyone has some pointers on the app bundle merge problem, feel free to help out!

@rolfbjarne
Copy link
Member

@enclave-alistair as a first step (to see if there are other problems), you could try creating a non-universal app.

You can do that by setting RuntimeIdentifier:

<PropertyGroup>
    <RuntimeIdentifier>maccatalyst-x64</RuntimeIdentifier>
</PropertyGroup>

@enclave-alistair
Copy link
Author

Ah, good tip @rolfbjarne; tried that, and the build fails later in ComputeCodeSignItems when it looks for the app extension in Enclave.Maui.Tray/bin/Release/net7.0-maccatalyst/maccatalyst-x64/Enclave.Maui.Tray.app/Contents/PlugIns/Enclave.Fabric.MacCatalyst.Extension.appex

I actually think that ComputeCodeSignItems step is correct though, because app extensions for MacCatalyst should definitely be in that path, but the _PlaceAppExtensions target is incorrectly setting _AppExtensionRoot to the root of the bundle rather than the Contents folder, probably because that macOS platform check doesn't include MacCatalyst?

<PropertyGroup>
<_AppExtensionRoot Condition="'$(_PlatformName)' == 'macOS'">$(_AppBundlePath)Contents\</_AppExtensionRoot>
<_AppExtensionRoot Condition="'$(_PlatformName)' != 'macOS'">$(_AppBundlePath)</_AppExtensionRoot>
</PropertyGroup>

I'll try fixing that Condition statement tomorrow and see how far that gets me.

Updated binlog: macbinlog.zip

@rolfbjarne
Copy link
Member

probably because that macOS platform check doesn't include MacCatalyst?

Yes, that sounds like the reason for this failure.

@enclave-alistair
Copy link
Author

Even more progress now, everything builds successfully, the appex is copied into the right folder, and codesigning appears to take place correctly. 🥳

However, when I try to launch the app locally on my Mac, it crashes instantly with:

Exception Type:  EXC_CRASH (SIGKILL (Code Signature Invalid))
Exception Codes: 0x0000000000000000, 0x0000000000000000
Termination Reason: CODESIGNING 1 Taskgated Invalid Signature

Full crash report: crashreport.txt

I manually verified the signatures of both the Maui App, and the extension:

alistairevans@Alistairs-MBP maccatalyst-x64 % codesign -vvv --deep --verify  Enclave.Maui.Tray.app
--prepared:/Users/alistairevans/git/fabric-clone/src/tray/Enclave.Maui.Tray/bin/Release/net7.0-maccatalyst/maccatalyst-x64/Enclave.Maui.Tray.app/Contents/PlugIns/Enclave.Fabric.MacCatalyst.Extension.appex
--validated:/Users/alistairevans/git/fabric-clone/src/tray/Enclave.Maui.Tray/bin/Release/net7.0-maccatalyst/maccatalyst-x64/Enclave.Maui.Tray.app/Contents/PlugIns/Enclave.Fabric.MacCatalyst.Extension.appex
--prepared:/Users/alistairevans/git/fabric-clone/src/tray/Enclave.Maui.Tray/bin/Release/net7.0-maccatalyst/maccatalyst-x64/Enclave.Maui.Tray.app/Contents/Frameworks/Sentry.framework/Versions/Current/.
--validated:/Users/alistairevans/git/fabric-clone/src/tray/Enclave.Maui.Tray/bin/Release/net7.0-maccatalyst/maccatalyst-x64/Enclave.Maui.Tray.app/Contents/Frameworks/Sentry.framework/Versions/Current/.
Enclave.Maui.Tray.app: valid on disk
Enclave.Maui.Tray.app: satisfies its Designated Requirement

alistairevans@Alistairs-MBP maccatalyst-x64 % codesign -vvv --deep -d  Enclave.Maui.Tray.app
Executable=/Users/alistairevans/git/fabric-clone/src/tray/Enclave.Maui.Tray/bin/Release/net7.0-maccatalyst/maccatalyst-x64/Enclave.Maui.Tray.app/Contents/MacOS/Enclave.Maui.Tray
Identifier=io.enclave.agent
Format=app bundle with Mach-O thin (x86_64)
CodeDirectory v=20400 size=756 flags=0x0(none) hashes=13+7 location=embedded
Hash type=sha256 size=32
CandidateCDHash sha256=be7e7f3ec87254e6002a032df753fccd70291cee
CandidateCDHashFull sha256=be7e7f3ec87254e6002a032df753fccd70291ceebaf3dc4177410255311e98de
Hash choices=sha256
CMSDigest=be7e7f3ec87254e6002a032df753fccd70291ceebaf3dc4177410255311e98de
CMSDigestType=2
Launch Constraints:
        None
CDHash=be7e7f3ec87254e6002a032df753fccd70291cee
Signature size=4795
Authority=Apple Development: Alistair Evans (7346U5WL85)
Authority=Apple Worldwide Developer Relations Certification Authority
Authority=Apple Root CA
Signed Time=15 Feb 2023 at 08:05:42
Info.plist entries=33
TeamIdentifier=RKJ69PR2S5
Sealed Resources version=2 rules=13 files=119
Nested=PlugIns/Enclave.Fabric.MacCatalyst.Extension.appex
Nested=Frameworks/Sentry.framework
Internal requirements count=1 size=180

alistairevans@Alistairs-MBP maccatalyst-x64 % cd Enclave.Maui.Tray.app/Contents/PlugIns/      
                                                 
alistairevans@Alistairs-MBP PlugIns % codesign -vvv --deep --verify  Enclave.Fabric.MacCatalyst.Extension.appex 
Enclave.Fabric.MacCatalyst.Extension.appex: valid on disk
Enclave.Fabric.MacCatalyst.Extension.appex: satisfies its Designated Requirement

alistairevans@Alistairs-MBP PlugIns % codesign -vvv --deep -d  
Enclave.Fabric.MacCatalyst.Extension.appex 
Executable=/Users/alistairevans/git/fabric-clone/src/tray/Enclave.Maui.Tray/bin/Release/net7.0-maccatalyst/maccatalyst-x64/Enclave.Maui.Tray.app/Contents/PlugIns/Enclave.Fabric.MacCatalyst.Extension.appex/Contents/MacOS/Enclave.Fabric.MacCatalyst.Extension
Identifier=io.enclave.agent.extension
Format=bundle with Mach-O thin (x86_64)
CodeDirectory v=20400 size=50462 flags=0x0(none) hashes=1566+7 location=embedded
Hash type=sha256 size=32
CandidateCDHash sha256=d8e9ccc120f2d08ce6fe974bb8c16c7303a39c0f
CandidateCDHashFull sha256=d8e9ccc120f2d08ce6fe974bb8c16c7303a39c0fa0e806499ada7b8f7fe254df
Hash choices=sha256
CMSDigest=d8e9ccc120f2d08ce6fe974bb8c16c7303a39c0fa0e806499ada7b8f7fe254df
CMSDigestType=2
Launch Constraints:
        None
CDHash=d8e9ccc120f2d08ce6fe974bb8c16c7303a39c0f
Signature size=4795
Authority=Apple Development: Alistair Evans (7346U5WL85)
Authority=Apple Worldwide Developer Relations Certification Authority
Authority=Apple Root CA
Signed Time=15 Feb 2023 at 08:05:41
Info.plist entries=26
TeamIdentifier=RKJ69PR2S5
Sealed Resources version=2 rules=13 files=62
Internal requirements count=1 size=192
alistairevans@Alistairs-MBP PlugIns % 

I wondered if it might be an issue with the entitlements, but those all look OK.

Something I did notice, but might be a red herring, is a difference between the codesigning outputs of iOS vs MacCatalyst; when the iOS UI app is signed, it's _CodeSignature/CodeResources file contains all the signatures of the app extension directly, e.g.:

<key>PlugIns/Enclave.Fabric.iOS.Extension.appex/Enclave.ARSoft.Tools.Net.aotdata.arm64</key>
<data>
HbcYDQhEt5kLt6DBKEt8Q+LIFwk=
</data>
<key>PlugIns/Enclave.Fabric.iOS.Extension.appex/Enclave.ARSoft.Tools.Net.dll</key>
<data>
R3QOp/A+xSrGAOqClaA2+P0Fmw0=
</data>
<key>PlugIns/Enclave.Fabric.iOS.Extension.appex/Enclave.Fabric.Abstractions.aotdata.arm64</key>
<data>
9jHIlKXQZUZGv6zVoe5hF19i3cM=
</data>
<key>PlugIns/Enclave.Fabric.iOS.Extension.appex/Enclave.Fabric.Abstractions.dll</key>
<data>
7NXQh4Gf6D08RaCU8Iy1i8R7Vio=
</data>

However for the maccatalyst build the signature looks like it's included "byref", using a cdhash:

<key>PlugIns/Enclave.Fabric.MacCatalyst.Extension.appex</key>
<dict>
	<key>cdhash</key>
	<data>
	2OnMwSDy0Izm/pdLuMFscwOjnA8=
	</data>
	<key>requirement</key>
	<string>identifier "io.enclave.agent.extension" and anchor apple generic and certificate leaf[subject.CN] = "Apple Development: Alistair Evans (7346U5WL85)" and certificate 1[field.1.2.840.113635.100.6.2.1] /* exists */</string>
</dict>

This is most likely not a problem, and just a difference in how macOS and iOS signing works, but wanted to include it just in case.

I'll try to do more digging into the signature problem, but now I'm outside msbuild my direction becomes less clear.
macbinlog-successful-build.zip

Oh, and here's the latest binlog (of the successful build): macbinlog-successful-build.zip

@rolfbjarne
Copy link
Member

However, when I try to launch the app locally on my Mac, it crashes instantly with:

Sometimes the system Console will give more information about the crash.

Since this is the main app crashing, and not the extension, one idea might be to try to narrow it down: does it still crash if you remove the extension from the main project?

Another option is to modify the .app bundle directly, and then replay the codesigning commands (you can find the codesign command in the binlog) - for instance you could remove the Contents/PlugIns subdirectory, resign, and see if the main app launches then.

@enclave-alistair
Copy link
Author

Urgh, thanks for the tip re:Console, realised my local provisioning profile didn't have my device in it for some reason; fixed that and app launches correctly.

App extension fails to start though with a "ASP: Security policy would not allow process" error which I'm digging in to now.

@enclave-alistair
Copy link
Author

Figured it out, all app extensions on MacCatalyst appear to need the com.apple.security.app-sandbox entitlement.

Once I added that, and the network client + server entitlements needed in the sandbox (more info), everything started to work! 🎉

image

Perhaps .NET should automatically add the com.apple.security.app-sandbox entitlement to all MacCatalyst extensions?

@rolfbjarne
Copy link
Member

Once I added that, and the network client + server entitlements needed in the sandbox (more info), everything started to work! 🎉

That's great news!

Perhaps .NET should automatically add the com.apple.security.app-sandbox entitlement to all MacCatalyst extensions?

Yes, that's not a bad idea, but it can be done separately from adding support to Mac Catalyst extensions (note that unless it's documented by Apple, we'll have to assume that the requirements might differ between different types of extensions).

@enclave-alistair
Copy link
Author

Fair point. Can be solved with documentation anyway.

What's the path to me raising a PR from my branch? I haven't merged a change in this repo before, are there tests and/or test projects that need creating?

@enclave-alistair
Copy link
Author

Oh, just remembered I need to figure out the universal bundle problem I skipped over, need to resolve that before this can be considered fixed.

@rolfbjarne
Copy link
Member

@enclave-alistair were you able to make this work? Would it be possible to get access to your source code to create a test project we can use?

@enclave-alistair
Copy link
Author

Hi @rolfbjarne; I was able to do everything except the universal bundling problem, I couldn't figure that out and ran out of time. My branch commits:

Made the build work for the individual architectures.

I just wasn't able to figure out the universal bundling for MacCatalyst.

A full app project with network extension and UI example may be tricky, but I'll see what I can do.

@rolfbjarne rolfbjarne changed the title Cannot build MacCatalyst App Extension - '_CollectBundleResources' does not exist; apparent typo in Xamarin.Shared.Sdk.targets Add support for network extension on Mac Catalyst Sep 12, 2023
@rolfbjarne rolfbjarne modified the milestones: .NET 8, .NET 9 Sep 12, 2023
@enclave-alistair
Copy link
Author

enclave-alistair commented Sep 18, 2023

Apologies for the delay on getting the example project to you, once I allocated some time for it it took a while to strip out all of our specific code down to the minimum, and then when I wanted to update it to net8-rc1 I ran into a few regressions I wasn't expecting.

You can find a demo net8 iOS app with a network extension here: https://github.com/enclave-alistair/dotnet-ios-netextension.

This runs OK (ish) on net8 iOS, but will not build on MacCatalyst. Now that the typo has been fixed, the remaining errors seem to be around missing targets file.

There was also an unnerving regression seemingly related to dotnet/linker#3165 that means you can't build AppExtensions in MAUI at all without a workaround (as far as I can tell). @rolfbjarne, shall I raise that as a separate issue, since it's a regression not related to maccatalyst?

@rolfbjarne
Copy link
Member

Apologies for the delay on getting the example project to you, once I allocated some time for it it took a while to strip out all of our specific code down to the minimum, and then when I wanted to update it to net8-rc1 I ran into a few regressions I wasn't expecting.

You can find a demo net8 iOS app with a network extension here: enclave-alistair/dotnet-ios-netextension.

That's wonderful!

This runs OK (ish) on net8 iOS, but will not build on MacCatalyst. Now that the typo has been fixed, the remaining errors seem to be around missing targets file.

There was also an unnerving regression seemingly related to dotnet/linker#3165 that means you can't build AppExtensions in MAUI at all without a workaround (as far as I can tell). @rolfbjarne, shall I raise that as a separate issue, since it's a regression not related to maccatalyst?

Yes, please file an issue in this repo and I'll have a look.

@enclave-alistair
Copy link
Author

Wanted to update this issue as I've done some more investigation, thought I'd drop my findings here.

The remaining issue when building for MacCatalyst comes from the _MergeAppBundles task, specifically where it merges the app extensions from the runtime-specific build for each app extension into the single generic bundle for the app.

I've got an otherwise working branch here.

In more detail, here's what actually happens when I kick off a dotnet build -c Release -f net8.0-maccatalyst on the app.

The app splits the build of itself for x64 and arm64 into separate runtime-specific builds (_RunRidSpecificBuild).

Each runtime-specific build identifies a dependency of the app extension and builds the app extension for a specific architecture.

Each runtime-specific build of the app identifies that there is an app extension build for that runtime, and correctly places it under the Plugins folder of the runtime-specific version of the app.

The runtime-specific builds complete and generates the app bundle; at this point the output file tree of the app build looks a bit like (some files/folders excluded):

- bin/Release/net8.0-maccatalyst
  - maccatalyst-arm64
    - Enclave.Maui.Tray.app
      - MacOS
        - Enclave.Maui.Tray (MachO)
      - Plugins
        - Enclave.Fabric.iOS.Extension.appex/Contents
          - MacOS/Enclave.Fabric.iOS.Extension (MachO)
          - MonoBundle
          - Info.plist        
      - Info.plist
  - maccatalyst-x64
    - Enclave.Maui.Tray.app
      - MacOS
        - Enclave.Maui.Tray (MachO)
      - Plugins
        - Enclave.Fabric.iOS.Extension.appex/Contents
          - MacOS/Enclave.Fabric.iOS.Extension (MachO)
          - MonoBundle
          - Info.plist        
      - Info.plist

The architecture-specific-version of the app contains the architecture-specific version of the app extension.

We then come to _MergeAppBundles.

This process would work just fine if it wasn't for the Info.plist file for the extension. I note that the Info.plist file for the app is specifically excluded from the merge because the Info.plist files cannot be merged, and a separate one is generated in _WriteAppManifest.

However, the app extension Info.plist files are not excluded. This causes the error:

Xamarin.Shared.Sdk.targets(400,3): Unable to merge the file 'Contents/PlugIns.Extension.appex/Contents/Info.plist', it's different between the input app bundles. [App.csproj]

The individual MachO files are merged correctly, and a multi-arch MachO file does end up in the merged bundle for the plugin.

I see two options to solve this:

  1. When we do a multi-runtime build of an app that references app extensions, also perform the full multi-runtime build of the app extension so we get a correctly merged .appex file for the extension. Basically run _MergeAppBundles for each extension separately. Then, when merging the app bundle, ignore the entire Plugins tree and pull in each merged extension directly, bypassing the merge.

  2. Add to the ignore list for _MergeAppBundles all the Info.plist files for each app extension we know about, then subsequently run _WriteAppManifest somehow for each extension. I tried to do this already, but found that the multi-runtime build of the app loses any knowledge about linked app extensions when it comes out of the runtime-specific builds back into the multi-runtime one.

Any thoughts/suggestions @rolfbjarne would obviously be most welcome.

@enclave-alistair
Copy link
Author

Hi @rolfbjarne, apologies for a chasing comment, I just wondered if you'd seen my prior comment around the issue with merging bundles, now we're a fair way past the .NET 8 release?

Happy to continue more work on this fix if you have some direction on how to resolve the merging problem.

@rolfbjarne
Copy link
Member

@enclave-alistair yes, I saw your comment, I've been a bit busy with other work for a while.

To your two options: I think the first one sounds like the cleanest approach. In fact I might have done part of the work already, because we now (in the main branch) only build project reference once per build (as opposed to once per RuntimeIdentifier): 9d9e5a0.

In any case my plan is to look at this in the .NET 9 timeframe.

@enclave-alistair
Copy link
Author

Appreciate the response, I know you've got a lot of other things going on. I'll take a look at the build again based off main.

Also appreciate the planned .NET9 timeframe if I fail to make any progress!

@github-project-automation github-project-automation bot moved this to Technical Debt in .NET 9 Aug 27, 2024
@rolfbjarne rolfbjarne modified the milestones: .NET 9, .NET 10 Sep 27, 2024
@rolfbjarne rolfbjarne removed this from .NET 9 Sep 27, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
app-extensions feature A feature to be implemented
Projects
None yet
Development

No branches or pull requests

2 participants