[net7.0] Make MAUI into a workload based on NuGet packs

[mattleibow]

Description

Backport of:

• Make MAUI into 1 workload and lots of NuGets instead of a full workload #11206
• Add implicit package reference for Compat #13427
• Add some conditions so we could skip if we wanted #14310
• Add a new Sdk.props for the existing Sdk.targets #14300
• Add Controls.Compatibility to local cache #14567
• Support both Update and Include for maui #14576

This change moves most - if not all - the logic, assemblies, build tasks, targets and props into NuGet packages. This allows .NET MAUI to be usable without having to care about what version of the workload is installed.

There are still a few things - mostly the automagic and Windows fixes - in the workload pack, but this is probably temporary and/or non-essential for the working of the build. Things like the project capabilities are still in the workload as this is needed for the IDE and cannot be in NuGet packages.

Everything else is now inside a NuGet package that can be upgraded, pinned and otherwise used without requiring VS to have installed the workload to match. There is the benefit of the NuGet packages being installed on disk and thus reducing/removing the need to download. But that is minimal now since we have smaller and fewer packages.

Risks

Since we are doing a fairly major change very late into the .NET 7 release of MAUI, this PR could cause issues to appear that we do not know about. Users do many things in many different ways so we cannot foresee them all. This PR is ready to be merged but that is only because the issues I have thought of have been made known/fixed.

Since the current SR is going out soon, this will have to be in the next one. But maybe there is a way we can do a preview workload that allows for testing.

PR Testing

Since this is a PR that makes fundamental changes, it cannot be used via a simple <MauiVersion> property, however, it is quite easy:

1. Make sure you are on the latest stable VS
2. Run this dotnet CLI command:
   dotnet workload install maui --skip-sign-check --from-rollback-file https://aka.ms/maui-net7-nugets/rollback.json --source https://api.nuget.org/v3/index.json --source https://aka.ms/maui-net7-nugets/index.json

User Scenarios

With the PR for .NET 8, it was simple because if net8 then nuget else workload is the story, but this is doing a fairly major change to a released version and may have issues. To try and detect them, I will be testing:

#	Scenario		Error	Reason	Mitigation
1	Starting with a stable install (7.0.59), Can I use <MauiVersion> to update to a newer version from NuGet?	❌	There are several errors similar to: "error NETSDK1112: The runtime pack for Microsoft.Maui.Core.Runtime.maccatalyst was not downloaded."	This is because those packages no longer exist at all. They have all been merged into a single "Microsoft.Maui.Core" NuGet package.	We can't add a pre-restore error because we haven't restored yet, but we can maybe make dummy packages that do nothing and satisfy the restore.
2	Starting with this PR, Can I use <MauiVersion> to update to an older version from NuGet?	✅	None.	This works because of a few reasons, but effectively thi PR has logic that detects the older version and uses a different set of NuGet packages and FrameworkReferences.	None.
3	Upgrading from a stable workload to a "NuGet-based" workload using the CLI still works.	✅	None.	Updating workloads uninstall the old one and install the new one, so there is no overlap. And we only have the 7.0.100 band.	None.
4	"NuGet-based" .NET 7 workloads still allow for multi-targeting of .NET 6 and .NET 7 without issues.	✅	None.	The .NET 7 workload in this PR has nothing in the AutoImports and only imports .NET 7 things if the TFM is .NET 7, so there is no overlap.	None.
5	"NuGet-based" .NET 7 workloads can be multi-targeted with .NET 8.	✅	None.	Since both .NET 7and .NET 8 are NuGet-based, the workloads do not actually have any significant code. As a result, the workload acts as a proxy to import NuGets (and set some Windows workarounds)	None.
6	"NuGet-based" .NET 7 and .NET 8 can be multi-targeted together in a project SDK without the need for any maui workloads to be installed.	✅	None.	This is only doable because all significant code is moved out to NuGet. Any workload code is now simple and can be duplicated in the Project SDK: https://github.com/mattleibow/Microsoft.NET.Sdk.Maui/tree/nugets	None.

Notes

Notes on Scenario 2

This scenario is a special case for the .NET 7 backport as it is not relevant for .NET 8 because .NET 8 never shipped with workloads. However, this logic is slightly different than before:

• The logic is to use the same method as .NET 8 if the MauiVersion is a CI version (7.0.0-*) or if it is a post-PR version (maybe the next release after 7.0.81).
• If there are any issues, then the "compatibility logic" can all be disabled with <MauiIgnoreWorkloadPackagingType>true</MauiIgnoreWorkloadPackagingType>.
• If you look at the versions in the BundledVersions.in.targets, you will see some packagesare $(MauiVersion) and others are $(MauiWorkloadVersion) and this is by design
  • Workloads had all their targets loaded first so the <MauiVersion> only influenced the actual assemblies referenced.
  • The current logic uses the workload's build tasks and dependencies, but falls back to the framework references for the assemblies - as it always did. This is the core reason why <MauiVersion> is not 100% a great thing.
• In order to get this to work and not have to do 2 sets of targets/props, I cheated and added a net7.0-branch-only change and that is to add a new NuGet: Microsoft.Maui.Core.Build.Tasks.
  • This package just forms the basis of sharing the build tasks with both the installed workload and the MauiVersion workload. We need to do this because as mentioned before the build tasks are shared regardless of maui version.
  • It also is the part that brings in the NuGet dependencies - such as AndroidX - allowing people to upgrade without implicit packages and other annoying things.

Notes on Scenario 4

It would appear that multi-targeting net7;net8 is the same as multi-targeting net6;net7, however there is a significant difference:

• .NET 6 does not import any "significant" NuGet packages in the BundledVersions.targets. It just imports "dependency-only" packages: Microsoft.Maui.Extensions, Microsoft.Maui.Dependencies and then Microsoft.AspNetCore.Components.WebView.Maui, Microsoft.Windows.SDK.BuildTools.
• .NET 7 moved all the transitive dependencies into the BundledVersions.targets and as a result the .csproj directly imports a bunch of packages.
• As a result of this difference, when the .csproj includes the .NET 7 version of Microsoft.Extensions.Logging.Debug, the transitive dependencies of the Microsoft.Maui.Extensions package automatically upgrades and works seamlessly.
• However, in .NET 8 projects, when we target .NET 7, we end up with duplicate package and thus a collision of versions where the .csproj imports 2 versions of the same package. The final package collection is that the .csproj imports the Logging.Debug v8.0 directly and the Logging v7.0 via the BundledVersions.targets:

  Assembly 'Microsoft.Extensions.Logging.Debug' with identity 'Microsoft.Extensions.Logging.Debug, Version=8.0.0.0, Culture=neutral, PublicKeyToken=adb9793829ddae60' uses 'Microsoft.Extensions.Logging, Version=8.0.0.0, Culture=neutral, PublicKeyToken=adb9793829ddae60' which has a higher version than referenced assembly 'Microsoft.Extensions.Logging' with identity 'Microsoft.Extensions.Logging, Version=7.0.0.0, Culture=neutral, PublicKeyToken=adb9793829ddae60'

• With regards to the packages imported by .NET 6, there are 2 that are also lib-containing. However, these are excepted from the rules of support because (like all the other .NET Frameworks) we do not support a .NET 7 MAUI component in a .NET 6 app, and vice versa. So to update/reference the Microsoft.AspNetCore.Components.WebView.Maui package in a net6;net7 project, you have to condition this import to only import when the TFM is net7. The same will happen when adding .NET 8.
• There is an exception to this exception, and that is "Microsoft.Windows.SDK.BuildTools" - this is an implicit package reference and typically should not be changed. This is partially considered to be part of the MAUI SDK - at least for .NET 6 - so it will also have to be conditioned. However, however... If this is seen as a serious issue, we can add it as a dependency of "Microsoft.Maui.Dependencies" as a bug was fixed recently that will allow it to be added as a transitive dependency.
• If we merged and release [net6.0] Microsoft.Windows.SDK.BuildTools can be transitive #14435, then this last point no longer applies and we will have no non-maui dependencies.

Notes on Scenario 6

As mentioned in the "Reason", we can create a special SDK that can be referenced directly in the <Project Sdk="Microsoft.NET.Sdk.Maui"> because everything is in a NuGet. The only code that is not in a NuGet is several <ProjectCapability> items as well as some things for Windows - like setting up RID support and various other NuGet-restore-influencing properties.
As of today, my testing has required the use of the /<version> in the Sdk, but I was also able to get this to work automatically using the MSBuild workload resolver if this SDK is registered as part of the workload manifest. This means that we could potentially have support for 2 ways of using .NET MAUI:

1. <UseMaui>true</UseMaui>
2. <Project Sdk="Microsoft.NET.Sdk.Maui">

In both cases, the "default" version is set by the installed workload - but can be overridden by simply installing a newer NuGet package version in the csproj - no need for any workload updates.
See more at https://github.com/mattleibow/Microsoft.NET.Sdk.Maui/tree/nugets

[jsuarezruiz]

/azp run

[azure-pipelines]

Azure Pipelines successfully started running 2 pipeline(s).

[mattleibow]

I have a test repo here with all the things just working using nugets: https://github.com/mattleibow/Microsoft.NET.Sdk.Maui/tree/nugets

[mattleibow]

/azp run MAUI

[azure-pipelines]

No pipelines are associated with this pull request.

[SF-Simon]

I have a test repo here with all the things just working using nugets: https://github.com/mattleibow/Microsoft.NET.Sdk.Maui/tree/nugets

Hello, @mattleibow

We urgently need this solution that can use the specified Nuget version, but we have not understood the writing method of version 7.0. x. Can you provide a .csproj file so that we can understand how to set it up? For example, sample7.csproj.

Thank you very much. This idea is very good.

We use the version of the Main branch to compile the Nuget file and version 7.0.0-ci.net7.16393 under the dotnet7-feed folder.

The error thrown:
Microsoft.NET.Sdk.Maui-nugets\sample\sample.csproj : error : The project file cannot be opened by the project system, because it is missing some critical imports or the referenced SDK cannot be found.

The content of the .csproj:

<Project Sdk="Microsoft.Maui.Sdk/7.0.100-dev">

	<PropertyGroup>
		<TargetFrameworks>net7.0-android;net7.0-ios;net7.0-maccatalyst</TargetFrameworks>
		<TargetFrameworks Condition="$([MSBuild]::IsOSPlatform('windows'))">$(TargetFrameworks);net7.0-windows10.0.19041.0</TargetFrameworks>
...
		<UseMaui>true</UseMaui>
...
	</PropertyGroup>
...

	<ItemGroup Condition="$(TargetFramework.StartsWith('net8'))">
		<PackageReference Include="Microsoft.Extensions.Logging.Debug" Version="8.0.0-preview.2.23128.3" />
	</ItemGroup>

	<ItemGroup Condition="$(TargetFramework.StartsWith('net8'))">
		<PackageReference Include="Microsoft.Maui.Controls" Version="8.0.0-preview.3.*" />
	</ItemGroup>
	<ItemGroup Condition="$(TargetFramework.StartsWith('net7'))">
		<PackageReference Include="Microsoft.Maui.Controls" Version="7.0.0-ci.net7.16393" />
	</ItemGroup>

	<ItemGroup>
		<ProjectCapability Include="DiagnoseCapabilities" />
	</ItemGroup>

</Project>

[MartyIX]

@mattleibow Would it be worth mentioning if this PR affects #12953 in any way?

[mattleibow]

This PR is a backport of things in net8 and I mentioned in the other issue how you can actually disable all nugets and take control.

Not sure if that helps?

[MartyIX]

Hopefully, it does. I'll try. Thank you!

[vhugogarcia]

Hello @mattleibow ,

I'm wondering, if Maui Essentials is now a NuGet pack?, so I can download the source code and contribute with it, if yes, where can I find it. Also to fix an issue I'm facing personally with the WebAuthenticator, since I need to overwrite it to fix an issue on Android devices using a different browser as default.

#13798 (comment)

Thanks in advance.

[mattleibow]

The maui essentials will still be in this repo: https://github.com/dotnet/maui/tree/main/src/Essentials

We are more just changing how you get this into the app. Previously it was via a workload magic, now you can technically update this separately via nuget or a PR build.

[vhugogarcia]

The maui essentials will still be in this repo: https://github.com/dotnet/maui/tree/main/src/Essentials

We are more just changing how you get this into the app. Previously it was via a workload magic, now you can technically update this separately via nuget or a PR build.

Thanks @mattleibow for the response. That clarifies my question.

[vhugogarcia]

One last question @mattleibow , is it possible to convert the WebUtils class to be public, so we can access the methods? Please. I am working/contributing on a fix for the WebAuthenticator from MAUI Essentials and when I try to overwrite some methods from the WebAuthenticator class I get an error saying that the WebUtils is not accesible due to its protection level.

Thanks in advance. 🙂

[mattleibow]

Closing this as we will not be backporting this to net7.0. It is fairly risky - especially with the fact that we are doing this halfway through the net7 life.

This is all merged and the only/default way in .net8.