[build] bootstrap a local .\bin\dotnet\ with .NET Workloads

Based on: https://github.com/jonathanpeppers/maui-workload

This adds a new `DotNet.csproj` that provisions a local .NET 6 install
into `.\bin\dotnet\`.

Next, it uses versions defined in `eng/Versions.props` as required by
.NET version bumping infrastructure (called Darc):

    <Project>
      <PropertyGroup>
        <MicrosoftNETSdkPackageVersion>6.0.100-preview.2.21155.3</MicrosoftNETSdkPackageVersion>
        <MicrosoftAndroidSdkPackageVersion>11.0.200-ci.main.148</MicrosoftAndroidSdkPackageVersion>
        <MicrosoftMacCatalystSdkPackageVersion>14.3.100-ci.main.337</MicrosoftMacCatalystSdkPackageVersion>
        <MicrosoftiOSSdkPackageVersion>14.4.100-ci.main.1192</MicrosoftiOSSdkPackageVersion>
      </PropertyGroup>
    </Project>

Next, we can use these versions to consume NuGet packages for workloads:

    <PackageDownload Include="Microsoft.NET.Workload.Android"     Version="[$(MicrosoftAndroidSdkPackageVersion)]" />
    <PackageDownload Include="Microsoft.NET.Workload.MacCatalyst" Version="[$(MicrosoftMacCatalystSdkPackageVersion)]" />
    <PackageDownload Include="Microsoft.NET.Workload.iOS"         Version="[$(MicrosoftiOSSdkPackageVersion)]" />

Then the other packs they depend on:

    <PackageDownload Include="Microsoft.Android.Ref"            Version="[$(MicrosoftAndroidSdkPackageVersion)]" />
    <PackageDownload Include="Microsoft.Android.Sdk.win-x64"    Version="[$(MicrosoftAndroidSdkPackageVersion)]" Condition="$([MSBuild]::IsOSPlatform('windows'))" />
    <PackageDownload Include="Microsoft.Android.Sdk.osx-x64"    Version="[$(MicrosoftAndroidSdkPackageVersion)]" Condition="$([MSBuild]::IsOSPlatform('osx'))" />
    <PackageDownload Include="Microsoft.Android.Sdk.BundleTool" Version="[$(MicrosoftAndroidSdkPackageVersion)]" />
    <PackageDownload Include="Microsoft.MacCatalyst.Ref"        Version="[$(MicrosoftMacCatalystSdkPackageVersion)]" />
    <PackageDownload Include="Microsoft.MacCatalyst.Sdk"        Version="[$(MicrosoftMacCatalystSdkPackageVersion)]" />
    <PackageDownload Include="Microsoft.iOS.Ref"                Version="[$(MicrosoftiOSSdkPackageVersion)]" />
    <PackageDownload Include="Microsoft.iOS.Sdk"                Version="[$(MicrosoftiOSSdkPackageVersion)]" />

After doing this, I can build .NET 6 projects with:

    > .\bin\dotnet\dotnet.exe build .\src\Controls\samples\Controls.Sample.SingleProject\Maui.Controls.Sample.SingleProject.csproj

I can even build MacCatalyst apps on Windows!

~~ Other Changes ~~

* Rework CI setup to use .\bin\dotnet\dotnet
* We don't need boots or URLs anymore
* Fixed `MSBuildTests` to use .\bin\dotnet\dotnet if found and fall
  back to the system `dotnet`

~~ TODO ~~

This obviously will change the Maui team's workflow, so:

* Need docs explaining how to do local dev, etc.

~~ What problems does this solve? ~~

* MacCatalyst builds on Windows! (offline build that checks C#)
* Building Maui always gets you the right things installed.
* Maui becoming a .NET workload will be a breeze. We can copy files
  into `.\bin\dotnet\sdk-manifests` and `.\bin\dotnet\packs`.
* We can use Darc to bump dependencies within .NET:

https://github.com/dotnet/arcade/blob/main/Documentation/Darc.md

.nuspec/package.ps1

@@ -1,6 +1,17 @@
-param ($configuration)
+param ([string] $configuration = 'Debug')
 
-dotnet pack $PSScriptRoot\..\Microsoft.Maui-net6.sln `
+$ext = if ($IsWindows) { ".exe" } else { "" }
+$dotnet = Join-Path $PSScriptRoot ../bin/dotnet/dotnet$ext
+$sln = Join-Path $PSScriptRoot ../Microsoft.Maui-net6.sln
+$artifacts = Join-Path $PSScriptRoot ../artifacts
+
+if (-not (Test-Path $dotnet))
+{
+    $csproj = Join-Path $PSScriptRoot ../src/DotNet/DotNet.csproj
+    & dotnet build $csproj -bl:$artifacts/dotnet.binlog
+}
+
+& $dotnet pack $sln `
     -c:$configuration `
     -p:SymbolPackageFormat=snupkg `
-    -bl:$PSScriptRoot/../artifacts/maui.binlog
+    -bl:$artifacts/maui.binlog

Directory.Build.props

@@ -1,4 +1,5 @@
 <Project>
+  <Import Project="eng\Version.props" />
   <PropertyGroup>
     <_MauiBuildTasksLocation>$(_MauiBuildTasksLocation)</_MauiBuildTasksLocation>
     <_MauiBuildTasksLocation Condition="'$(_MauiBuildTasksLocation)' == ''">$(MSBuildThisFileDirectory).nuspec\</_MauiBuildTasksLocation>

eng/Version.props

@@ -0,0 +1,9 @@
+<Project>
+  <!--Package versions-->
+  <PropertyGroup>
+    <MicrosoftNETSdkPackageVersion>6.0.100-preview.2.21155.3</MicrosoftNETSdkPackageVersion>
+    <MicrosoftAndroidSdkPackageVersion>11.0.200-ci.main.148</MicrosoftAndroidSdkPackageVersion>
+    <MicrosoftMacCatalystSdkPackageVersion>14.3.100-ci.main.337</MicrosoftMacCatalystSdkPackageVersion>
+    <MicrosoftiOSSdkPackageVersion>14.4.100-ci.main.1192</MicrosoftiOSSdkPackageVersion>
+  </PropertyGroup>
+</Project>

eng/dogfood.ps1

@@ -0,0 +1,93 @@
+<#
+  .SYNOPSIS
+  *Windows-only* Launches Visual Studio with environment variables to use the local ./bin/dotnet/dotnet.exe.
+
+  .DESCRIPTION
+  *Windows-only* Launches Visual Studio with environment variables to use the local ./bin/dotnet/dotnet.exe.
+  Script based on:
+  https://github.com/dotnet/runtime/blob/1be117d8e7c0cd29ebc55cbcff2a7fa70604ed39/eng/build.ps1#L186-L208
+  https://github.com/dotnet/runtime/blob/1be117d8e7c0cd29ebc55cbcff2a7fa70604ed39/eng/common/tools.ps1#L109
+
+  .PARAMETER vs
+  The path to Visual Studio, defaults to:
+  "C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\Common7\IDE\devenv.exe".
+
+  .PARAMETER sln
+  The path to a .sln or .project file, defaults to "Microsoft.Maui-net6.sln".
+
+  .PARAMETER modify
+  Modify the environment variables in the current session. This would
+  allow the "dotnet" command to work instead of ".\bin\dotnet\dotnet".
+  However, it would be good to do this in a new terminal session,
+  since you would have trouble running "dotnet build" if it needs to
+  provision .NET 6 and the iOS/Android workloads again.
+
+  .EXAMPLE
+  PS> .\scripts\dogfood.ps1
+
+  .EXAMPLE
+  PS> .\scripts\dogfood.ps1 -vs "C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\Common7\IDE\devenv.exe"
+
+  .EXAMPLE
+  PS> .\scripts\dogfood.ps1 -sln .\path\to\MySolution.sln
+#>
+
+param(
+  [string]$vs = "${env:ProgramFiles(x86)}\Microsoft Visual Studio\2019\Enterprise\Common7\IDE\devenv.exe",
+  [string]$sln,
+  [switch]$modify
+)
+
+$eng=Split-Path -Parent $MyInvocation.MyCommand.Path
+$dotnet=Join-Path $eng ../bin/dotnet/
+$dotnet=(Get-Item $dotnet).FullName
+
+if (-Not $sln) {
+    $sln=Join-Path $eng ../Microsoft.Maui-net6.sln
+    $sln=(Get-Item $sln).FullName
+}
+
+# Modify global.json, so the IDE can load
+$globaljson = Join-Path $eng ../global.json
+[xml] $xml = Get-Content (Join-Path $eng Version.props)
+$json = Get-Content $globaljson | ConvertFrom-Json
+$json | Add-Member sdk (New-Object -TypeName PSObject)
+$json.sdk | Add-Member version $xml.Project.PropertyGroup.MicrosoftNETSdkPackageVersion
+$json | ConvertTo-Json | Set-Content $globaljson
+
+# NOTE: I've not found a better way to do this
+# see: https://github.com/PowerShell/PowerShell/issues/3316
+$oldDOTNET_INSTALL_DIR=$env:DOTNET_INSTALL_DIR
+$oldDOTNET_ROOT=$env:DOTNET_ROOT
+$oldDOTNET_MSBUILD_SDK_RESOLVER_CLI_DIR=$env:DOTNET_MSBUILD_SDK_RESOLVER_CLI_DIR
+$oldDOTNET_MULTILEVEL_LOOKUP=$env:DOTNET_MULTILEVEL_LOOKUP
+$oldPATH=$env:PATH
+
+try {
+    $env:DOTNET_INSTALL_DIR=$dotnet
+
+    # This tells .NET to use the bootstrapped runtime
+    $env:DOTNET_ROOT=$env:DOTNET_INSTALL_DIR
+
+    # This tells MSBuild to load the SDK from the directory of the bootstrapped SDK
+    $env:DOTNET_MSBUILD_SDK_RESOLVER_CLI_DIR=$env:DOTNET_ROOT
+
+    # This tells .NET not to go looking for .NET in other places
+    $env:DOTNET_MULTILEVEL_LOOKUP=0
+
+    # Put our local dotnet.exe on PATH first so Visual Studio knows which one to use
+    $env:PATH=($env:DOTNET_ROOT + ";" + $env:PATH)
+
+    # Launch VS
+    & "$vs" "$sln"
+} finally {
+    if (-Not $modify) {
+        $env:DOTNET_INSTALL_DIR = $oldDOTNET_INSTALL_DIR
+        $env:DOTNET_ROOT=$oldDOTNET_ROOT
+        $env:DOTNET_MSBUILD_SDK_RESOLVER_CLI_DIR=$oldDOTNET_MSBUILD_SDK_RESOLVER_CLI_DIR
+        $env:DOTNET_MULTILEVEL_LOOKUP=$oldDOTNET_MULTILEVEL_LOOKUP
+        $env:PATH=$oldPATH
+    }
+}
+
+exit 0

eng/pipelines/common/build-windows.yml

@@ -16,7 +16,9 @@ steps:
       provisioning_script: $(provisionator.path)
       provisioning_extra_args: $(provisionator.extraArguments)
 
-  - template: dotnet-install.yml
+  - powershell: |
+      & dotnet build src\DotNet\DotNet.csproj -bl:${{ parameters.artifactsTargetFolder }}\$(BuildConfiguration)-dotnet.binlog
+    displayName: install .NET
 
   - powershell: |
       $(System.DefaultWorkingDirectory)/build.ps1 --target provision --TeamProject="$(System.TeamProject)"
@@ -67,7 +69,7 @@ steps:
     displayName: 'Copy Files dlls'
     inputs:
       Contents: |
-        **/bin/**/*.dll
+        src/**/bin/**/*.dll
       TargetFolder: ${{ parameters.artifactsTargetFolder }}
 
   - task: PublishBuildArtifacts@1

eng/pipelines/handlers.yml

@@ -46,7 +46,6 @@ schedules:
 
 variables:
   - template: /eng/pipelines/common/variables.yml
-  - template: /eng/pipelines/common/variables-net6.yml
   - name: LogDirectory
     value: $(Build.ArtifactStagingDirectory)/logs
   - name: provisionator.xcode
@@ -55,6 +54,8 @@ variables:
     value: '$(System.DefaultWorkingDirectory)/eng/provisioning/provisioning.csx'
   - name: provisionator.extraArguments
     value: '--v'
+  - name: DotNet.Path
+    value: $(System.DefaultWorkingDirectory)/bin/dotnet/dotnet
 
 parameters:
   - name: BuildEverything
@@ -188,27 +189,7 @@ stages:
           pool:
             name: ${{ BuildPlatform.poolName }}
             vmImage: ${{ BuildPlatform.vmImage }}
-          ${{ if eq(BuildPlatform.name, 'macos') }}:
-            variables:
-              DotNet.Root: /usr/local/share/dotnet/
-              DotNet.Tools: ~/.dotnet/tools
           steps:
-            - ${{ if eq(BuildPlatform.name, 'macos') }}:
-              - task: UseDotNet@2
-                displayName: install .NET Core 3.1
-                inputs:
-                  version: 3.1.x
-                  installationPath: $(DotNet.Root)
-            - template: common/dotnet-install.yml
-            - pwsh: |
-                dotnet tool install --global boots
-                boots ${{ BuildPlatform.bootsAndroid }}
-                boots ${{ BuildPlatform.bootsiOS }}
-                if ('${{ BuildPlatform.bootsMacCatalyst }}' -ne '') {
-                  boots ${{ BuildPlatform.bootsMacCatalyst }}
-                }
-              displayName: install .NET workloads
-              errorActionPreference: stop
             - ${{ if eq(BuildPlatform.name, 'macos') }}:
               - bash: |
                   set -x
@@ -253,27 +234,7 @@ stages:
             pool:
               name: ${{ BuildPlatform.poolName }}
               vmImage: ${{ BuildPlatform.vmImage }}
-            ${{ if eq(BuildPlatform.name, 'macos') }}:
-              variables:
-                DotNet.Root: /usr/local/share/dotnet/
-                DotNet.Tools: ~/.dotnet/tools
             steps:
-              - ${{ if eq(BuildPlatform.name, 'macos') }}:
-                - task: UseDotNet@2
-                  displayName: install .NET Core 3.1
-                  inputs:
-                    version: 3.1.x
-                    installationPath: $(DotNet.Root)
-              - template: common/dotnet-install.yml
-              - pwsh: |
-                  dotnet tool install --global boots
-                  boots ${{ BuildPlatform.bootsAndroid }}
-                  boots ${{ BuildPlatform.bootsiOS }}
-                  if ('${{ BuildPlatform.bootsMacCatalyst }}' -ne '') {
-                    boots ${{ BuildPlatform.bootsMacCatalyst }}
-                  }
-                displayName: install .NET workloads
-                errorActionPreference: stop
               - ${{ if eq(BuildPlatform.name, 'macos') }}:
                 - bash: |
                     set -x
@@ -283,8 +244,9 @@ stages:
                     cat ~/Library/Preferences/Xamarin/Settings.plist || true
                   displayName: configure vsmac xcode
               - pwsh: |
-                  dotnet build Microsoft.Maui.BuildTasks-net6.sln -c $(BuildConfiguration) -bl:$(LogDirectory)/$(BuildConfiguration)-buildtasks.binlog
-                  dotnet build Microsoft.Maui-net6.sln -c $(BuildConfiguration) -bl:$(LogDirectory)/$(BuildConfiguration).binlog
+                  & dotnet build src\DotNet\DotNet.csproj -bl:$(LogDirectory)\$(BuildConfiguration)-dotnet.binlog
+                  & $(DotNet.Path) build Microsoft.Maui.BuildTasks-net6.sln -c $(BuildConfiguration) -bl:$(LogDirectory)/$(BuildConfiguration)-buildtasks.binlog
+                  & $(DotNet.Path) build Microsoft.Maui-net6.sln -c $(BuildConfiguration) -bl:$(LogDirectory)/$(BuildConfiguration).binlog
                 displayName: build samples
                 errorActionPreference: stop
               - task: PublishBuildArtifacts@1

src/Compatibility/Core/src/Compatibility-net6.csproj

@@ -1,7 +1,6 @@
 <Project Sdk="Microsoft.NET.Sdk">
   <PropertyGroup>
-    <TargetFrameworks>net6.0-ios;net6.0-android</TargetFrameworks>
-    <TargetFrameworks Condition=" '$(OS)' != 'Windows_NT'">$(TargetFrameworks);net6.0-maccatalyst</TargetFrameworks>
+    <TargetFrameworks>net6.0-ios;net6.0-android;net6.0-maccatalyst</TargetFrameworks>
     <RootNamespace>Microsoft.Maui.Controls.Compatibility</RootNamespace>
     <AssemblyName>Microsoft.Maui.Controls.Compatibility</AssemblyName>
     <Nullable>disable</Nullable>

src/Controls/samples/Controls.Sample.SingleProject/Maui.Controls.Sample.SingleProject.csproj

@@ -1,7 +1,6 @@
 <Project Sdk="Microsoft.NET.Sdk">
   <PropertyGroup>
-    <TargetFrameworks>net6.0-android;net6.0-ios</TargetFrameworks>
-    <TargetFrameworks Condition=" '$(OS)' != 'Windows_NT'">$(TargetFrameworks);net6.0-maccatalyst</TargetFrameworks>
+    <TargetFrameworks>net6.0-android;net6.0-maccatalyst;net6.0-ios</TargetFrameworks>
     <RuntimeIdentifier Condition="'$(TargetFramework)' == 'net6.0-ios'">ios-x64</RuntimeIdentifier>
     <RuntimeIdentifier Condition="'$(TargetFramework)' == 'net6.0-maccatalyst'">maccatalyst-x64</RuntimeIdentifier>
     <OutputType>Exe</OutputType>

src/Controls/samples/Controls.Sample/Maui.Controls.Sample-net6.csproj

@@ -1,8 +1,7 @@
 <Project Sdk="Microsoft.NET.Sdk">
 
   <PropertyGroup>
-    <TargetFrameworks>net6.0-android;net6.0-ios</TargetFrameworks>
-    <TargetFrameworks Condition=" '$(OS)' != 'Windows_NT'">$(TargetFrameworks);net6.0-maccatalyst</TargetFrameworks>
+    <TargetFrameworks>net6.0-android;net6.0-ios;net6.0-maccatalyst</TargetFrameworks>
     <RootNamespace>Maui.Controls.Sample</RootNamespace>
     <AssemblyName>Maui.Controls.Sample</AssemblyName>
     <IsPackable>false</IsPackable>

src/Controls/src/Core/Controls.Core-net6.csproj

@@ -1,7 +1,6 @@
 <Project Sdk="Microsoft.NET.Sdk">
 	<PropertyGroup>
-		<TargetFrameworks>netstandard2.0;net6.0;net6.0-android;net6.0-ios</TargetFrameworks>
-		<TargetFrameworks Condition=" '$(OS)' != 'Windows_NT'">$(TargetFrameworks);net6.0-maccatalyst</TargetFrameworks>
+		<TargetFrameworks>netstandard2.0;net6.0;net6.0-android;net6.0-ios;net6.0-maccatalyst</TargetFrameworks>
     <RootNamespace>Microsoft.Maui.Controls</RootNamespace>
     <AssemblyName>Microsoft.Maui.Controls</AssemblyName>
     <Nullable>disable</Nullable>

src/Controls/src/Xaml/Controls.Xaml-net6.csproj

@@ -1,7 +1,6 @@
 <Project Sdk="Microsoft.NET.Sdk">
 	<PropertyGroup>
-		<TargetFrameworks>netstandard2.0;net6.0;net6.0-android;net6.0-ios</TargetFrameworks>
-		<TargetFrameworks Condition=" '$(OS)' != 'Windows_NT'">$(TargetFrameworks);net6.0-maccatalyst</TargetFrameworks>
+		<TargetFrameworks>netstandard2.0;net6.0;net6.0-android;net6.0-ios;net6.0-maccatalyst</TargetFrameworks>
 		<AssemblyName>Microsoft.Maui.Controls.Xaml</AssemblyName>
 		<RootNamespace>Microsoft.Maui.Controls.Xaml</RootNamespace>
 		<NoWarn>$(NoWarn);CA2200;</NoWarn>

src/Controls/tests/Xaml.UnitTests/MSBuild/MSBuildTests.cs

@@ -2,6 +2,7 @@
 using System.Diagnostics;
 using System.IO;
 using System.Linq;
+using System.Runtime.InteropServices;
 using System.Text;
 using System.Xml.Linq;
 using Mono.Cecil;
@@ -184,9 +185,25 @@ void onData(object s, DataReceivedEventArgs e)
 					}
 			};
 
+			var ext = RuntimeInformation.IsOSPlatform(OSPlatform.Windows) ? ".exe" : "";
+			var dotnet = IOPath.Combine(testDirectory, "..", "..", "..", "..", "..", "..", "..", "bin", "dotnet", $"dotnet{ext}");
+			if (!File.Exists (dotnet))
+			{
+				Console.WriteLine($"Using 'dotnet', did not find: {dotnet}");
+
+				// If we don't have .\bin\dotnet\dotnet, try the system one
+				dotnet = "dotnet";
+			}
+			else
+			{
+				dotnet = IOPath.GetFullPath(dotnet);
+
+				Console.WriteLine($"Using '{dotnet}'");
+			}
+
 			var psi = new ProcessStartInfo
 			{
-				FileName = "dotnet",
+				FileName = dotnet,
 				Arguments = $"build -v:{verbosity} -nologo {projectFile} -t:{target} -bl {additionalArgs}",
 				CreateNoWindow = true,
 				WindowStyle = ProcessWindowStyle.Hidden,

src/Core/src/Core-net6.csproj

@@ -1,7 +1,6 @@
 <Project Sdk="Microsoft.NET.Sdk">
   <PropertyGroup>
-    <TargetFrameworks>netstandard2.1;netstandard2.0;net6.0-android;net6.0-ios</TargetFrameworks>
-    <TargetFrameworks Condition=" '$(OS)' != 'Windows_NT'">$(TargetFrameworks);net6.0-maccatalyst</TargetFrameworks>
+    <TargetFrameworks>netstandard2.1;netstandard2.0;net6.0-android;net6.0-ios;net6.0-maccatalyst</TargetFrameworks>
     <RootNamespace>Microsoft.Maui</RootNamespace>
     <AssemblyName>Microsoft.Maui</AssemblyName>
     <Nullable>enable</Nullable>

src/DotNet/Dependencies/Directory.Build.props

@@ -0,0 +1,3 @@
+<Project>
+  <Import Project="../../../eng/Version.props" />
+</Project>

src/DotNet/Dependencies/Directory.Build.targets

@@ -0,0 +1 @@
+<Project />

src/DotNet/Dependencies/Packs.csproj

@@ -0,0 +1,15 @@
+<Project Sdk="Microsoft.Build.NoTargets">
+  <PropertyGroup>
+    <TargetFramework>netstandard2.0</TargetFramework>
+  </PropertyGroup>
+  <ItemGroup>
+    <PackageDownload Include="Microsoft.Android.Ref"            Version="[$(MicrosoftAndroidSdkPackageVersion)]" />
+    <PackageDownload Include="Microsoft.Android.Sdk.win-x64"    Version="[$(MicrosoftAndroidSdkPackageVersion)]" Condition="$([MSBuild]::IsOSPlatform('windows'))" />
+    <PackageDownload Include="Microsoft.Android.Sdk.osx-x64"    Version="[$(MicrosoftAndroidSdkPackageVersion)]" Condition="$([MSBuild]::IsOSPlatform('osx'))" />
+    <PackageDownload Include="Microsoft.Android.Sdk.BundleTool" Version="[$(MicrosoftAndroidSdkPackageVersion)]" />
+    <PackageDownload Include="Microsoft.MacCatalyst.Ref"        Version="[$(MicrosoftMacCatalystSdkPackageVersion)]" />
+    <PackageDownload Include="Microsoft.MacCatalyst.Sdk"        Version="[$(MicrosoftMacCatalystSdkPackageVersion)]" />
+    <PackageDownload Include="Microsoft.iOS.Ref"                Version="[$(MicrosoftiOSSdkPackageVersion)]" />
+    <PackageDownload Include="Microsoft.iOS.Sdk"                Version="[$(MicrosoftiOSSdkPackageVersion)]" />
+  </ItemGroup>
+</Project>