Make host tests use single copy of shared framework (#100588)

- Only create one .NET install layout to be shared by all host tests
-  Add `pretest.proj` for `host.pretest` subset that builds all test project assets and creates the single .NET install layout
- Fix `NativeHostApis` tests that were editing the .NET install layout directly (instead of creating a copy to edit)
  - Remove some unnecessary copying/creating of SDKs and frameworks by sharing the fixture across tests
- Update host testing doc with simpler setup instructions and more details around investigating test failures

Author: elinor-fung

docs/workflow/testing/host/testing.md

@@ -37,16 +37,16 @@ dotnet build src\installer\tests\HostActivation.Tests
 
 The host tests depend on:
   1. Pre-built [test project](/src/installer/tests/Assets/Projects) output which will be copied and run by the tests. The `host.pretest` subset builds these projects.
-  2. Product binaries in a directory layout matching that of a .NET install
-  3. TestContextVariables.txt file with property and value pairs which will be read by the tests
+  2. Product binaries in a directory layout matching that of a .NET install. The `host.pretest` subset creates this layout.
+  3. TestContextVariables.txt files with property and value pairs which will be read by the tests. The `host.tests` subset creates these files as part of building the tests.
 
 When [running all tests](#running-all-tests), the build is configured such that these are created/performed before the start of the test run.
 
 In order to create (or update) these dependencies without running all tests:
-  1. Build the `host.pretest` subset. By default, this is included in the `host` subset. This corresponds to (1) above.
-  2. Run the `SetUpSharedFrameworkPublish` and `SetupTestContextVariables` targets for the desired test project. This corresponds to (2) and (3) above. For example:
+  1. Build the `host.pretest` subset. By default, this is included in the `host` subset. This corresponds to (1) and (2) above.
+  2. Build the desired test project. This corresponds to (3) above. Building the test itself will run the `SetupTestContextVariables` target, but it can also be run independently - for example:
   ```
-  dotnet build src\installer\tests\HostActivation.Tests -t:SetUpSharedFrameworkPublish;SetupTestContextVariables -p:RuntimeConfiguration=Release -p:LibrariesConfiguration=Release
+  dotnet build src\installer\tests\HostActivation.Tests -t:SetupTestContextVariables -p:RuntimeConfiguration=Release -p:LibrariesConfiguration=Release
   ```
 
 ## Running tests
@@ -86,6 +86,15 @@ If you built the runtime or libraries with a different configuration from the ho
 build.cmd -vs Microsoft.DotNet.CoreSetup -rc Release -lc Release
 ```
 
+## Investigating failures
+
+When [running all tests](#running-all-tests), reports with results will be generated under `<repo_root>\artifacts\TestResults`. When [running individual tests](#running-specific-tests), results will be output to the console by default and can be configured via [`dotnet test` options](https://learn.microsoft.com/dotnet/core/tools/dotnet-test#options).
+
+In order to test the hosting components, the tests launch a separate process (e.g. `dotnet`, apphost, native host) and validate the expected output (standard output and error) of the launched process. This usually involves copying or creating test artifacts in the form of an application to run or a .NET install to run against.
+
+On failure, tests will report the file, arguments, and environment for the launched process that failed validation. With [preserved test artifacts](#preserving-test-artifacts), this information can be used to directly debug the specific scenario that the test was running.
+
 ### Preserving test artifacts
 
-In order to test the hosting components, the tests launch a separate process (e.g. `dotnet`, apphost, native host) and validate the expected output (standard output and error) of the launched process. This usually involves copying or creating test artifacts in the form of an application to run or a .NET install to run against. The tests will delete these artifacts after the test finishes. To allow inspection or usage after the test finishes, set the environment variable `PRESERVE_TEST_RUNS=1` to avoid deleting the test artifacts.
+The tests will delete any generated test artifacts after the test finishes. To allow inspection or usage after the test finishes, set the environment variable `PRESERVE_TEST_RUNS=1` to avoid deleting the test artifacts.
+

eng/Subsets.props

@@ -500,7 +500,7 @@
 
   <!-- Host.pretest subset (consumes live built libraries assets so needs to come after libraries) -->
   <ItemGroup Condition="$(_subset.Contains('+host.pretest+'))">
-    <ProjectToBuild Include="$(InstallerProjectRoot)tests\Assets\Projects\**\*.csproj" Category="host" />
+    <ProjectToBuild Include="$(InstallerProjectRoot)tests\pretest.proj" Category="host" />
   </ItemGroup>
 
   <!-- Host.tests subset (consumes live built libraries assets so needs to come after libraries) -->

src/installer/tests/Directory.Build.targets

@@ -1,25 +1,5 @@
 <Project>
 
-  <Target Name="SetUpSharedFrameworkPublish"
-          DependsOnTargets="DetermineTestOutputDirectory"
-          BeforeTargets="RunTests">
-
-    <!--
-      Explicitly restore before PublishToDisk. IsRestoring property ensures that this evaluation won't be reused.
-      See https://github.com/dotnet/msbuild/issues/2811
-    -->
-    <MSBuild Projects="$(InstallerProjectRoot)pkg\sfx\Microsoft.NETCore.App\Microsoft.NETCore.App.Runtime.sfxproj"
-             Targets="Restore"
-             Properties="IsRestoring=true" />
-    <!--
-      Set up the shared framework copy this set of tests should use. There's no known reason to have
-      one per test project, but RepoDirectoriesProvider may need some tweaking to share.
-    -->
-    <MSBuild Projects="$(InstallerProjectRoot)pkg\sfx\bundle\Microsoft.NETCore.App.Bundle.bundleproj"
-             Targets="PublishToDisk"
-             Properties="OutputPath=$(TestsOutputDir)sharedFrameworkPublish/" />
-  </Target>
-
   <Target Name="SetupTestContextVariables"
           Condition="'$(IsTestProject)' == 'true'"
           DependsOnTargets="