Snap master -> release/3.0 for end of Preview 7 (#1099)

Author: vatsan-madhavan

Documentation/api-compat.md

@@ -0,0 +1,73 @@
+# API Compatibility
+API compatibility is a build-time check that ensures that an `implementation` assembly implements the API surface area of a `contract` assembly.
+
+For `WPF on .NET Core`, this means the following:
+* All `WPF on .NET Core` reference assemblies contain **at least** the API surface area contained by `WPF on .NET Framework 4.8` reference assemblies.
+* All hand-crafted reference assemblies for `WPF on .NET Core` contain **exactly** the needed API surface area defined by their corresponding runtime assemblies.
+
+This is accomplished by the use of the [Arcade API Compatibility tool](https://github.com/dotnet/arcade/blob/master/src/Microsoft.DotNet.ApiCompat) with some modifications to fit our specific needs.
+
+## [ApiCompat.props](/eng/WpfArcadeSdk/tools/ApiCompat.props)
+This props file implements necessary elements to trigger and control the usage of API Compatibility checks.
+### Net48CompatNeededProjects
+This property contains a list of projects that should have their reference assemblies compared against reference assemblies for `WPF on .NET Framework 4.8`.
+### Net48RefAssembliesDir
+This property points to the directory where reference assemblies for `WPF on .NET Framework 4.8` will be downloaded during native tool acquisition. In order
+to avoid requiring the  [.NET Framework 4.8 Developer Pack](https://dotnet.microsoft.com/download/dotnet-framework/net48) to be installed on all machines that build `WPF on .NET Core`, a private tools zip is used that
+contains a copy of these assemblies.
+### RefApiCompatNeededProjects
+This property contains a list of projects that have hand-crafted references assemblies that must be compared against their corresponding runtime assemblies during API Compatibility checks.
+
+## [ApiCompat.targets](/eng/WpfArcadeSdk/tools/ApiCompat.targets)
+This targets file implements necessary targets to run API compatibility checks.
+### Properties
+#### RunNetFrameworkApiCompat
+Controls if a project's reference assembly is checked for API compatibility against the reference assemblies for `WPF on .NET Framework 4.8`.
+#### RunRefApiCompat
+Controls if a project's hand-crafted reference assembly is checked for API compatibility against its corresponding runtime assembly.
+#### RunApiCompat
+Controls if Arcade's default API compatibility targets will run.  WPF turns this off.
+#### ApiCompatBaseline
+Controls the location of the API compatibility baseline file for a specific project and API compatibility check.
+### Items
+These MSBuild Items are important to the setup of the API compatibility checks.
+#### ResolvedMatchingContract
+This points to the assembly that contains the contract API surface to validate against.
+#### ResolvedImplementationAssembly
+This points to the assembly whose API surface is being validated.
+### Targets
+The various targets both setup and execute API compatibility checks.
+#### ResolveNetFrameworkApiCompatItems
+Sets up [ApiCompatBaseline](#ApiCompatBaseline), [ResolvedMatchingContract](#ResolvedMatchingContract), and [ResolvedImplementationAssembly](#ResolvedImplementationAssembly) for projects that require
+API compatibility checks against `WPF on .NET Framework 4.8`.  This is run before [WpfValidateApiCompatForNetFramework](#WpfValidateApiCompatForNetFramework) to
+ensure that the necessary configuration is availabe for the check.
+#### ResolveRefApiCompatItems
+Sets up [ApiCompatBaseline](#ApiCompatBaseline), [ResolvedMatchingContract](#ResolvedMatchingContract), and [ResolvedImplementationAssembly](#ResolvedImplementationAssembly) for projects that require
+API compatibility checks between runtime assemblies and hand-crafted reference assemblies.  This is run before [WpfValidateApiCompatForRef](#WpfValidateApiCompatForRef) to
+ensure that the necessary configuration is availabe for the check.
+#### WpfValidateApiCompatForNetFramework
+Calls the API compatibility tool in order to validate a particular project's reference assembly against the corresponding reference assembly for `WPF on .NET Framework 4.8`.
+The [ResolvedMatchingContract](#ResolvedMatchingContract) is the `.NET Framework 4.8` assembly and the [ResolvedImplementationAssembly](#ResolvedImplementationAssembly) is the
+`.NET Core` assembly.  This will generate and MSBuild error for each compatibility issue found.  A developer can examine the current [baseline files](#Baseline-Files) to get
+an idea of the kinds of errors that can be reported.
+
+If the tool fails completely, an error of the form "ApiCompat failed for..." will be generated.  If this occurs, please [file an issue](https://github.com/dotnet/wpf/issues/new/choose) and include a link to your fork and branch that failed.
+#### WpfValidateApiCompatForRef
+Calls the API compatibility tool in order to validate a particular project's hand-crafted reference assembly against the corresponding runtime assembly.
+The [ResolvedMatchingContract](#ResolvedMatchingContract) is the runtime assembly and the [ResolvedImplementationAssembly](#ResolvedImplementationAssembly) is the
+hand-crafted reference assembly.  This will generate and MSBuild error for each compatibility issue found.  A developer can examine the current [baseline files](#Baseline-Files) to get
+an idea of the kinds of errors that can be reported.
+
+If the tool fails completely, an error of the form "ApiCompat failed for..." will be generated.  If this occurs, please [file an issue](https://github.com/dotnet/wpf/issues/new/choose) and include a link to your fork and branch that failed.
+## [Baseline Files](/src/Microsoft.DotNet.Wpf/ApiCompat/Baselines)
+This directory contains the aggregate baseline files for all initial API compatibility checks.  The filenames are of the general form
+"{Project}-{APICompatType}.baseline.txt", where `APICompatType` is either `Net48` or `ref`.
+
+Errors listed in a baseline file are ignored by the API compatibility tool on subsequent runs.
+
+These baselined errors are, generally, one of the following:
+* Intential API changes that diverge from `WPF on .NET Framework 4.8`
+* Errors resulting from changes to underlying assemblies in `.NET Core` that do not adversely affect product functionality
+* Errors due to build-specific architecture at the time of baselining (e.g. the split nature of WPF's product build).
+
+A developer can re-baseline the entirety of the product by setting the property `BaselineAllAPICompatError` to `true` during a build.

Documentation/developer-guide.md

@@ -23,10 +23,39 @@ We use the following workflow for building and testing features and fixes.
 
 You first need to [Fork](https://github.com/dotnet/corefx/wiki/Checking-out-the-code-repository#fork-the-repository) and [Clone](https://github.com/dotnet/corefx/wiki/Checking-out-the-code-repository#clone-the-repository) this WPF repository. This is a one-time task.
 
+
+### Running DRTs locally ###
+In order to run the set of DRTs on your local machine, pass the `-test` parameter to the `build.cmd` script. At the end of the run, you should see something like this:
+
+```
+  A total of 1 test Infos were processed, with the following results.
+   Passed: 1
+   Failed (need to analyze): 0
+   Failed (with BugIDs): 0
+   Ignore: 0
+
+```
+If there were any failures, you can cd into $(RepoRoot)\artifacts\test\$(Configuration)\$(Platform)\Test and run the tests manually with the `/debugtests` flag using the `RunDrts.cmd` script. Note that you do not run the `RunDrtsDebug` script, as this will debug the test infrastructure, `QualityVault`. When you pass the `/debugtests` flag, a cmd window will open where you can open the test executable in Visual Studio and debug it. When the cmd pops up, you will see instructions for debugging using a few different commands, however these commands will enable you to debug the `Simple Test Invocation` executable, `sti.exe`, which simply launches the test executable you are most likely interested in debugging. Using `DrtXaml.exe` as an example, this is how you can debug the test executable. Any MSBuild style properties should be replaced with actual values:
+
+1. `$(RepoRoot)\artifacts\test\$(Configuration)\$(Platform)\Test\RunDrts.cmd /name=DrtXaml /debugtests`
+2. Enter following command into the cmd window that pops up:
+`"%ProgramFiles%\Microsoft Visual Studio\2019\Preview\Common7\IDE\devenv.exe" DrtXaml.exe`
+3. Once Visual Studio is open, go to `Debug-> DrtXaml Properties` and do the following:
+    - Manually change the `Debugger Type` from `Auto` to `Mixed (CoreCLR)`.
+    - Change the `Environment` from `Default` to a custom one that properly defines the `DOTNET_ROOT` variable so that the host is able to locate the install of `Microsoft.NETCore.App`.
+      - x86 (Default): Name: `DOTNET_ROOT(x86)` Value: `$(RepoRoot).dotnet\x86`
+      - x64 (/p:Platform=x64): Name: `DOTNET_ROOT` Value: `$(RepoRoot).dotnet` 
+4. From there you can F5 and the test will execute.
+
+*Note: To run a specific test, you can pass the name of the test like this: `/name=DrtXaml`. The names of these tests are contained in DrtList.xml.*
+
+**NOTE: This requires being run from an admin window at the moment. Removing this restriction is tracked by https://github.com/dotnet/wpf/issues/816. **
+
 ### Testing Locally built WPF assemblies (excluding PresentationBuildTasks)
 This section of guide is intended to discuss the different approaches for ad-hoc testing of WPF assemblies,
-and not automated testing. There are a few different ways this can be done, and for the most part,
-it can be a matter of personal preference on which workflow you choose.
+and not automated testing. For automated testing, see the [Running DRTs locally](#Running-DRTs-locally) section above. There are a few different ways this can be done, and for the most part, it depends on what you are trying to accomplish. This section tries to lay out which scenarios require which workflow.
+
+*NOTE: You should build locally with the `-pack` param to ensure the binaries are put in the correct location for manual testing.*
 
 #### Copying binaries to publish location of a self-contained application
 The simplest approach is to publish your sample app using `dotnet publish -r <rid> --self-contained`.
@@ -39,23 +68,25 @@ Then to copy the WPF assemblies to this published location, simply run the copy-
 located in the `eng` folder of the repo and point it to the location of your test application:
 > eng\copy-wpf.ps1 -destination "c:\mysampleproj"
 
-#### Copying binaries to local dotnet installation
+#### Copying binaries to test host installation of dotnet
 
-If you want/need to test an existing application that targets the shared installation, 
-it is safest to setup a test host, rather than trying to copy assemblies over the shared installation.
-The arcade infrastructure creates a local dotnet installation in the `.dotnet` folder contained at the root
-of the repository when you do a full build using the `build.cmd` or `build.sh` script.
-You can run the copy-wpf.ps1 script again, except you can leave out the destination and be sure to pass in the 
-the `-testhost` parameter:
-> eng\copy-wpf.ps1 -testhost  
-```cmd eng\copy-wpf.ps1 -testhost  ```
+If you want/need to test an existing application that targets the shared installation, it is safest to setup a test host, rather than trying to copy assemblies over the shared installation. A test host is a complete install of dotnet (host and runtimes) used for testing applications and can be setup by using the [dotnet install script](https://docs.microsoft.com/en-us/dotnet/core/tools/dotnet-install-script). This method is also effective for internal contributors who are working on porting our current test corpus from .NET Framework to .NET Core and wants to run the tests against locally built assemblies. Note that there is nothing fundamentally different between a testhost installation of dotnet and the one installed in `$env:ProgramFiles`. However the dotnet host dll won't be able to find the testhost install if the appropriate environment variables aren't set. Note that these environment variables are set for you by copy-wpf.ps1 
 
-You need to set environment variables so that your testhost installation is used when launching the application.
-Once these are set, you should be able to launch the executable from the command line and then your assemblies
-will be used.
+You can run the copy-wpf.ps1 script again and be sure to pass in the `-testhost` parameter:
+> eng\copy-wpf.ps1 -testhost -destination "c:\testhost\x86"
 
-- DOTNET_ROOT=<path_to_wpf_repo>\\.dotnet
-- DOTNET_MULTILEVEL_LOOKUP=0
+If your testhost directory has multiple versions of the `Microsoft.WindowsDesktop.App` shared runtime in it, you can use the `-version` parameter to specify which one you want:
+
+> eng\copy-wpf.ps1 -testhost -destination "c:\testhost\x86" -version "3.0.0-preview6-27728-04"  
+
+If there are multiple versions, you will see a warning and the last installed runtime will be selected. You can backup the folder by creating a copy of it, and the script will ensure that this folder isn't touched as long as the word "Copy" is in the path. This was chosen because the default for Windows Explorer is to append "- Copy" to the folder. This allows you to easily backup folders containing the runtime assemblies knowing you can restore them to their original state if needed. 
+
+If you are installing to a special test host location, you will see output from the script that confirms the appropriate environment variables are set:
+
+```
+** Setting env:DOTNET_ROOT(x86) to c:\testhost\x86 **
+** Setting env:DOTNET_MULTILEVEL_LOOKUP to 0 **
+```
 
 **How to find location of the exe to test?**
 If you are testing an application and don't know where the executable is located, the easiest thing to do
@@ -69,7 +100,6 @@ When the C# compiler detects a collision with assembly references, the assembly
 higher version number is chosen. Assuming our locally built binaries are newer than what is
 installed, we can then simply reference those local binaries directly from the project file, like this:
 
-*Note: you should build locally with the `-pack` param to ensure the binaries are put in the correct location.*
 ```xml
   <PropertyGroup>
      <!-- Change this value based on where your local repo is located -->
@@ -82,21 +112,17 @@ installed, we can then simply reference those local binaries directly from the p
     <RuntimeIdentifier>win-x86</RuntimeIdentifier>
   </PropertyGroup>
   <ItemGroup>
-    <Reference Include="$(WpfRepoRoot)\artifacts\packaging\$(WpfConfig)\Microsoft.DotNet.Wpf.GitHub\ref\netcoreapp3.0\*.dll" Private="false" />
-    <ReferenceCopyLocalPaths Include="$(WpfRepoRoot)\artifacts\packaging\$(WpfConfig)\Microsoft.DotNet.Wpf.GitHub\lib\netcoreapp3.0\*.dll" />
+    <Reference Include="$(WpfRepoRoot)\artifacts\packaging\$(WpfConfig)\Microsoft.DotNet.Wpf.GitHub\lib\netcoreapp3.0\*.dll" />
     <ReferenceCopyLocalPaths Include="$(WpfRepoRoot)\artifacts\packaging\$(WpfConfig)\Microsoft.DotNet.Wpf.GitHub\lib\$(RuntimeIdentifier)\*.dll" />
   </ItemGroup>
 ```
 
 ### Testing specific versions of the Microsoft.WindowsDesktop.App runtime
-At times, it is necessary to install and test specific versions of the runtime. This can be helpful if you are trying to root cause when an issue started occuring.
-
-For testing different versions of the runtime, you can install a specific version of the runtimes via the dotnet install script: https://docs.microsoft.com/en-us/dotnet/core/tools/dotnet-install-script
-**Note**: These install the versions to your %user% directory, so you can use the DOTNET_ROOT environment variables to ensure these get used as described above. Otherwise, you can point them to install in %programfiles% and specify which version of the runtime should be picked up.
+At times, it is necessary to install and test specific versions of the runtime. This can be helpful if you are trying to root cause when an issue started occuring, or need to compare functionality between two different versions.
 
-Below is an example powershell script of how you can use the `dotnet-install.ps1` script:
+For testing different versions of the runtime, you can install a specific version of the runtimes via the [dotnet install script](https://docs.microsoft.com/en-us/dotnet/core/tools/dotnet-install-script). Below is an example powershell script of how you can use the `dotnet-install.ps1` script that will install both 32-bit and 64-bit versions of the `Microsoft.WindowsDesktop.App` runtime into the specified folder:
 
-```
+```ps1
 $dotnet_install = "$env:TEMP\dotnet-install.ps1"
 $x64InstallDir  = "$env:ProgramFiles\dotnet"
 $x86InstallDir  = "${env:ProgramFiles(x86)}\dotnet"
@@ -126,14 +152,16 @@ In this example, the version of `Microsoft.WindowsDesktop.App` associated with t
 **Note**: The ability to install the WindowsDesktop runtime via the dotnet install script is being tracked by: https://github.com/dotnet/cli/issues/11115 
 
 #### Specifying which version of the runtime to use
-If you can build directly from source, you can add this to your project file to pick up the version of the shared runtime you want to test:
+If you can build directly from source, and want to test your application against a certain version of the `Microsoft.WindowsDesktop.App` shared runtime, you can add this to your project file to pick up the version of the shared runtime you want to test:
 ```xml
  <PropertyGroup>
     <MicrosoftWindowsDesktopAppVersion>3.0.0-preview5-27619-18</MicrosoftWindowsDesktopAppVersion>
- <PropertyGroup>
- <FrameworkReference Update="Microsoft.WindowsDesktop.App">
-    <TargetingPackVersion>$(MicrosoftWindowsDesktopAppVersion)</TargetingPackVersion>
- </FrameworkReference>
+ </PropertyGroup>
+ <ItemGroup>
+   <FrameworkReference Update="Microsoft.WindowsDesktop.App">
+      <TargetingPackVersion>$(MicrosoftWindowsDesktopAppVersion)</TargetingPackVersion>
+   </FrameworkReference>
+ </ItemGroup>
 ```
 
 If you don't have the ability to build from source, you can update the *.runtimeconfig.json file located next to the executable to pick up your version:
@@ -162,3 +190,4 @@ Follow the steps defined [here](https://github.com/dotnet/arcade/blob/master/Doc
 * [up-for-grabs WPF issues](https://github.com/dotnet/wpf/issues?q=is%3Aopen+is%3Aissue+label%3Aup-for-grabs)
 * [easy WPF issues](https://github.com/dotnet/wpf/issues?utf8=%E2%9C%93&q=is%3Aopen+is%3Aissue+label%3Aeasy)
 * [Code generation in dotnet/wpf](codegen.md)
+* [Testing in Helix](testing-in-helix.md)