Change Load Behavior for NuGet Components

[brianrob]

Contributes to dotnet/sdk#15558.

Context

In a single MSBuild process, most of the NuGet binaries are being loaded into more than one AssemblyLoadContext. The ready to run code associated with these binaries can only be used in a single AssemblyLoadContext, which means that usage of the assembly in all other contexts must be jitted. This shows up as an extra 736 methods that get jitted (measured as 147.6ms).

Changes Made

1. Change NuGetFrameworkWrapper to use Assembly.LoadFrom instead of Assembly.LoadFile when using reflection to load NuGet.Frameworks.dll. This ensures that the assembly is not loaded into the NULL loader context, and unifies the load with later loads of the same assembly.
2. Add all of the NuGet assemblies to the list of well-known assemblies in MSBuildLoadContext, forcing them to all be loaded in the default AssemblyLoadContext. This is an important part of the fix because these loads occur after the load that occurs in NuGetFrameworkWrapper, and so by MSBuild taking a dependency on these NuGet assemblies, they should be treated as part of MSBuild itself, and not as part of a task using the task loader behavior.

Testing

• Verified that basic build scenarios work.
• Used COMPlus_ReadyToRunLogFile to confirm that R2R code is not rejected due to multiple loads in different AssemblyLoadContexts.
• Captured before/after profiles to confirm wins.

Notes

This PR contains the following performance wins:

• -736 drop in method jitting overall.
• 147.6ms improvement in JIT time (15.5%)
• 5.1% wall-clock latency improvement overall

cc: @stephentoub, @DamianEdwards, @marcpopMSFT

[stephentoub]

Nice!

[brianrob]

Thanks for the review @Forgind. Can you merge at your convenience? I don't appear to have permissions.

[ladipro]

Thank you!

[ericstj]

This introduced regressions in our build tasks. This is sharing NuGet unconditionally regardless of version which will pin NuGet to the version in the SDK. It’s also causing problems because NewtonsoftJson is not pinned but is a dependency of NuGet. This causes Newtonsoft to load in the tasks ALC and it doesn’t unify with the one used by NuGet even if the same version. This results in Type loading issues (method not found in the case we saw). Why not instead remove NuGet from the tasks which are carrying extra copies to achieve this perf improvement? That will have the added benefit of reducing disk footprint, assuming this unification is indeed legitimate.

Barring that it seems like some additional work needs to be done to selectively reuse based on some version rules and ensure the entire assembly closure will be reused, including Newtonsoft.Json.