.Net SDK Validation

[MattFromRVA]

Fixes #2199

[timcassell]

MonoAotLLVMToolChain and CsProjClassicNetToolchain also need the validation.

[timcassell]

Another thing you could do is integrate the Toolchain.IsCliPathInvalid into this validator.

[timcassell]

Almost there, few more comments.

[timcassell]

Tests are failing

Failed CanDisassembleAllMethodCalls(jit: LegacyJit, platform: X86, runtime: .NET Framework 4.6.2) [13 ms]
  Error Message:
   The "Summary" should have NOT "HasCriticalValidationErrors"
  Stack Trace:
     at BenchmarkDotNet.IntegrationTests.BenchmarkTestExecutor.CanExecute(Type type, IConfig config, Boolean fullValidation) in D:\a\BenchmarkDotNet\BenchmarkDotNet\tests\BenchmarkDotNet.IntegrationTests\BenchmarkTestExecutor.cs:line 70
   at BenchmarkDotNet.IntegrationTests.DisassemblyDiagnoserTests.CanDisassembleAllMethodCalls(Jit jit, Platform platform, Runtime runtime) in D:\a\BenchmarkDotNet\BenchmarkDotNet\tests\BenchmarkDotNet.IntegrationTests\DisassemblyDiagnoserTests.cs:line 101
  Standard Output Messages:
 // Validating benchmarks:
 // Benchmark WithCalls.Benchmark: Dry(Jit=LegacyJit, Platform=X86, Runtime=.NET Framework 4.6.2, IterationCount=1, LaunchCount=1, RunStrategy=ColdStart, UnrollFactor=1, WarmupCount=1) [someArgument=2147483647]
 //    * The required .NET Framework SDK version 4.6.2 for runtime moniker Net462 is not installed.

Maybe some faulty logic with the Framework sdk validation (maybe newer Framework sdks are backwards-compatible?).

[MattFromRVA]

Tests are failing

Failed CanDisassembleAllMethodCalls(jit: LegacyJit, platform: X86, runtime: .NET Framework 4.6.2) [13 ms]
  Error Message:
   The "Summary" should have NOT "HasCriticalValidationErrors"
  Stack Trace:
     at BenchmarkDotNet.IntegrationTests.BenchmarkTestExecutor.CanExecute(Type type, IConfig config, Boolean fullValidation) in D:\a\BenchmarkDotNet\BenchmarkDotNet\tests\BenchmarkDotNet.IntegrationTests\BenchmarkTestExecutor.cs:line 70
   at BenchmarkDotNet.IntegrationTests.DisassemblyDiagnoserTests.CanDisassembleAllMethodCalls(Jit jit, Platform platform, Runtime runtime) in D:\a\BenchmarkDotNet\BenchmarkDotNet\tests\BenchmarkDotNet.IntegrationTests\DisassemblyDiagnoserTests.cs:line 101
  Standard Output Messages:
 // Validating benchmarks:
 // Benchmark WithCalls.Benchmark: Dry(Jit=LegacyJit, Platform=X86, Runtime=.NET Framework 4.6.2, IterationCount=1, LaunchCount=1, RunStrategy=ColdStart, UnrollFactor=1, WarmupCount=1) [someArgument=2147483647]
 //    * The required .NET Framework SDK version 4.6.2 for runtime moniker Net462 is not installed.

Maybe some faulty logic with the Framework sdk validation (maybe newer Framework sdks are backwards-compatible?).

You are correct. Framework sdks are backwards-compatible and it only shows the latest in the registry. I have updated my code to reflect that.

[timcassell]

Thanks a lot @MattFromRVA!

[caaavik-msft]

This change causes wasm runs to fail (and potentially others I haven't tested) because not all runtime monikers are accounted for in GetSdkVersionFromMoniker. I have a PR that adds an integration test for wasm runs which reproduced this and I have included a fix for it by updating GetSdkVersionFromMoniker there. I think there are some other monikers missing such as the MonoAotLLVM?

[timcassell]

This change causes wasm runs to fail (and potentially others I haven't tested) because not all runtime monikers are accounted for in GetSdkVersionFromMoniker. I have a PR that adds an integration test for wasm runs which reproduced this and I have included a fix for it by updating GetSdkVersionFromMoniker there. I think there are some other monikers missing such as the MonoAotLLVM?

Oops. This is why we need those integration tests. If someone sends another PR with that fix, I can approve and merge (I can't merge my own PRs).

[caaavik-msft]

PR up now: #2538