Dotnet test doesn't run separate TargetFrameworks in parallel

Issue Description

Attempting to run multiple unit tests in parallel for a test project with multiple target frameworks. However, there is no way to do this out of the box. The DispatchToInnerBuildsWithVSTestTarget target in Microsoft.Common.CrossTargeting.targets\ImportAfter\Microsoft.TestPlatform.CrossTargeting.targets that recursively calls VSTest does not set the BuildInParallel option on the MSBuild task. Exerpt from version 5.0.302 of the sdk:

  <Target Name="DispatchToInnerBuildsWithVSTestTarget" Returns="@(InnerOutput)">
    <ItemGroup>
      <_TargetFramework Include="$(TargetFrameworks)" />
    </ItemGroup>
    <MSBuild Projects="$(MSBuildProjectFile)"
             Condition="'$(TargetFrameworks)' != '' "
             Targets="$(InnerVSTestTargets)"
             Properties="TargetFramework=%(_TargetFramework.Identity);VSTestNoBuild=true"
             ContinueOnError="ErrorAndContinue">
      <Output ItemName="InnerOutput" TaskParameter="TargetOutputs" />
    </MSBuild>
  </Target>

Steps to Reproduce

1. Create a new mstest project, e.g. dotnet new mstest
2. enable multiple target frameworks for the test, with <TargetFrameworks>net462;netcoreapp2.1;net5.0</TargetFrameworks>
3. run dotnet test or dotnet msbuild /restore /t:VSTest /p:TestInParallel=true

Expected

Tests for each targetframework run in parallel, as long as the user selects the appropriate command line option or configuration.

Actual

Tests never run in parallel.

Data

Analysis

Versions & Configurations

aaronla@aaronla-rabbit:PowerPlatform$ dotnet --version
5.0.302

aaronla@aaronla-rabbit:PowerPlatform$ dotnet msbuild --nologo --version
16.10.2.30804

Regression?

Attach a binlog

Recommended fix:

<Target Name="DispatchToInnerBuildsWithVSTestTarget" Returns="@(InnerOutput)">
    <PropertyGroup>
      <!-- Emulating technique found in Microsoft.Build.Traversal for deciding whether to run tests in parallel -->
      <_TestInParallel>$([MSBuild]::ValueOrDefault('$(TestInParallel)', '$(BuildInParallel)'))</_TestInParallel>
    </PropertyGroup>
    <ItemGroup>
      <_TargetFramework Include="$(TargetFrameworks)" />
      <_MSBuildProjectFile Include="$(MSBuildProjectFile)" Properties="TargetFramework=%(_TargetFramework.Identity);VSTestNoBuild=true" />
    </ItemGroup>
    <Message Importance="High" Text="Starting custom DispatchToInnerBuildsWithVSTestTarget" />
    <MSBuild Projects="@(_MSBuildProjectFile)"
             Condition="'$(TargetFrameworks)' != '' "
             Targets="$(InnerVSTestTargets)"
             ContinueOnError="ErrorAndContinue"
             BuildInParallel="$(_TestInParallel)">
      <Output ItemName="InnerOutput" TaskParameter="TargetOutputs" />
    </MSBuild>
  </Target>

I couldn't figure out the best area label to add to this issue. If you have write-permissions please help me learn by adding exactly one area label.

[AR-May]

Seems like target "DispatchToInnerBuildsWithVSTestTarget" lives in this repo. Please, take a look.
cc @nohwnd

I should clarify, I understand the desire for test to run serially by default. The request is for an option to enable running in parallel.

When we manually made this change in our repo, we saw an additional 45% speedup in test execution times, bringing test times down from over 6 minutes to about 4 minutes.

[wli3]

@nohwnd could you look into this?

[0xced]

I'd love to see this issue addressed. Currently, I'm running tests in parallel across different target frameworks by passing the built dlls + the --parallel option to dotnet test instead of the sln or csproj file (so that --parallel is forwarded to vstest).

For example:

dotnet test test\Serilog.Settings.Configuration.Tests\bin\Release\*\Serilog.Settings.Configuration.Tests.dll --parallel

This has the drawback of not reporting the correct target framework in the test summary, as explained by @nohwnd in microsoft/vstest#2037 (comment).

It would be much better if we could do this instead and get correct target framework in the test summary:

dotnet test test\Serilog.Settings.Configuration.Tests --parallel --configuration Release