Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Honor --arch switch for arm64 on Windows and Mac #3100

Merged
merged 44 commits into from
Oct 27, 2021

Conversation

MarcoRossignoli
Copy link
Contributor

@MarcoRossignoli MarcoRossignoli commented Oct 13, 2021

@MarcoRossignoli
Copy link
Contributor Author

MarcoRossignoli commented Oct 20, 2021

Manual testing on Darwin XXXXX Darwin Kernel Version 20.5.0: Sat May 8 05:10:31 PDT 2021; root:xnu-7195.121.3~9/RELEASE_ARM64_T8101 arm64

Installed sdk(in this order):
sudo installer -verboseR -dumplog -pkg .../installers/dotnet-sdk-5.0.402-osx-x64.pkg -target / -> this is broken, it installs runtime on native global folder close to arm one.
sudo installer -verboseR -dumplog -pkg .../installers/dotnet-sdk-6.0.100-rc.2.21505.57-osx-arm64.pkg -target /
sudo installer -verboseR -dumplog -pkg .../installers/dotnet-sdk-6.0.100-rc.2.21505.57-osx-x64.pkg -target /

Global install locations /etc/dotnet/install_location-install_location_{arch}
cat /etc/dotnet/install_location -> /usr/local/share/dotnet/x64
cat /etc/dotnet/install_location_x64 -> /usr/local/share/dotnet/x64
cat /etc/dotnet/install_location_arm64 -> /usr/local/share/dotnet

Sample project(using local TP build)

<Project Sdk="Microsoft.NET.Sdk">
  <PropertyGroup>
    <TargetFrameworks>net6.0;net5.0</TargetFrameworks>
    <IsPackable>false</IsPackable>
  </PropertyGroup>
  <ItemGroup>
    <PackageReference Include="Microsoft.NET.Test.Sdk" Version="17.1.0-dev" />
    <PackageReference Include="MSTest.TestAdapter" Version="2.2.7" />
    <PackageReference Include="MSTest.TestFramework" Version="2.2.7" />
    <PackageReference Include="coverlet.collector" Version="3.0.2" />
  </ItemGroup>
</Project>

Source

using Microsoft.VisualStudio.TestTools.UnitTesting;

namespace TestProjectNetcore
{
    [TestClass]
    public class UnitTest1
    {
        [TestMethod]
        public void TestMethod1()
        {
           System.Console.WriteLine("OSArchitecture " + System.Runtime.InteropServices.RuntimeInformation.OSArchitecture.ToString());
           System.Console.WriteLine("ProcessArchitecture " + System.Runtime.InteropServices.RuntimeInformation.ProcessArchitecture.ToString());
           Assert.IsTrue(false);
        }
    }
}

We did some update to the sdk for the test verb, to be able to run manual test I need to set a temporary custom env var called VSTEST_TMP_SWITCH_DOTNETROOTS_ENVVARS to move DOTNET_ROOT/DOTNET_ROOT(x86) vars to a new VSTEST only vars VSTEST_WINAPPHOST_DOTNET_ROOT/VSTEST_WINAPPHOST_DOTNET_ROOT(x86). In this way searching alg won't be affected. The update is already in draft on SDK side.
We're injecting local TP build using -p:VSTestConsolePath

Scenario 1: use global arm64 sdk installation

dotnet --info
.NET SDK (reflecting any global.json):
 Version:   6.0.100-rc.2.21505.57
 Commit:    ab39070116

Runtime Environment:
 OS Name:     Mac OS X
 OS Version:  11.4
 OS Platform: Darwin
 RID:         osx.11.0-arm64
 Base Path:   /usr/local/share/dotnet/sdk/6.0.100-rc.2.21505.57/

Host (useful for support):
  Version: 6.0.0-rc.2.21480.5
  Commit:  6b11d64e7e

.NET SDKs installed:
  6.0.100-rc.2.21505.57 [/usr/local/share/dotnet/sdk]

Command line

 dotnet test -p:VSTestConsolePath=".../TP/Debug/netcoreapp2.1/vstest.console.dll" TestProjectMac

Results

Determining projects to restore...
  All projects are up-to-date for restore.
  You are using a preview version of .NET. See: https://aka.ms/dotnet-core-preview
  TestProjectNetcore -> .../TestProjectMac/TestProjectNetcore/bin/Debug/net6.0/TestProjectNetcore.dll
  You are using a preview version of .NET. See: https://aka.ms/dotnet-core-preview
  TestProjectNetcore -> .../TestProjectMac/TestProjectNetcore/bin/Debug/net5.0/TestProjectNetcore.dll
Test run for .../TestProjectMac/TestProjectNetcore/bin/Debug/net6.0/TestProjectNetcore.dll (.NETCoreApp,Version=v6.0)
Forwarding DOTNET_ROOT to VSTEST_WINAPPHOST_DOTNET_ROOT '/usr/local/share/dotnet'
Current DOTNET_ROOT ''
Microsoft (R) Test Execution Command Line Tool Version 17.1.0-dev
Copyright (c) Microsoft Corporation.  All rights reserved.

Starting test execution, please wait...
Logging Vstest Diagnostics in file: .../log.txt
A total of 1 test files matched the specified pattern.
  Failed TestMethod1 [63 ms]
  Error Message:
   Assert.IsTrue failed.
  Stack Trace:
     at TestProjectNetcore.UnitTest1.TestMethod1() in .../TestProjectMac/TestProjectNetcore/UnitTest1.cs:line 13

  Standard Output Messages:
 OSArchitecture Arm64
 ProcessArchitecture Arm64



Failed!  - Failed:     1, Passed:     0, Skipped:     0, Total:     1, Duration: 71 ms - .../TestProjectMac/TestProjectNetcore/bin/Debug/net6.0/TestProjectNetcore.dll (net6.0)
Test run for .../TestProjectMac/TestProjectNetcore/bin/Debug/net5.0/TestProjectNetcore.dll (.NETCoreApp,Version=v5.0)
Forwarding DOTNET_ROOT to VSTEST_WINAPPHOST_DOTNET_ROOT '/usr/local/share/dotnet'
Current DOTNET_ROOT ''
Microsoft (R) Test Execution Command Line Tool Version 17.1.0-dev
Copyright (c) Microsoft Corporation.  All rights reserved.

Starting test execution, please wait...
Logging Vstest Diagnostics in file: .../log.txt
A total of 1 test files matched the specified pattern.
  Failed TestMethod1 [86 ms]
  Error Message:
   Assert.IsTrue failed.
  Stack Trace:
     at TestProjectNetcore.UnitTest1.TestMethod1() in .../TestProjectMac/TestProjectNetcore/UnitTest1.cs:line 13

  Standard Output Messages:
 OSArchitecture X64
 ProcessArchitecture X64



Failed!  - Failed:     1, Passed:     0, Skipped:     0, Total:     1, Duration: 111 ms - .../TestProjectMac/TestProjectNetcore/bin/Debug/net5.0/TestProjectNetcore.dll (net5.0)

Run tests with --arch x64

dotnet test -p:VSTestConsolePath=".../TP/Debug/netcoreapp2.1/vstest.console.dll" TestProjectMac --arch x64

Result

Determining projects to restore...
  Restored .../TestProjectMac/TestProjectNetcore/TestProjectNetcore.csproj (in 511 ms).
  You are using a preview version of .NET. See: https://aka.ms/dotnet-core-preview
  TestProjectNetcore -> .../TestProjectMac/TestProjectNetcore/bin/Debug/net6.0/osx-x64/TestProjectNetcore.dll
  You are using a preview version of .NET. See: https://aka.ms/dotnet-core-preview
  TestProjectNetcore -> .../TestProjectMac/TestProjectNetcore/bin/Debug/net5.0/osx-x64/TestProjectNetcore.dll
Test run for .../TestProjectMac/TestProjectNetcore/bin/Debug/net6.0/osx-x64/TestProjectNetcore.dll (.NETCoreApp,Version=v6.0)
Forwarding DOTNET_ROOT to VSTEST_WINAPPHOST_DOTNET_ROOT '/usr/local/share/dotnet'
Current DOTNET_ROOT ''
Microsoft (R) Test Execution Command Line Tool Version 17.1.0-dev
Copyright (c) Microsoft Corporation.  All rights reserved.

Starting test execution, please wait...
Logging Vstest Diagnostics in file: .../log.txt
A total of 1 test files matched the specified pattern.
  Failed TestMethod1 [84 ms]
  Error Message:
   Assert.IsTrue failed.
  Stack Trace:
     at TestProjectNetcore.UnitTest1.TestMethod1() in .../TestProjectMac/TestProjectNetcore/UnitTest1.cs:line 13

  Standard Output Messages:
 OSArchitecture X64
 ProcessArchitecture X64



Failed!  - Failed:     1, Passed:     0, Skipped:     0, Total:     1, Duration: 110 ms - .../TestProjectMac/TestProjectNetcore/bin/Debug/net6.0/osx-x64/TestProjectNetcore.dll (net6.0)
Test run for .../TestProjectMac/TestProjectNetcore/bin/Debug/net5.0/osx-x64/TestProjectNetcore.dll (.NETCoreApp,Version=v5.0)
Forwarding DOTNET_ROOT to VSTEST_WINAPPHOST_DOTNET_ROOT '/usr/local/share/dotnet'
Current DOTNET_ROOT ''
Microsoft (R) Test Execution Command Line Tool Version 17.1.0-dev
Copyright (c) Microsoft Corporation.  All rights reserved.

Starting test execution, please wait...
Logging Vstest Diagnostics in file: .../log.txt
A total of 1 test files matched the specified pattern.
  Failed TestMethod1 [50 ms]
  Error Message:
   Assert.IsTrue failed.
  Stack Trace:
     at TestProjectNetcore.UnitTest1.TestMethod1() in .../TestProjectMac/TestProjectNetcore/UnitTest1.cs:line 13

  Standard Output Messages:
 OSArchitecture X64
 ProcessArchitecture X64



Failed!  - Failed:     1, Passed:     0, Skipped:     0, Total:     1, Duration: 86 ms - .../TestProjectMac/TestProjectNetcore/bin/Debug/net5.0/osx-x64/TestProjectNetcore.dll (net5.0)

Scenario 2: private x64 installation

./dotnetX64/dotnet --info
.NET SDK (reflecting any global.json):
 Version:   6.0.100-rc.2.21505.57
 Commit:    ab39070116

Runtime Environment:
 OS Name:     Mac OS X
 OS Version:  11.4
 OS Platform: Darwin
 RID:         osx.11.0-x64
 Base Path:   .../dotnetX64/sdk/6.0.100-rc.2.21505.57/

Host (useful for support):
  Version: 6.0.0-rc.2.21480.5
  Commit:  6b11d64e7e

.NET SDKs installed:
  6.0.100-rc.2.21505.57 [.../dotnetX64/sdk]

.NET runtimes installed:
  Microsoft.AspNetCore.App 6.0.0-rc.2.21480.10 [.../dotnetX64/shared/Microsoft.AspNetCore.App]
  Microsoft.NETCore.App 6.0.0-rc.2.21480.5 [.../dotnetX64/shared/Microsoft.NETCore.App]

Command line

 ./dotnetX64/dotnet test -p:VSTestConsolePath=".../TP/Debug/netcoreapp2.1/vstest.console.dll" TestProjectMac

Result

./dotnetX64/dotnet test -p:VSTestConsolePath=".../TP/Debug/netcoreapp2.1/vstest.console.dll" TestProjectMac --diag:test.txt /bl
.../dotnetX64/sdk/6.0.100-rc.2.21505.57/MSBuild.dll -nologo -distributedlogger:Microsoft.DotNet.Tools.MSBuild.MSBuildLogger,.../dotnetX64/sdk/6.0.100-rc.2.21505.57/dotnet.dll*Microsoft.DotNet.Tools.MSBuild.MSBuildForwardingLogger,.../dotnetX64/sdk/6.0.100-rc.2.21505.57/dotnet.dll -maxcpucount -nodereuse:false -property:VSTestDiag=.../test.txt -p:VSTestConsolePath=.../TP/Debug/netcoreapp2.1/vstest.console.dll -restore -target:VSTest -verbosity:m /bl TestProjectMac/Sample.sln
  Determining projects to restore...
  Restored .../TestProjectMac/TestProjectNetcore/TestProjectNetcore.csproj (in 603 ms).
  You are using a preview version of .NET. See: https://aka.ms/dotnet-core-preview
  TestProjectNetcore -> .../TestProjectMac/TestProjectNetcore/bin/Debug/net6.0/TestProjectNetcore.dll
  You are using a preview version of .NET. See: https://aka.ms/dotnet-core-preview
  TestProjectNetcore -> .../TestProjectMac/TestProjectNetcore/bin/Debug/net5.0/TestProjectNetcore.dll
Test run for .../TestProjectMac/TestProjectNetcore/bin/Debug/net6.0/TestProjectNetcore.dll (.NETCoreApp,Version=v6.0)
Microsoft (R) Test Execution Command Line Tool Version 17.1.0-dev
Copyright (c) Microsoft Corporation.  All rights reserved.

Starting test execution, please wait...
Logging Vstest Diagnostics in file: .../test.txt
A total of 1 test files matched the specified pattern.
  Failed TestMethod1 [61 ms]
  Error Message:
   Assert.IsTrue failed.
  Stack Trace:
     at TestProjectNetcore.UnitTest1.TestMethod1() in .../TestProjectMac/TestProjectNetcore/UnitTest1.cs:line 13

  Standard Output Messages:
 OSArchitecture X64
 ProcessArchitecture X64



Failed!  - Failed:     1, Passed:     0, Skipped:     0, Total:     1, Duration: 86 ms - .../TestProjectMac/TestProjectNetcore/bin/Debug/net6.0/TestProjectNetcore.dll (net6.0)
Test run for .../TestProjectMac/TestProjectNetcore/bin/Debug/net5.0/TestProjectNetcore.dll (.NETCoreApp,Version=v5.0)
Microsoft (R) Test Execution Command Line Tool Version 17.1.0-dev
Copyright (c) Microsoft Corporation.  All rights reserved.

Starting test execution, please wait...
Logging Vstest Diagnostics in file: .../test.txt
A total of 1 test files matched the specified pattern.
Testhost process exited with error: . Please check the diagnostic logs for more information.

Test Run Aborted.

net5.0 is failing because we're using private net6.0/x64 installation

Run tests with --arch arm64

./dotnetX64/dotnet test -p:VSTestConsolePath=".../TP/Debug/netcoreapp2.1/vstest.console.dll" TestProjectMac --arch arm64

Result

Determining projects to restore...
  All projects are up-to-date for restore.
  You are using a preview version of .NET. See: https://aka.ms/dotnet-core-preview
  TestProjectNetcore -> .../TestProjectMac/TestProjectNetcore/bin/Debug/net6.0/osx-arm64/TestProjectNetcore.dll
  You are using a preview version of .NET. See: https://aka.ms/dotnet-core-preview
  TestProjectNetcore -> .../TestProjectMac/TestProjectNetcore/bin/Debug/net5.0/osx-arm64/TestProjectNetcore.dll
Test run for .../TestProjectMac/TestProjectNetcore/bin/Debug/net6.0/osx-arm64/TestProjectNetcore.dll (.NETCoreApp,Version=v6.0)
Forwarding DOTNET_ROOT to VSTEST_WINAPPHOST_DOTNET_ROOT '.../dotnetX64'
Current DOTNET_ROOT ''
Microsoft (R) Test Execution Command Line Tool Version 17.1.0-dev
Copyright (c) Microsoft Corporation.  All rights reserved.

Starting test execution, please wait...
Logging Vstest Diagnostics in file: .../log.txt
A total of 1 test files matched the specified pattern.
  Failed TestMethod1 [63 ms]
  Error Message:
   Assert.IsTrue failed.
  Stack Trace:
     at TestProjectNetcore.UnitTest1.TestMethod1() in .../TestProjectMac/TestProjectNetcore/UnitTest1.cs:line 13

  Standard Output Messages:
 OSArchitecture Arm64
 ProcessArchitecture Arm64



Failed!  - Failed:     1, Passed:     0, Skipped:     0, Total:     1, Duration: 71 ms - .../TestProjectMac/TestProjectNetcore/bin/Debug/net6.0/osx-arm64/TestProjectNetcore.dll (net6.0)
Test run for .../TestProjectMac/TestProjectNetcore/bin/Debug/net5.0/osx-arm64/TestProjectNetcore.dll (.NETCoreApp,Version=v5.0)
Forwarding DOTNET_ROOT to VSTEST_WINAPPHOST_DOTNET_ROOT '.../dotnetX64'
Current DOTNET_ROOT ''
Microsoft (R) Test Execution Command Line Tool Version 17.1.0-dev
Copyright (c) Microsoft Corporation.  All rights reserved.

Starting test execution, please wait...
Logging Vstest Diagnostics in file: .../log.txt
A total of 1 test files matched the specified pattern.
Testhost process exited with error: . Please check the diagnostic logs for more information.

Test Run Aborted.

Scenario 3: private arm64 installation

./dotnet/dotnet --info
.NET SDK (reflecting any global.json):
 Version:   6.0.100-rc.2.21505.57
 Commit:    ab39070116

Runtime Environment:
 OS Name:     Mac OS X
 OS Version:  11.4
 OS Platform: Darwin
 RID:         osx.11.0-arm64
 Base Path:   .../dotnet/sdk/6.0.100-rc.2.21505.57/

Host (useful for support):
  Version: 6.0.0-rc.2.21480.5
  Commit:  6b11d64e7e

.NET SDKs installed:
  6.0.100-rc.2.21505.57 [.../dotnet/sdk]

.NET runtimes installed:
  Microsoft.AspNetCore.App 6.0.0-rc.2.21480.10 [.../dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.NETCore.App 6.0.0-rc.2.21480.5 [.../dotnet/shared/Microsoft.NETCore.App]

Command line

 ./dotnet/dotnet test -p:VSTestConsolePath=".../TP/Debug/netcoreapp2.1/vstest.console.dll" TestProjectMac

Result

Determining projects to restore...
  Restored .../TestProjectMac/TestProjectNetcore/TestProjectNetcore.csproj (in 394 ms).
  You are using a preview version of .NET. See: https://aka.ms/dotnet-core-preview
  TestProjectNetcore -> .../TestProjectMac/TestProjectNetcore/bin/Debug/net6.0/TestProjectNetcore.dll
  You are using a preview version of .NET. See: https://aka.ms/dotnet-core-preview
  TestProjectNetcore -> .../TestProjectMac/TestProjectNetcore/bin/Debug/net5.0/TestProjectNetcore.dll
Test run for .../TestProjectMac/TestProjectNetcore/bin/Debug/net6.0/TestProjectNetcore.dll (.NETCoreApp,Version=v6.0)
Forwarding DOTNET_ROOT to VSTEST_WINAPPHOST_DOTNET_ROOT '.../dotnet'
Current DOTNET_ROOT ''
Microsoft (R) Test Execution Command Line Tool Version 17.1.0-dev
Copyright (c) Microsoft Corporation.  All rights reserved.

Starting test execution, please wait...
Logging Vstest Diagnostics in file: .../log.txt
A total of 1 test files matched the specified pattern.
  Failed TestMethod1 [63 ms]
  Error Message:
   Assert.IsTrue failed.
  Stack Trace:
     at TestProjectNetcore.UnitTest1.TestMethod1() in .../TestProjectMac/TestProjectNetcore/UnitTest1.cs:line 13

  Standard Output Messages:
 OSArchitecture Arm64
 ProcessArchitecture Arm64



Failed!  - Failed:     1, Passed:     0, Skipped:     0, Total:     1, Duration: 71 ms - .../TestProjectMac/TestProjectNetcore/bin/Debug/net6.0/TestProjectNetcore.dll (net6.0)
Test run for .../TestProjectMac/TestProjectNetcore/bin/Debug/net5.0/TestProjectNetcore.dll (.NETCoreApp,Version=v5.0)
Forwarding DOTNET_ROOT to VSTEST_WINAPPHOST_DOTNET_ROOT '.../dotnet'
Current DOTNET_ROOT ''
Microsoft (R) Test Execution Command Line Tool Version 17.1.0-dev
Copyright (c) Microsoft Corporation.  All rights reserved.

Starting test execution, please wait...
Logging Vstest Diagnostics in file: .../log.txt
A total of 1 test files matched the specified pattern.
  Failed TestMethod1 [51 ms]
  Error Message:
   Assert.IsTrue failed.
  Stack Trace:
     at TestProjectNetcore.UnitTest1.TestMethod1() in .../TestProjectMac/TestProjectNetcore/UnitTest1.cs:line 13

  Standard Output Messages:
 OSArchitecture X64
 ProcessArchitecture X64



Failed!  - Failed:     1, Passed:     0, Skipped:     0, Total:     1, Duration: 75 ms - .../TestProjectMac/TestProjectNetcore/bin/Debug/net5.0/TestProjectNetcore.dll (net5.0)

Run tests with --arch x64

./dotnet/dotnet test -p:VSTestConsolePath=".../TP/Debug/netcoreapp2.1/vstest.console.dll" TestProjectMac --arch x64

Result

Determining projects to restore...
  Restored .../TestProjectMac/TestProjectNetcore/TestProjectNetcore.csproj (in 502 ms).
  You are using a preview version of .NET. See: https://aka.ms/dotnet-core-preview
  TestProjectNetcore -> .../TestProjectMac/TestProjectNetcore/bin/Debug/net6.0/osx-x64/TestProjectNetcore.dll
  You are using a preview version of .NET. See: https://aka.ms/dotnet-core-preview
  TestProjectNetcore -> .../TestProjectMac/TestProjectNetcore/bin/Debug/net5.0/osx-x64/TestProjectNetcore.dll
Test run for .../TestProjectMac/TestProjectNetcore/bin/Debug/net6.0/osx-x64/TestProjectNetcore.dll (.NETCoreApp,Version=v6.0)
Forwarding DOTNET_ROOT to VSTEST_WINAPPHOST_DOTNET_ROOT '.../dotnet'
Current DOTNET_ROOT ''
Microsoft (R) Test Execution Command Line Tool Version 17.1.0-dev
Copyright (c) Microsoft Corporation.  All rights reserved.

Starting test execution, please wait...
Logging Vstest Diagnostics in file: .../log.txt
A total of 1 test files matched the specified pattern.
  Failed TestMethod1 [53 ms]
  Error Message:
   Assert.IsTrue failed.
  Stack Trace:
     at TestProjectNetcore.UnitTest1.TestMethod1() in .../TestProjectMac/TestProjectNetcore/UnitTest1.cs:line 13

  Standard Output Messages:
 OSArchitecture X64
 ProcessArchitecture X64



Failed!  - Failed:     1, Passed:     0, Skipped:     0, Total:     1, Duration: 78 ms - .../TestProjectMac/TestProjectNetcore/bin/Debug/net6.0/osx-x64/TestProjectNetcore.dll (net6.0)
Test run for .../TestProjectMac/TestProjectNetcore/bin/Debug/net5.0/osx-x64/TestProjectNetcore.dll (.NETCoreApp,Version=v5.0)
Forwarding DOTNET_ROOT to VSTEST_WINAPPHOST_DOTNET_ROOT '.../dotnet'
Current DOTNET_ROOT ''
Microsoft (R) Test Execution Command Line Tool Version 17.1.0-dev
Copyright (c) Microsoft Corporation.  All rights reserved.

Starting test execution, please wait...
Logging Vstest Diagnostics in file: .../log.txt
A total of 1 test files matched the specified pattern.
  Failed TestMethod1 [51 ms]
  Error Message:
   Assert.IsTrue failed.
  Stack Trace:
     at TestProjectNetcore.UnitTest1.TestMethod1() in .../TestProjectMac/TestProjectNetcore/UnitTest1.cs:line 13

  Standard Output Messages:
 OSArchitecture X64
 ProcessArchitecture X64



Failed!  - Failed:     1, Passed:     0, Skipped:     0, Total:     1, Duration: 75 ms - .../TestProjectMac/TestProjectNetcore/bin/Debug/net5.0/osx-x64/TestProjectNetcore.dll (net5.0)

Scenario 4: only private install using env vars

dotnet
zsh: command not found: dotnet

Private arm64 installation

./dotnet/dotnet test -p:VSTestConsolePath=".../TP/Debug/netcoreapp2.1/vstest.console.dll" TestProjectMac

Result

./dotnet/dotnet test -p:VSTestConsolePath=".../TP/Debug/netcoreapp2.1/vstest.console.dll" TestProjectMac
  Determining projects to restore...
  Restored .../TestProjectMac/TestProjectNetcore/TestProjectNetcore.csproj (in 398 ms).
  You are using a preview version of .NET. See: https://aka.ms/dotnet-core-preview
  TestProjectNetcore -> .../TestProjectMac/TestProjectNetcore/bin/Debug/net6.0/TestProjectNetcore.dll
  You are using a preview version of .NET. See: https://aka.ms/dotnet-core-preview
  TestProjectNetcore -> .../TestProjectMac/TestProjectNetcore/bin/Debug/net5.0/TestProjectNetcore.dll
Test run for .../TestProjectMac/TestProjectNetcore/bin/Debug/net6.0/TestProjectNetcore.dll (.NETCoreApp,Version=v6.0)
Forwarding DOTNET_ROOT to VSTEST_WINAPPHOST_DOTNET_ROOT '.../dotnet'
Current DOTNET_ROOT ''
Microsoft (R) Test Execution Command Line Tool Version 17.1.0-dev
Copyright (c) Microsoft Corporation.  All rights reserved.

Starting test execution, please wait...
A total of 1 test files matched the specified pattern.
  Failed TestMethod1 [62 ms]
  Error Message:
   Assert.IsTrue failed.
  Stack Trace:
     at TestProjectNetcore.UnitTest1.TestMethod1() in .../TestProjectMac/TestProjectNetcore/UnitTest1.cs:line 13

  Standard Output Messages:
 OSArchitecture Arm64
 ProcessArchitecture Arm64



Failed!  - Failed:     1, Passed:     0, Skipped:     0, Total:     1, Duration: 72 ms - .../TestProjectMac/TestProjectNetcore/bin/Debug/net6.0/TestProjectNetcore.dll (net6.0)
Test run for .../TestProjectMac/TestProjectNetcore/bin/Debug/net5.0/TestProjectNetcore.dll (.NETCoreApp,Version=v5.0)
Forwarding DOTNET_ROOT to VSTEST_WINAPPHOST_DOTNET_ROOT '.../dotnet'
Current DOTNET_ROOT ''
Microsoft (R) Test Execution Command Line Tool Version 17.1.0-dev
Copyright (c) Microsoft Corporation.  All rights reserved.

Starting test execution, please wait...
A total of 1 test files matched the specified pattern.
Could not find 'dotnet' host for the 'x64' architecture.  Make sure that 'dotnet' for 'x64' architecture is installed on the machine.

Test Run Aborted.

Forwarding for net5.0 fails because no x64 found.
Set env

export DOTNET_ROOT_X64=.../dotnetX64

After

Determining projects to restore...
  All projects are up-to-date for restore.
  You are using a preview version of .NET. See: https://aka.ms/dotnet-core-preview
  TestProjectNetcore -> .../TestProjectMac/TestProjectNetcore/bin/Debug/net6.0/TestProjectNetcore.dll
  You are using a preview version of .NET. See: https://aka.ms/dotnet-core-preview
  TestProjectNetcore -> .../TestProjectMac/TestProjectNetcore/bin/Debug/net5.0/TestProjectNetcore.dll
Test run for .../TestProjectMac/TestProjectNetcore/bin/Debug/net6.0/TestProjectNetcore.dll (.NETCoreApp,Version=v6.0)
Forwarding DOTNET_ROOT to VSTEST_WINAPPHOST_DOTNET_ROOT '.../dotnet'
Current DOTNET_ROOT ''
Microsoft (R) Test Execution Command Line Tool Version 17.1.0-dev
Copyright (c) Microsoft Corporation.  All rights reserved.

Starting test execution, please wait...
Logging Vstest Diagnostics in file: .../log.txt
A total of 1 test files matched the specified pattern.
  Failed TestMethod1 [63 ms]
  Error Message:
   Assert.IsTrue failed.
  Stack Trace:
     at TestProjectNetcore.UnitTest1.TestMethod1() in .../TestProjectMac/TestProjectNetcore/UnitTest1.cs:line 13

  Standard Output Messages:
 OSArchitecture Arm64
 ProcessArchitecture Arm64



Failed!  - Failed:     1, Passed:     0, Skipped:     0, Total:     1, Duration: 71 ms - .../TestProjectMac/TestProjectNetcore/bin/Debug/net6.0/TestProjectNetcore.dll (net6.0)
Test run for .../TestProjectMac/TestProjectNetcore/bin/Debug/net5.0/TestProjectNetcore.dll (.NETCoreApp,Version=v5.0)
Forwarding DOTNET_ROOT to VSTEST_WINAPPHOST_DOTNET_ROOT '.../dotnet'
Current DOTNET_ROOT ''
Microsoft (R) Test Execution Command Line Tool Version 17.1.0-dev
Copyright (c) Microsoft Corporation.  All rights reserved.

Starting test execution, please wait...
Logging Vstest Diagnostics in file: .../log.txt
A total of 1 test files matched the specified pattern.
Testhost process exited with error: . Please check the diagnostic logs for more information.

Test Run Aborted.

Like the above net5.0 fails because we're redirecting to x64 and it's present but doesn't contain net5.0 runtime
Run tests with --arch x64

./dotnet/dotnet test -p:VSTestConsolePath=".../TP/Debug/netcoreapp2.1/vstest.console.dll" TestProjectMac  --arch x64

Result

Determining projects to restore...
  Restored .../TestProjectMac/TestProjectNetcore/TestProjectNetcore.csproj (in 489 ms).
  You are using a preview version of .NET. See: https://aka.ms/dotnet-core-preview
  TestProjectNetcore -> .../TestProjectMac/TestProjectNetcore/bin/Debug/net6.0/osx-x64/TestProjectNetcore.dll
  You are using a preview version of .NET. See: https://aka.ms/dotnet-core-preview
  TestProjectNetcore -> .../TestProjectMac/TestProjectNetcore/bin/Debug/net5.0/osx-x64/TestProjectNetcore.dll
Test run for .../TestProjectMac/TestProjectNetcore/bin/Debug/net6.0/osx-x64/TestProjectNetcore.dll (.NETCoreApp,Version=v6.0)
Forwarding DOTNET_ROOT to VSTEST_WINAPPHOST_DOTNET_ROOT '.../dotnet'
Current DOTNET_ROOT ''
Microsoft (R) Test Execution Command Line Tool Version 17.1.0-dev
Copyright (c) Microsoft Corporation.  All rights reserved.

Starting test execution, please wait...
Logging Vstest Diagnostics in file: .../log.txt
A total of 1 test files matched the specified pattern.
  Failed TestMethod1 [52 ms]
  Error Message:
   Assert.IsTrue failed.
  Stack Trace:
     at TestProjectNetcore.UnitTest1.TestMethod1() in .../TestProjectMac/TestProjectNetcore/UnitTest1.cs:line 13

  Standard Output Messages:
 OSArchitecture X64
 ProcessArchitecture X64



Failed!  - Failed:     1, Passed:     0, Skipped:     0, Total:     1, Duration: 77 ms - .../TestProjectMac/TestProjectNetcore/bin/Debug/net6.0/osx-x64/TestProjectNetcore.dll (net6.0)
Test run for .../TestProjectMac/TestProjectNetcore/bin/Debug/net5.0/osx-x64/TestProjectNetcore.dll (.NETCoreApp,Version=v5.0)
Forwarding DOTNET_ROOT to VSTEST_WINAPPHOST_DOTNET_ROOT '.../dotnet'
Current DOTNET_ROOT ''
Microsoft (R) Test Execution Command Line Tool Version 17.1.0-dev
Copyright (c) Microsoft Corporation.  All rights reserved.

Starting test execution, please wait...
Logging Vstest Diagnostics in file: .../log.txt
A total of 1 test files matched the specified pattern.
Testhost process exited with error: . Please check the diagnostic logs for more information.

Test Run Aborted.

The net5.0 fails because we use private net6.0 x64 and muxer is not able to find the runtime, like the above
Unset arch specific env var and use DOTNET_ROOT

unset VSTEST_TMP_SWITCH_DOTNETROOTS_ENVVARS -> remove env var redirection
unset DOTNET_ROOT_X64
export DOTNET_ROOT=.../dotnetX64

Result

./dotnet/dotnet test -p:VSTestConsolePath=".../TP/Debug/netcoreapp2.1/vstest.console.dll" TestProjectMac --arch x64 --diag:test.log
zsh: no matches found: *.txt
  Determining projects to restore...
  All projects are up-to-date for restore.
  You are using a preview version of .NET. See: https://aka.ms/dotnet-core-preview
  TestProjectNetcore -> .../TestProjectMac/TestProjectNetcore/bin/Debug/net6.0/osx-x64/TestProjectNetcore.dll
  You are using a preview version of .NET. See: https://aka.ms/dotnet-core-preview
  TestProjectNetcore -> .../TestProjectMac/TestProjectNetcore/bin/Debug/net5.0/osx-x64/TestProjectNetcore.dll
Test run for .../TestProjectMac/TestProjectNetcore/bin/Debug/net6.0/osx-x64/TestProjectNetcore.dll (.NETCoreApp,Version=v6.0)
Microsoft (R) Test Execution Command Line Tool Version 17.1.0-dev
Copyright (c) Microsoft Corporation.  All rights reserved.

Starting test execution, please wait...
Logging Vstest Diagnostics in file: .../test.log
A total of 1 test files matched the specified pattern.
  Failed TestMethod1 [53 ms]
  Error Message:
   Assert.IsTrue failed.
  Stack Trace:
     at TestProjectNetcore.UnitTest1.TestMethod1() in .../TestProjectMac/TestProjectNetcore/UnitTest1.cs:line 13

  Standard Output Messages:
 OSArchitecture X64
 ProcessArchitecture X64



Failed!  - Failed:     1, Passed:     0, Skipped:     0, Total:     1, Duration: 77 ms - .../TestProjectMac/TestProjectNetcore/bin/Debug/net6.0/osx-x64/TestProjectNetcore.dll (net6.0)
Test run for .../TestProjectMac/TestProjectNetcore/bin/Debug/net5.0/osx-x64/TestProjectNetcore.dll (.NETCoreApp,Version=v5.0)
Microsoft (R) Test Execution Command Line Tool Version 17.1.0-dev
Copyright (c) Microsoft Corporation.  All rights reserved.

Starting test execution, please wait...
Logging Vstest Diagnostics in file: .../test.log
A total of 1 test files matched the specified pattern.
Testhost process exited with error: . Please check the diagnostic logs for more information.

Test Run Aborted.

The net5.0 fails because x64 private is net6.0 only
Private install x64

./dotnetX64/dotnet test -p:VSTestConsolePath=".../TP/Debug/netcoreapp2.1/vstest.console.dll" TestProjectMac

Result

Determining projects to restore...
  Restored .../TestProjectMac/TestProjectNetcore/TestProjectNetcore.csproj (in 584 ms).
  You are using a preview version of .NET. See: https://aka.ms/dotnet-core-preview
  You are using a preview version of .NET. See: https://aka.ms/dotnet-core-preview
  TestProjectNetcore -> .../TestProjectMac/TestProjectNetcore/bin/Debug/net6.0/TestProjectNetcore.dll
  TestProjectNetcore -> .../TestProjectMac/TestProjectNetcore/bin/Debug/net5.0/TestProjectNetcore.dll
Test run for .../TestProjectMac/TestProjectNetcore/bin/Debug/net6.0/TestProjectNetcore.dll (.NETCoreApp,Version=v6.0)
Forwarding DOTNET_ROOT to VSTEST_WINAPPHOST_DOTNET_ROOT '.../dotnetX64'
Current DOTNET_ROOT ''
Microsoft (R) Test Execution Command Line Tool Version 17.1.0-dev
Copyright (c) Microsoft Corporation.  All rights reserved.

Starting test execution, please wait...
Logging Vstest Diagnostics in file: .../log.txt
A total of 1 test files matched the specified pattern.
  Failed TestMethod1 [54 ms]
  Error Message:
   Assert.IsTrue failed.
  Stack Trace:
     at TestProjectNetcore.UnitTest1.TestMethod1() in .../TestProjectMac/TestProjectNetcore/UnitTest1.cs:line 13

  Standard Output Messages:
 OSArchitecture X64
 ProcessArchitecture X64



Failed!  - Failed:     1, Passed:     0, Skipped:     0, Total:     1, Duration: 79 ms - .../TestProjectMac/TestProjectNetcore/bin/Debug/net6.0/TestProjectNetcore.dll (net6.0)
Test run for .../TestProjectMac/TestProjectNetcore/bin/Debug/net5.0/TestProjectNetcore.dll (.NETCoreApp,Version=v5.0)
Forwarding DOTNET_ROOT to VSTEST_WINAPPHOST_DOTNET_ROOT '.../dotnetX64'
Current DOTNET_ROOT ''
Microsoft (R) Test Execution Command Line Tool Version 17.1.0-dev
Copyright (c) Microsoft Corporation.  All rights reserved.

Starting test execution, please wait...
Logging Vstest Diagnostics in file: .../log.txt
A total of 1 test files matched the specified pattern.
Testhost process exited with error: . Please check the diagnostic logs for more information.

Test Run Aborted.

The net5.0 fails because x64 private is net6.0 only
Run tests with --arch arm64

 ./dotnetX64/dotnet test -p:VSTestConsolePath=".../TP/Debug/netcoreapp2.1/vstest.console.dll" TestProjectMac --arch arm64 --diag:log.txt /bl

.../dotnetX64/sdk/6.0.100-rc.2.21505.57/MSBuild.dll -nologo -distributedlogger:Microsoft.DotNet.Tools.MSBuild.MSBuildLogger,.../dotnetX64/sdk/6.0.100-rc.2.21505.57/dotnet.dll*Microsoft.DotNet.Tools.MSBuild.MSBuildForwardingLogger,.../dotnetX64/sdk/6.0.100-rc.2.21505.57/dotnet.dll -maxcpucount -nodereuse:false -property:VSTestDiag=.../log.txt -property:RuntimeIdentifier=osx-arm64 -property:SelfContained=false -p:VSTestConsolePath=.../TP/Debug/netcoreapp2.1/vstest.console.dll -restore -target:VSTest -verbosity:m /bl TestProjectMac/Sample.sln
  Determining projects to restore...
  Restored .../TestProjectMac/TestProjectNetcore/TestProjectNetcore.csproj (in 747 ms).
  You are using a preview version of .NET. See: https://aka.ms/dotnet-core-preview
  TestProjectNetcore -> .../TestProjectMac/TestProjectNetcore/bin/Debug/net6.0/osx-arm64/TestProjectNetcore.dll
  You are using a preview version of .NET. See: https://aka.ms/dotnet-core-preview
  TestProjectNetcore -> .../TestProjectMac/TestProjectNetcore/bin/Debug/net5.0/osx-arm64/TestProjectNetcore.dll
Test run for .../TestProjectMac/TestProjectNetcore/bin/Debug/net6.0/osx-arm64/TestProjectNetcore.dll (.NETCoreApp,Version=v6.0)
Forwarding DOTNET_ROOT to VSTEST_WINAPPHOST_DOTNET_ROOT '.../dotnetX64'
Current DOTNET_ROOT ''
Microsoft (R) Test Execution Command Line Tool Version 17.1.0-dev
Copyright (c) Microsoft Corporation.  All rights reserved.

Starting test execution, please wait...
Logging Vstest Diagnostics in file: .../log.txt
A total of 1 test files matched the specified pattern.
Could not find 'dotnet' host for the 'ARM64' architecture.  Make sure that 'dotnet' for 'ARM64' architecture is installed on the machine.

Test Run Aborted.
Test run for .../TestProjectMac/TestProjectNetcore/bin/Debug/net5.0/osx-arm64/TestProjectNetcore.dll (.NETCoreApp,Version=v5.0)
Forwarding DOTNET_ROOT to VSTEST_WINAPPHOST_DOTNET_ROOT '.../dotnetX64'
Current DOTNET_ROOT ''
Microsoft (R) Test Execution Command Line Tool Version 17.1.0-dev
Copyright (c) Microsoft Corporation.  All rights reserved.

Starting test execution, please wait...
Logging Vstest Diagnostics in file: .../log.txt
A total of 1 test files matched the specified pattern.
Could not find 'dotnet' host for the 'ARM64' architecture.  Make sure that 'dotnet' for 'ARM64' architecture is installed on the machine.

Test Run Aborted.

Now set the arch env

export DOTNET_ROOT_ARM64=.../dotnet/dotnet

Result

./dotnetX64/dotnet test -p:VSTestConsolePath=".../TP/Debug/netcoreapp2.1/vstest.console.dll" TestProjectMac --arch arm64 --diag:log.txt /bl

.../dotnetX64/sdk/6.0.100-rc.2.21505.57/MSBuild.dll -nologo -distributedlogger:Microsoft.DotNet.Tools.MSBuild.MSBuildLogger,.../dotnetX64/sdk/6.0.100-rc.2.21505.57/dotnet.dll*Microsoft.DotNet.Tools.MSBuild.MSBuildForwardingLogger,.../dotnetX64/sdk/6.0.100-rc.2.21505.57/dotnet.dll -maxcpucount -nodereuse:false -property:VSTestDiag=.../log.txt -property:RuntimeIdentifier=osx-arm64 -property:SelfContained=false -p:VSTestConsolePath=.../TP/Debug/netcoreapp2.1/vstest.console.dll -restore -target:VSTest -verbosity:m /bl TestProjectMac/Sample.sln
  Determining projects to restore...
  All projects are up-to-date for restore.
  You are using a preview version of .NET. See: https://aka.ms/dotnet-core-preview
  TestProjectNetcore -> .../TestProjectMac/TestProjectNetcore/bin/Debug/net6.0/osx-arm64/TestProjectNetcore.dll
  You are using a preview version of .NET. See: https://aka.ms/dotnet-core-preview
  TestProjectNetcore -> .../TestProjectMac/TestProjectNetcore/bin/Debug/net5.0/osx-arm64/TestProjectNetcore.dll
Test run for .../TestProjectMac/TestProjectNetcore/bin/Debug/net6.0/osx-arm64/TestProjectNetcore.dll (.NETCoreApp,Version=v6.0)
Forwarding DOTNET_ROOT to VSTEST_WINAPPHOST_DOTNET_ROOT '.../dotnetX64'
Current DOTNET_ROOT ''
Microsoft (R) Test Execution Command Line Tool Version 17.1.0-dev
Copyright (c) Microsoft Corporation.  All rights reserved.

Starting test execution, please wait...
Logging Vstest Diagnostics in file: .../log.txt
A total of 1 test files matched the specified pattern.
  Failed TestMethod1 [62 ms]
  Error Message:
   Assert.IsTrue failed.
  Stack Trace:
     at TestProjectNetcore.UnitTest1.TestMethod1() in .../TestProjectMac/TestProjectNetcore/UnitTest1.cs:line 13

  Standard Output Messages:
 OSArchitecture Arm64
 ProcessArchitecture Arm64



Failed!  - Failed:     1, Passed:     0, Skipped:     0, Total:     1, Duration: 70 ms - .../TestProjectMac/TestProjectNetcore/bin/Debug/net6.0/osx-arm64/TestProjectNetcore.dll (net6.0)
Test run for .../TestProjectMac/TestProjectNetcore/bin/Debug/net5.0/osx-arm64/TestProjectNetcore.dll (.NETCoreApp,Version=v5.0)
Forwarding DOTNET_ROOT to VSTEST_WINAPPHOST_DOTNET_ROOT '.../dotnetX64'
Current DOTNET_ROOT ''
Microsoft (R) Test Execution Command Line Tool Version 17.1.0-dev
Copyright (c) Microsoft Corporation.  All rights reserved.

Starting test execution, please wait...
Logging Vstest Diagnostics in file: .../log.txt
A total of 1 test files matched the specified pattern.
Testhost process exited with error: . Please check the diagnostic logs for more information.

Test Run Aborted.

Unset arch specific env var and use DOTNET_ROOT

unset VSTEST_TMP_SWITCH_DOTNETROOTS_ENVVARS -> remove env var redirection
unset DOTNET_ROOT_ARM64
export DOTNET_ROOT=.../dotnet

Result

./dotnetX64/dotnet test -p:VSTestConsolePath=".../TP/Debug/netcoreapp2.1/vstest.console.dll" TestProjectMac --arch arm64 --diag:log.txt /bl

.../dotnetX64/sdk/6.0.100-rc.2.21505.57/MSBuild.dll -nologo -distributedlogger:Microsoft.DotNet.Tools.MSBuild.MSBuildLogger,.../dotnetX64/sdk/6.0.100-rc.2.21505.57/dotnet.dll*Microsoft.DotNet.Tools.MSBuild.MSBuildForwardingLogger,.../dotnetX64/sdk/6.0.100-rc.2.21505.57/dotnet.dll -maxcpucount -nodereuse:false -property:VSTestDiag=.../log.txt -property:RuntimeIdentifier=osx-arm64 -property:SelfContained=false -p:VSTestConsolePath=.../TP/Debug/netcoreapp2.1/vstest.console.dll -restore -target:VSTest -verbosity:m /bl TestProjectMac/Sample.sln
  Determining projects to restore...
  All projects are up-to-date for restore.
  You are using a preview version of .NET. See: https://aka.ms/dotnet-core-preview
  TestProjectNetcore -> .../TestProjectMac/TestProjectNetcore/bin/Debug/net6.0/osx-arm64/TestProjectNetcore.dll
  You are using a preview version of .NET. See: https://aka.ms/dotnet-core-preview
  TestProjectNetcore -> .../TestProjectMac/TestProjectNetcore/bin/Debug/net5.0/osx-arm64/TestProjectNetcore.dll
Test run for .../TestProjectMac/TestProjectNetcore/bin/Debug/net6.0/osx-arm64/TestProjectNetcore.dll (.NETCoreApp,Version=v6.0)
Microsoft (R) Test Execution Command Line Tool Version 17.1.0-dev
Copyright (c) Microsoft Corporation.  All rights reserved.

Starting test execution, please wait...
Logging Vstest Diagnostics in file: .../log.txt
A total of 1 test files matched the specified pattern.
  Failed TestMethod1 [63 ms]
  Error Message:
   Assert.IsTrue failed.
  Stack Trace:
     at TestProjectNetcore.UnitTest1.TestMethod1() in .../TestProjectMac/TestProjectNetcore/UnitTest1.cs:line 13

  Standard Output Messages:
 OSArchitecture Arm64
 ProcessArchitecture Arm64



Failed!  - Failed:     1, Passed:     0, Skipped:     0, Total:     1, Duration: 71 ms - .../TestProjectMac/TestProjectNetcore/bin/Debug/net6.0/osx-arm64/TestProjectNetcore.dll (net6.0)
Test run for .../TestProjectMac/TestProjectNetcore/bin/Debug/net5.0/osx-arm64/TestProjectNetcore.dll (.NETCoreApp,Version=v5.0)
Microsoft (R) Test Execution Command Line Tool Version 17.1.0-dev
Copyright (c) Microsoft Corporation.  All rights reserved.

Starting test execution, please wait...
Logging Vstest Diagnostics in file: .../log.txt
A total of 1 test files matched the specified pattern.
Testhost process exited with error: . Please check the diagnostic logs for more information.

Test Run Aborted.

Scenario 5: global x64 installation

In case of global x64 installation muxer won't be available on PATH and so we need to refer to it directly

dotnet
zsh: command not found: dotnet

/usr/local/share/dotnet/x64/dotnet --info
.NET SDK (reflecting any global.json):
 Version:   6.0.100-rc.2.21505.57
 Commit:    ab39070116

Runtime Environment:
 OS Name:     Mac OS X
 OS Version:  11.4
 OS Platform: Darwin
 RID:         osx.11.0-x64
 Base Path:   /usr/local/share/dotnet/x64/sdk/6.0.100-rc.2.21505.57/

Host (useful for support):
  Version: 6.0.0-rc.2.21480.5
  Commit:  6b11d64e7e

.NET SDKs installed:
  6.0.100-rc.2.21505.57 [/usr/local/share/dotnet/x64/sdk]

.NET runtimes installed:
  Microsoft.AspNetCore.App 6.0.0-rc.2.21480.10 [/usr/local/share/dotnet/x64/shared/Microsoft.AspNetCore.App]
  Microsoft.NETCore.App 5.0.11 [/usr/local/share/dotnet/x64/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 6.0.0-rc.2.21480.5 [/usr/local/share/dotnet/x64/shared/Microsoft.NETCore.App]
/usr/local/share/dotnet/x64/dotnet test -p:VSTestConsolePath=".../TP/Debug/netcoreapp2.1/vstest.console.dll" TestProjectMac

Result

/usr/local/share/dotnet/x64/dotnet test -p:VSTestConsolePath=".../TP/Debug/netcoreapp2.1/vstest.console.dll" TestProjectMac --diag:test.txt /bl
/usr/local/share/dotnet/x64/sdk/6.0.100-rc.2.21505.57/MSBuild.dll -nologo -distributedlogger:Microsoft.DotNet.Tools.MSBuild.MSBuildLogger,/usr/local/share/dotnet/x64/sdk/6.0.100-rc.2.21505.57/dotnet.dll*Microsoft.DotNet.Tools.MSBuild.MSBuildForwardingLogger,/usr/local/share/dotnet/x64/sdk/6.0.100-rc.2.21505.57/dotnet.dll -maxcpucount -nodereuse:false -property:VSTestDiag=.../test.txt -p:VSTestConsolePath=.../TP/Debug/netcoreapp2.1/vstest.console.dll -restore -target:VSTest -verbosity:m /bl TestProjectMac/Sample.sln
  Determining projects to restore...
  Restored .../TestProjectMac/TestProjectNetcore/TestProjectNetcore.csproj (in 950 ms).
  You are using a preview version of .NET. See: https://aka.ms/dotnet-core-preview
  You are using a preview version of .NET. See: https://aka.ms/dotnet-core-preview
  TestProjectNetcore -> .../TestProjectMac/TestProjectNetcore/bin/Debug/net5.0/TestProjectNetcore.dll
  TestProjectNetcore -> .../TestProjectMac/TestProjectNetcore/bin/Debug/net6.0/TestProjectNetcore.dll
Test run for .../TestProjectMac/TestProjectNetcore/bin/Debug/net6.0/TestProjectNetcore.dll (.NETCoreApp,Version=v6.0)
Microsoft (R) Test Execution Command Line Tool Version 17.1.0-dev
Copyright (c) Microsoft Corporation.  All rights reserved.

Starting test execution, please wait...
Logging Vstest Diagnostics in file: .../test.txt
A total of 1 test files matched the specified pattern.
  Failed TestMethod1 [56 ms]
  Error Message:
   Assert.IsTrue failed.
  Stack Trace:
     at TestProjectNetcore.UnitTest1.TestMethod1() in .../TestProjectMac/TestProjectNetcore/UnitTest1.cs:line 13

  Standard Output Messages:
 OSArchitecture X64
 ProcessArchitecture X64



Failed!  - Failed:     1, Passed:     0, Skipped:     0, Total:     1, Duration: 81 ms - .../TestProjectMac/TestProjectNetcore/bin/Debug/net6.0/TestProjectNetcore.dll (net6.0)
Test run for .../TestProjectMac/TestProjectNetcore/bin/Debug/net5.0/TestProjectNetcore.dll (.NETCoreApp,Version=v5.0)
Microsoft (R) Test Execution Command Line Tool Version 17.1.0-dev
Copyright (c) Microsoft Corporation.  All rights reserved.

Starting test execution, please wait...
Logging Vstest Diagnostics in file: .../test.txt
A total of 1 test files matched the specified pattern.
  Failed TestMethod1 [86 ms]
  Error Message:
   Assert.IsTrue failed.
  Stack Trace:
     at TestProjectNetcore.UnitTest1.TestMethod1() in .../TestProjectMac/TestProjectNetcore/UnitTest1.cs:line 13

  Standard Output Messages:
 OSArchitecture X64
 ProcessArchitecture X64



Failed!  - Failed:     1, Passed:     0, Skipped:     0, Total:     1, Duration: 120 ms - .../TestProjectMac/TestProjectNetcore/bin/Debug/net5.0/TestProjectNetcore.dll (net5.0)

Run tests with --arch arm64

/usr/local/share/dotnet/x64/dotnet test -p:VSTestConsolePath=".../TP/Debug/netcoreapp2.1/vstest.console.dll" TestProjectMac --arch arm64 --diag:test.txt /bl
/usr/local/share/dotnet/x64/sdk/6.0.100-rc.2.21505.57/MSBuild.dll -nologo -distributedlogger:Microsoft.DotNet.Tools.MSBuild.MSBuildLogger,/usr/local/share/dotnet/x64/sdk/6.0.100-rc.2.21505.57/dotnet.dll*Microsoft.DotNet.Tools.MSBuild.MSBuildForwardingLogger,/usr/local/share/dotnet/x64/sdk/6.0.100-rc.2.21505.57/dotnet.dll -maxcpucount -nodereuse:false -property:VSTestDiag=.../test.txt -property:RuntimeIdentifier=osx-arm64 -property:SelfContained=false -p:VSTestConsolePath=.../TP/Debug/netcoreapp2.1/vstest.console.dll -restore -target:VSTest -verbosity:m /bl TestProjectMac/Sample.sln
  Determining projects to restore...
  Restored .../TestProjectMac/TestProjectNetcore/TestProjectNetcore.csproj (in 753 ms).
  You are using a preview version of .NET. See: https://aka.ms/dotnet-core-preview
  You are using a preview version of .NET. See: https://aka.ms/dotnet-core-preview
  TestProjectNetcore -> .../TestProjectMac/TestProjectNetcore/bin/Debug/net6.0/osx-arm64/TestProjectNetcore.dll
  TestProjectNetcore -> .../TestProjectMac/TestProjectNetcore/bin/Debug/net5.0/osx-arm64/TestProjectNetcore.dll
Test run for .../TestProjectMac/TestProjectNetcore/bin/Debug/net6.0/osx-arm64/TestProjectNetcore.dll (.NETCoreApp,Version=v6.0)
Microsoft (R) Test Execution Command Line Tool Version 17.1.0-dev
Copyright (c) Microsoft Corporation.  All rights reserved.

Starting test execution, please wait...
Logging Vstest Diagnostics in file: .../test.txt
A total of 1 test files matched the specified pattern.
Could not find 'dotnet' host for the 'ARM64' architecture.  Make sure that 'dotnet' for 'ARM64' architecture is installed on the machine.

Test Run Aborted.
Test run for .../TestProjectMac/TestProjectNetcore/bin/Debug/net5.0/osx-arm64/TestProjectNetcore.dll (.NETCoreApp,Version=v5.0)
Microsoft (R) Test Execution Command Line Tool Version 17.1.0-dev
Copyright (c) Microsoft Corporation.  All rights reserved.

Starting test execution, please wait...
Logging Vstest Diagnostics in file: .../test.txt
A total of 1 test files matched the specified pattern.
Could not find 'dotnet' host for the 'ARM64' architecture.  Make sure that 'dotnet' for 'ARM64' architecture is installed on the machine.

Test Run Aborted.

Now set arm64 on env vars

export DOTNET_ROOT_ARM64=.../dotnet

Result

/usr/local/share/dotnet/x64/dotnet test -p:VSTestConsolePath=".../TP/Debug/netcoreapp2.1/vstest.console.dll" TestProjectMac --arch arm64 --diag:test.txt /bl
/usr/local/share/dotnet/x64/sdk/6.0.100-rc.2.21505.57/MSBuild.dll -nologo -distributedlogger:Microsoft.DotNet.Tools.MSBuild.MSBuildLogger,/usr/local/share/dotnet/x64/sdk/6.0.100-rc.2.21505.57/dotnet.dll*Microsoft.DotNet.Tools.MSBuild.MSBuildForwardingLogger,/usr/local/share/dotnet/x64/sdk/6.0.100-rc.2.21505.57/dotnet.dll -maxcpucount -nodereuse:false -property:VSTestDiag=.../test.txt -property:RuntimeIdentifier=osx-arm64 -property:SelfContained=false -p:VSTestConsolePath=.../TP/Debug/netcoreapp2.1/vstest.console.dll -restore -target:VSTest -verbosity:m /bl TestProjectMac/Sample.sln
  Determining projects to restore...
  All projects are up-to-date for restore.
  You are using a preview version of .NET. See: https://aka.ms/dotnet-core-preview
  TestProjectNetcore -> .../TestProjectMac/TestProjectNetcore/bin/Debug/net6.0/osx-arm64/TestProjectNetcore.dll
  You are using a preview version of .NET. See: https://aka.ms/dotnet-core-preview
  TestProjectNetcore -> .../TestProjectMac/TestProjectNetcore/bin/Debug/net5.0/osx-arm64/TestProjectNetcore.dll
Test run for .../TestProjectMac/TestProjectNetcore/bin/Debug/net6.0/osx-arm64/TestProjectNetcore.dll (.NETCoreApp,Version=v6.0)
Microsoft (R) Test Execution Command Line Tool Version 17.1.0-dev
Copyright (c) Microsoft Corporation.  All rights reserved.

Starting test execution, please wait...
Logging Vstest Diagnostics in file: .../test.txt
A total of 1 test files matched the specified pattern.
  Failed TestMethod1 [65 ms]
  Error Message:
   Assert.IsTrue failed.
  Stack Trace:
     at TestProjectNetcore.UnitTest1.TestMethod1() in .../TestProjectMac/TestProjectNetcore/UnitTest1.cs:line 13

  Standard Output Messages:
 OSArchitecture Arm64
 ProcessArchitecture Arm64



Failed!  - Failed:     1, Passed:     0, Skipped:     0, Total:     1, Duration: 73 ms - .../TestProjectMac/TestProjectNetcore/bin/Debug/net6.0/osx-arm64/TestProjectNetcore.dll (net6.0)
Test run for .../TestProjectMac/TestProjectNetcore/bin/Debug/net5.0/osx-arm64/TestProjectNetcore.dll (.NETCoreApp,Version=v5.0)
Microsoft (R) Test Execution Command Line Tool Version 17.1.0-dev
Copyright (c) Microsoft Corporation.  All rights reserved.

Starting test execution, please wait...
Logging Vstest Diagnostics in file: .../test.txt
A total of 1 test files matched the specified pattern.
Testhost process exited with error: . Please check the diagnostic logs for more information.

Test Run Aborted.

The net5.0 fails because it's not supported on arm64
Now using archless env var

export DOTNET_ROOT=.../dotnet

Result

/usr/local/share/dotnet/x64/dotnet test -p:VSTestConsolePath=".../TP/Debug/netcoreapp2.1/vstest.console.dll" TestProjectMac --arch arm64 --diag:test.txt /bl
/usr/local/share/dotnet/x64/sdk/6.0.100-rc.2.21505.57/MSBuild.dll -nologo -distributedlogger:Microsoft.DotNet.Tools.MSBuild.MSBuildLogger,/usr/local/share/dotnet/x64/sdk/6.0.100-rc.2.21505.57/dotnet.dll*Microsoft.DotNet.Tools.MSBuild.MSBuildForwardingLogger,/usr/local/share/dotnet/x64/sdk/6.0.100-rc.2.21505.57/dotnet.dll -maxcpucount -nodereuse:false -property:VSTestDiag=.../test.txt -property:RuntimeIdentifier=osx-arm64 -property:SelfContained=false -p:VSTestConsolePath=.../TP/Debug/netcoreapp2.1/vstest.console.dll -restore -target:VSTest -verbosity:m /bl TestProjectMac/Sample.sln
  Determining projects to restore...
  All projects are up-to-date for restore.
  You are using a preview version of .NET. See: https://aka.ms/dotnet-core-preview
  TestProjectNetcore -> .../TestProjectMac/TestProjectNetcore/bin/Debug/net6.0/osx-arm64/TestProjectNetcore.dll
  You are using a preview version of .NET. See: https://aka.ms/dotnet-core-preview
  TestProjectNetcore -> .../TestProjectMac/TestProjectNetcore/bin/Debug/net5.0/osx-arm64/TestProjectNetcore.dll
Test run for .../TestProjectMac/TestProjectNetcore/bin/Debug/net6.0/osx-arm64/TestProjectNetcore.dll (.NETCoreApp,Version=v6.0)
Microsoft (R) Test Execution Command Line Tool Version 17.1.0-dev
Copyright (c) Microsoft Corporation.  All rights reserved.

Starting test execution, please wait...
Logging Vstest Diagnostics in file: .../test.txt
A total of 1 test files matched the specified pattern.
  Failed TestMethod1 [65 ms]
  Error Message:
   Assert.IsTrue failed.
  Stack Trace:
     at TestProjectNetcore.UnitTest1.TestMethod1() in .../TestProjectMac/TestProjectNetcore/UnitTest1.cs:line 13

  Standard Output Messages:
 OSArchitecture Arm64
 ProcessArchitecture Arm64



Failed!  - Failed:     1, Passed:     0, Skipped:     0, Total:     1, Duration: 74 ms - .../TestProjectMac/TestProjectNetcore/bin/Debug/net6.0/osx-arm64/TestProjectNetcore.dll (net6.0)
Test run for .../TestProjectMac/TestProjectNetcore/bin/Debug/net5.0/osx-arm64/TestProjectNetcore.dll (.NETCoreApp,Version=v5.0)
Microsoft (R) Test Execution Command Line Tool Version 17.1.0-dev
Copyright (c) Microsoft Corporation.  All rights reserved.

Starting test execution, please wait...
Logging Vstest Diagnostics in file: .../test.txt
A total of 1 test files matched the specified pattern.
Testhost process exited with error: . Please check the diagnostic logs for more information.

Test Run Aborted.

The net5.0 fails because it's not supported on arm64

--END TESTS--

@richlander these are the manual test on M1, pls can you double check if I've covered all scenario?
In case of unsupported framework should we print Architecture ARM64 does not support .NETCoreApp,Version=v5.0 framework. or .NETCoreApp,Version=v5.0 framework is not supported on ARM64?

cc: @vitek-karas

@richlander
Copy link
Member

richlander commented Oct 20, 2021

In case of unsupported framework should we print

Ideally, dotnet run and dotnet test produce the same error messages for this situation. dotnet run doesn't inform you of any policy just that it cannot find a given runtime version. While that behavior isn't necessarily the best we can imagine, it is the simplest (which is why I like it).

@MarcoRossignoli and I talked offline. The main thing we need to validate in those private scenarios is that the dotnet test behavior of using the global muxer/runtime matches dotnet run.

The behavior initially felt like multi-level-lookup. I can see on further thought that there are two aspects to this:

  • Which SDK is used to run dotnet build (impliclitly) and dotnet test (explicitly)? I assume this is the private one in the all the private examples.
  • Which runtime is used for the test runner? I assume this is a mix of the private and global ones in the private examples.

There should be a way to disable global lookup for runtimes. Is that MLL?

Here's a good MLL test (all Arm64):

  • Install .NET 6 globally
  • Install .NET 7 SDK privately
  • Run a net6.0 test with the private install

With MLL disabled, I would expect dotnet test to fail to find a runtime.

@richlander
Copy link
Member

richlander commented Oct 20, 2021

Another scenario is using the global x64 SDK.

It would be nice to see a working .NET 5 x64 runtime (just move it to the right place) so that we see more success cases. It looks like the right things are happening, however, success is still better.

@MarcoRossignoli
Copy link
Contributor Author

MarcoRossignoli commented Oct 21, 2021

Ideally, dotnet run and dotnet test produce the same error messages for this situation. dotnet run doesn't inform you of any policy just that it cannot find a given runtime version. While that behavior isn't necessarily the best we can imagine, it is the simplest (which is why I like it).

I've removed custom error message, now will fail like in dotnet run and will be visible inside logs(in case of startup error of test host by design the platform print Testhost process exited with error: . Please check the diagnostic logs for more information. and the real issue is visible in logs).

It would be nice to see a working .NET 5 x64 runtime (just move it to the right place) so that we see more success cases. It looks like the right things are happening, however, success is still better.

I've moved net5.0 in the correct folder, now works as expected(scenario outcome updated)

@MarcoRossignoli and I talked offline. The main thing we need to validate in those private scenarios is that the dotnet test behavior of using the global muxer/runtime matches dotnet run.

I've sent to @vitek-karas some unexpected behavior for dotnet run. I got failure for private x64 and global arm64 for --arch arm64, I need to understand if it's related to some not up to date version or if it's expected, atm I made it work for test.

I'll complete missing scenarios asap(MLL, global x64 SDK).

@MarcoRossignoli
Copy link
Contributor Author

MarcoRossignoli commented Oct 22, 2021

Another scenario is using the global x64 SDK.

Added global x64 Scenario 5

@vitek-karas
Copy link
Member

The error message:

Could not find 'dotnet' host for the 'x64' architecture. Make sure that 'dotnet' for 'x64' architecture is installed on the machine.

The first part is OK, but the suggestion should use product terminology, so something like "Please install .NET x64 on the machine" or something along those lines. Also once we have stable download links, we should provide a link to download. For now we can at least point people to the main download page. Alternatively we could generate similar URL to what apphost does (which includes the desired architecture and version).
It should potentially also mention DOTNET_ROOT since if that one is set, we won't even look at the global install.

For the tests - just to be 100% sure, can you please modify the test to print out the location of the runtime? typeof(object).Assembly.Location - and validate that when running from private install, it picks the private install for the correct arch and that it picks the global install for the other arch.

In case of unsupported framework

I agree with Rich that this should have a similar behavior to dotnet run. It should simply fail - in this case just let the host fail, the error will be very similar to what you get with dotnet run. It also already contains a download link which should work pretty well.

Multi level lookup

There's no such thing as multi level lookup (MLL) on macOS. Setting the DOTNET_MULTILEVEL_LOOKUP will do nothing on macOS (and linux).
On Windows MLL is only used when searching for frameworks, not when looking for the install location itself. This means that apphost itself doesn't react to MLL in any way. Once the hostfxr to be used is determined, then MLL can be used to resolve framework versions (and which runtime will actually be loaded).
Since dotnet test is trying to emulate the behavior of apphost it should not react to MLL either.

Which runtime will be used to actually run the test then follow the exact same rules as apphost (it's actually running the exact some code in hostfxr). Which means that on Windows only, if MLL is set to true (or not set as true is the default) global installs will be considered and can be used depending on version resolution. Setting MLL to false will disable this and force the resolution to stick to the private install.

@MarcoRossignoli MarcoRossignoli marked this pull request as ready for review October 26, 2021 15:02
@MarcoRossignoli MarcoRossignoli enabled auto-merge (squash) October 26, 2021 16:07
Copy link
Member

@nohwnd nohwnd left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, blocked on the case sensitive check for architecture parameter, and hiding existing public api. Those are breaking changes.

There are many changes in the logging, I suggest batching the changes up before comitting using the add suggestion to batch buttons.

@MarcoRossignoli MarcoRossignoli merged commit 1950e32 into microsoft:main Oct 27, 2021
@MarcoRossignoli MarcoRossignoli deleted the arm64 branch October 27, 2021 15:09
This was referenced Jan 13, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
4 participants