Regression - After an update to 2.2.3 no tests are discovered anymore due to netstandard2.0-dependency

[N-Olbert]

Description

After updating MSTest from version 2.1.2 to 2.2.3 no tests are discovered anymore on the build server (TFS 2017).

The following error is shown:

2021-04-28T07:42:54.8596234Z Starting test execution, please wait...
2021-04-28T07:42:56.2034731Z [MSTest][Discovery][path\to\myTest.dll] MSTestAdapter failed to discover tests in class 'SomeTestClass.cs' of assembly 'path\to\myTest.dll' because Could not load file or assembly 'netstandard, Version=2.0.0.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51' or one of its dependencies. The system cannot find the file specified..

The projects target .NET 4.6.2, which is also the only runtime, which is installed on the server.

Note: On a local machine, which has VS 2019 and therefore .NET 5 installed, all tests are discovered and run as expected.

The packages.config of an affected test project looks like so:

<?xml version="1.0" encoding="utf-8"?>
<packages>
  <package id="MSTest.TestAdapter" version="2.1.2" targetFramework="net462" />
  <package id="MSTest.TestFramework" version="2.1.2" targetFramework="net462" />
</packages>

Expected behavior

Tests should still be discovered upgrading MSTest.

I assume that the error is related to the fact that it is assumed that .NETStandard.Library is either already installed or referenced in an according version (it seems like the netstandard2.0-dependency was introduced by #796). Anyways, if a reference to NETStandard.Library is required by MSTest (+ the Adapter) the PackageReference should be a part of MSTest - changing a local solution (in my case over 30 projects) just to make MSTest work is a bit nasty.

[N-Olbert]

By the way according to nuget this package should not depend on netstandard 2.0

.NETCoreApp 1.0
NETStandard.Library (>= 1.6.1)
System.Diagnostics.TextWriterTraceListener (>= 4.3.0)

.NETFramework 4.5
No dependencies.

.NETStandard 1.5
NETStandard.Library (>= 1.6.1)
System.Diagnostics.TextWriterTraceListener (>= 4.3.0)

UAP 10.0
No dependencies.

Source: https://www.nuget.org/packages/MSTest.TestAdapter/2.2.3

[nohwnd]

Would you be able to give a simple full repro please? You can zip a project and drop it here.

[N-Olbert]

Hi @nohwnd

Any test will work, what is important here is the environment. You can use any test project, but you need to run it within an environment where the highest possible .NET-Version is below .NET 4.7.

F.E. Take a virtual machine; install Windows Server 2012 R2 and Microsoft TFS 2017 (f. e. Version 15.117.27024.0). Then try to run the tests on that machine and the error will occur. It should also be possible to reproduce this without installing TFS - simply sticking to .NET below 4.7 should do the job. The hard thing is that virtually everywhere a newer version than 4.7 is installed (f. e. because it is a Visual Studio requirement)

I understand that this is a very specific problem – most people will in fact have installed a recent .NET-version. Therefore I can also understand if this is not going to be investigated further. For our part we simply stick with 2.1.2 in affected projects.

[N-Olbert]

@nohwnd With 2.2.8 the test execution works fine again (we did not change anything on the build server).

Most likely due to #980