Azure DevOps Release Pipeline - Error Message: get_DataRow .. does not have an implementation

[MartinHatch80]

Description

I am getting an error message when trying to execute unit tests using a DevOps RELEASE pipeline.

[MSTest][Discovery][D:\a\r1\a\ExtractedFiles\Framework\MartinHatch.Azure.Functions.v1.Tests.dll] MSTestAdapter failed to discover tests in class 'MartinHatch.Azure.Functions.v1.Tests.TextExtract_IntegrationTests' of assembly 'D:\a\r1\a\ExtractedFiles\Framework\MartinHatch.Azure.Functions.v1.Tests.dll' because Method 'get_DataRow' in type 'Microsoft.VisualStudio.TestPlatform.MSTestAdapter.PlatformServices.TestContextImplementation' from assembly 'Microsoft.VisualStudio.TestPlatform.MSTestAdapter.PlatformServices, Version=14.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' does not have an implementation.

The tests all execute without any issues at all when running from either:

• Visual Studio 2022 Test Explorer
• Azure DevOps BUILD Pipeline

I have checked and checked, and the settings between the Build and Release pipelines are IDENTICAL.

Steps to reproduce

Create .NET Framework 4.8 "Microsoft Unit Test" project using

• MSTest.TestFramework 2.2.8
• MSTest.TestAdapter 2.2.8

Create a release pipeline in Azure DevOps running Unit Tests

Expected behavior

The unit tests run

Actual behavior

It fails to detect any unit tests with the above error message

Environment

Microsoft Unit Test project
.NET Framework 4.8
Azure DevOps RELEASE Pipeline
Pipeline agent running Windows-2019

This is the YAML of the Unit Test in DevOps

steps:
- task: VSTest@2
  displayName: 'Framework - Automated Integration Tests'
  inputs:
    testAssemblyVer2: |
     **\Framework\*.Tests.dll
     !**\obj\**
     !**\bin\**\ref\**
    runSettingsFile: '$(System.DefaultWorkingDirectory)/ExtractedFiles/Framework/Test Parameters/IntegrationSettings.runsettings'
    overrideTestrunParameters: '-BuildNumber "$(Build.BuildNumber)" -ReleaseNumber "$(Release.ReleaseName)" -Environment "Test"'
    runInParallel: false
    codeCoverageEnabled: false
  continueOnError: true

[MartinHatch80]

As a side note - without changing anything else .. I have added the NUnit Test Framework and Test Adapter, and effectively copy-pasted my test code (with a few minor changes to update the syntax from MS Test to nUnit).

The new nUnit tests are being identified and running in the DevOps Release pipeline without any issues.

[Shanayara]

I'm getting the same error on my debug pipeline, but under different circumstances and was wondering whether they were the same as yours @MartinHatch80. Do you per chance have test projects for both Framework and Core in your pipeline?

We have encountered the same error message. We've long had a running VSTest pipeline for our projects unit tests, which target Framework 4.8 and run MSTest.

The YAML of the unit test on DevOps is very minimal:

- name: DebugBuildFolder
  default: '$(DebugBuildFolder)'
- name: DebugBuildArtifactName
  default: '$(DebugBuildArtifactName)'

steps:
  - task: DownloadPipelineArtifact@2
    displayName: 'Download ${{ parameters.DebugBuildArtifactName }} artifact'
    inputs:
      buildType: 'current'
      artifactName: ${{ parameters.DebugBuildArtifactName }}
      targetPath: '$(Build.SourcesDirectory)'
  - task: VSTest@2
    displayName: 'UnitTests on debug build'
    inputs:
      testAssemblyVer2: '${{ parameters.DebugBuildFolder }}\UnitTests.dll'

We recently added a new MSTest project to our solution, which already gets built and procudes a different .dll as its artifact. It was just an empty MSTest project, targeting Core 6 instead:

  <PropertyGroup>
    <TargetFramework>net6.0</TargetFramework>

    <IsPackable>false</IsPackable>

    <PlatformTarget>x64</PlatformTarget>
  </PropertyGroup>

  <ItemGroup>
    <PackageReference Include="Microsoft.NET.Test.Sdk" Version="17.0.0" />
    <PackageReference Include="MSTest.TestAdapter" Version="2.2.8" />
    <PackageReference Include="MSTest.TestFramework" Version="2.2.8" />
    <PackageReference Include="coverlet.collector" Version="3.1.1">
      <PrivateAssets>all</PrivateAssets>
      <IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
    </PackageReference>
  </ItemGroup>

</Project>

With that project simply present, the above error message appears and not a single unit test is discovered. Despite that new MSTest project not being referenced by any other project, and despite VSTest correctly only discovering the same UnitTest.dll as before. For every test class xxxTests, the log contains a message of the form:

[MSTest][Discovery][D:\a\1\s\Build\Debug\UnitTests.dll] MSTestAdapter failed to discover tests in class UnitTests.xxxTests' of assembly 'D:\a\1\s\Build\Debug\UnitTests.dll' because Method 'get_DataRow' in type 'Microsoft.VisualStudio.TestPlatform.MSTestAdapter.PlatformServices.TestContextImplementation' from assembly 'Microsoft.VisualStudio.TestPlatform.MSTestAdapter.PlatformServices, Version=14.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' does not have an implementation..

and at the end there is a message:

No test is available in D:\a\1\s\Build\Debug\UnitTests.dll. Make sure that test discoverer & executors are registered and platform & framework version settings are appropriate and try again.

Notably, no project in the solution references the new MSTest project or it's single (empty, default) unit test, as should be. As soon as we remove the new MSTest project from the solution, all tests get discovered again. Weird!

Now, I was able to disocver the potential root of the problem:

I did notice that the Microsoft.VisualStudio.TestPlatform.MSTestAdapter.PlatformServices.dll in the build output folder has a different size when the Core MSTest project is present - it's just 59kb, compared to109kb before. I then discovered that because of the Core 6 project, the NuGet reference to MSTest.TestAdapter - which both shows as version 2.2.8 - will actually resolve to different dlls than before. Namely, it uses the dll from mstest.testadapter\2.2.8\build\netstandard1.5 instead of mstest.testadapter\2.2.8\build_common, which targets Framework 4.5

The class TestContextImplementation in Framework 4.5 contains a public DataRow attribute, while the TestContextImplementation in Standard 1.5 does not. The moment the Core project is added to the solution, MSBuild will copy the Standard 1.5 dll into the output directory. But the Framework 4.5 UnitTest.dll was linked against the Framework 4.5 version, where the DataRow attribute IS present. When VSTest loads that library it discovers that the implementation for DataRow is missing and quits.

I don't currently know how to solve this problem - there seems to be no way to force the 4.8 project to also use the Standard 1.5 version for the MSTest package, thereby linking against the same library that the other project uses, see also:
dotnet/sdk#1791
NuGet/Home#7416

But maybe - just maybe - you are seeing this behaviour on your pipeline because it's not actually well-defined which Microsoft.VisualStudio.TestPlatform.MSTestAdapter.PlatformServices.dll gets copied to the output folder of your build, so maybe you have a similar situation and one pipeline always builts the Framework project before the Core one, and the other does it the other way around, and so a different Microsoft.VisualStudio.TestPlatform.MSTestAdapter.PlatformServices.dll is present in the build folder in the end? It would be great if you could confirm or deny this.

In any case, this doesn't seem to be a VSTest issue, but maybe rather an MSTest issue. I have opened an issue over there:
microsoft/testfx#1035

[vinayakmsft]

Hi @Shanayara is this issue resolved?

[jbcutting]

I'm still seeing this issue with VS2022 and MSBuild 17.5 when running tests from a unit test project that uses DataRow and targets both .NET 7 and .NET 4.8. I don't think it's resolved.

[MSTest][Discovery][C:\dev\4\aaa\bin\Debug\net7.0\company.product.DataAdapter.Test.dll] MSTestAdapter failed to discover tests in class 'company.product.DataAdapter.Test.SourceCalendarTests' of assembly 'C:\dev\4\aaa\bin\Debug\net7.0\company.product.DataAdapter.Test.dll' because Method 'get_DataRow' in type 'Microsoft.VisualStudio.TestPlatform.MSTestAdapter.PlatformServices.TestContextImplementation' from assembly 'Microsoft.VisualStudio.TestPlatform.MSTestAdapter.PlatformServices, Version=14.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' does not have an implementation..

[Evangelink]

Hey there,

Have you started to get the issue after an update of MSTest or VSTest or did it suddenly started to fail? Would you mind sharing your project configuration (at least version of MSTest and SDK)?

[jbcutting]

Sorry, I just noticed the ADO part on this issue (I was also looking at microsoft/testfx#1035, which is closed). This isn't only an issue with the VSTS ADO task, I also see it with dotnet test and Visual Studio.
We're using latest of everything.

    <PackageVersion Include="Microsoft.NET.Test.Sdk" Version="17.5.0" />
    <PackageVersion Include="MSTest.TestFramework" Version="3.0.2" />
    <PackageVersion Include="MSTest.TestAdapter" Version="3.0.2" />

I have other dev and test projects that are multi-targeting 4.8 and 7.0 - those are working. Only the multi-targeted test project that uses DataRow is failing.

[Evangelink]

@jbcutting Thanks for the details. Are you able to create a repro we could look at? So far we only had cases where the issue was occuring after an update and got cleared after some cleanup so we could not have a proper investigation.

[jbcutting]

Apologies for the delayed response. While trying to create an isolated repro, I figured out the following. I'm sharing details in case they help anyone else who comes across this page in the future.

• I was mistaken - there were no with DataRow usage and the issue occurred even if I removed all references and reduced the project to a single test
• I determined the issue only appeared when I set the BaseOutputPath property in our Directory.Build.props file (or in the project itself). However, other test projects using the same props file weren't affected.
• Ultimately, I'm not sure what, but something must have gotten into the output folder that caused this. Maybe from the build in VS, not sure. After wiping out everything, I was able to get things working again.

Thanks for trying to help me out, @Evangelink.

[Evangelink]

Thanks for the details @jbcutting! Glag to hear the issue self cleared :)

And at the same time, that's too bad for us to investigate 😢 I will try to have a look at BaseOutputPath to see if it can change anything in TP or MSTest behaviors.

[bmacombe]

I had this same issue with a Net 7 and Net48 test project in the same solution, worked locally fine, but could not run in DevOps.

Resolved by forcing the net48 project test adapter reference.

<Reference Include="Microsoft.VisualStudio.TestPlatform.MSTest.TestAdapter">
		<HintPath>$(PkgO2S_MSTest_TestAdapter\build\net462\Microsoft.VisualStudio.TestPlatform.MSTest.TestAdapter.dll</HintPath>
		<Private>True</Private>
</Reference>

[nohwnd]

I cannot reproduce and there is no clear way to reproduce this. Please let me know if you find a way to reproduce reliably so I can investigate.