Support test target for desktop msbuild

[pvlakshm]

Description

User should be able to run tests specifying the test target with desktop msbuild. For example

> MSBuild testapp.csproj /t:VSTest

This target should first build the app and then run tests for all the outputs (similar to how dotnet test behaves today).

To get this target, user may need to add a package reference to Microsoft.NET.Test.Sdk. This package will include the target and the related task.

Related customer ask: dotnet/sdk#546

[codito]

The testplatform targets and task are part of Microsoft.TestPlatform.Build nuget. They're not in a form that nuget's build extensibility hooks can discover and use them. In the short term, we can refactor this package?

@Faizan2304 @harshjain2 thoughts?

[TheRealPiotrP]

We should not bifurcate the project format between MSBuild and CLI. We had talked about making test a first class SDK in CLI and VS. Once this is done, we can bundle test into the "how do we make SDK's available to MSBuild directly" problem. Until we solve that one, /t:Test will not be able to execute a build since the project under test won't be able to find SDK="Microsoft.NET.Sdk"

[clairernovotny]

Until this is done, some projects cannot successfully run tests this way. dotnet test cannot execute a test if the csproj contains targets that use a desktop based Task. There are many existing useful build tasks that can be used to generate a netcoreapp project but they must be run using desktop msbuild.

[tmat]

I don't think MSBuild testapp.csproj /t:VSTest should force build on the project. If I wanted to build the project I would have used /t:Build;VSTest.

[tmat]

Also I'd consider calling the target Test instead of VSTest, to be consistent with dotnet test.

[clairernovotny]

Is there a workaround for this? I have a multi-targeted test library that can only be built with desktop msbuild. I want to run tests!

[tmat]

@onovotny Check this one out:
https://github.com/dotnet/roslyn-tools/blob/master/src/RepoToolset/Test.props
https://github.com/dotnet/roslyn-tools/blob/master/src/RepoToolset/Test.targets

You should be able to use these in your project.

[tmat]

(the .props file might not be needed)

[clairernovotny]

Thanks, will check it out. Do you use anything for Code Coverage? I was using opencover but that seems broken lately and def doesn't support ppdb's or embedded ppdb's yet.

[clairernovotny]

@tmat your targets had something that might actually work for me -- after running the build normally with msbuild /t:build, I tried using dotnet test --no-build. I think that actually worked! The issue is that I'm using GitVersionTask in some of my projects and those tasks don't yet support coreclr. So once it's built, it doesn't matter.

[pchalamet]

Looks similar to issue #1155 I raised. Is there any update when "msbuild /t:VSTest" will be able to test both dncore and regular desktop app ? Thanks!

[pvlakshm]

I have removed the question tag. We will use this item as an input when we take up planning for Q2 of next year.

[clairernovotny]

Bump. Using dotnet test --no-build no longer works for certain scenarios. Really need a way to invoke the targets via an MSBuild target.

[clairernovotny]

I should add that as-of MSBuild 15.6. we can now have SDK's distributed via NuGet, so it should be a lot easier to support adding this as a test sdk out-of-band for both msbuild and dotnet.

[jnevins-gcm]

Anyone planning to fix this? Building via dotnet msbuild CLI and MSBuild.exe is generally inconsistent.

[clairernovotny]

Bump?

[smadala]

@onovotny We haven't decided when to pick this yet. If you would like to contribute, Let me know I'm happy to help you.

[redoz]

Has this been picked up yet? Just ran into this today as I was setting up our new build system.

[shtratos]

is there at least a workaround for this? e.g. can I somehow tell msbuild.exe to import the targets explicitly?

[MeikTranel]

Y'all should probably close this issue if the future milestone gets removed without a replacement. As much as i'd like to have the feature i'd rather take a set answer on whether its something y'all want to take on.

[rainersigwald]

@novacole, this is what you're looking at with dotnet/msbuild#9193, right?

[novacole]

@novacole, this is what you're looking at with dotnet/msbuild#9193, right?

Yes, that's correct! @rainersigwald

[nohwnd]

This is a new feature and won't be implemented, we are focusing on adding new features to Testing.Platform instead. https://aka.ms/testingplatform