dotnet vstest -lt questions

[Gustav-Eikaas]

Description

Not sure if this is the correct place to ask this but just close the issue if its not.

Im trying to build a neovim plugin for dotnet development and I am having some issues with discovering MsTests. If a test has the attribute [TestMethod] which I understand is a MsTest attribute. The output of dotnet test -t strips the namespace and only gives me the name of the test. Is this intentional? xUnit and nUnit seems to give full namespace

MsTest output for tests with [TestMethod]

dotnet vstest -lt .\bin\Debug\net8.0\Project.Tests.dll
The following Tests are available:
    GetOrCreate_ShouldReturnCachedValue
    GetOrCreate_ShouldReuseCachedValue_BeforeExpirationExpired
    GetOrCreate_ShouldReplaceCachedValue_AfterExpirationExpired

Normally outputted from xUnit and nUnit using [Fact], [Theory] etc..

  Project.Tests.Tests.CacheManagerTests.Get_ShouldReturnCachedValue_WhenKnownKey
  Project.Tests.Tests.ViewModels.GetReleaseControlApiModelTests.TestEmptyDataOk

Is there a way to get the namespace as part of the output?

[nohwnd]

Yes this is intentional.

dotnet test calls vstest which then calls MSTest. The display name of a test is used to show the test in the list.

There is another mode nowadays, which is testing.platform. We are currently in the process of integrating it with dotnet test and have an issue created here to improve the output of --list-tests here: #3221

If this will also impact VSTest I don't know, but personally I'd like to avoid it.

For VSTest there is also a better way of automating the platform. The TestPlatform.TranslationLayer package this package creates a session with vstest.console and allows you to automate it without parsing output. You can see an example (and a easy way to experiment with this here in vstest repo

This is how VS and VSCode automate vstest to get tests for their interactive views.

Fot the new testing platform the same capabilities are there, but there is no client in a nuget package yet (that I know of).

[Gustav-Eikaas]

Thank you for replying. The TransitionLayer package looks good, I will test it out and see if it solves my problem

You can close this issue if you want.

[Gustav-Eikaas]

@nohwnd Okay so I tested a bit and wrote an F# script to discover tests in a project. It works great! Only issue/question I have is, the playground example requires a path to the vstest.console.exe. Is there any way to avoid this? It is my understanding that this console is a part of Visual studio installation and not dotnet CLI.

Does VSCode install Visual studio test tools automatically?

[nohwnd]

there is no way to avoid providing path to vstest.console. Vstest.console does all the work, but you can use the one shipped with dotnet SDK.

Or you can bundle up with your extension one of the redistributables we produce.

This is inserted into VS: https://www.nuget.org/packages/Microsoft.TestPlatform
This is inserted into dotnet sdk: https://www.nuget.org/packages/Microsoft.TestPlatform.CLI

[Gustav-Eikaas]

Referring to the one bundled with the sdk is perfect! I just wanted to avoid having people installing more than the dotnet sdk if they didnt have to

[nohwnd]

You might also find this reply by @rouke-broersma, where he describes how stryker-dotnet uses test platform and why they chose not to use the one in dotnet sdk helpful.

stryker-mutator/stryker-net#1962 (comment)

I would also like to bring your attention to the new testing platform that comes bundled with mstest. We don't offer any public client (as described above), but we would be interested in your use case and the kind of data you need, so you could run "natively" on the new testing platform :)

[GustavEikaas]

but we would be interested in your use case and the kind of data you need, so you could run "natively" on the new testing platform :)

So basically I tried C# development in Neovim and it was horrible, dotnet has such a massive ecosystem of tooling. So i decided to try and write a dotnet plugin to bring a lot of this tooling into neovim. https://github.com/GustavEikaas/easy-dotnet.nvim. Adding basic functionality for finding sln files, csproj files etc wasnt too hard. But one thing I wasnt satisfied with was the dotnet test output. Way too hard to read and work with, especially if more than one test was failing. So I tried to make something similiar to the rider/vs testrunner. Working with lua, dispatching dotnet system commands and parsing output is pretty hard so utilizing F# and fsi makes it a bit easier. Your playground link really helped me improve the testrunner

Down the road im thinking it would be nice if I could run the tests using F# instead of dispatching dotnet test --filter... --logger=trx... commands
Seems like this is possible given the links you shared

[nohwnd]

That looks great :)

Down the road im thinking it would be nice if I could run the tests using F# instead of dispatching dotnet test --filter... --logger=trx... commands Seems like this is possible given the links you shared

Yeah, this code has all the basic things: discovery, passing tests to run to the runner, or just giving it dll to run.

https://github.com/microsoft/vstest/blob/main/playground/TestPlatform.Playground/Program.cs#L145

[rouke-broersma]

If interested here is our plumbing to drive the vstest test platform using the translationlayer package:

https://github.com/stryker-mutator/stryker-net/tree/master/src/Stryker.Core/Stryker.Core/TestRunners/VsTest

[GustavEikaas]

If interested here is our plumbing to drive the vstest test platform using the translationlayer package:

Yes very!
This is great, running the tests using F# would relieve a lot of headache with parsing results in lua! 😅

[Evangelink]

@GustavEikaas I'll move the discussion as closed, please feel free to comment/reopen if you consider this as not fixed.

[GustavEikaas]

@Evangelink yes, no problem! This issue helped me tremendeously. Thanks for all the help! :D