Specifying extremely long filters doesn't work

[ffMathy]

I get an error when passing an extremely long filter as the --filter argument.

Is it possible to circumvent this limitation? Can I specify a filter from a file, or pass it in from stdin?

Or is there another way to execute extremely long filters?

We are talking probably 50.000 characters long.

[Sanan07]

@ffMathy Could you please show what error are you getting and the command which you ran.

[nohwnd]

Probably this:

$50k = 1..50000 | % { "a" }; dotnet test --filter $50k
ResourceUnavailable: Program 'dotnet.exe' failed to run: The filename or extension is too long.At line:1 char:30
+ $50k = 1..50000 | % { "a" }; dotnet test --filter $50k
+                              ~~~~~~~~~~~~~~~~~~~~~~~~~.

Seems like issue with the command line length. Not sure how to work around that, probably via environment variables, or runsettings.

[ffMathy]

Yes @nohwnd. That's exactly the reason.

You may wonder why I want to have such a large filter.

I'm building a CLI which can run in watch mode, and run tests automatically as you save your files. However, it only runs tests that ran through the lines you changed (affected tests).

It's similar to DotCover's automatic testing, or Visual Studio's Live Unit Testing feature, but works well with Visual Studio Code and the Coverage Gutters extension.

You can read more here: https://github.com/pruner/cli

[ffMathy]

What I'd like to run, is a set of tests that have been affected by the lines that the developer changed (based on previous coverage reports), but I'd also want to run new tests that the developer may have added, that are new compared to the old coverage report.

So let's say the previous coverage report contains coverage for tests A, B, C, D, E and F.

Now, the user changes some lines that tests A, B and C run through. In addition, the user adds a new test, Q.

The filter I'd then like to run is this: (FullyQualifiedName=A|FullyQualifiedName=B|FullyQualifiedName=C)|(FullyQualifiedName!=D&FullyQualifiedName!=E&FullyQualifiedName!=F)

The reason for the last part of that filter, is that the only way to the test Q is to make a negation filter of the tests that are known (FullyQualifiedName!=D&FullyQualifiedName!=E&FullyQualifiedName!=F).

[ffMathy]

And as you can imagine, that filter quickly becomes huge.

[Sanan07]

Why not to run test discovery and get new tests instead of filtering already known tests?

[ffMathy]

The problem with test discovery (I'm assuming you mean --list) is that it doesn't return the namespace of tests - only the method name. In other words, there may be a conflict if two tests are named the same.

[ffMathy]

And it still doesn't solve the problem if the developer has changed a lot of files, leading to a large filter for the affected tests (the FullyQualifiedName=A|FullyQualifiedName=B|FullyQualifiedName=C part).

[ThomasWillumsen]

@nohwnd you mention runsettings - is it possible to make filters in runsettings already?

[ffMathy]

Hmmm it seems this PR describes something that should be possible, but it's not working. Does that still work for .NET 5?

#2356

[ffMathy]

That worked. Added the missing documentation here. The issue can be closed.

MicrosoftDocs/visualstudio-docs#6139