Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Microsoft.Testing.Platform: not equal operator not working in mstest runner graph query filtering #4302

Open
sergey-rybalkin opened this issue Dec 10, 2024 · 1 comment
Assignees
Labels
Area: Testing Platform Belongs to the Microsoft.Testing.Platform core library

Comments

@sergey-rybalkin
Copy link

Describe the bug

Using new Microsoft.Testing.Platform library with TUnit framework. It seems like not all of the test filtering operators documented in graph query filtering are supported. For example, when using TUnit with --treenode-filter command line parameter the not equal != operator doesn't seem to be working for me.

Steps To Reproduce

Consider the following code snippet:

public class MyTests
{
    [Test]
    [Category("Smoke")]
    public void TestWithCategory()
    {
        // ...
    }

    [Test]
    public void TestWithoutCategory()
    {
        // ...
    }
}

The following command runs only the first test, as expected dotnet run --treenode-filter /*/*/*/*[Category=Smoke]:

Test run summary: Passed! - bin\Debug\net9.0\Engine.Tests.dll (net9.0|x64)
  total: 1
  failed: 0
  succeeded: 1
  skipped: 0
  duration: 91ms

However, dotnet run --treenode-filter /*/*/*/*[Category!=Smoke] doesn't seem to find anything:

Test run summary: Zero tests ran - bin\Debug\net9.0\Engine.Tests.dll (net9.0|x64)
  total: 0
  failed: 0
  succeeded: 0
  skipped: 0
  duration: 88ms

Expected behavior

Filter should exclude tests with Category property set to Smoke.

Actual behavior

Filter excludes all tests.

Additional context

Seems like filter parser doesn't support != operator at all - https://github.com/microsoft/testfx/blob/main/src/Platform/Microsoft.Testing.Platform/Requests/TreeNodeFilter/TreeNodeFilter.cs#L56
Related issue in TUnit repository - thomhurst/TUnit#1381

SDK version: 9.0.101
OS version: Microsoft Windows [Version 10.0.26100.2454]
TUnit version: 0.4.86
Microsoft.Testing.Platform version: 1.4.3

@Evangelink
Copy link
Member

I don't recall how much was actually implemented in the current version of the tree filter but that's definitely supported by our design: https://github.com/microsoft/testfx/blob/main/docs/mstest-runner-graphqueryfiltering/graph-query-filtering.md

Hence marking this as a bug.

@Evangelink Evangelink added Area: Testing Platform Belongs to the Microsoft.Testing.Platform core library and removed Needs: Triage 🔍 labels Dec 10, 2024
@Evangelink Evangelink added this to the MSTest 3.8 / Platform 1.6 milestone Dec 10, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Area: Testing Platform Belongs to the Microsoft.Testing.Platform core library
Projects
None yet
Development

No branches or pull requests

3 participants