-
Notifications
You must be signed in to change notification settings - Fork 49
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
OuterLoop
, and SkipOnPlatform
attributes conflict, and don't work correctly when used together
#908
Comments
I think this is something that is actually implemented in runtime? My understanding was that you provide the filters and also this file with ignores/excludes and XHarness only triggers the regular Xunit runner over that? But @akoeplinger might know more |
That doc is mostly relevant for the regular xunit console runner, but we implement most of that logic ourselves, e.g. xharness/src/Microsoft.DotNet.XHarness.TestRunners.Xunit/WasmApplicationEntryPoint.cs Lines 30 to 31 in 5a55841
It's most likely some issue there. |
Closing this as this is not implemented in xharness but in runtime |
This is still an issue, reopening. |
Consider
https://github.com/dotnet/runtime/blob/978df67ced885aeca5f7e75379451bc1a57cc219/src/libraries/System.Net.WebSockets/tests/WebSocketCreateTest.cs#L40-L45
This has both
OuterLoop
, andSkipOnPlatform
attributes. This should run when usingcategory=outerloop
, but get skipped if it is running on the browser. Instead, the tests run anyway, and fail because the operations are not supported on browser.I can reproduce this with
./dotnet.sh build /bl src/libraries/System.Net.WebSockets/tests/System.Net.WebSockets.Tests.csproj /p:Configuration=Release /p:TargetOS=Browser /p:TargetArchitecture=wasm /t:Test -p:Scenario=WasmTestOnBrowser -p:TestScope=outerloop -p:XUnitClassName=System.Net.WebSockets.Tests.WebSocketTests
this is the xharness command line:
wasm test-browser --app=. --output-directory=/workspaces/runtime/artifacts/bin/System.Net.WebSockets.Tests/Release/net7.0/browser-wasm/AppBundle/xharness-output --browser-arg=--no-sandbox -s dotnet.js.symbols --symbol-patterns wasm-symbol-patterns.txt --browser-arg=--no-sandbox -- --run WasmTestRunner.dll System.Net.WebSockets.Tests.dll -class System.Net.WebSockets.Tests.WebSocketTests -trait category=OuterLoop -notrait category=failing
.I don't think this is wasm, or
SkipOnPlatform
specific.cc @akoeplinger @premun
The text was updated successfully, but these errors were encountered: