-
-
Notifications
You must be signed in to change notification settings - Fork 6.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Expect test to fail (xfail style) #10030
Comments
+1 on this. Sometimes we have frontend functionality which needs to be hard-disabled due to a backend/API bug. Often times this is code which is tested and works, but the data isn't reliable enough to show it to customers. After stubbing the code we can use it.skip we can disable these tests, but there is nothing which forces them to be re-enabled after the upstream services are working and the code is re-enabled (someone unaware of the test may leave the it.skip in - after all, "the tests are passing"). An xfail like interface would start failing after the feature was re-enabled in this case, ensuring that the test is also turned on in conjunction. |
I just came here to make a similar request, I was going to call it test.quarantine but the functionality is the same. updated: I previously added some workarounds, none of which were very good. I've since come up with a "better" one (although it's pretty unpleasant code).
used as
which shows
I'll leave this here in case it's useful to anyone. |
The example doesn't work if you have any afterAll/afterEach/beforeEach/beforeAll which screws up the state. Haven't figured out a solution for that. I tried:
but |
Duplicate of #4627, which shipped in |
This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs. |
🚀 Feature Proposal
The ability to mark a test as expected to fail probably with the following syntax:
And suggested output:
Motivation
Many testing frameworks allow you to specify a test as a expected failure. This can be very useful for long-term TDD and for testing known-bugs are still bugs without implying that it's intended behaviour.
Example
In the above output, an engineer should be surprised (and pleased) by the
2 unexpected passes
and change those tests totest
fromtest.xfail
if their commit caused the fixes. Otherwise they can leave as is for an engineer who understands those tests to mark as resolved.Pitch
I feel this is a core feature as it's a fundamentally new result status for tests.
See some great comments from an older issue #8317
The text was updated successfully, but these errors were encountered: