-
Notifications
You must be signed in to change notification settings - Fork 55
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Use unittest discover for running tests #515
Conversation
I've been running the test suite in many different ways to check if the behaviour of It might be due to test interactions. Investigating it now but it might take a while as it's not an easy thing to debug. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good, and seems equivalent to what nose does, so good from that point of view. We need to fix the imports done in pyface/__init__.py
as they will be imported whenever pyface
is imported for the first time as a side-effect.
I have some concerns about the "collect and then filter" approach, particularly if we are running the tests in an environment with both wxPython and a Qt installed. If we get unlucky in that situation then we may end up with a Qt app (or vice-versa a Wx app) up and running before the backend we really want gets initialized. But we can prevent that by importing pyface.toolkit
as an early line in load_tests
.
It might even be possible to use the pyface.toolkit
value to select the filter rather than having to rely on an environment variable (but maybe still have the choice overridable by an envirionment variable for safety, if you do this). But isn't needed for this PR.
Wx test failures are because:
|
With #510 merged, this needs a merge of master which will hopefully then make the tests pass. |
Codecov Report
@@ Coverage Diff @@
## master #515 +/- ##
==========================================
+ Coverage 36.84% 36.95% +0.11%
==========================================
Files 470 470
Lines 25921 25951 +30
Branches 3931 3937 +6
==========================================
+ Hits 9550 9590 +40
+ Misses 15955 15943 -12
- Partials 416 418 +2
Continue to review full report at Codecov.
|
The tests on Travis seem to work fine so I'm merging this PR... |
Closes #514
The PR adds a custom test loader functions that is used by
unittest discover
which allows excluding tests that match with a regex pattern specified through environment variableEXCLUDE_TESTS
.This is an attempt to simulate the behaviour of
--exclude
flag in nosetests.Caveat: unittest attempts to import all test modules during loading, therefore, it will try to import test modules in
ui/qt4
when running tests forwx
. The import will fail and the loader will return aunittest.loader._FailedTest
which has the path to the test module in its ID. Because the path contains excluded patternqt
the test will be filtered out.