Refactor test_models to use pytest #3697
Conversation
| @@ -132,25 +124,16 @@ def remove_prefix_suffix(text, prefix, suffix): | |||
|
|
|||
| return expected_file | |||
|
|
|||
| def assertExpected(self, output, subname=None, prec=None, strip_suffix=None): | |||
| def assertExpected(self, output, name, prec=None): | |||
There was a problem hiding this comment.
I had to tweak this a little because in the previous version, the check name was based on self.id(), which changes now that the tests are parametrized with pytest.
In fact I simplified it a bit because all what assertExpected needs to know is the name of the model, so I removed strip_suffix (which we don't need anymore) and also subname which was actually never used.
There was a problem hiding this comment.
As a general note I would like us to re-use as much as possible the testing utils from PyTorch, if it makes sense. This was originally taken from PyTorch tests, and now it is exposed via torch.testing I believe
| @@ -132,25 +124,16 @@ def remove_prefix_suffix(text, prefix, suffix): | |||
|
|
|||
There was a problem hiding this comment.
The munged_id variable in the exact above line should have also been removed.
| try: | ||
| # We first try to assert the entire output if possible. This is not | ||
| # only the best way to assert results but also handles the cases | ||
| # where we need to create a new expected result. | ||
| self.assertExpected(output, prec=prec, strip_suffix=strip_suffix) | ||
| self.assertExpected(output, name, prec=prec) | ||
| raise AssertionError |
There was a problem hiding this comment.
@NicolasHug I think this was left here by accident. This will cause the tests to be considered partially validated and marked as skipped.
Summary: * refactor test_models to use pytest * Also xfail the detection models * Remove xfail and just comment out expected failing parts * Comment out some more * put back commented checks * cleaning + comment * docs * void unnecessary changes * r2plus1d_18 seems to segfault on linux gpu?? * put back test, failure is unrelated Reviewed By: NicolasHug Differential Revision: D28169138 fbshipit-source-id: dc1332bf48a0fdf51158efc401df9c82a83e76f4 Co-authored-by: Francisco Massa <fvsmassa@gmail.com>
Pytest is now supported in the FB internal repo. We can start parametrizing tests!
This PR cleans up the
test_modelstests by relying on pytest.