-
Notifications
You must be signed in to change notification settings - Fork 3.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Bugfix/_has_len #2307
Bugfix/_has_len #2307
Conversation
…nd corrected match string for raised exception
Hello @thschaaf! Thanks for updating this PR. There are currently no PEP 8 issues detected in this Pull Request. Cheers! 🍻 Comment last updated at 2020-06-23 14:56:23 UTC |
…duce runtime for continuous integration
…to reduce test time
…o reduce test time
Codecov Report
@@ Coverage Diff @@
## master #2307 +/- ##
======================================
Coverage 88% 88%
======================================
Files 70 70
Lines 5502 5502
======================================
+ Hits 4835 4844 +9
+ Misses 667 658 -9 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
remove all
@pytest.mark.skip('TODO: speed up this test')
…omNotImplementedErrorDataloader stops to speedup test.
@Borda I commented out all The local test ran successful on my laptop. |
@Borda For your information the added test passed on all machines that showed "canceled" (e.g. https://github.com/PyTorchLightning/pytorch-lightning/runs/792966810?check_suite_focus=true). However I assume that the overall runtime was too long. Ideally the runtime for the test should be increased. What do you recommend to get the PR merged? |
tests/trainer/test_dataloaders.py
Outdated
@@ -374,7 +374,7 @@ def test_inf_train_dataloader(tmpdir, check_interval): | |||
|
|||
|
|||
@pytest.mark.parametrize('check_interval', [50, 1.0]) | |||
@pytest.mark.skip('TODO: speed up this test') | |||
# @pytest.mark.skip('TODO: speed up this test') | |||
def test_not_implemented_error_train_dataloader(tmpdir, check_interval): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
this and the following test in which your removed the skip are taking too long because the model runs for the whole epoch, right? But the test suggests that there should be a test for a raise.
I think that's the problem.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
we can also set just nb steps to 3 or so...
tests/trainer/test_dataloaders.py
Outdated
@pytest.mark.skip('TODO: speed up this test') | ||
def test_not_implemented_error_dataloader(tmpdir, check_interval): | ||
# @pytest.mark.skip('TODO: speed up this test') | ||
def test_not_implemented_error_val_dataloader(tmpdir, check_interval): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
same here, no test for "not implemented error"
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The name of the test refers CustomNotImplementedErrorDataloader which raises a NotImplementedError when len is called. In this configuration training should take place. I am open for name suggestions.
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
is this ready to review? :]
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
trying to get some more descriptive names
This pull request is now in conflict... :( |
This pull request is now in conflict... :( |
@Borda Did you have time to look at my previous questions. (inlined below). In particular the one to create a new issue. |
I hope I 'll get to this today, maybe we can contact on slack? |
Mind describes your suspicion in a new issue so we can split the PR fix from others
@williamFalcon have you experienced something similar?
but they were skipped by your PR, right?
maybe I am missing something, what are you not sure about this tests repair?
yes, that I would do... cc: @awaelchli |
Sure. I have slack installed for work. How do I contact you or join the right slack channel or workspace? |
Yes. I just locally enabled them and the inf_dataloader test don't take long at all. |
Not so sure about the num_workers = 0 in the test. Sooner or later someone is going to change that and then has to debug these tests. I would rather not set it if we can't explain why it does not work. Maybe it has to do with raising the StopIteration? |
@thschaaf This raises an NotImplementedError now on the master branch, could you have a look? |
@awaelchli will do. |
What does this PR do?
Enable tests for dataloader where len is defined but can raise NotImplementedError (e.g. in some torchtext cases).
Continuation of PR #2293
Fixes #2277 #2293
Before submitting
PR review
Anyone in the community is free to review the PR once the tests have passed.
If we didn't discuss your PR in Github issues there's a high chance it will not be merged.
Did you have fun?
Make sure you had fun coding 🙃