-
Notifications
You must be signed in to change notification settings - Fork 904
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
implementing changes for exposing Lightning Tuner's LR finder #1609
Conversation
Check out this pull request on See visual diffs & provide feedback on Jupyter Notebooks. Powered by ReviewNB |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hi @solalatus and thanks for this PR! Looks good already :)
I made a suggestion for adding a new private method that handles setting up the model for training, to avoid adapting the fit API and users having to call fit() with setup_only=True before lr_find.
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #1609 +/- ##
==========================================
- Coverage 94.14% 94.07% -0.08%
==========================================
Files 125 125
Lines 11364 11370 +6
==========================================
- Hits 10699 10696 -3
- Misses 665 674 +9 ☔ View full report in Codecov by Sentry. 🚨 Try these New Features:
|
Hi @solalatus, just wanted to check in how it's going with this PR. |
@dennisbader thanks for checking. Sadly I am completely overwhelmed, so will not be able to make progress on this. Can anyone please take over and push it through the "finish line"? |
Yes, I can take over from here. Thanks for the work so far! |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM, I find this approach quite elegant!
Co-authored-by: madtoinou <32447896+madtoinou@users.noreply.github.com>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good now :) Thanks @solalatus for the contribution!
@dennisbader thanks for pushing it through! Cheers! |
…o#1609) * implementing changes for exposing Lightning Tuner's LR finder * decouple dataset/loader generation from training * improve decoupling * add unit test and docs * add plotting from unittest * remove lr_finder example * fix minor mistakes in docs * final updates for PR * Update darts/tests/models/forecasting/test_torch_forecasting_model.py Co-authored-by: madtoinou <32447896+madtoinou@users.noreply.github.com> * fix Tuner import for PTL<2.0.0 --------- Co-authored-by: Dennis Bader <dennis.bader@gmx.ch> Co-authored-by: madtoinou <32447896+madtoinou@users.noreply.github.com>
Fixes #1503.
Summary
As proposed in the linked issue here, this PR implements a lightweight setting for the
fit()
methods of theTorchForecastingModel
s to just do a "setup" run, and return gracefully without being interrupted and without running the main training loop. This enabled the usage of Lightning Tuner to find optimal learning rates. For this a new method is exposed, that wraps Lightning's method and provides sane defaults.Other Information
I have not done unittests for this functionality, since I am basically exposing some already present functionality. None the less I included a Notebook in the examples here that demonstrates the usage of the feature.
Updates (from @dennisbader):
fit_from_dataset
and_train
setup from fitting the model. like this we can setup the model without having to fit the model, or having to add a parametersetup_only