Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix lr finder for optimizers with states #3897

Merged

Conversation

SkafteNicki
Copy link
Member

What does this PR do?

Fixes #3340
Move the construction of optimizer and special scheduler that happens in the learning rate finder to a later state such that optimizer that have an internal state are constructed on the correct device.

Before submitting

  • Was this discussed/approved via a Github issue? (no need for typos and docs improvements)
  • Did you read the contributor guideline, Pull Request section?
  • Did you make sure your PR does only one thing, instead of bundling different changes together? Otherwise, we ask you to create a separate PR for every change.
  • Did you make sure to update the documentation with your changes?
  • Did you write any new necessary tests?
  • Did you verify new and existing tests pass locally with your changes?
  • If you made a notable change (that affects users), did you update the CHANGELOG?

PR review

Anyone in the community is free to review the PR once the tests have passed.
If we didn't discuss your PR in Github issues there's a high chance it will not be merged.

Did you have fun?

Make sure you had fun coding 🙃

@mergify mergify bot requested a review from a team October 6, 2020 10:27
@mergify
Copy link
Contributor

mergify bot commented Oct 6, 2020

This pull request is now in conflict... :(

@codecov
Copy link

codecov bot commented Oct 6, 2020

Codecov Report

Merging #3897 into master will not change coverage.
The diff coverage is n/a.

@@          Coverage Diff           @@
##           master   #3897   +/-   ##
======================================
  Coverage      86%     86%           
======================================
  Files         117     117           
  Lines        9353    9353           
======================================
  Hits         8074    8074           
  Misses       1279    1279           

@williamFalcon williamFalcon merged commit 3ab43dd into Lightning-AI:master Oct 6, 2020
@Borda Borda added this to the 0.10.0 milestone Oct 7, 2020
@SkafteNicki SkafteNicki deleted the lr_finder/optim_with_state_fix branch October 8, 2020 14:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Learning rate finder fails for optimizer with internal state
3 participants