Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: CosineAnnealingLR is missing #757

Merged
merged 9 commits into from
Nov 15, 2021
Merged

Conversation

hoangtnm
Copy link
Contributor

@hoangtnm hoangtnm commented Oct 23, 2021

According to Moco_v2 paper, cosine lr scheduler is used but the lighting-bolts's implementation has not implemented it yet.

What does this PR do?

Add CosineAnnealingLR as the learning rate scheduler to follow Moco_v2 paper

Fixes # (issue)

Before submitting

  • Was this discussed/approved via a Github issue? (no need for typos and docs improvements)
  • Did you read the contributor guideline, Pull Request section?
  • Did you make sure your PR does only one thing, instead of bundling different changes together?
  • Did you make sure to update the documentation with your changes?
  • Did you write any new necessary tests? [not needed for typos/docs]
  • Did you verify new and existing tests pass locally with your changes?
  • If you made a notable change (that affects users), did you update the CHANGELOG?

PR review

  • Is this pull request ready for review? (if not, please submit in draft mode)

Anyone in the community is free to review the PR once the tests have passed.
If we didn't discuss your PR in Github issues there's a high chance it will not be merged.

Did you have fun?

Make sure you had fun coding 🙃

According to [Moco_v2 paper](https://arxiv.org/abs/2003.04297), `cosine lr scheduler` is used but the lighting-bolts's implementation has not implemented it yet.
@ananyahjha93
Copy link
Contributor

@hoangtnm thanks for the contribution

Also, we are now in the process of deprecating the bolts implementations of self-supervised learning models and directly support vissl integration in flash: https://lightning-flash.readthedocs.io/en/latest/reference/image_embedder.html

So, we won't be maintaining these implementations any further. You are welcome to contribute to our self-supervised learning vissl integration in flash.

@Borda Borda enabled auto-merge (squash) October 25, 2021 19:03
@mergify mergify bot added the ready label Nov 8, 2021
Copy link

@tchaton tchaton left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@mergify mergify bot removed the ready label Nov 8, 2021
@mergify mergify bot added the ready label Nov 15, 2021
@Borda Borda merged commit 533c09b into Lightning-Universe:master Nov 15, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants