Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

After resuming traing scheduler.step() will not update optimzer's learning rate #17539

Closed
zxydi1992 opened this issue May 1, 2023 · 1 comment
Labels
bug Something isn't working needs triage Waiting to be triaged by maintainers ver: 1.9.x

Comments

@zxydi1992
Copy link

zxydi1992 commented May 1, 2023

Bug description

Not able to update learning rate after resuming is a critical bug, and make training longer jobs with Pytorch Lightning very hard.

#12812

What version are you seeing the problem on?

v1_9

How to reproduce the bug

NA

Error messages and logs

# Error messages and logs here please

Environment

Current environment
#- Lightning Component (e.g. Trainer, LightningModule, LightningApp, LightningWork, LightningFlow):
#- PyTorch Lightning Version (e.g., 1.5.0):
#- Lightning App Version (e.g., 0.5.2):
#- PyTorch Version (e.g., 2.0):
#- Python version (e.g., 3.9):
#- OS (e.g., Linux):
#- CUDA/cuDNN version:
#- GPU models and configuration:
#- How you installed Lightning(`conda`, `pip`, source):
#- Running environment of LightningApp (e.g. local, cloud):

More info

No response

@zxydi1992 zxydi1992 added bug Something isn't working needs triage Waiting to be triaged by maintainers labels May 1, 2023
@Borda Borda changed the title After resuming traing scheduler.step() will not update optimzer's learning rate #12812 is a critical bug After resuming traing scheduler.step() will not update optimzer's learning rate May 3, 2023
@Borda
Copy link
Member

Borda commented May 3, 2023

as here we do not have any more details, let's close it and re-classify the #12812 as bug

@Borda Borda closed this as completed May 3, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working needs triage Waiting to be triaged by maintainers ver: 1.9.x
Projects
None yet
Development

No branches or pull requests

2 participants