Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

lr_scheduler does not work when "interval": "step" #20436

Closed
lucl13 opened this issue Nov 21, 2024 · 0 comments
Closed

lr_scheduler does not work when "interval": "step" #20436

lucl13 opened this issue Nov 21, 2024 · 0 comments
Labels
bug Something isn't working needs triage Waiting to be triaged by maintainers ver: 2.4.x

Comments

@lucl13
Copy link

lucl13 commented Nov 21, 2024

Bug description

lr_scheduler does not work when "interval": "step." No changes in lr were observed within one epoch, i think it still uses "interval": "epoch"

def configure_optimizers(self):
    #optimizer = torch.optim.Adam(self.parameters(), lr=self.learning_rate)
    optimizer = torch.optim.AdamW(self.parameters(), lr=self.learning_rate, weight_decay=1e-2)
    
    total_steps = self.trainer.estimated_stepping_batches
    scheduler = torch.optim.lr_scheduler.OneCycleLR(
        optimizer, 
        max_lr=1e-2, 
        total_steps=total_steps
    )
    
    return {
        "optimizer": optimizer,
        "lr_scheduler": {
            "scheduler": scheduler,
            "interval": "step", 
            "frequency": 1
        },
    }
```****

### What version are you seeing the problem on?

v2.4

### How to reproduce the bug

_No response_

### Error messages and logs

_No response_

### Environment

<details>
  <summary>Current environment</summary>

#- PyTorch Lightning Version (e.g., 2.4.0):
#- PyTorch Version (e.g., 2.4):
#- Python version (e.g., 3.12):
#- OS (e.g., Linux):
#- CUDA/cuDNN version:
#- GPU models and configuration:
#- How you installed Lightning(conda, pip, source):


</details>


### More info

_No response_
@lucl13 lucl13 added bug Something isn't working needs triage Waiting to be triaged by maintainers labels Nov 21, 2024
@lucl13 lucl13 closed this as completed Nov 22, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working needs triage Waiting to be triaged by maintainers ver: 2.4.x
Projects
None yet
Development

No branches or pull requests

1 participant