Skip to content

Commit

Permalink
Fix type check for non-standard schedulers in horovod (#14215)
Browse files Browse the repository at this point in the history
  • Loading branch information
awaelchli authored Aug 18, 2022
1 parent 401eb2c commit 047f0aa
Show file tree
Hide file tree
Showing 2 changed files with 8 additions and 3 deletions.
6 changes: 6 additions & 0 deletions src/pytorch_lightning/CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -62,6 +62,12 @@ The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/).
- Removed the experimental `pytorch_lightning.utiltiies.meta` functions in favor of built-in https://github.com/pytorch/torchdistx support ([#13868](https://github.com/Lightning-AI/lightning/pull/13868))


### Fixed

- Fixed an assertion error when using a `ReduceOnPlateau` scheduler with the Horovod strategy ([#14215](https://github.com/Lightning-AI/lightning/pull/14215))



## [1.7.2] - 2022-08-17

### Added
Expand Down
5 changes: 2 additions & 3 deletions src/pytorch_lightning/strategies/horovod.py
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,6 @@
from pytorch_lightning.utilities.exceptions import MisconfigurationException
from pytorch_lightning.utilities.imports import _HOROVOD_AVAILABLE
from pytorch_lightning.utilities.rank_zero import rank_zero_only
from pytorch_lightning.utilities.types import _LRScheduler

if _HOROVOD_AVAILABLE:
import horovod.torch as hvd
Expand Down Expand Up @@ -114,8 +113,8 @@ def _unpack_lightning_optimizer(opt: Optimizer) -> Optimizer:
lr_scheduler_configs = self.lr_scheduler_configs
for config in lr_scheduler_configs:
scheduler = config.scheduler
assert isinstance(scheduler, _LRScheduler)
scheduler.base_lrs = [lr * self.world_size for lr in scheduler.base_lrs]
if hasattr(scheduler, "base_lrs"):
scheduler.base_lrs = [lr * self.world_size for lr in scheduler.base_lrs] # type: ignore[union-attr]

assert self.lightning_module is not None
# Horovod: broadcast parameters & optimizer state to ensure consistent initialization
Expand Down

0 comments on commit 047f0aa

Please sign in to comment.