Fix user warning produced by apex + scheduler combination

CHANGELOG.md

@@ -20,6 +20,8 @@ The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/).
 
 ### Fixed
 
+- Fixed user warning when apex was used together with learning rate schedulers ([#1873](https://github.com/PyTorchLightning/pytorch-lightning/pull/1873))
+
 ## [0.7.6] - 2020-05-16
 
 ### Added

pytorch_lightning/trainer/distrib_data_parallel.py

@@ -373,6 +373,14 @@ def ddp_train(self, process_idx, model):
             model, optimizers = model.configure_apex(amp, model, self.optimizers, self.amp_level)
             self.optimizers = optimizers
 
+            # Reinitialize optimizer.step properties added by schedulers
+            for lr_scheduler in self.lr_schedulers:
+                for optimizer in self.optimizers:
+                    scheduler = lr_scheduler['scheduler']
+                    # check that we dont mix users optimizers and schedulers
+                    if scheduler.optimizer == optimizer:
+                        scheduler.__class__.__mro__[-2].__init__(scheduler, optimizer)
+
         # DDP2 uses all GPUs on the machine
         if self.distributed_backend == 'ddp':
             device_ids = [self.root_gpu]