Disable lr_scheduler.step() in manual optimization

[akihironitta]

What does this PR do?

Fixes #6379. Once #6567 and this PR are merged, users can call lr_scheduler.step() at arbitrary intervals in manual optimization for ultimate flexibility.

Before submitting

• Was this discussed/approved via a GitHub issue? (not for typos and docs)
• Did you read the contributor guideline, Pull Request section?
• Did you make sure your PR does only one thing, instead of bundling different changes together?
• Did you make sure to update the documentation with your changes? (if necessary)
• Did you write any new necessary tests? (not for typos and docs)
• Did you verify new and existing tests pass locally with your changes?
• Did you update the CHANGELOG? (not for typos, docs, test updates, or internal minor changes/refactorings)

PR review

Anyone in the community is free to review the PR once the tests have passed.
Before you start reviewing make sure you have read Review guidelines. In short, see the following bullet-list:

• Is this pull request ready for review? (if not, please submit in draft mode)
• Check that all items from Before submitting are resolved
• Make sure the title is self-explanatory and the description concisely explains the PR
• Add labels and milestones (and optionally projects) to the PR so it can be classified

Did you have fun?

Make sure you had fun coding 🙃

[codecov]

Codecov Report

Merging #6825 (0ff4fea) into master (d1529c2) will decrease coverage by 1%.
The diff coverage is 100%.

@@           Coverage Diff           @@
##           master   #6825    +/-   ##
=======================================
- Coverage      92%     91%    -1%     
=======================================
  Files         197     197            
  Lines       12588   12821   +233     
=======================================
+ Hits        11610   11696    +86     
- Misses        978    1125   +147     

[tchaton]

Let's update the doc with this change + add a warning if we detect the scheduler was on step or epoch, but never triggered.

[akihironitta]

@tchaton I updated the docs and added a warning when scheduler keys (e.g. "interval", "frequency", "monitor"), which are invalid in manual optimization, are provided. Would you mind having a look at the changes?

[tchaton]

Looks good. Could we add in the doc how to use the scheduler for interval=epoch.

 def training_step(self, batch, batch_idx)

     # do optimization

      if self.trainer.is_last_batch: # is_last_batch doesn't exist for the trainer, could be added.
             self.lr_schedulers[0].step()

[akihironitta]

@tchaton I added self.trainer.is_last_batch and updated the docs. Could you have a look again?

[tchaton]

Great work there !

[tchaton]

@akihironitta. There is a bug with Pipe.

Better to access trainer.train_loop.automatic_optimization

[akihironitta]

I think the failing tests on Windows are not related to the changes here as they're also failing on master.

tests/trainer/test_data_loading.py::test_dataloader_warnings[0] FAILED   [ 63%]
tests/trainer/test_data_loading.py::test_dataloader_warnings[1] FAILED   [ 63%]

[Borda]

seems like all Windows tests are failing

[carmocca]

Windows errors are caused by #6762

[akihironitta]

@Borda The failing windows tests should be fixed now by @carmocca in #6930, so would you mind having a look again?

[akihironitta]

Just for the note, as #6907 has been merged already, there's no need to change any docs here, so I reverted the docs change.

Windows tests are fixed