Support/Features for step-based models

[srush]

🐛 Bug+Feature

For models like transformer, we utilize step-based learning rates and evaluation.

It would be nice to have several features along this line.

1. Support for step-based schedulers. Right now we use cannot give the scheduler to lightning, because it calls scheduler.step(epochs=epoch) internally which resets the scheduler.

2. Fix docs for step based evaluation and checkpointing. It seems like it exists but it is hard to tell which is epochs / steps.

3. Add a max_steps option for stopping training.

4. Helper functions for converting between steps and epochs. For instance, from the number of epochs and parallelism, get the steps for configure_optimizers

[peteriz]

Hi @srush
For 1. I have opened issue #806 to discuss about it and I will contribute this soon.
For 3. I added PR #728 that adds max/min steps for training such models.

[Borda]

@srush good suggestions, would you mind to send a PR for 2 and 4?

[williamFalcon]

we just merged a pr #728 about this

[Borda]

feel free to reopen if needed 🤖