fast_dev_run should set max_steps

[indigoviolet]

🐛 Bug

Trainer(fast_dev_run=True) should set max_steps accordingly, but max_steps is None

Please reproduce using the BoringModel and post here

https://colab.research.google.com/drive/1PFoozULca2zFtw0Ljor5V8AAGsGqe1Y4#scrollTo=quj4LUDgmFvj

To Reproduce

Expected behavior

max_steps should be set to 1 if fast_dev_run is True else fast_dev_run

Environment

Note: Bugs with code are solved faster ! Colab Notebook should be made public !

• IDE: Please, use our python bug_report_model.py template.

• Colab Notebook: Please copy and paste the output from our environment collection script (or fill out the checklist below manually).

You can get the script and run it with:

wget https://raw.githubusercontent.com/PyTorchLightning/pytorch-lightning/master/tests/collect_env_details.py
# For security purposes, please check the contents of collect_env_details.py before running it.
python collect_env_details.py

• CUDA:
  • GPU:
    • Tesla T4
  • available: True
  • version: 10.1
• Packages:
  • numpy: 1.18.5
  • pyTorch_debug: True
  • pyTorch_version: 1.7.0+cu101
  • pytorch-lightning: 1.1.0
  • tqdm: 4.41.1
• System:
  • OS: Linux
  • architecture:
    • 64bit
  • processor: x86_64
  • python: 3.6.9
  • version: Proposal for help #1 SMP Thu Jul 23 08:00:38 PDT 2020

Additional context

[github-actions]

Hi! thanks for your contribution!, great first issue!

[rohitgr7]

max_steps should be set to 1 if fast_dev_run is True else fast_dev_run

Do you mean?
max_steps should be set to 1 if fast_dev_run is True else max_steps

also val_check_interval, check_val_every_n_epoch be set to 1.0 and 1 respectively as well??

in the video here it says, fast_dev_run won't create any logs or save checkpoints but unfortunately it does both. Not sure what's the intended behavior.

[awaelchli]

fast_dev_run won't create any logs or save checkpoints but unfortunately it does both

I noticed it too, it's a bug (I tried to explain here #4629). it should really just be a test that the training loop runs. logging is still executed to some extend but it should not get sent to the actual logger object, no files should be saved. The motivation is to be able to debug the script without polluting the working dir with tons of files :)

[rohitgr7]

yeah, I believe a simple fix would be to disable loggers, checkpoint callback, earlystopping, ... maybe more in the init itself.

[indigoviolet]

Will that fix the original bug filed here - ie. max_steps not being set, which can break the optimizers/schedulers that refer to it?

[Borda]

@tchaton ^^

[rohitgr7]

which can break the optimizers/schedulers that refer to it?

@indigoviolet mind explain this a bit more, what do you mean here?

[indigoviolet]

I meant that the lightning module might need to refer to self.trainer.max_steps to configure its optimizers/schedulers, and this breaks in fast_dev_run.

[rohitgr7]

yeah then max_steps should be set correctly too in such a case. But fast_dev_run is meant to debug, I don't think anyone should set it to a value > max_steps (even if someone does, it will stop at max_steps only) so scheduler might get a higher number of iterations > fast_dev_run, but it won't break in such a case too.