Trainer.check

[justusschock]

🚀 Feature

Add a Trainer.check(...) interface that checks all your subclassed methods for (programatically) correctness.

Use a manually defined batch to avoid already creating the data loader here.

Discussed offline with @Borda and @ananthsub

[gianscarpe]

I find this feature extremely interesting and I'd like to contribute!

[gianscarpe]

Default check function inside LightningModule. This function uses inspect
library to check that each overriden function of the LightningModule subclass
respect the correct signature.

E.g., training_epoch_end, if defines, should have two inputs (self:
LightningModule and outputs: List) and should return None

The same check can be defined for LightiningCallbacks and
LightningDataModule. trainer.check should call each check function and raise
an specific exception (something like ProgrammaticallyError exception) if some
check fail.

Note: we can exploit object-orientation; the user can defined is own check
function for its own module

What do you @williamFalcon @Borda @edenlightning @justusschock ? Any suggestions are really appreciated! :)

[gianscarpe]

Up @Borda @williamFalcon @justusschock :) Before working on this I think we need a deeper discussion on how to implement the check itself. I suggest using inspect library to check the correctness of arguments and overridden functions! :)

[Borda]

Pls be aware of #4945 and #4948

[carmocca]

Closing this issue in favor of #6029