Bug report: This line causes error when using MADGRAD optimizer & resume training

[circlecrystal]

https://github.com/PyTorchLightning/pytorch-lightning/blob/8c07d8bf905e395bbd2142b5df7b185b8e936c41/pytorch_lightning/trainer/connectors/checkpoint_connector.py#L284

The reported error is:
File "/home/tiger/.local/lib/python3.7/site-packages/pytorch_lightning/trainer/connectors/checkpoint_connector.py", line 283, in restore_optimizers for k, v in state.items(): AttributeError: 'Tensor' object has no attribute 'items'

For MADGRAD, optimizer.state["k"] is tensor-type instead of mapping type. Therefore, we need to fix this problem for making Pytorch Lightning compatible with such kind of optimizers.

The correct way of fixing this issue is changing it to:

cc @awaelchli @ananthsub @ninginthecloud @rohitgr7

[awaelchli]

Hello @circlecrystal
Thanks for bringing this up.
I don't see anything immediately wrong with your suggestion. Since you found the fix already, would you be interested in sending a fix in a PR? We would then need to extend one of our tests to check against this type of optimizer.

cheers

[circlecrystal]

Hello @circlecrystal Thanks for bringing this up. I don't see anything immediately wrong with your suggestion. Since you found the fix already, would you be interested in sending a fix in a PR? We would then need to extend one of our tests to check against this type of optimizer.

cheers

I have finally finished a PR for fixing this bug. Please take a look at here:
#11757

[awaelchli]

Re-opening. We would need to add a test to complete the bugfix.

[stale]

This issue has been automatically marked as stale because it hasn't had any recent activity. This issue will be closed in 7 days if no further activity occurs. Thank you for your contributions, Pytorch Lightning Team!