Full recovery from a restart point

[Benzoin96485]

We found that a perfect restart from a checkpoint is still not working. For example, loading a checkpoint of a trained model and continuing results in a significantly higher training loss than the last epoch in the last training. This may be fixed by a thorough check of what parameters are loaded, adding optimizer and scheduler state dicts, and distinguish the "restart" loading and "pretrain-finetune" loading