early stopping callback requires load_best_model_at_end to be True

[winglian]

there are a lot of factors that affect how load_best_model_at_end gets set from eval steps and save_steps. we need to figure out what the best way to handle this is.

[winglian]

I think a good solution is allow explicit disabling of load_best_model_at_end via configuration to override all those, and if it's not specifically disabled, then add some warning in validation https://github.com/OpenAccess-AI-Collective/axolotl/blob/main/src/axolotl/utils/validation.py
if cfg.load_best_model_at_end is not False and cfg. early_stopping_patience:

[flotos]

I also got this error, my config worked well with eval_steps: 5, but when changed to eval_steps:30 I receive this error.

[winglian]

this can be set explicitly now.