support evaluation data during ppo training

[akk-123]

support evaluation data add save best checkpoint(have highest reward) during ppo training

[younesbelkada]

Hi @akk-123
Thanks for the issue, currently you can push to hub the best model (highest reward) since: #275
Let us know if this works for your usecase or no

[akk-123]

thanks for you reply, can you supprt pass evaluation data during ppo training like trlx, and save the best model(highest reward in evaluation data) in the local machine

[younesbelkada]

@akk-123 I believe this is not supported yet, however you can implement your own loop to do so. After calling ppo_trainer.step() you can manually compute the reward of the model on the evaluation data and save it if the reward is higher than in the previous steps.