Add the possibility to skip prepare_model_for_kbit_training

[hugoabonizio]

Feature request

I'd like to be able to skip the prepare_model_for_kbit_training call that occurs inside DPOTrainer.

Motivation

I noticed a significant speedup by skipping the prepare_model_for_kbit_training call in DPOTrainer without degrading training quality by properly setting the gradient checkpointing configurations, as mentioned in this and this comments.

Your contribution

I can open a PR that adds a flag to allow users to skip this call when desired.

[qgallouedec]

Thanks for this suggestion.
Can you quantify the speedup?
Any idea how to properly set the gradient checkpointing configurations?

Can we reproduce the speedup with a very simple code example?