Remove
if tokenizer.chat_template is None:
tokenizer.chat_template = SIMPLE_CHAT_TEMPLATE
present in most scripts.
Adding a chat template only makes sense when you SFT the model (instruction tuning). Doing it before DPO, CPO etc, doesn't make sense, so I'd remove them all.