README.md

DPO Training

Follow set up in setup environment Data setup:

bash setup/setup_train_data.sh

Data includes 300k frames used for SFT and DPO, plus 17k preference data.

DPO script

bash dpo_scripts/train_dpo.sh