hardcode refmodel.checkpoint.folder to empty

[DNXie]

Solving #413

After this, the checkpoint for reference model will be load only from initial_load_path. User still have the flexibility to change ref model by modifying initial_load_path.

Tested on grpo/main. ref_model now will always load from the initial_load_path regardless of trainer's checkpoint config.

[JenniferWang]

Where does ref model get the initial weights after the change?

[DNXie]

from initial_load_path

[JenniferWang]

I see. So maybe add a validator on the ref model's config and only allow fields that differs per model. for example, Also put enable: true and initial_load_in_hf: true in the override together with the folder.

[pbontrager]

Can you add a todo to make this configurable in the future? Sometimes you do want to update your reference model just very infrequently.

[pbontrager]

@JenniferWang that would have to be on the titan side right? Also, I think this module will be reusable for any "forward pass" model in the near future and not just for reference model. So I don't know if it's worth doing anything more than this now.

[allenwang28]

can this not just be accomplished by setting this in the config?

  checkpoint:
    enable: true
    initial_load_path: hf://${model}
    folder: "" 

[DNXie]

@allenwang28 Yes, it is the same. But I thought it would be a little confusing to the users.
@pbontrager If user want to change the reference model, they can just update the initial_load_path in the config. Does it resolve some of your concern?