[Question] add_generation_prompt=True on prompt

[Galaxy-Husky]

Hi,

I noticed that from v0.11.0, maybe_apply_chat_template added generation prompt on the prompt of the example, which was different from the previous version.

trl/trl/data_utils.py

Lines 86 to 88 in 0238d96

	# Apply the chat template to the prompt, adding the generation prompt
	if "prompt" in example:
	prompt = tokenizer.apply_chat_template(example["prompt"], tokenize=False, add_generation_prompt=True)

Was it a little bug? Will the generation prompt influence the final result?

A prompt reply is appreciated.

[qgallouedec]

Can you point the "previous version" you are refering to?

[qgallouedec]

I think it has been like this from the initial implementation (see #2020)

[Galaxy-Husky]

I think it has been like this from the initial implementation (see #2020)

Sorry, I didn't say that right. I mean before v0.11.0, there was no maybe_apply_chat_template back then. For example, the dpo dataset was preprocessed like:

trl/examples/scripts/dpo.py

Lines 156 to 160 in 55cc4b1

	def process(row):
	row["prompt"] = tokenizer.apply_chat_template(row["chosen"][:-1], tokenize=False)
	row["chosen"] = tokenizer.apply_chat_template([row["chosen"][-1]], tokenize=False)
	row["rejected"] = tokenizer.apply_chat_template([row["rejected"][-1]], tokenize=False)
	return row

Since the code has been refactored , I'm not sure if there was generation prompt or not. If so, could you please point out where it was implemented?

[qgallouedec]

Yes the example code was wrong, you need to add a generation prompt at the end of the prompt.

[Galaxy-Husky]

Yes the example code was wrong, you need to add a generation prompt at the end of the prompt.

I see. Thanks a lot!