Update dataset_formats.mdx

[August-murr]

What does this PR do?

Fixes #2219
The trainers on TRL docs from the website have links attached, but the markdown file in the repo didn't contain any of the links. So, I wasn't sure If I should add the GKDTrainer docs link to the table, please let me know if I need to add it to this PR.

Before submitting

• This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
• Did you read the contributor guideline,
  Pull Request section?
• Was this discussed/approved via a GitHub issue? Please add a link
  to it if that's the case.
• Did you make sure to update the documentation with your changes? Here are the
  documentation guidelines.
• Did you write any new necessary tests?

Who can review?

@qgallouedec

[qgallouedec]

conversational is a type, not a format. I think GKD needs prompt-only but you'll need to double check

[qgallouedec]

please also sort it alphabeticaly

[kashif]

no GKD needs prompts as well as full conversations in the "messages" key, the prompts are typically the messages without the very last assistant reply... depending on the mode... we need the prompts to figure out labels for the completion in the full message and to generate from the teacher/student for the online case

[August-murr]

I think GKD needs prompt-only but you'll need to double check

no GKD needs prompts as well as full conversations in the "messages" key,

in GKDTrainer docs the expected data format is described as:
The dataset should be formatted as a list of “messages” where each message is a list of dictionaries with the following keys:

• role: either system, assistant or user
• content: the message content

like:

messages = [
    {"role": "user", "content": "Hello, how are you?"},
    {"role": "assistant", "content": "I'm doing great. How can I help you today?"},
    {"role": "user", "content": "I'd like to show off how chat templating works!"},
]

which is why I wrote it as Conversational

is the GKDTrainer docs incorrect?

[kashif]

no its good! that is fine and if there is no "prompt" then it makes it fro the messages

[qgallouedec]

which is why I wrote it as Conversational

"conversational" is a type not a format, please refer to the doc.

Thanks for the clarification @kashif. From what I understand, the correct format is then "prompt-completion".
@kashif, to be 100% correct, we will have to modify GKDTrainer so that it concats prompt and completion into a new column "messages". Ideally we want to have

prompt = [{"content": "Why do stars last so long?", "role": "user"}]
completion = [{"content": "Stars last a long time due to the process of nuclear fusion that occurs []...]", "role": "assistant"}]

instead of

prompt = [{"content": "Why do stars last so long?", "role": "user"}]
messages = [
    {"content": "Why do stars last so long?", "role": "user"},
    {"content": "Stars last a long time due to the process of nuclear fusion that occurs []...]", "role": "assistant"},
]

[qgallouedec]

is the GKDTrainer docs incorrect?

it's not incorrect but it has to be updated. Don't worry about that, we will make this clarification in a follow-up PR

[qgallouedec]

The trainers on TRL docs from the website have links attached, but the markdown file in the repo didn't contain any of the links. So, I wasn't sure If I should add the GKDTrainer docs link to the table, please let me know if I need to add it to this PR.

When you write [`GKDTrainer`], the link is automatically created, no need to add it

[qgallouedec]

Lgtm thanks @August-murr!