Data Format and Mixtures

[ayush714]

Hi, I have seen in T5 Mesh, our data should have the format like this <input>/t<target> , So I have data where I have two inputs which is question and context and one target, so it is like <question>, <context>, <answer_text> So, for making one input, I combined question and context giving \\n in between, So, It's like <question \\n context> and then <target>.

So, Is it correct way to do?, my another question related to this is about Mixtures, I am following the example t5-trivia notebook and there I don't want to build qna model without context, I want to build with context, so which mixture should I use and how I can prepare it?

Plz make sure my data is <question>, <context>, <answer_text>. I am juggling with this from last 5 days, Thanks In Advance.

And Why we need the second dataset, I am following this notebook:- https://colab.research.google.com/github/google-research/text-to-text-transfer-transformer/blob/main/notebooks/t5-trivia.ipynb#scrollTo=y4_1gpcK9i3W, Here they have taken the second dataset as well? why it has been taken?

[craffel]

If you are using a TSV file, the inputs and targets can't have newlines in them since newlines delimit examples. You don't need to create a mixture if you are training on a single task.

[ayush714]

Hi @craffel Okay, Thanks for your clarification, but here I have two inputs first is <question> and second is <context>, so how to deal with it?

[ayush714]

and @craffel After training for 15 fine tuning steps, I got these checkpoints created, What are they? I just want to be confirm, and How I can generate predictions using these checkpoint if they are model and which one to choose?