What is Multi-Tasking in t5 mixtures?

[puraminy]

what does multi-tasking mean in T5 mixtures? Is it just sampling from different datasets and fed into a model? Is it equivalent to merge and shuffle these datasets beforehand and then train the model?

What is the role of prompts like "a task: input". Is it just an arbitrary constant prefix, or should it be in the format of "prefix : input"? Is colon important? I mean does the model do some calculations on prefix?

[craffel]

Is it just sampling from different datasets and fed into a model? Is it equivalent to merge and shuffle these datasets beforehand and then train the model?

Yes, though you are able to specify how often to sample from each dataset.

Is it just an arbitrary constant prefix

Yes.

Is colon important? I mean does the model do some calculations on prefix?

No.