Implement img2text widget

[mishig25]

Implement Text-to-Image widget

huggingface/transformers#18821 (comment)

input (identical to ImageClassificationWidget):

const requestBody = { file }; // img file

output (identical to TextGenerationWidget):

Array<{ generated_text: string; }>

example:

[
  {
    "generated_text": "Some text"
  }
]

Note:

In transformers implementation, I see that the pipeline is called image-to-text-generation, while in the hub, we already defined it as image-to-text without the generation part. Please let me know if it is an issue: @Narsil @osanseviero

todos:

• test when api-inference is up @Narsil
• document widget input sample

[osanseviero]

Related to

In transformers implementation, I see that the pipeline is called image-to-text-generation, while in the hub, we already defined it as image-to-text without the generation part. Please let me know if it is an issue:

This should take care of it huggingface/transformers#18864

[Narsil]

LGTM. I don't think we chose List[{"generated_text": str}] for the output but {"generated_text": str} only (I think).

I think being aligned with what we already have is better.

@OlivierDehaene In general, unfortunately since the pipelines were created at different times by different people originally, the exact output types are not super normalized, especially on the List, List of List stuff. This is something I would like to change on v5 (whenever we decide to go for it and I don't think we have plans) so that the pipeline code could be more regular.

For text-generation it can really generate multiple texts (it's controlled by num_return_sequences) and I think enabling the list here also enables more uses cases (even for captioning you might be interested to generate several at once and choose the best).

However, I don't think we should by highly strict in v4 since every pipeline is ever so slightly different than it's neighbor.

[osanseviero]

Closing as this now lives in https://github.com/huggingface/hub-docs.