Add ORTModelForVision2Seq for VisionEncoderDecoder models inference

[mht-sharma]

What does this PR do?

This PR enables the inference of the VisionEncoderDecoder models using ONNXRuntime. The PR adds ORTModelForVision2Seq for doing inference similar to AutoModelForVision2Seq by changing just a few lines.

Usage

>>> from PIL import Image
>>> from transformers import GPT2TokenizerFast, ViTImageProcessor
->>>  from transformers import AutoModelForVision2Seq
+>>>  from optimum.onnxruntime import ORTModelForVision2Seq
>>> import requests

>>> model_name = "nlpconnect/vit-gpt2-image-captioning"
->>> model = AutoModelForVision2Seq.from_pretrained(model_name)
+>>> model = ORTModelForVision2Seq.from_pretrained(model_name, from_transformers=True)
>>> tokenizer = GPT2TokenizerFast.from_pretrained(model_name)
>>> image_processor = ViTImageProcessor.from_pretrained(model_name)

>>> url = "http://images.cocodataset.org/val2017/000000039769.jpg"
>>> image = Image.open(requests.get(url, stream=True).raw)
>>> pixel_values = image_processor(image, return_tensors="pt").pixel_values

>>> generated_ids = model.generate(pixel_values)
>>> generated_text = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0]
>>> print(generated_text)

Limitations

• Donut model not supported
• TrOCR model not supported with use_cache=True.
• IObinding not supported

Before submitting

• This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
• Did you make sure to update the documentation with your changes?
• Did you write any new necessary tests?

[HuggingFaceDocBuilderDev]

The documentation is not available anymore as the PR was closed or merged.

[fxmarty]

LGTM as long as the tests pass, thank you for the addition! Could you also make sure that the code snippet in the documentation is valid?

[fxmarty]

Suggested change

	past_key_values (`tuple(tuple(torch.FloatTensor), *optional*)`
	past_key_values (`tuple(tuple(torch.FloatTensor), *optional*)`

We should not put *optional* in Optimum following this discussion: https://huggingface.slack.com/archives/C02P0559X9S/p1669628754896019

Can rather precise what the default is? Although, for the type: Tuple[Tuple[torch.FloatTensor]]

(edit: this is copy paste, so could you edit it as well in the rest?)

[mht-sharma]

Added default to None as in the function sig

[fxmarty]

Just for my knowledge, why is this not supported? What will happen if an user tries to use ORTModelForVision2Seq with trocr with use_cache = True?

[mht-sharma]

The model currently not output any past_key_values when use_cache=True. So need to figure out why. Probably something wrong in the modeling code.

| What will happen if an user tries to use ORTModelForVision2Seq with trocr with use_cache = True?

For this I have added an error message during export.

[michaelbenayoun]

LGTM!

What's the status for the pat key / values?

[mht-sharma]

LGTM!

What's the status for the pat key / values?

Past key values is not supported only for TrOCR, would look into this after Donut