Quantization support for CausalVisualLMs

[nikita-savelyevv]

What does this PR do?

Add support for quantization of CausalVisualLMs (llava). When quantization_config is given, language model will be compressed according to it. Vision and text embeddings models are compression to int8_asym.

Example:

import requests
from PIL import Image
from transformers import AutoProcessor

from optimum.intel.openvino import OVModelForVisualCausalLM, OVWeightQuantizationConfig
from optimum.intel.openvino.configuration import OVQuantizationMethod


model_id = "llava-hf/llava-v1.6-mistral-7b-hf"

data_aware_config = OVWeightQuantizationConfig(
    bits=4,
    sym=True,
    quant_method=OVQuantizationMethod.AWQ,
    num_samples=32,
    dataset="contextual",
    processor=model_id,
)

model = OVModelForVisualCausalLM.from_pretrained("llava/fp32", compile=False)
OVQuantizer(model).quantize(ov_config=OVConfig(quantization_config=data_aware_config))

image_file = "http://images.cocodataset.org/val2017/000000039769.jpg"
processor = AutoProcessor.from_pretrained(model_id)

conversation = [
    {
        "role": "user",
        "content": [
            {"type": "text", "text": "What are these?"},
            {"type": "image"},
        ],
    },
]
prompt = processor.apply_chat_template(conversation, add_generation_prompt=True)
raw_image = Image.open(requests.get(image_file, stream=True).raw)
inputs = processor(images=raw_image, text=prompt, return_tensors="pt")

output = model.generate(**inputs, max_new_tokens=50, do_sample=False)
print(processor.decode(output[0], skip_special_tokens=True))

Before submitting

• This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
• Did you make sure to update the documentation with your changes?
• Did you write any new necessary tests?

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.