[Bug]: "TypeError: StableDiffusionPipeline.__init__() got an unexpected keyword argument 'text_encoder_2'" when using OpenVINO script

[TheWonderfulTartiflette]

Is there an existing issue for this?

• I have searched the existing issues and checked the recent builds/commits

What happened?

With OpenVINO only, I'm getting this error. I'm only able to use v1.5 pruned ema only, and t3_Ver112 models, every other model I have are erroring this (epicrealismXL_vRPhotoreal, control_v1p_sd15_qrcode_monster, controlnetPreTrained_cannyV10, sd_xl_base_1.0 etc). This happens either with the CPU or GPU. Sometimes, it'll be KeyError: 'time_embed.0.weight'
i5-13600K, Arc A770, Windows 11 23H2

Steps to reproduce the problem

1. Try to use any model, except v1.5pruned_ema_ony or t3_Ver112 or others
2. Generate

What should have happened?

Simply not erroring

Sysinfo

Sysinfo 03.03.2024.txt

What browsers do you use to access the UI ?

No response

Console logs

Loading weights [c772a1a690] from E:\AI\Image\Stable Diffusion\stable-diffusion-webui\models\Stable-diffusion\epicrealismXL_v4Photoreal.safetensors
Creating model from config: E:\AI\Image\Stable Diffusion\stable-diffusion-webui\repositories\generative-models\configs\inference\sd_xl_base.yaml
Applying attention optimization: InvokeAI... done.
Model loaded in 4.5s (create model: 0.4s, apply weights to model: 2.1s, apply float(): 1.0s, calculate empty prompt: 0.9s).
{}
Loading weights [c772a1a690] from E:\AI\Image\Stable Diffusion\stable-diffusion-webui\models\Stable-diffusion\epicrealismXL_v4Photoreal.safetensors
OpenVINO Script:  created model from config : E:\AI\Image\Stable Diffusion\stable-diffusion-webui\repositories\generative-models\configs\inference\sd_xl_base.yaml
*** Error completing request
*** Arguments: ('task(b8536hgdinynca8)', 'a beautiful and powerful mysterious sorceress, smile, sitting on a rock, lightning magic, hat, detailed leather clothing with gemstones, dress, castle background, digital art, hyperrealistic, fantasy, dark art, artstation, highly detailed, sharp focus, sci-fi, dystopian', '', [], 20, 'DPM++ 2M Karras', 1, 1, 7, 512, 512, False, 0.7, 2, 'Latent', 0, 0, 0, 'Use same checkpoint', 'Use same sampler', '', '', [], <gradio.routes.Request object at 0x00000216057F05E0>, 1, False, '', 0.8, -1, False, -1, 0, 0, 0, 'None', 'None', 'GPU', True, 'Euler a', True, False, 'None', 0.8, False, False, 'positive', 'comma', 0, False, False, '', 1, '', [], 0, '', [], 0, '', [], True, False, False, False, 0, False) {}
    Traceback (most recent call last):
      File "E:\AI\Image\Stable Diffusion\stable-diffusion-webui\modules\call_queue.py", line 57, in f
        res = list(func(*args, **kwargs))
      File "E:\AI\Image\Stable Diffusion\stable-diffusion-webui\modules\call_queue.py", line 36, in f
        res = func(*args, **kwargs)
      File "E:\AI\Image\Stable Diffusion\stable-diffusion-webui\modules\txt2img.py", line 52, in txt2img
        processed = modules.scripts.scripts_txt2img.run(p, *args)
      File "E:\AI\Image\Stable Diffusion\stable-diffusion-webui\modules\scripts.py", line 601, in run
        processed = script.run(p, *script_args)
      File "E:\AI\Image\Stable Diffusion\stable-diffusion-webui\scripts\openvino_accelerate.py", line 1228, in run
        processed = process_images_openvino(p, model_config, vae_ckpt, p.sampler_name, enable_caching, openvino_device, mode, is_xl_ckpt, refiner_ckpt, refiner_frac)
      File "E:\AI\Image\Stable Diffusion\stable-diffusion-webui\scripts\openvino_accelerate.py", line 895, in process_images_openvino
        shared.sd_diffusers_model = get_diffusers_sd_model(model_config, vae_ckpt, sampler_name, enable_caching, openvino_device, mode, is_xl_ckpt, refiner_ckpt, refiner_frac)
      File "E:\AI\Image\Stable Diffusion\stable-diffusion-webui\scripts\openvino_accelerate.py", line 591, in get_diffusers_sd_model
        sd_model = StableDiffusionPipeline.from_single_file(checkpoint_path, original_config_file=checkpoint_config, use_safetensors=True, variant="fp32", dtype=torch.float32)
      File "E:\AI\Image\Stable Diffusion\stable-diffusion-webui\venv\lib\site-packages\diffusers\loaders.py", line 2822, in from_single_file
        pipe = download_from_original_stable_diffusion_ckpt(
      File "E:\AI\Image\Stable Diffusion\stable-diffusion-webui\venv\lib\site-packages\diffusers\pipelines\stable_diffusion\convert_from_ckpt.py", line 1740, in download_from_original_stable_diffusion_ckpt
        pipe = pipeline_class(
    TypeError: StableDiffusionPipeline.__init__() got an unexpected keyword argument 'text_encoder_2'

Additional information

No response