Errors in Learning-based Method

[zijinY]

Thanks for your excellent work! But when I run the Learning-based Method, I met following errors:

[2024-04-02 16:40:02,359][__main__][INFO] - Starting optimization...
Optimizing:   0%|                                                                                                                                                                                              | 0/1000 [02:46<?, ?it/s]
Error executing job with overrides: ['device=cuda:0', 'model=sd15', 'prompts=people/age']
Traceback (most recent call last):
  File "/home/yinzijin/attribute-control/learn_delta.py", line 81, in main
    eps_p = model.predict_eps(prompts_embedded['prompt_positive'], x_t, t_relative).unsqueeze(0)
  File "/home/yinzijin/attribute-control/attribute_control/model/model.py", line 223, in predict_eps
    return self._get_eps_pred(t, start_sample, self.pipe.unet(start_sample, t, encoder_hidden_states=self._get_pipe_kwargs(embs, embs_neg=None, start_sample=None)['prompt_embeds']).sample)
  File "/home/yinzijin/attribute-control/attribute_control/model/model.py", line 164, in _get_eps_pred
    return (sample - beta_prod_t ** (0.5) * model_output) / alpha_prod_t ** (0.5)
RuntimeError: The size of tensor a (2) must match the size of tensor b (64) at non-singleton dimension 3

I have tried sdxl and sd15, but the error remains. My environment is:

torch                     2.2.0+cu121
torchvision               0.17.0+cu121
diffusers                 0.27.2
accelerate                0.28.0

Do you have any ideas on how to fix this ?

[stefan-baumann]

Hey, sorry for that and thanks for the feedback! This is probably related to some diffusers API changes between 0.26 and 0.27 that I tried to account for but apparently didn't do completely correctly. I will look into this asap.

[zijinY]

Appriciate your quick response, older diffusers version seems not helpful.

[stefan-baumann]

Fixed this now, took me longer than expected to get around to looking into it. I validated the fix for both learning methods and both notebooks on diffusers==0.27.0 and diffusers==0.27.2 (latest, like you used) on an Nvidia A100 and it works for me. Could you please verify that it also works for you now?