Train Prior With CLIP problem

[yufeng9819]

Hey,

Sorry to disturb you again. @ZENGXH

I finally got a reasonable result. I choose VAE checkpoint with epoch=1599 as final VAE training result. The results are as follows:

I visualize results with epochs larger than 1599 and find that the reconstruction results seem to get worse.

Then I start to train prior with clip feature following train_prior_clip.sh and I get an error. I debug the code and find that it is caused by ddpm.num_steps=1. I want to know is there any problem if I set ddpm.num_steps to 1000?

Looking forward to your reply.

[ZENGXH]

Nice! This vae results look reasonable to me.
Setting to 1000 should be ok. You can set it as a relatively smaller number (like 500 or so) in training to save some eval time in training.
Let me know if it doesn't work ;)

[Javiergonb]

@yufeng9819 How are you plotting the latent points of each sample? Are you using the z_local tensor? I've been trying to do the same but have failed.

[ZENGXH]

@Javiergonb what's the failure you met? the training code by default will plot the latent points and save it as well.