New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Loss Function #11

Open

dongqian0206 opened this issue Jun 27, 2018 · 2 comments

dongqian0206 commented Jun 27, 2018

Hi,

Would you like to tell me why you added scalar '79' in front of the cross entropy (RVAE, Line 110)?
loss = 79 * cross_entropy + kld_coef(i) * kld

best

nguyenvo09 commented Sep 4, 2018

when I remove 79, kld can go to zero very fast.

Author

dongqian0206 commented Sep 4, 2018

Hi, thanks for your reply. We have to use kl annealing to alleviate kl vanishing issue in this model. I think it is the same.

best

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment