-
Notifications
You must be signed in to change notification settings - Fork 43
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Mis-inplementation of JS divergence #27
Comments
Thank you a lot for pointing this out. We are looking into it. |
@greatwallet Thank you again for pointing this bug out! |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Hi, according to definition of JS divergence (as mentioned in your supp file), JS divergence is calculated as the difference of
entropy of average probabilities and average of entropies.
However in your code, the first term of JS, aka the difference of entropy of average probabilities is implemented as:
datasetGAN_release/datasetGAN/train_interpreter.py
Line 304 in dee6d7d
where
mean_seg
is defined as average segmentation map of 10 outputs of ensembledpixel_classifier
s.Specifically, I have traced the implementation of
mean_seg
-->datasetGAN_release/datasetGAN/train_interpreter.py
Line 302 in dee6d7d
-->
datasetGAN_release/datasetGAN/train_interpreter.py
Lines 291 to 294 in dee6d7d
-->
img_seg
datasetGAN_release/datasetGAN/train_interpreter.py
Lines 282 to 284 in dee6d7d
In fact,
img_seg
are all unnormalized probabilities, aka logits defined in pytorch distribution's argument. I think in the code you attempted to do average upon logits instead of probabilies (since you have commented outSigmoid
inpixel_classifier
)datasetGAN_release/datasetGAN/train_interpreter.py
Lines 68 to 92 in dee6d7d
TL; DR
The unlawful commutation of
softmax
and linear operation leads to mis-implementation of JS divergence.The text was updated successfully, but these errors were encountered: