Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

May I ask how you obtained the checkpoint files for the diffusion model; ckpt.pth? #22

Open
xbyym opened this issue Dec 12, 2023 · 2 comments

Comments

@xbyym
Copy link

xbyym commented Dec 12, 2023

Great work!Thank you @XzwHan @JegZheng for this wonderful contribution. I have read the complete paper and code, but I encountered a problem when training on my own. I used your provided snapshot_self_supervised.ckpt pre-training file, then proceeded with the training of the diffusion model, resulting in two checkpoint files: ckpt_best.pth and ckpt.pth. Then I used these checkpoint files for evaluation testing, and the results were very abnormal, as follows:

INFO - card_classification.py - 2023-12-12 18:39:20,375 - Apply default temperature scaling parameter T with a value of 0.1623
40it [10:36, 15.91s/it]
INFO - card_classification.py - 2023-12-12 18:49:56,841 - Mean accuracy of all samples at test instance level is 9.8780%.

INFO - card_classification.py - 2023-12-12 18:49:56,841 -
Now we compute metrics related to predicted probability quantiles for all classes...
INFO - card_classification.py - 2023-12-12 18:49:56,843 - We predict the label by the class with the 1-th narrowest CI width,
and obtain a test accuracy of 10.0900% through the entire test set.
INFO - card_classification.py - 2023-12-12 18:49:56,845 - We predict the label by the class with the 2-th narrowest CI width,
and obtain a test accuracy of 9.7700% through the entire test set.
INFO - card_classification.py - 2023-12-12 18:49:56,846 - We predict the label by the class with the 3-th narrowest CI width,
and obtain a test accuracy of 9.6300% through the entire test set.
INFO - card_classification.py - 2023-12-12 18:49:56,847 - We predict the label by the class with the 4-th narrowest CI width,
and obtain a test accuracy of 9.8100% through the entire test set.
INFO - card_classification.py - 2023-12-12 18:49:56,848 - We predict the label by the class with the 5-th narrowest CI width,
and obtain a test accuracy of 9.8200% through the entire test set.
INFO - card_classification.py - 2023-12-12 18:49:56,850 - We predict the label by the class with the 6-th narrowest CI width,
and obtain a test accuracy of 9.5800% through the entire test set.
INFO - card_classification.py - 2023-12-12 18:49:56,851 - We predict the label by the class with the 7-th narrowest CI width,
and obtain a test accuracy of 10.3100% through the entire test set.
INFO - card_classification.py - 2023-12-12 18:49:56,852 - We predict the label by the class with the 8-th narrowest CI width,
and obtain a test accuracy of 10.1700% through the entire test set.
INFO - card_classification.py - 2023-12-12 18:49:56,852 - We predict the label by the class with the 9-th narrowest CI width,
and obtain a test accuracy of 10.3200% through the entire test set.
INFO - card_classification.py - 2023-12-12 18:49:56,853 - We predict the label by the class with the 10-th narrowest CI width,
and obtain a test accuracy of 10.5000% through the entire test set.

My majority voting class prediction results are only at 10%, and I would like to know where my problem lies.

@xbyym
Copy link
Author

xbyym commented Dec 12, 2023

I know you've provided your own checkpoint file: ckpt.pth. But how should I go about creating the right checkpoint files for my own pre-trained model, huh?

@xbyym
Copy link
Author

xbyym commented Jan 29, 2024

Hello @XzwHan and @JegZheng,
I previously posted this question and haven't received a response yet. I understand that you might be busy, but your insights on this matter would be really valuable to me. Any guidance or response you could provide would be greatly appreciated. Thanks for your time and consideration!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant