Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Loss collapse #50

Open
aisagarw opened this issue Oct 5, 2022 · 2 comments
Open

Loss collapse #50

aisagarw opened this issue Oct 5, 2022 · 2 comments

Comments

@aisagarw
Copy link

aisagarw commented Oct 5, 2022

I am trying to pretrain the SimSiam model on mscoco dataset.. but the loss collapses to -1 very quickly.. What are the possible reasons behind and some suggestions to solve the same?
@PatrickHua
Copy link
Owner

https://alexanderli.com/noncontrastive-ssl/ a recent paper shows that simsiam is extremely sensitive to dataset and model size.

@liuzuo-byte
Copy link

我使用高光谱数据做对比学习，损失也一下子就到了-0.99，坍塌很快，我怀疑是有些数据不适用与simsiam这个框架

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@PatrickHua @aisagarw @liuzuo-byte