About Contrastive Loss in your method

[AprilT0621]

Hello! I am currently attempting to replicate the work presented in your paper and I am very grateful for the code you have provided. However, I have encountered an issue regarding the contrastive loss.

During my training process, the contrastive loss remains consistently at 0. After discovering this problem, I checked the inputs in the contrastive loss function (soft_image_embeds[i], anchor_image_embeds[i], soft_res_masks[i], anchor_res_masks[i]) and found that the values ​​of soft_image_embeds[i] and anchor_image_embeds[i] are almost the same, the same goes for soft_res_masks[i] and anchor_res_masks[i].
loss_contra += contra_loss(soft_image_embeds[i], anchor_image_embeds[i], soft_res_masks[i].clone().detach(), anchor_res_masks[i].clone().detach())
Here is my tensorboard visualization of the contrastive loss using the ISIC dataset and 'box' prompt, the model is ViT-b:

I was wondering if you could provide some insights into possible reasons for this occurrence or directions I could explore for troubleshooting? Perhaps I may have encountered some common pitfalls, but I am currently unable to ascertain them.

I look forward to your guidance and suggestions.

[zhang-haojie]

In cases with only one target in an image, the absence of negative samples in the contrast loss construction results in a constant 0 loss. We have used a method to treat the background as a special instance, and the code will be updated soon.

[lfxx]

In cases with only one target in an image, the absence of negative samples in the contrast loss construction results in a constant 0 loss. We have used a method to treat the background as a special instance, and the code will be updated soon.

looking forward to this update since i can not get 80.01 miou on ISIC,i just get 76.1.

[xmutly]

In cases with only one target in an image, the absence of negative samples in the contrast loss construction results in a constant 0 loss. We have used a method to treat the background as a special instance, and the code will be updated soon.

looking forward to this update since i can not get 80.01 miou on ISIC,i just get 76.1.

me too

[JianghaoWu]

In cases with only one target in an image, the absence of negative samples in the contrast loss construction results in a constant 0 loss. We have used a method to treat the background as a special instance, and the code will be updated soon.

looking forward to this update since i can not get 80.01 miou on ISIC,i just get 76.1.

May I ask if the result of your point as a prompt is normal? I seem to be lower than the beginning.