Class-wise contrastive learning loss

[Kevin-Planolles]

The code is not clear about the CCL mentionned in the paper. It is not mentionned in the losses.py file. Is it possible to further explain where in the code this particular loss is used ?

[ChangyaoTian]

Sure. The CCL loss is implicitly implemented via the following functions:

• The labels2idxs first generates the target matrix for the input samples, where samples from the same class will have positive target values;
• Then the CCL loss is calculated in the PretrainSentLoss module here, where the base_criterion, (i.e. the LabelSmoothingCrossEntropy module) computes the loss of visual and linguistics branches respectively.