about DINO+MS-DETR

[ms12]

I tried to add MS-DETR to DINO and found that the performance dropped a lot compared to before I added it, it seems to be due to o2m supervision affecting o2o, does the author have any idea what might be causing this?

[ZhaoChuyang]

Yes, we find on the original DINO repo, o2m seems not compatible. We incorporate the contrastive denoising queries (CDN) from DINO into the Deformable-DETR code and it works with o2m supervision. I think the reason may lie in the loss weights and the way to average the loss, which is different in DINO and Deformable-DETR. But we did not further explore it.