Which dataset was used to train EVF-SAM2?

[iseunghoon]

Which dataset was used to train EVF-SAM2?

[CoderZhangYx]

Same as EVF-SAM1, we used RefCOCO/+/g, ADE20K, Objects365(filtered & machine annotated), PartImageNet, Humanparsing, Pascal-part.

[yunjeongch]

Hi,

I’m a bit confused about the training scheme of EVF-SAM2.

From what I see in the code, the EvfSam2Model class is implemented in both evf_sam2.py and evf_sam2_video.py. As far as I understand, the difference between them lies in the visual model: SAM2Base in evf_sam2.py and SAM2VideoPredictor in evf_sam2_video.py.

Did you train the evf_sam2.py version with SAM2Base using the aforementioned image dataset and then perform inference with evf_sam2_video.py using the trained parameters?

Thanks in advance!

[CoderZhangYx]

Yes, you are right. Both SAM2Base and SAM2VideoPredictor are wrappers of SAM2 components (image encoder + prompt encoder + mask decoder). We train EVF-SAM2 using image datasets by keeping all SAM2 params frozen, then the model is able to perform zero-shot video prediction.

[yunjeongch]

Thanks for the clarification!