Use your own pre-trained model for downstream segmentation

[yyy1998-i]

I would like to ask you a question, I pre-train on my medical image data set (6k) according to the official pre-training code, and then the downstream task is segmentation, but the fine-tuning results with my own pre-training model is always inferior to the author's pre-training model with imagenet, which is why, is my pre-training data is still too small?

[Jivitesh2001]

Hey,
Your dataset seems too small compared to Imagenet-1K. A suggestion - you should initialize your model using the pre-trained model weights of Imagenet and then continue pre-training on your medical image dataset, which you can use for your downstream task. Please let me know how this goes for you.

[yyy1998-i]

Hey, Your dataset seems too small compared to Imagenet-1K. A suggestion - you should initialize your model using the pre-trained model weights of Imagenet and then continue pre-training on your medical image dataset, which you can use for your downstream task. Please let me know how this goes for you.

Ok, thank you very much for your reply. Next, I will try according to what you said. Thanks again

[Jivitesh2001]

Hello,
Have you tried this approach? and Can you tell me what were your results?

[yyy1998-i]

Hello, Have you tried this approach? and Can you tell me what were your results?

Sorry for the late reply. This is my latest training situation, and it is still not as good as the weight of imagenet.

[Jivitesh2001]

Hey!
Have you tried different augmentation techniques to oversample your medical dataset? I think the idea should be a) Load the ImageNet Pre-Trained Checkpoint, b) Then use the pre-trained checkpoint to fine-tune the model (which seems to be a classification model) on your dataset for your downstream task. Have you tried this approach?

[yyy1998-i]

Hey! Have you tried different augmentation techniques to oversample your medical dataset? I think the idea should be a) Load the ImageNet Pre-Trained Checkpoint, b) Then use the pre-trained checkpoint to fine-tune the model (which seems to be a classification model) on your dataset for your downstream task. Have you tried this approach?

Next, I will try to use data enhancement (the previous training was in accordance with the official default Settings). In the pre-training process, I found that without using normalize,MAE's reconstruction effect is better, and the loss is also very low (0.0822). When fine-tuning, I used mean and std which I used to fine-tune the data set. I found that this also affected the effectiveness of the downstream classification task.