MSANet: Multi-Similarity and Attention Guidance for Boosting Few-Shot Segmentation

Dependencies

• Python 3.9
• PyTorch 1.11.0
• cuda 11.0
• torchvision 0.8.1
• tensorboardX 2.14

Datasets

• PASCAL-5ⁱ: VOC2012 + SBD

• COCO-20ⁱ: COCO2014

  Download the data and put them into the MSANet/lists directory.

Models

• Download the pre-trained backbones from here and put them into the MSANet/initmodel directory.
• Download our trained base learners from Drive and put them under initmodel/PSPNet.
• We provide all trained MSANet models for performance evaluation. Backbone: VGG16 & ResNet50; Dataset: PASCAL-5ⁱ & COCO-20ⁱ; Setting: 1-shot & 5-shot.

Scripts

• Change configuration and add weight path to .yaml files in MSHNet/config , then run the test.py file for testing.

Performance

Performance comparison with the state-of-the-art approaches (i.e., [HSNet], [BAM] and [VAT] in terms of average mIoU across all folds.

1. PASCAL-5ⁱ

   Backbone	Method	1-shot	5-shot
   VGG16	BAM	64.41	68.76
   	MSANet(ours)	65.76 (+1.35)	70.40 (+1.64)
   ResNet50	BAM	67.81	70.91
   	MSANet(ours)	68.52 (+0.71)	72.60 (+1.69)
   ResNet101	VAT	67.50	71.60
   	MSANet(ours)	69.13 (+1.63)	73.99 (+2.39)

2. COCO-20ⁱ

   Backbone	Method	1-shot	5-shot
   ResNet50	BAM	46.23	51.16
   	MSANet(ours)	48.03 (+1.8)	53.67 (+2.51)
   ResNet101	HSNet	41.20	49.50
   	MSANet(ours)	51.09 (+9.89)	56.80 (+7.30)

Visualization

References

BibTeX