More ResNet backbones for Faster R-CNN and Mask R-CNN

[davidnvq]

🚀 Feature

More ResNet backbones (e.g., resnet18, resnet34, and resnet101) for Object Detection and Instance Segmentation.

Motivation

Detectron2 provides a curated list of pre-trained detectors with having different backbones. However, I believe that the implementation of object detectors in torchvision is intriguingly easy to play around and plug into users' systems to solve their problems. Still, there is a limited choice of backbone for Faster R-CNN and Mask R-CNN.

Is there any reason behind not supporting pre-trained models with such backbones or any future plan?

Many thanks for making educationally understandable and clean codebases, and I'm really looking forward to your reply.

[oke-aditya]

Hello, @davidnvq Resnet 18, 34, 101, 151, as well as ResNext backbones, can be used with FPNs for Object detection from torchvision. Both FRCNN as well as Retina Net support these.
Torchvision does not provide pre-trained weights over COCO dataset for these but you can use these with ImageNet Weights.

For Segmentation models it is possible to use ResNet50 and Resnet101 whose weights are provided as well.
For other Resnet18, Resnet34, etc I think we need Dilation block #2115 and support can be extended.

[davidnvq]

Thanks for your reply.

For Segmentation models it is possible to use ResNet50 and Resnet101 whose weights are provided as well.

Do you mean the pre-trained weights on the COCO dataset? If so, where can I find those weights for ResNet101? So far, I have found the torchvision version of Mask R-CNN with backbone ResNet50 that was pretrained on the COCO dataset.

[oke-aditya]

You can have a read here in docs.
ResNet 101 is pretrained using COCO 2017 for Semantic Segmentation model.s

[davidnvq]

Thank you for your pointer. Indeed, I'm looking for Instance Segmentation Models like Mask R-CNN rather than Semantic Segmentation. I'm still not sure the reason why they haven't provided the pre-trained Mask R-CNN models with ResNet101 backbone.

[oke-aditya]

I'm not sure why they haven't provided or they have plans to extend with Resnet101 backbone (for detection and instance segmentation models).
You can train on COCO2017 using ImageNet weights. Also, torchvision provides reference scripts which were used to obtain these weights. You can use them to train for Resnet101 over COCO.

[SriniMaiya]

from torchvision.models.detection import MaskRCNN
from torchvision.models.detection.backbone_utils import resnet_fpn_backbone



backbone  = resnet_fpn_backbone("resnet101", pretrained=True, trainable_layers=3)
print(backbone.eval())
model = MaskRCNN(backbone=backbone, num_classes=2)

I was able to create the model using this approach. I checked the documentation of resnet_fpn_backbone for the creation of maskrcnn_resnet50_fpn and modified it for resnet101 accordingly.

This was just a trial and I am yet to change the config parameters of MaskRCNN

[AhmedShahinAI]

Thanks @SriniMaiya for your comment. Any one here added resnet18 with maskrcnn?