Update RCNN-family docstrings

[rizavelioglu]

This issue is about docstrings in MaskRCNN, FasterRCNN, RegionProposalNetwork classes. Since it's not a bug, or a documentation (../index.html) issue, I opened a blank issue.

Issue

• I think the rpn_score_thresh argument is wrongly defined in both MaskRCNN & FasterRCNN docstrings;
  

  vision/torchvision/models/detection/mask_rcnn.py

  Lines 87 to 88 in 6f0deb9

  	rpn_score_thresh (float): during inference, only return proposals with a classification score
  	greater than rpn_score_thresh

  
  probably because of simple copy/paste from box_score_thresh argument, which is correctly defined:

  vision/torchvision/models/detection/mask_rcnn.py

  Lines 94 to 95 in 6f0deb9

  	box_score_thresh (float): during inference, only return proposals with a classification score
  	greater than box_score_thresh

• and the score_thresh argument is missing in the RegionProposalNetwork docstring:
  

  vision/torchvision/models/detection/rpn.py

  Line 158 in 6f0deb9

  score_thresh: float = 0.0,

Possible fix:

• Update both docstrings in mask_rcnn.py and faster_rcnn.py for the argument:

"""
rpn_score_thresh (float): only return proposals with an objectness score greater than rpn_score_thresh
"""

• And in rpn.py, add the following argument inside the docstring:

"""
score_thresh (float): only return proposals with an objectness score greater than score_thresh
"""

Explanation:

Both arguments (rpn_score_thresh & box_score_thresh) are first passed to FasterRCNN:

vision/torchvision/models/detection/mask_rcnn.py

Lines 227 to 252 in 6f0deb9

	super().__init__(
	backbone,
	num_classes,
	# transform parameters
	min_size,
	max_size,
	image_mean,
	image_std,
	# RPN-specific parameters
	rpn_anchor_generator,
	rpn_head,
	rpn_pre_nms_top_n_train,
	rpn_pre_nms_top_n_test,
	rpn_post_nms_top_n_train,
	rpn_post_nms_top_n_test,
	rpn_nms_thresh,
	rpn_fg_iou_thresh,
	rpn_bg_iou_thresh,
	rpn_batch_size_per_image,
	rpn_positive_fraction,
	rpn_score_thresh,
	# Box parameters
	box_roi_pool,
	box_head,
	box_predictor,
	box_score_thresh,

where rpn_score_thresh is passed to RegionProposalNetwork:

vision/torchvision/models/detection/faster_rcnn.py

Lines 234 to 244 in 6f0deb9

	rpn = RegionProposalNetwork(
	rpn_anchor_generator,
	rpn_head,
	rpn_fg_iou_thresh,
	rpn_bg_iou_thresh,
	rpn_batch_size_per_image,
	rpn_positive_fraction,
	rpn_pre_nms_top_n,
	rpn_post_nms_top_n,
	rpn_nms_thresh,
	score_thresh=rpn_score_thresh,

and box_score_thresh is passed to RoIHeads:

vision/torchvision/models/detection/faster_rcnn.py

Lines 259 to 269 in 6f0deb9

	roi_heads = RoIHeads(
	# Box
	box_roi_pool,
	box_head,
	box_predictor,
	box_fg_iou_thresh,
	box_bg_iou_thresh,
	box_batch_size_per_image,
	box_positive_fraction,
	bbox_reg_weights,
	box_score_thresh,

In RegionProposalNetwork (here the score_thresh argument is missing in the docstring), rpn_score_thresh is used inside filter_proposals function to filter proposals based on the objectness score:

vision/torchvision/models/detection/rpn.py

Lines 271 to 284 in 6f0deb9

	objectness_prob = torch.sigmoid(objectness)
	final_boxes = []
	final_scores = []
	for boxes, scores, lvl, img_shape in zip(proposals, objectness_prob, levels, image_shapes):
	boxes = box_ops.clip_boxes_to_image(boxes, img_shape)
	# remove small boxes
	keep = box_ops.remove_small_boxes(boxes, self.min_size)
	boxes, scores, lvl = boxes[keep], scores[keep], lvl[keep]
	# remove low scoring boxes
	# use >= for Backwards compatibility
	keep = torch.where(scores >= self.score_thresh)[0]

and the filter_proposals function is used during both training and inference:

vision/torchvision/models/detection/rpn.py

Line 372 in 6f0deb9

boxes, scores = self.filter_proposals(proposals, objectness, images.image_sizes, num_anchors_per_level)

Hence, the docstring should change from: "during inference, only return proposals with a classification score greater than rpn_score_thresh" to: "only return proposals with an objectness score greater than rpn_score_thresh"

---

In RoIHeads, box_score_thresh is used inside postprocess_detections:

vision/torchvision/models/detection/roi_heads.py

Lines 707 to 708 in 6f0deb9

	# remove low scoring boxes
	inds = torch.where(scores > self.score_thresh)[0]

and the postprocess_detections is only executed during inference:

vision/torchvision/models/detection/roi_heads.py

Lines 767 to 775 in 6f0deb9

	if self.training:
	if labels is None:
	raise ValueError("labels cannot be None")
	if regression_targets is None:
	raise ValueError("regression_targets cannot be None")
	loss_classifier, loss_box_reg = fastrcnn_loss(class_logits, box_regression, labels, regression_targets)
	losses = {"loss_classifier": loss_classifier, "loss_box_reg": loss_box_reg}
	else:
	boxes, scores, labels = self.postprocess_detections(class_logits, box_regression, proposals, image_shapes)

Hence, the docstring is correct for box_score_thresh.

---

In case my observations are correct, I would be glad to submit a PR.

[NicolasHug]

Thanks a lot for the detailed report @rizavelioglu, I think you're 100% correct! We'd love to review your PR. On top of MaskRCNN and FasterRCNN, it looks like this is affecting MaskRCNN (edit: KeypointRCNN!) as well?