Feature request: Support for aug_test.

[ypwhs]

Difference of mmdetection and mmdeploy at aut_test

Now mmdeploy is to directly take the first image prediction, which is different from the behavior of mmdetection: https://github.com/open-mmlab/mmdeploy/blob/v0.2.0/mmdeploy/codebase/mmdet/deploy/object_detection_model.py#L191

input_img = img[0].contiguous()
outputs = self.forward_test(input_img, img_metas, *args, **kwargs)

mmdetction will combine the results of multiple scale results and do nms: https://github.com/open-mmlab/mmdetection/blob/v2.20.0/mmdet/models/roi_heads/test_mixins.py#L138-L176

def aug_test_bboxes(self, feats, img_metas, proposal_list, rcnn_test_cfg):
    """Test det bboxes with test time augmentation."""
    aug_bboxes = []
    aug_scores = []
    for x, img_meta in zip(feats, img_metas):
        # only one image in the batch
        img_shape = img_meta[0]['img_shape']
        scale_factor = img_meta[0]['scale_factor']
        flip = img_meta[0]['flip']
        flip_direction = img_meta[0]['flip_direction']
        # TODO more flexible
        proposals = bbox_mapping(proposal_list[0][:, :4], img_shape,
                                 scale_factor, flip, flip_direction)
        rois = bbox2roi([proposals])
        bbox_results = self._bbox_forward(x, rois)
        bboxes, scores = self.bbox_head.get_bboxes(
            rois,
            bbox_results['cls_score'],
            bbox_results['bbox_pred'],
            img_shape,
            scale_factor,
            rescale=False,
            cfg=None)
        aug_bboxes.append(bboxes)
        aug_scores.append(scores)
    # after merging, bboxes will be rescaled to the original image size
    merged_bboxes, merged_scores = merge_aug_bboxes(
        aug_bboxes, aug_scores, img_metas, rcnn_test_cfg)
    if merged_bboxes.shape[0] == 0:
        # There is no proposal in the single image
        det_bboxes = merged_bboxes.new_zeros(0, 5)
        det_labels = merged_bboxes.new_zeros((0, ), dtype=torch.long)
    else:
        det_bboxes, det_labels = multiclass_nms(merged_bboxes,
                                                merged_scores,
                                                rcnn_test_cfg.score_thr,
                                                rcnn_test_cfg.nms,
                                                rcnn_test_cfg.max_per_img)
    return det_bboxes, det_labels

Feature request

Using multi-scale prediction can improve the accuracy for oversized and undersized objects. I want mmdeploy to implement the same aug_test as mmdetection.

[RunningLeon]

@ypwhs Hi, we have not planned to add aug_test in mmdeploy for any codebase by far. As we understand, once scale test on a dataset is fair enough to verify the correctness of deployment models.

[ypwhs]

@ypwhs Hi, we have not planned to add aug_test in mmdeploy for any codebase by far. As we understand, once scale test on a dataset is fair enough to verify the correctness of deployment models.

Using multiple scales can improve the accuracy of small objects, which has good application scenarios in industry. If not, we need to manually predict at multiple scales and then call NMS, which complicates the inference process.

[RunningLeon]

Well, welcome PR from you if multis scales testing on a deployed model is necessary.