some questions on default datasetmapper with inference in detectron2

[liguang-ops]

Recently I was working on reproducing mask2former. When I dug into its source code, I found that by default, defaultTraniner calls build_detectron2_test_loader to create a test loader, and the mapper defaults to DatasetMapper. It includes an image enhancement during testing: ResizeShortEdge, which causes the image to change. What is puzzling is that the return value of DatasetMapper does not include the enhanced mask, but when using COCOEvaluator for evaluation, it reloads the ground_truth from the disk. In my understanding, shouldn't the enhanced image and mask of DatasetMapper be used as ground_truth?

[liguang-ops]

here are the DatasetMapper.from_config

@classmethod
    def from_config(cls, cfg, is_train: bool = True):
        augs = utils.build_augmentation(cfg, is_train)
        ...

def build_augmentation(cfg, is_train):
    """
    Create a list of default :class:`Augmentation` from config.
    Now it includes resizing and flipping.

    Returns:
        list[Augmentation]
    """
    if is_train:
        min_size = cfg.INPUT.MIN_SIZE_TRAIN
        max_size = cfg.INPUT.MAX_SIZE_TRAIN
        sample_style = cfg.INPUT.MIN_SIZE_TRAIN_SAMPLING
    else:
        min_size = cfg.INPUT.MIN_SIZE_TEST
        max_size = cfg.INPUT.MAX_SIZE_TEST
        sample_style = "choice"
    augmentation = [T.ResizeShortestEdge(min_size, max_size, sample_style)]
    if is_train and cfg.INPUT.RANDOM_FLIP != "none":
        augmentation.append(
            T.RandomFlip(
                horizontal=cfg.INPUT.RANDOM_FLIP == "horizontal",
                vertical=cfg.INPUT.RANDOM_FLIP == "vertical",
            )
        )
    return augmentation

[liguang-ops]

I did a simple test to see the output of the created test_loader which used the DatasetMapper:

    args = default_argument_parser().parse_args()
    cfg = setup(args)
    dl = build_detection_test_loader(cfg, cfg.DATASETS.TEST) #mapper = InstanceValidationDatasetMapper(cfg, False))
    for i , data in enumerate(dl):
        for k, v in data[0].items():
            print(k)
        break

here are the output:

file_name
image_id
height
width
image

the value of the key image is an enhanced tensor, casued by ReizeShortEdge. The output does not contain the new mask.