Observation transformation registry for Habitat baselines to provide input cropping, resizing, etc.

[mathfac]

🚀 Feature

Observation transformation registry for Habitat baselines to provide input cropping, resizing, etc.

Motivation

To make training pipeline agnostic to observation resolution or experiment with grayscale observations instead of RGB that would be great to have a list of supported transformations.

Reference Resize and crop PR.

Pitch

1. Implement Transformations registry here.
2. Add nodes to the habitat baselines config:

OBSERVATION_TRANSFORMATIONS: ["RESIZE", "CENTER_CROP", "GRAYSCALE"]
TRANSFORMATIONS:
    RESIZE:
        WIDTH: 640
        HEIGHT: 480
   CROP:
        WIDTH: 250
        HEIGHT: 250        

3. Add a call of the transformations chain to training and evaluation general code.
4. Transformations will change Observations space for models and should be addressed in the code.

Alternative way

Same transformation can be implemented in Models hierarchy.

[srama2512]

Would it be better to have this in the models hierarchy? The data augmentation must not impact the visualization aspect of the trainer, right?

[Skylion007]

@srama2512 I think it might be best to have this as part of the policy instantiation introduced by @rpartsey in #452 and #453 .

[srama2512]

I haven't been following the policy registry issue. Just checked it out. That's a brilliant feature! And yes, this would fit perfectly there.

[Skylion007]

@srama2512 That's not entirely true, there definitely are some 'data' augmentations you want to visualize. Like creating a fake depth sensor that is just a neural network that is fed RGB input. Seeing the results of that would be desirable, so it's not quite that simple.

[srama2512]

@Skylion007 - Wouldn't that just be a part of the model definition though? That is, having a depth-prediction module. In this case, the network would only take an RGB input.

[Skylion007]

@srama2512 Your comments would be appreciated on #478

[Skylion007]

Closed by #478