IDTReeS - Clip boxes outside of image bounds

[isaaccorley]

This PR

• clips the bounding box coordinates to the image size (essentially clips boxes that go outside of the image bounds) and filters any boxes that have zero area after clipping.
• fixed bug where id of some geometries were being overriden resulting in some images have no objects.

Fixes part of #684

For MLBS_5 the boxes before

tensor([[107, -57, 165, -10],
        [157, -48, 190, -10],
        [127,  24, 198,  92],
        [149, 122, 199, 174],
        [ 30,  70,  70, 109],
        [ -6, -80,  48, -10],
        [ 47, 121, 114, 197],
        [119,  84, 144, 129],
        [108,  74, 129, 101],
        [ 36,  21,  83,  73],
        [137,  73, 187, 128],
        [ 38, -63,  81, -10],
        [ 19,  93,  62, 152],
        [ 71,  81, 117, 133]])

and after filtering

tensor([[127,  24, 198,  92],
        [149, 122, 199, 174],
        [ 30,  70,  70, 109],
        [ 47, 121, 114, 197],
        [119,  84, 144, 129],
        [108,  74, 129, 101],
        [ 36,  21,  83,  73],
        [137,  73, 187, 128],
        [ 19,  93,  62, 152],
        [ 71,  81, 117, 133]])

Visual result:

[ashnair1]

The coordinate swap was already done in #683

[isaaccorley]

@ashnair1 Thanks for pointing that out. Totally missed it.

[isaaccorley]

Created generic torchgeo.datasets.utils.filter_boxes function in case this occurs in other object detection datasets. Also refactored to not hardcode the image size but to use the current sample image (h, w).

[weiji14]

Code logic and docstrings seems to be sound. Did you check the image and box alignment? I'm not quite sure how to interpret the sample image you posted at #760 (comment), the colors look a bit weird 😅

[isaaccorley]

Code logic and docstrings seems to be sound. Did you check the image and box alignment? I'm not quite sure how to interpret the sample image you posted at #760 (comment), the colors look a bit weird 😅

Yes. I don't select the colors. torchvision's draw_bounding_boxes will randomly select box colors based on the number of classes in the image since I don't provide any. See #684 for more info.

[isaaccorley]

Just wanted to add some notes for everyone's clarity on the dataset.

• The test set - task 1 has no boxes or labels only RGB/HSI/CHM images and LAS point clouds
• The test set - task 2 shapefiles (e.g. task2/ITC/test_MLBS.shp) contains the image filename the boxes belong to in the properties, e.g. below in properties["plotID"] the image file is MLBS_1.tif.

{
  'geometry': {'coordinates': [[(541886.6194198467, 4136580.9452913683),
                                 (541886.6194198467, 4136588.0890556537),
                                 (541891.3025542134, 4136588.0890556537),
                                 (541891.3025542134, 4136580.9452913683),
                                 (541886.6194198467, 4136580.9452913683)]],
                'type': 'Polygon'},
   'id': '0',
   'properties': OrderedDict([('indvdID', 'MLBSE00048'),
                              ('plotID', 'MLBS_1.tif')]),
   'type': 'Feature'
}

• The train set shapefiles (e.g. train/ITC/train_MLBS.shp) contain the boxes but does not contain the image filename. However, it does contain a unique geometry id properties["id"].

{
  'geometry': {'coordinates': [[(542071.1868343, 4134996.0734410095),
                                 (542071.1868343, 4134999.0),
                                 (542074.96994003, 4134999.0),
                                 (542074.96994003, 4134996.0734410095),
                                 (542071.1868343, 4134996.0734410095)]],
                'type': 'Polygon'},
   'id': '0',
   'properties': OrderedDict([('id', 57), ('indvdID', 'MLBS01681')]),
   'type': 'Feature'
}

The image <-> geometry id mapping is inside train/Field/itc_rsFile.csv.

"id","indvdID","rsFile"
57,"MLBS01681","MLBS_1.tif"
45,"MLBS01679","MLBS_1.tif"
49,"MLBS01674","MLBS_1.tif"
63,"MLBS01678","MLBS_1.tif"
...

Note that the test set does not have any csv files only the train set. Because of this, the implementation for parsing the dataset can become quite verbose and requires different logic for the the train set, test set - task1, and test set - task2.

Hopefully this clarifies some of the implementation decisions.

[adamjstewart]

Okay, I think I finally understand the code well enough to review this now!

So if I understand correctly, this PR only solves issue (3) in #684. Once this PR is merged, A) all bounding boxes will be clipped to the image bounds which will fix plotting, and B) bounding boxes won't be accidentally deleted from the testing set due to integer overlap.

My only change request is to replace the current id = len(features) + 1 logic with a counter i = 0, i += 1 so we aren't computing the length of a list in each iteration.

[isaaccorley]

Okay, I think I finally understand the code well enough to review this now!

So if I understand correctly, this PR only solves issue (3) in #684. Once this PR is merged, A) all bounding boxes will be clipped to the image bounds which will fix plotting, and B) bounding boxes won't be accidentally deleted from the testing set due to integer overlap.

My only change request is to replace the current id = len(features) + 1 logic with a counter i = 0, i += 1 so we aren't computing the length of a list in each iteration.

I've made the requested change. Also note that this PR doesn't just fix plotting but also allows a user to actually train on the dataset whereas before you would have issues dealing with boxes with negative values.