Add Points to TVTensors

[Alexandre-SCHOEPP]

🚀 The feature

A new TVTensor subclass that represents a (N, 2) tensor.

It can be used to represent points, and should be affected by the augmentation pipelines as such (for example, RandomRotation should also affect the points coordinates)

Motivation, pitch

TVTensors allow the easy integration of augmentation pipelines with semi-complex data structure, by tagging tensors as being linked to the augmentation and being affected by it. However, TVTensors is far from covering the entire wealth of complexity.

Allowing a set of points to be manipulated as such would allow more complex structures, like skeletons, polygons, rotated bboxes, and tracking points to be handled ¹.

The signature should be similar to all others TVTensors, which, as a whole, are very usable.

Alternatives

No response

Additional context

No response

Footnotes

1. Specific support is, IMO better, but supporting arbitrary points and then building on them is an easily implementable trajectory that may lead to wider support later. ↩

[NicolasHug]

Thank you for the feature request @Alexandre-SCHOEPP .

Supporting a KeyPoints TVTensor class is something we treated as lower-priority initially, but it is definitely in scope. It's a lot of work though, since we would need to implement keypoint kernels for all / most existing transforms. That can probably be done in sequences though.

I won't be able to add this to my stack for now, i.e. I cannot lead the implementation myself. But I can spend some time reviewing if you're interested in adding those.

[Alexandre-SCHOEPP]

I can spend some time reviewing if you're interested in adding those.

I don't have the time nowadays, so, unless someone gets on it, it'll have to wait some more for now. If I ever get the time, why not !

[Alexandre-SCHOEPP]

@NicolasHug I ended up having some time to dedicate to it, I will be making a PR so that we can discuss my early changes