Inconsistent Behavior of _check_coplanar in iou_box3d

[GoodarzMehr]

🐛 Bugs / Unexpected behaviors

I have encoutered instances where the _check_coplanar function of iou_box3d.py behaves inconsistently across GPUs when the number of boxes is 1.

Instructions To Reproduce the Issue:

I'm using Pytorch3D v0.6.2 and Torch 1.10.1+cu113. Looking at the code I don't see any changes that have been made to the _check_coplanar function in newer versions.

Let

boxes = torch.tensor([[[-115.3384,   31.8733,    0.2705],
                       [-115.3384,   31.8733,    4.5234],
                       [-109.9031,   23.1563,    4.5234],
                       [-109.9031,   23.1563,    0.2705],
                       [-118.6852,   29.7865,    0.2705],
                       [-118.6852,   29.7865,    4.5234],
                       [-113.2499,   21.0695,    4.5234],
                       [-113.2499,   21.0695,    0.2705]]])

when it is passed through _check_coplanar, if the tensor is on CPU (Ryzen 3900X) or a GPU (RTX 2080 Ti) the value of mat1.bmm(mat2).abs() is 0, but if the tensor is on another GPU (RTX 3090) the value of mat1.bmm(mat2).abs() becomes 0.0108, which raises a non-coplanar error.

On the RTX 3090, I've only experienced this when the number of boxes is 1, everything works fine for box numbers more than that.

[bottler]

This could just be numerical differences in libraries or hardware lower down than pytorch3d. Is the dtype of boxes the same in all cases? (e.g. could the default floating point type be different).

[GoodarzMehr]

In all cases the tensors use the default dtype, which is torch.float32. I tried passing torch.double tensors but I got an error saying that dtype is not allowed.

[bottler]

I don't know for certain if this matters, but the box vertices are not the right way round. If principal_axes is boxes[0,[1,3,4]]-boxes[0,0] then I expect principal_axes to have a positive determinant.

[GoodarzMehr]

Does it matter though? Considering the box description in the code, my box is the result of a 90 deg CCW rotation along the (0, 4) axis and a reflection across the (0, 1, 2, 3) face, so the vertices on planes as defined in _box_planes should remain invariant.

It feels really odd that it works on one GPU but not another, and that it only happens when B (the number of boxes) is 1.

[GoodarzMehr]

Here's some supplementary information:

If I define (essentially two of the same box)

box=torch.tensor([[[-115.3384,   31.8733,    0.2705],
         [-115.3384,   31.8733,    4.5234],
         [-109.9031,   23.1563,    4.5234],
         [-109.9031,   23.1563,    0.2705],
         [-118.6852,   29.7865,    0.2705],
         [-118.6852,   29.7865,    4.5234],
         [-113.2499,   21.0695,    4.5234],
         [-113.2499,   21.0695,    0.2705]],

        [[-115.3384,   31.8733,    0.2705],
         [-115.3384,   31.8733,    4.5234],
         [-109.9031,   23.1563,    4.5234],
         [-109.9031,   23.1563,    0.2705],
         [-118.6852,   29.7865,    0.2705],
         [-118.6852,   29.7865,    4.5234],
         [-113.2499,   21.0695,    4.5234],
         [-113.2499,   21.0695,    0.2705]]], device='cuda:0')

then box3d_overlap(box, box) gives me

(tensor([[172.3125, 172.3125],
        [172.3125, 172.3125]], device='cuda:0'), tensor([[1., 1.],
        [1., 1.]], device='cuda:0'))

However, if I define

box = torch.tensor([[[-115.3384,   31.8733,    0.2705],
         [-115.3384,   31.8733,    4.5234],
         [-109.9031,   23.1563,    4.5234],
         [-109.9031,   23.1563,    0.2705],
         [-118.6852,   29.7865,    0.2705],
         [-118.6852,   29.7865,    4.5234],
         [-113.2499,   21.0695,    4.5234],
         [-113.2499,   21.0695,    0.2705]]], device='cuda:0')

then box3d_overlap(box, box) gives me:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.8/dist-packages/pytorch3d/ops/iou_box3d.py", line 161, in box3d_overlap
    _check_coplanar(boxes1, eps)
  File "/usr/local/lib/python3.8/dist-packages/pytorch3d/ops/iou_box3d.py", line 67, in _check_coplanar
    raise ValueError(msg)
ValueError: Plane vertices are not coplanar