[Bug] active_rotated_filter op get a wrong value on cuda

[dflhw]

Prerequisite

• I have searched Issues and Discussions but cannot get the expected help.
• The bug has not been fixed in the latest version(https://github.com/open-mmlab/mmcv).

Environment

mmcv = 1.7.1
pytorch = 1.13.1
cuda = 11.2
python = 3.9.15

Reproduces the problem - code sample

import torch
import numpy as np
from mmcv.ops import active_rotated_filter

device = torch.device("cpu")
print('===============>cpu')
feature = torch.tensor([[[[[385., 658., 852.],
[619., 716., 106.],
[189., 706., 143.]],
[[394., 779., 399.],
[268., 891., 822.],
[262., 163., 958.]]]]], dtype=torch.float32, device=device)
indices = torch.tensor([[[[5, 4], [6, 5], [2, 3]], [[3, 2], [1, 6], [7, 7]], [[4, 8], [9, 1], [8, 9]]],
[[[7, 1], [2, 5], [3, 4]], [[5, 2], [1, 7], [9, 8]], [[4, 3], [8, 6], [6, 9]]]],
dtype=torch.int32, device=device)
cpu_out = active_rotated_filter(feature, indices).cpu().numpy()
print("cpu_out: ", cpu_out[0])

print('===============>cuda')
feature = feature.cuda()
indices = indices.cuda()
cuda_out = active_rotated_filter(feature, indices).cpu().numpy()
print("cuda_out: ", cuda_out[0])

assert np.allclose(cuda_out, cpu_out, atol=1e-4)

Reproduces the problem - command or script

When num_orientations is 2, the calculation result of active_rotated_filter op on cpu is inconsistent with that on cuda.

Reproduces the problem - error message

Assertion Error

Additional information

No response