Maybe a bug about multi_scale_deform_attn

[pixeli99]

Hi,
I got an error when executing this line of code.

mmcv/mmcv/ops/multi_scale_deform_attn.py

Line 324 in 4558bfb

sampling_offsets = self.sampling_offsets(query).view(

RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:0 and cpu!

I personally feel that there is a problem with initializing weights. You can see here, the bias of sampling_offsets may be put on cpu?

mmcv/mmcv/ops/multi_scale_deform_attn.py

Line 249 in 4558bfb

self.sampling_offsets.bias.data = grid_init.view(-1)

The device of grid_init is cpu on my machine.

thetas = torch.arange(
            self.num_heads,
            dtype=torch.float32) * (2.0 * math.pi / self.num_heads)
grid_init = torch.stack([thetas.cos(), thetas.sin()], -1)

[zhouzaida]

Calling model.init_weight() after model.to('cuda:0') will cause this error you mentioned above. Are you willing to create a PR to resolve it?

[pixeli99]

OK, but where was the model.to('cuda:0') called?

I'm not sure where to fix it

[zhouzaida]

Oh, and there's already a PR (#2158) to fix this bug.

[pixeli99]

OK, thank you for your reply.😊

[zhouzaida]

Oh, and there's already a PR (#2158) to fix this bug.

Done