torch.clamp issue due to update of pytorch 1.12.0

[CryptoSalamander]

Since torch.clamp was updated in 1.12.0, the latest version of Pytorch, torch.clamp's min, max argument should be loaded on same device with input tensor. pytorch/pytorch#77035

I got an error with PyTorch 1.12.0 in this line,

Swin-Transformer/models/swin_transformer_v2.py

Line 156 in b720b41

logit_scale = torch.clamp(self.logit_scale, max=torch.log(torch.tensor(1. / 0.01))).exp()

Error :

backbones/SwinV2.py:153, in WindowAttention.forward(self, x, mask)
    151 # cosine attention
    152 attn = (F.normalize(q, dim=-1) @ F.normalize(k, dim=-1).transpose(-2, -1))
--> 153 logit_scale = torch.clamp(self.logit_scale, max=torch.log(torch.tensor(1. / 0.01))).exp()
    154 attn = attn * logit_scale
    156 relative_position_bias_table = self.cpb_mlp(self.relative_coords_table).view(-1, self.num_heads)

RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:0 and cpu! (when checking argument for argument max in method wrapper_clamp_Tensor)

In 1.11.0 this line works without problems because there was no argument-type promotion before 1.12.0!
but now, guess it should be fixed.

[jaehyunnn]

I have simply solved it as follows:

153 logit_scale = torch.clamp(self.logit_scale, max=torch.log(torch.tensor(1. / 0.01)).to(self.logit_scale.get_device())).exp()

[haraldger]

I solved it as follows:

A fix to this problem would be very useful.