Question about the initialization of sampling_offsets

[Mayy1994]

Dear authors,

In lib.models/ops/modules/projattn.py, I noticed that the weights of self.sampling_offsets is set to constant 0, and the bias has no gradient backpropagation (line 94- line 105).

mvp/lib/models/ops/modules/projattn.py

Line 94 in 80eecd0

def _reset_parameters(self):

In my opinion, if the weights are set to 0 and the bias has no gradient, the sampled offsets will be always the same across different training samples. But it seems the offsetted points are informatively selected according to Figure 5 in your paper.

On the other hand, in the provided pretrained model, the weights and the bias are different from what they are initialized.
Could you please tell me what is the final initialization method of self.sampling_offsets?
Thank you very much!