It seems that the code does not just do softmax for the neighbor nodes, but just adds a bias_mat. I don't understand this step very well.

[HelloCxz]

It seems that the code does not just do softmax for the neighbor nodes, but just adds a bias_mat. I don't understand this step very well.

[PetarV-]

Hello,

Thanks for your issue.

The bias matrix has entries of 0 in cells (i, j) where there exists an edge, and entries of -1e9 (approximation to negative infinity) when there does not exist an edge. When a softmax is applied to such a sum of logits and bias, this will effectively "mask out" non neighbouring nodes, as softmax of negative infinity is zero.

Hope that helps!

Thanks,
Petar