Some questions about HypAgg

[nikita-petrashen]

Hello, my name is Nikita and I'm a second year MSc student at Skolkovo Institute of Science and Technology, Moscow, Russia. I am kinda new to the area of Hyperbolic Deep Learning and I have some questions regarding your paper. I have noticed that in the forward pass of the HypAgg module there is a loop over the 0-th dimension in the case of local aggregation. I wonder if this loop can be avoided in any way and why is there no such loop in the case of aggregation at T_0? The second question is why is there (in the same forward pass) a manifold.proj after manifold.exp? Doesn't manifold.exp guarantee that the result is on the manifold? Or is there some kind of an approximation in the computation of manifold.exp?

Thank you in advance!

[Chuan1997]

For the second question, the paper states that projection is needed to constrain embeddings and tangent vectors to remain on the manifold and tangent spaces.

[JamesDYX]

For the second question, the paper states that projection is needed to constrain embeddings and tangent vectors to remain on the manifold and tangent spaces.

But in mainfolds/hyperboloid.py, before return from the function, each manifold.exp has a manifold.proj after it, so exp has already guarantee that the result is on the manifold space. And meanwhile, in HypAGG or other hyperbolic layers, an extra manifold.proj exists. This also makes me confused.

[ruilialice]

For the second question, the paper states that projection is needed to constrain embeddings and tangent vectors to remain on the manifold and tangent spaces.

But in mainfolds/hyperboloid.py, before return from the function, each manifold.exp has a manifold.proj after it, so exp has already guarantee that the result is on the manifold space. And meanwhile, in HypAGG or other hyperbolic layers, an extra manifold.proj exists. This also makes me confused.

But it seems that even if the code calls manifold.proj twice, the result is the same as it calls manifold.proj once. The result will not change if the code calls manifold.proj one more time, because the function manifold.proj only make sure that input x stays in the hyperboloid manifold with negative curvature -1/c.

[fratajcz]

Hi Nikita, I doubt that you are still interested in this question but I have seen that this repo is still frequented by users and I was also first puzzled by this, so I will give my impression of what is happening in that for loop in case it helps someone.

If we don't use local aggregation, we can just map all hyperbolic vectors to the euclidean space by using the logarithmic map at the origin (manifold.logmap0(x, c))

x_tangent = self.manifold.logmap0(x, c=self.c)

However, local aggregation means we want to map each node j in the neighborhood of node i using the "local" logarithmic map at point i instead of the origin (see the lower index of the outer exp function in formula 9 in the paper). Since the information which node is in the neighborhood of which node is stored in the adjacency matrix which comes into play later, the line you mentioned simply computes each node j's logarithmic mapping with respect to each other node i:

x_local_tangent = []
for i in range(x.size(0)):
    x_local_tangent.append(self.manifold.logmap(x[i], x, c=self.c))
x_local_tangent = torch.stack(x_local_tangent, dim=0)

note that his leaves us with a tensor of shape (n, n, d) where n is the number of nodes and d the dimension. This is also the reason why there are some issues posted here that it runs out of memory for n>15k.

The neighborhood aggregation then comes in three lines later and collapses the whole thing back to a tensor of shape (n, d):

support_t = torch.sum(adj_att.unsqueeze(-1) * x_local_tangent, dim=1)

If someone reads this and sees a mistake in my reasoning (perhaps even @ines-chami ) or came to the same conclusion and can confirm my reasoning, please leave a comment.

[a912289748]

Hi Nikita, I doubt that you are still interested in this question but I have seen that this repo is still frequented by users and I was also first puzzled by this, so I will give my impression of what is happening in that for loop in case it helps someone.

If we don't use local aggregation, we can just map all hyperbolic vectors to the euclidean space by using the logarithmic map at the origin (manifold.logmap0(x, c))

x_tangent = self.manifold.logmap0(x, c=self.c)

However, local aggregation means we want to map each node j in the neighborhood of node i using the "local" logarithmic map at point i instead of the origin (see the lower index of the outer exp function in formula 9 in the paper). Since the information which node is in the neighborhood of which node is stored in the adjacency matrix which comes into play later, the line you mentioned simply computes each node j's logarithmic mapping with respect to each other node i:

x_local_tangent = []
for i in range(x.size(0)):
    x_local_tangent.append(self.manifold.logmap(x[i], x, c=self.c))
x_local_tangent = torch.stack(x_local_tangent, dim=0)

note that his leaves us with a tensor of shape (n, n, d) where n is the number of nodes and d the dimension. This is also the reason why there are some issues posted here that it runs out of memory for n>15k.

The neighborhood aggregation then comes in three lines later and collapses the whole thing back to a tensor of shape (n, d):

support_t = torch.sum(adj_att.unsqueeze(-1) * x_local_tangent, dim=1)

If someone reads this and sees a mistake in my reasoning (perhaps even @ines-chami ) or came to the same conclusion and can confirm my reasoning, please leave a comment.

i meet the same question