New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

No relative position embedding? #4

Open

BrandonHanx opened this issue Dec 12, 2020 · 0 comments

BrandonHanx commented Dec 12, 2020

Hi, @leaderj1001

Thanks for your code. However, I am a little bit confused about this line

LambdaNetworks/model.py

Line 48 in 3a51b6a

lambda_p = torch.einsum('ku,bvun->bkvn', self.embedding, values)

It seems like did not use relative position embedding when strategy is not local context. What's the meaning of this line?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment