Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Misc] Remove Gemma RoPE #7638

Merged
merged 4 commits into from
Aug 19, 2024
Merged

[Misc] Remove Gemma RoPE #7638

merged 4 commits into from
Aug 19, 2024

Conversation

WoosukKwon
Copy link
Collaborator

This PR removes GemmaRotaryEmbedding because it is actually the same as the normal RotaryEmbedding.

This is because

torch.arange(0, self.rotary_dim, 2, dtype=torch.float)

is equal to

torch.arange(0, self.rotary_dim, 2, dtype=torch.int64).float()

as rotary_dim is less than or equal to head_size (i.e., 256).

Copy link

👋 Hi! Thank you for contributing to the vLLM project.
Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run fastcheck CI which consists a small and essential subset of CI tests to quickly catch errors. You can run other CI tests on top of default ones by unblocking the steps in your fast-check build on Buildkite UI.

Once the PR is approved and ready to go, please make sure to run full CI as it is required to merge (or just use auto-merge).

To run full CI, you can do one of these:

  • Comment /ready on the PR
  • Add ready label to the PR
  • Enable auto-merge.

🚀

@WoosukKwon WoosukKwon added the ready ONLY add when PR is ready to merge/full CI is needed label Aug 18, 2024
@WoosukKwon WoosukKwon mentioned this pull request Aug 19, 2024
@simon-mo simon-mo merged commit df845b2 into main Aug 19, 2024
47 checks passed
@WoosukKwon WoosukKwon deleted the gemma-rope branch August 19, 2024 16:40
zifeitong pushed a commit to zifeitong/vllm that referenced this pull request Aug 20, 2024
fialhocoelho pushed a commit to opendatahub-io/vllm that referenced this pull request Aug 22, 2024
omrishiv pushed a commit to omrishiv/vllm that referenced this pull request Aug 26, 2024
Alvant pushed a commit to compressa-ai/vllm that referenced this pull request Oct 26, 2024
Signed-off-by: Alvant <alvasian@yandex.ru>
KuntaiDu pushed a commit to KuntaiDu/vllm that referenced this pull request Nov 20, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ready ONLY add when PR is ready to merge/full CI is needed
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants