Enable SDPA without kv cache #8950

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Open

iseeyuan wants to merge 1 commit into pytorch:main from iseeyuan:export-D70593177

Contributor

iseeyuan commented Mar 5, 2025

Summary: Sdpa custom op has been decoupled from kv cache by kimishpatel. Update the llama definition so that the sdpa op is applied both with and without kv cache.

Differential Revision: D70593177

iseeyuan requested review from lucylq and jackzhxng as code owners

March 5, 2025 00:12

pytorch-bot bot commented Mar 5, 2025 •

edited

Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/8950

📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 4 New Failures

As of commit d6cf34e with merge base f6c5959 ():

NEW FAILURES - The following jobs have failed:

pull / unittest / linux / linux-job (gh)
examples/models/llama/tests/test_static_attention.py::StaticAttentionTest::test_without_cache
pull / unittest / macos / macos-job (gh)
examples/models/llama/tests/test_static_attention.py::StaticAttentionTest::test_without_cache
pull / unittest-editable / linux / linux-job (gh)
examples/models/llama/tests/test_static_attention.py::StaticAttentionTest::test_without_cache
pull / unittest-editable / macos / macos-job (gh)
examples/models/llama/tests/test_static_attention.py::StaticAttentionTest::test_without_cache

This comment was automatically generated by Dr. CI and updates every 15 minutes.

facebook-github-bot added the CLA Signed label

Contributor

facebook-github-bot commented Mar 5, 2025

This pull request was exported from Phabricator. Differential Revision: D70593177

facebook-github-bot added the fb-exported label

Contributor

facebook-github-bot commented Mar 5, 2025

This pull request was exported from Phabricator. Differential Revision: D70593177

iseeyuan force-pushed the export-D70593177 branch from 12f83ab to 354159e Compare

March 5, 2025 00:19

Contributor Author

iseeyuan commented Mar 5, 2025

@pytorchbot label "topic: not user facing"

pytorch-bot bot added the topic: not user facing label

jackzhxng approved these changes

View reviewed changes

iseeyuan force-pushed the export-D70593177 branch from 354159e to d804e5c Compare

March 7, 2025 00:39

iseeyuan pushed a commit to iseeyuan/executorch-1 that referenced this pull request


          Enable SDPA without kv cache (pytorch#8950)

d804e5c

Summary:

Sdpa custom op has been decoupled from kv cache by kimishpatel. Update the llama definition so that the sdpa op is applied both with and without kv cache.

Reviewed By: kimishpatel

Differential Revision: D70593177

Contributor

facebook-github-bot commented Mar 7, 2025

This pull request was exported from Phabricator. Differential Revision: D70593177

iseeyuan pushed a commit to iseeyuan/executorch-1 that referenced this pull request


          Enable SDPA without kv cache (pytorch#8950)

eda8111

Summary:

Sdpa custom op has been decoupled from kv cache by kimishpatel. Update the llama definition so that the sdpa op is applied both with and without kv cache.

Reviewed By: kimishpatel

Differential Revision: D70593177

iseeyuan force-pushed the export-D70593177 branch from d804e5c to eda8111 Compare

March 7, 2025 00:54

Contributor

facebook-github-bot commented Mar 7, 2025

This pull request was exported from Phabricator. Differential Revision: D70593177

iseeyuan linked an issue

that may be closed by this pull request

SDPA custom op works without kv cach #9014

Open

iseeyuan pushed a commit to iseeyuan/executorch-1 that referenced this pull request


          Enable SDPA without kv cache (pytorch#8950)

627609e

Summary:

Sdpa custom op has been decoupled from kv cache by kimishpatel. Update the llama definition so that the sdpa op is applied both with and without kv cache.

Reviewed By: kimishpatel

Differential Revision: D70593177

iseeyuan force-pushed the export-D70593177 branch from eda8111 to 627609e Compare

March 7, 2025 14:31

Contributor

facebook-github-bot commented Mar 7, 2025

This pull request was exported from Phabricator. Differential Revision: D70593177

iseeyuan pushed a commit to iseeyuan/executorch-1 that referenced this pull request


          Enable SDPA without kv cache (pytorch#8950)

809f578

Summary:

Sdpa custom op has been decoupled from kv cache by kimishpatel. Update the llama definition so that the sdpa op is applied both with and without kv cache.

Reviewed By: kimishpatel

Differential Revision: D70593177

iseeyuan force-pushed the export-D70593177 branch from 627609e to 809f578 Compare

March 7, 2025 15:10

Contributor

facebook-github-bot commented Mar 7, 2025

This pull request was exported from Phabricator. Differential Revision: D70593177


          Enable SDPA without kv cache (pytorch#8950)

d6cf34e

Summary:
Pull Request resolved: pytorch#8950

Sdpa custom op has been decoupled from kv cache by kimishpatel. Update the llama definition so that the sdpa op is applied both with and without kv cache.

Reviewed By: kimishpatel

Differential Revision: D70593177

iseeyuan force-pushed the export-D70593177 branch from 809f578 to 2fee7ce Compare

March 7, 2025 21:18

iseeyuan pushed a commit to iseeyuan/executorch-1 that referenced this pull request


          Enable SDPA without kv cache (pytorch#8950)

2fee7ce

Summary:

Sdpa custom op has been decoupled from kv cache by kimishpatel. Update the llama definition so that the sdpa op is applied both with and without kv cache.

Reviewed By: kimishpatel

Differential Revision: D70593177

Contributor

facebook-github-bot commented Mar 7, 2025

This pull request was exported from Phabricator. Differential Revision: D70593177

1 similar comment

Contributor

facebook-github-bot commented Mar 7, 2025

This pull request was exported from Phabricator. Differential Revision: D70593177

iseeyuan force-pushed the export-D70593177 branch from 2fee7ce to d6cf34e Compare

March 7, 2025 21:20

github-actions bot mentioned this pull request

Weekly pr metrics report - 2025-03-01..2025-03-07 wdvr/pytorch#16

Open

This was referenced Mar 17, 2025

Weekly pr metrics report - 2025-03-01..2025-03-07 wdvr/pytorch#18

Open

Weekly pr metrics report - 2025-03-01..2025-03-07 wdvr/pytorch#20

Open

github-actions bot mentioned this pull request

Weekly pr metrics report - 2025-03-01..2025-03-07 wdvr/pytorch#22

Open

github-actions bot mentioned this pull request

Weekly pr metrics report - 2025-03-01..2025-03-07 wdvr/pytorch#26

Open

github-actions bot commented Aug 30, 2025

Looks like this PR hasn't been updated in a while so we're going to go ahead and mark this as Stale.
Feel free to remove the Stale label if you feel this was a mistake.
If you are unable to remove the Stale label please contact a maintainer in order to do so.
If you want the bot to never mark this PR stale again, add the no-stale label.
Stale pull requests will automatically be closed after 30 days of inactivity.

github-actions bot added the stale label

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CLA Signed fb-exported stale topic: not user facing