Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

flashinfer paged attention #2772

Open
wants to merge 22 commits into
base: main
Choose a base branch
from
Open

flashinfer paged attention #2772

wants to merge 22 commits into from

Commits on Feb 4, 2024

  1. prefill works, needs decoding impl

    @sumo43
    sumo43 committed Feb 4, 2024
    Configuration menu
    Copy the full SHA
    e150f3c View commit details
    Browse the repository at this point in the history

  2. add some decoding code

    @sumo43
    sumo43 committed Feb 4, 2024
    Configuration menu
    Copy the full SHA
    dc3e5d4 View commit details
    Browse the repository at this point in the history

  3. working decoding

    @sumo43
    sumo43 committed Feb 4, 2024
    Configuration menu
    Copy the full SHA
    55191f5 View commit details
    Browse the repository at this point in the history

  4. fix index, remove some ops

    @sumo43
    sumo43 committed Feb 4, 2024
    Configuration menu
    Copy the full SHA
    5b5fff0 View commit details
    Browse the repository at this point in the history

  5. 1.5x throughput 1000in/out. TODO fix cache update

    @sumo43
    sumo43 committed Feb 4, 2024
    Configuration menu
    Copy the full SHA
    34f1743 View commit details
    Browse the repository at this point in the history

Commits on Feb 5, 2024

  1. tweak kernel

    @sumo43
    sumo43 committed Feb 5, 2024
    Configuration menu
    Copy the full SHA
    a548101 View commit details
    Browse the repository at this point in the history

  2. fix ind

    @sumo43
    sumo43 committed Feb 5, 2024
    Configuration menu
    Copy the full SHA
    288717f View commit details
    Browse the repository at this point in the history

  3. fix kvcache?

    @sumo43
    sumo43 committed Feb 5, 2024
    Configuration menu
    Copy the full SHA
    7c8dfaf View commit details
    Browse the repository at this point in the history

  4. fix gqa shape

    @sumo43
    sumo43 committed Feb 5, 2024
    Configuration menu
    Copy the full SHA
    a6bb43a View commit details
    Browse the repository at this point in the history

Commits on Feb 6, 2024

  1. rm unnecessary files, add flashinfer as requirement

    @sumo43
    sumo43 committed Feb 6, 2024
    Configuration menu
    Copy the full SHA
    3922712 View commit details
    Browse the repository at this point in the history

  2. fix kv_cache indexing

    @sumo43
    sumo43 committed Feb 6, 2024
    Configuration menu
    Copy the full SHA
    7aa25fa View commit details
    Browse the repository at this point in the history

  3. batched kvcache indexing

    @sumo43
    sumo43 committed Feb 6, 2024
    Configuration menu
    Copy the full SHA
    a176b8d View commit details
    Browse the repository at this point in the history

Commits on Feb 7, 2024

  1. working version. last commit before formatting + tests

    @sumo43
    sumo43 committed Feb 7, 2024
    Configuration menu
    Copy the full SHA
    037a54a View commit details
    Browse the repository at this point in the history

  2. formatting and compatibility pt 1

    @sumo43
    sumo43 committed Feb 7, 2024
    Configuration menu
    Copy the full SHA
    8368d0b View commit details
    Browse the repository at this point in the history

  3. formatting and compatibility pt 2

    @sumo43
    sumo43 committed Feb 7, 2024
    Configuration menu
    Copy the full SHA
    8276872 View commit details
    Browse the repository at this point in the history

  4. fix reqs

    @sumo43
    sumo43 committed Feb 7, 2024
    Configuration menu
    Copy the full SHA
    4d06135 View commit details
    Browse the repository at this point in the history

  5. Merge branch 'main' into kvcache2

    @sumo43
    sumo43 authored Feb 7, 2024
    Configuration menu
    Copy the full SHA
    b0d5cff View commit details
    Browse the repository at this point in the history

  6. rm hardcoded values, passes tests w/ eager mode

    @sumo43
    sumo43 committed Feb 7, 2024
    Configuration menu
    Copy the full SHA
    57874d5 View commit details
    Browse the repository at this point in the history

Commits on Feb 8, 2024

  1. fix indexing

    @sumo43
    sumo43 committed Feb 8, 2024
    Configuration menu
    Copy the full SHA
    cddf14d View commit details
    Browse the repository at this point in the history

  2. rm prefill

    @sumo43
    sumo43 committed Feb 8, 2024
    Configuration menu
    Copy the full SHA
    35f3e18 View commit details
    Browse the repository at this point in the history

  3. remove file

    @sumo43
    sumo43 committed Feb 8, 2024
    Configuration menu
    Copy the full SHA
    f430e50 View commit details
    Browse the repository at this point in the history

Commits on Feb 9, 2024

  1. fix device for tp

    @sumo43
    sumo43 committed Feb 9, 2024
    Configuration menu
    Copy the full SHA
    af69cd6 View commit details
    Browse the repository at this point in the history