Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Implement single_query_cached_kv_attention kernel #3

Merged
merged 14 commits into from
Mar 1, 2023
Merged

Implement single_query_cached_kv_attention kernel #3

merged 14 commits into from
Mar 1, 2023

Commits on Feb 28, 2023

  1. [Bugfix] int -> torch.int

    @WoosukKwon
    WoosukKwon committed Feb 28, 2023
    Configuration menu
    Copy the full SHA
    6f5c391 View commit details
    Browse the repository at this point in the history

  2. [WIP] Add attention kernel

    @WoosukKwon
    WoosukKwon committed Feb 28, 2023
    Configuration menu
    Copy the full SHA
    5726be8 View commit details
    Browse the repository at this point in the history

Commits on Mar 1, 2023

  1. Fix a bug in loading QK

    @WoosukKwon
    WoosukKwon committed Mar 1, 2023
    Configuration menu
    Copy the full SHA
    1202c0a View commit details
    Browse the repository at this point in the history

  2. Raise an error for invalid block sizes

    @WoosukKwon
    WoosukKwon committed Mar 1, 2023
    Configuration menu
    Copy the full SHA
    b5f3c49 View commit details
    Browse the repository at this point in the history

  3. Add FP32 flags

    @WoosukKwon
    WoosukKwon committed Mar 1, 2023
    Configuration menu
    Copy the full SHA
    645a49e View commit details
    Browse the repository at this point in the history

  4. Change value cache layout

    @WoosukKwon
    WoosukKwon committed Mar 1, 2023
    Configuration menu
    Copy the full SHA
    ad13de6 View commit details
    Browse the repository at this point in the history

  5. Add single_query_cached_kv_attention

    @WoosukKwon
    WoosukKwon committed Mar 1, 2023
    Configuration menu
    Copy the full SHA
    3b416c0 View commit details
    Browse the repository at this point in the history

  6. Add TODO

    @WoosukKwon
    WoosukKwon committed Mar 1, 2023
    Configuration menu
    Copy the full SHA
    723c9fc View commit details
    Browse the repository at this point in the history

  7. Move

    @WoosukKwon
    WoosukKwon committed Mar 1, 2023
    Configuration menu
    Copy the full SHA
    78e048a View commit details
    Browse the repository at this point in the history

  8. Minor

    @WoosukKwon
    WoosukKwon committed Mar 1, 2023
    Configuration menu
    Copy the full SHA
    b7c7a60 View commit details
    Browse the repository at this point in the history

  9. Minor

    @WoosukKwon
    WoosukKwon committed Mar 1, 2023
    Configuration menu
    Copy the full SHA
    a0fbd6b View commit details
    Browse the repository at this point in the history

  10. Minor

    @WoosukKwon
    WoosukKwon committed Mar 1, 2023
    Configuration menu
    Copy the full SHA
    c89e5d3 View commit details
    Browse the repository at this point in the history

  11. Add a test for attention ops

    @WoosukKwon
    WoosukKwon committed Mar 1, 2023
    Configuration menu
    Copy the full SHA
    7d30bdc View commit details
    Browse the repository at this point in the history

  12. Minor fix in comment

    @WoosukKwon
    WoosukKwon committed Mar 1, 2023
    Configuration menu
    Copy the full SHA
    40a2f7b View commit details
    Browse the repository at this point in the history