Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Allows interweaving of arbitrary kinds of 'attention' layers, like sliding window, reuse prev layer kv cache etc. #1299

Merged
merged 86 commits into from
Jun 30, 2024
Merged

Allows interweaving of arbitrary kinds of 'attention' layers, like sliding window, reuse prev layer kv cache etc. #1299

merged 86 commits into from
Jun 30, 2024

Commits on Jun 21, 2024

  1. [WIP] Allows interweaving of arbitrary kinds of 'attention' layers, l… 

    …ike RNN, sliding window etc.
    @ShashankMosaicML
    ShashankMosaicML committed Jun 21, 2024
    Configuration menu
    Copy the full SHA
    8804472 View commit details
    Browse the repository at this point in the history

  2. lint

    @ShashankMosaicML
    ShashankMosaicML committed Jun 21, 2024
    Configuration menu
    Copy the full SHA
    a50755e View commit details
    Browse the repository at this point in the history

  3. applying overrides to blocks rather than just attentions

    @ShashankMosaicML
    ShashankMosaicML committed Jun 21, 2024
    Configuration menu
    Copy the full SHA
    fcc28a1 View commit details
    Browse the repository at this point in the history

  4. add docstring

    @ShashankMosaicML
    ShashankMosaicML committed Jun 21, 2024
    Configuration menu
    Copy the full SHA
    877d80e View commit details
    Browse the repository at this point in the history

  5. Merge branch 'main' into mixed_attention_modules

    @ShashankMosaicML
    ShashankMosaicML authored Jun 21, 2024
    Configuration menu
    Copy the full SHA
    1f11415 View commit details
    Browse the repository at this point in the history

  6. minor

    @ShashankMosaicML
    ShashankMosaicML committed Jun 21, 2024
    Configuration menu
    Copy the full SHA
    dd1c64b View commit details
    Browse the repository at this point in the history

  7. Merge branch 'mixed_attention_modules' of github.com:ShashankMosaicML… 

    …/llm-foundry into mixed_attention_modules
    @ShashankMosaicML
    ShashankMosaicML committed Jun 21, 2024
    Configuration menu
    Copy the full SHA
    d3b32e6 View commit details
    Browse the repository at this point in the history

Commits on Jun 22, 2024

  1. changing yaml specification style

    @ShashankMosaicML
    ShashankMosaicML committed Jun 22, 2024
    Configuration menu
    Copy the full SHA
    fc1bf0b View commit details
    Browse the repository at this point in the history

  2. ..

    @ShashankMosaicML
    ShashankMosaicML committed Jun 22, 2024
    Configuration menu
    Copy the full SHA
    81e2930 View commit details
    Browse the repository at this point in the history

  3. fixes

    @ShashankMosaicML
    ShashankMosaicML committed Jun 22, 2024
    Configuration menu
    Copy the full SHA
    9b3d813 View commit details
    Browse the repository at this point in the history

  4. fix

    @ShashankMosaicML
    ShashankMosaicML committed Jun 22, 2024
    Configuration menu
    Copy the full SHA
    aafcebb View commit details
    Browse the repository at this point in the history

  5. fix

    @ShashankMosaicML
    ShashankMosaicML committed Jun 22, 2024
    Configuration menu
    Copy the full SHA
    b46756f View commit details
    Browse the repository at this point in the history

  6. fix

    @ShashankMosaicML
    ShashankMosaicML committed Jun 22, 2024
    Configuration menu
    Copy the full SHA
    ad6ba32 View commit details
    Browse the repository at this point in the history

  7. refactoring

    @ShashankMosaicML
    ShashankMosaicML committed Jun 22, 2024
    Configuration menu
    Copy the full SHA
    3ea79fd View commit details
    Browse the repository at this point in the history

  8. add warning

    @ShashankMosaicML
    ShashankMosaicML committed Jun 22, 2024
    Configuration menu
    Copy the full SHA
    13802cb View commit details
    Browse the repository at this point in the history

  9. compute only query vector when reusing kv

    @ShashankMosaicML
    ShashankMosaicML committed Jun 22, 2024
    Configuration menu
    Copy the full SHA
    9b6ae9c View commit details
    Browse the repository at this point in the history

  10. refactor

    @ShashankMosaicML
    ShashankMosaicML committed Jun 22, 2024
    Configuration menu
    Copy the full SHA
    c774a4b View commit details
    Browse the repository at this point in the history

  11. fixing

    @ShashankMosaicML
    ShashankMosaicML committed Jun 22, 2024
    Configuration menu
    Copy the full SHA
    8dee35e View commit details
    Browse the repository at this point in the history

Commits on Jun 23, 2024

  1. adding test for reusing previous layer kv cache

    @ShashankMosaicML
    ShashankMosaicML committed Jun 23, 2024
    Configuration menu
    Copy the full SHA
    8ff15b4 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'main' into mixed_attention_modules

    @ShashankMosaicML
    ShashankMosaicML authored Jun 23, 2024
    Configuration menu
    Copy the full SHA
    b1ee62a View commit details
    Browse the repository at this point in the history

  3. adding error messages

    @ShashankMosaicML
    ShashankMosaicML committed Jun 23, 2024
    Configuration menu
    Copy the full SHA
    04e9888 View commit details
    Browse the repository at this point in the history

  4. ..

    @ShashankMosaicML
    ShashankMosaicML committed Jun 23, 2024
    Configuration menu
    Copy the full SHA
    5eee910 View commit details
    Browse the repository at this point in the history

  5. adding test

    @ShashankMosaicML
    ShashankMosaicML committed Jun 23, 2024
    Configuration menu
    Copy the full SHA
    2a6c986 View commit details
    Browse the repository at this point in the history

  6. add logging

    @ShashankMosaicML
    ShashankMosaicML committed Jun 23, 2024
    Configuration menu
    Copy the full SHA
    7bf89f2 View commit details
    Browse the repository at this point in the history

  7. adding logging

    @ShashankMosaicML
    ShashankMosaicML committed Jun 23, 2024
    Configuration menu
    Copy the full SHA
    dcc5cc0 View commit details
    Browse the repository at this point in the history

  8. minor

    @ShashankMosaicML
    ShashankMosaicML committed Jun 23, 2024
    Configuration menu
    Copy the full SHA
    06d03c1 View commit details
    Browse the repository at this point in the history

  9. bug fix, adding test

    @ShashankMosaicML
    ShashankMosaicML committed Jun 23, 2024
    Configuration menu
    Copy the full SHA
    ec42e72 View commit details
    Browse the repository at this point in the history

  10. minor

    @ShashankMosaicML
    ShashankMosaicML committed Jun 23, 2024
    Configuration menu
    Copy the full SHA
    cc1f2f3 View commit details
    Browse the repository at this point in the history

Commits on Jun 24, 2024

  1. Merge branch 'main' into mixed_attention_modules

    @ShashankMosaicML
    ShashankMosaicML authored Jun 24, 2024
    Configuration menu
    Copy the full SHA
    9f7b346 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'main' into mixed_attention_modules

    @ShashankMosaicML
    ShashankMosaicML authored Jun 24, 2024
    Configuration menu
    Copy the full SHA
    214456f View commit details
    Browse the repository at this point in the history

Commits on Jun 25, 2024

  1. Merge branch 'main' into mixed_attention_modules

    @ShashankMosaicML
    ShashankMosaicML authored Jun 25, 2024
    Configuration menu
    Copy the full SHA
    62a3030 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'main' into mixed_attention_modules

    @ShashankMosaicML
    ShashankMosaicML authored Jun 25, 2024
    Configuration menu
    Copy the full SHA
    c955b98 View commit details
    Browse the repository at this point in the history

  3. addressing some comments

    @ShashankMosaicML
    ShashankMosaicML committed Jun 25, 2024
    Configuration menu
    Copy the full SHA
    63f4196 View commit details
    Browse the repository at this point in the history

  4. Merge branch 'mixed_attention_modules' of github.com:ShashankMosaicML… 

    …/llm-foundry into mixed_attention_modules
    @ShashankMosaicML
    ShashankMosaicML committed Jun 25, 2024
    Configuration menu
    Copy the full SHA
    8e89db9 View commit details
    Browse the repository at this point in the history

  5. addressing some comments

    @ShashankMosaicML
    ShashankMosaicML committed Jun 25, 2024
    Configuration menu
    Copy the full SHA
    89bc22e View commit details
    Browse the repository at this point in the history

  6. setting absolute absolute value for reuse_kv_layer_idx

    @ShashankMosaicML
    ShashankMosaicML committed Jun 25, 2024
    Configuration menu
    Copy the full SHA
    0a3f1b4 View commit details
    Browse the repository at this point in the history

  7. lint

    @ShashankMosaicML
    ShashankMosaicML committed Jun 25, 2024
    Configuration menu
    Copy the full SHA
    b74330e View commit details
    Browse the repository at this point in the history

  8. adding tests for override_block_args

    @ShashankMosaicML
    ShashankMosaicML committed Jun 25, 2024
    Configuration menu
    Copy the full SHA
    5079d2e View commit details
    Browse the repository at this point in the history

Commits on Jun 26, 2024

  1. adding error if reusing kv cache from a mismatch layer

    @ShashankMosaicML
    ShashankMosaicML committed Jun 26, 2024
    Configuration menu
    Copy the full SHA
    2a7e948 View commit details
    Browse the repository at this point in the history

  2. fixing test

    @ShashankMosaicML
    ShashankMosaicML committed Jun 26, 2024
    Configuration menu
    Copy the full SHA
    1daf534 View commit details
    Browse the repository at this point in the history

  3. Merge branch 'main' into mixed_attention_modules

    @ShashankMosaicML
    ShashankMosaicML authored Jun 26, 2024
    Configuration menu
    Copy the full SHA
    a57af04 View commit details
    Browse the repository at this point in the history

  4. fixing code, test

    @ShashankMosaicML
    ShashankMosaicML committed Jun 26, 2024
    Configuration menu
    Copy the full SHA
    afa4c09 View commit details
    Browse the repository at this point in the history

  5. fix

    @ShashankMosaicML
    ShashankMosaicML committed Jun 26, 2024
    Configuration menu
    Copy the full SHA
    a8a8a8b View commit details
    Browse the repository at this point in the history

  6. ..

    @ShashankMosaicML
    ShashankMosaicML committed Jun 26, 2024
    Configuration menu
    Copy the full SHA
    6f468ef View commit details
    Browse the repository at this point in the history

  7. refactoring

    @ShashankMosaicML
    ShashankMosaicML committed Jun 26, 2024
    Configuration menu
    Copy the full SHA
    0f96a87 View commit details
    Browse the repository at this point in the history

  8. fix

    @ShashankMosaicML
    ShashankMosaicML committed Jun 26, 2024
    Configuration menu
    Copy the full SHA
    085de4c View commit details
    Browse the repository at this point in the history

  9. ..

    @ShashankMosaicML
    ShashankMosaicML committed Jun 26, 2024
    Configuration menu
    Copy the full SHA
    65b816e View commit details
    Browse the repository at this point in the history

  10. ..

    @ShashankMosaicML
    ShashankMosaicML committed Jun 26, 2024
    Configuration menu
    Copy the full SHA
    e919422 View commit details
    Browse the repository at this point in the history

  11. ..

    @ShashankMosaicML
    ShashankMosaicML committed Jun 26, 2024
    Configuration menu
    Copy the full SHA
    b2c9ee9 View commit details
    Browse the repository at this point in the history

  12. refactoring

    @ShashankMosaicML
    ShashankMosaicML committed Jun 26, 2024
    Configuration menu
    Copy the full SHA
    7e77979 View commit details
    Browse the repository at this point in the history

  13. ..

    @ShashankMosaicML
    ShashankMosaicML committed Jun 26, 2024
    Configuration menu
    Copy the full SHA
    0ff081a View commit details
    Browse the repository at this point in the history

  14. ..

    @ShashankMosaicML
    ShashankMosaicML committed Jun 26, 2024
    Configuration menu
    Copy the full SHA
    d15d34b View commit details
    Browse the repository at this point in the history

  15. ..

    @ShashankMosaicML
    ShashankMosaicML committed Jun 26, 2024
    Configuration menu
    Copy the full SHA
    114b42f View commit details
    Browse the repository at this point in the history

  16. adding test for _get_modules_order_expanded

    @ShashankMosaicML
    ShashankMosaicML committed Jun 26, 2024
    Configuration menu
    Copy the full SHA
    380e954 View commit details
    Browse the repository at this point in the history

  17. fixing test

    @ShashankMosaicML
    ShashankMosaicML committed Jun 26, 2024
    Configuration menu
    Copy the full SHA
    3e3c974 View commit details
    Browse the repository at this point in the history

  18. fixing test

    @ShashankMosaicML
    ShashankMosaicML committed Jun 26, 2024
    Configuration menu
    Copy the full SHA
    7f9ef5c View commit details
    Browse the repository at this point in the history

  19. lint

    @ShashankMosaicML
    ShashankMosaicML committed Jun 26, 2024
    Configuration menu
    Copy the full SHA
    18b77af View commit details
    Browse the repository at this point in the history

  20. lint

    @ShashankMosaicML
    ShashankMosaicML committed Jun 26, 2024
    Configuration menu
    Copy the full SHA
    f417e95 View commit details
    Browse the repository at this point in the history

  21. adding test

    @ShashankMosaicML
    ShashankMosaicML committed Jun 26, 2024
    Configuration menu
    Copy the full SHA
    72eb7e1 View commit details
    Browse the repository at this point in the history

  22. Merge branch 'main' into mixed_attention_modules

    @ShashankMosaicML
    ShashankMosaicML authored Jun 26, 2024
    Configuration menu
    Copy the full SHA
    89c3985 View commit details
    Browse the repository at this point in the history

  23. addressing comment

    @ShashankMosaicML
    ShashankMosaicML committed Jun 26, 2024
    Configuration menu
    Copy the full SHA
    391978c View commit details
    Browse the repository at this point in the history

Commits on Jun 27, 2024

  1. ..

    @ShashankMosaicML
    ShashankMosaicML committed Jun 27, 2024
    Configuration menu
    Copy the full SHA
    0659e32 View commit details
    Browse the repository at this point in the history

  2. fixing test

    @ShashankMosaicML
    ShashankMosaicML committed Jun 27, 2024
    Configuration menu
    Copy the full SHA
    180c004 View commit details
    Browse the repository at this point in the history

  3. Merge branch 'main' into mixed_attention_modules

    @ShashankMosaicML
    ShashankMosaicML authored Jun 27, 2024
    Configuration menu
    Copy the full SHA
    2f073f7 View commit details
    Browse the repository at this point in the history

  4. Merge branch 'main' into mixed_attention_modules

    @ShashankMosaicML
    ShashankMosaicML authored Jun 27, 2024
    Configuration menu
    Copy the full SHA
    4f9893a View commit details
    Browse the repository at this point in the history

  5. changing yaml format

    @ShashankMosaicML
    ShashankMosaicML committed Jun 27, 2024
    Configuration menu
    Copy the full SHA
    023070f View commit details
    Browse the repository at this point in the history

  6. Merge branch 'main' into mixed_attention_modules

    @ShashankMosaicML
    ShashankMosaicML authored Jun 27, 2024
    Configuration menu
    Copy the full SHA
    ca9a2e9 View commit details
    Browse the repository at this point in the history

  7. fix configuation

    @ShashankMosaicML
    ShashankMosaicML committed Jun 27, 2024
    Configuration menu
    Copy the full SHA
    dc9890d View commit details
    Browse the repository at this point in the history

  8. fixing test

    @ShashankMosaicML
    ShashankMosaicML committed Jun 27, 2024
    Configuration menu
    Copy the full SHA
    4be19c6 View commit details
    Browse the repository at this point in the history

  9. Merge branch 'main' into mixed_attention_modules

    @ShashankMosaicML
    ShashankMosaicML authored Jun 27, 2024
    Configuration menu
    Copy the full SHA
    d4a417a View commit details
    Browse the repository at this point in the history

  10. allowing repeat at top level

    @ShashankMosaicML
    ShashankMosaicML committed Jun 27, 2024
    Configuration menu
    Copy the full SHA
    a5298c3 View commit details
    Browse the repository at this point in the history

  11. Merge branch 'mixed_attention_modules' of github.com:ShashankMosaicML… 

    …/llm-foundry into mixed_attention_modules
    @ShashankMosaicML
    ShashankMosaicML committed Jun 27, 2024
    Configuration menu
    Copy the full SHA
    ecd560c View commit details
    Browse the repository at this point in the history

Commits on Jun 28, 2024

  1. allowing overriding error

    @ShashankMosaicML
    ShashankMosaicML committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    700ede1 View commit details
    Browse the repository at this point in the history

Commits on Jun 29, 2024

  1. Merge branch 'main' into mixed_attention_modules

    @ShashankMosaicML
    ShashankMosaicML authored Jun 29, 2024
    Configuration menu
    Copy the full SHA
    ff055a4 View commit details
    Browse the repository at this point in the history

  2. addressing comments

    @ShashankMosaicML
    ShashankMosaicML committed Jun 29, 2024
    Configuration menu
    Copy the full SHA
    ca047fa View commit details
    Browse the repository at this point in the history

  3. lint

    @ShashankMosaicML
    ShashankMosaicML committed Jun 29, 2024
    Configuration menu
    Copy the full SHA
    3ca3d55 View commit details
    Browse the repository at this point in the history

  4. addressing comments

    @ShashankMosaicML
    ShashankMosaicML committed Jun 29, 2024
    Configuration menu
    Copy the full SHA
    3fbec2e View commit details
    Browse the repository at this point in the history

  5. fix

    @ShashankMosaicML
    ShashankMosaicML committed Jun 29, 2024
    Configuration menu
    Copy the full SHA
    e7d85bb View commit details
    Browse the repository at this point in the history

  6. ..

    @ShashankMosaicML
    ShashankMosaicML committed Jun 29, 2024
    Configuration menu
    Copy the full SHA
    8d97354 View commit details
    Browse the repository at this point in the history

  7. ..

    @ShashankMosaicML
    ShashankMosaicML committed Jun 29, 2024
    Configuration menu
    Copy the full SHA
    54abbd2 View commit details
    Browse the repository at this point in the history

  8. Merge branch 'main' into mixed_attention_modules

    @ShashankMosaicML
    ShashankMosaicML authored Jun 29, 2024
    Configuration menu
    Copy the full SHA
    c6ac78a View commit details
    Browse the repository at this point in the history

  9. ..

    @ShashankMosaicML
    ShashankMosaicML committed Jun 29, 2024
    Configuration menu
    Copy the full SHA
    52536cd View commit details
    Browse the repository at this point in the history

  10. ..

    @ShashankMosaicML
    ShashankMosaicML committed Jun 29, 2024
    Configuration menu
    Copy the full SHA
    6d62c2c View commit details
    Browse the repository at this point in the history

  11. ..

    @ShashankMosaicML
    ShashankMosaicML committed Jun 29, 2024
    Configuration menu
    Copy the full SHA
    2b53237 View commit details
    Browse the repository at this point in the history

Commits on Jun 30, 2024

  1. addressing comment

    @ShashankMosaicML
    ShashankMosaicML committed Jun 30, 2024
    Configuration menu
    Copy the full SHA
    d53a1e6 View commit details
    Browse the repository at this point in the history

  2. fixing test

    @ShashankMosaicML
    ShashankMosaicML committed Jun 30, 2024
    Configuration menu
    Copy the full SHA
    6554a22 View commit details
    Browse the repository at this point in the history