Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add batched_generate_fn() #1702

Merged
merged 26 commits into from
Sep 5, 2024
Merged

Add batched_generate_fn() #1702

merged 26 commits into from
Sep 5, 2024

Commits on Aug 22, 2024

  1. Added batched next token.

    @apaz-cli
    apaz-cli committed Aug 22, 2024
    Configuration menu
    Copy the full SHA
    f6d9f13 View commit details
    Browse the repository at this point in the history

  2. Slight cleanup

    @apaz-cli
    apaz-cli committed Aug 22, 2024
    Configuration menu
    Copy the full SHA
    3092c1c View commit details
    Browse the repository at this point in the history

Commits on Aug 23, 2024

  1. Cleaned up kwargs and dtypes.

    @apaz-cli
    apaz-cli committed Aug 23, 2024
    Configuration menu
    Copy the full SHA
    cb06ade View commit details
    Browse the repository at this point in the history

  2. Use union over | for python 3.9.

    @apaz-cli
    apaz-cli committed Aug 23, 2024
    Configuration menu
    Copy the full SHA
    b269212 View commit details
    Browse the repository at this point in the history

  3. Pad to the left, rather than the right 

    Co-authored-by: Sebastian Raschka <mail@sebastianraschka.com>
    @apaz-cli @rasbt
    apaz-cli and rasbt authored Aug 23, 2024
    Configuration menu
    Copy the full SHA
    57e57ce View commit details
    Browse the repository at this point in the history

Commits on Aug 27, 2024

  1. Split off into separate branch.

    @apaz-cli
    apaz-cli committed Aug 27, 2024
    Configuration menu
    Copy the full SHA
    329526f View commit details
    Browse the repository at this point in the history

  2. Merge branch 'main' of https://github.com/lightning-ai/litgpt into ap… 

    …/batched_generate
    @apaz-cli
    apaz-cli committed Aug 27, 2024
    Configuration menu
    Copy the full SHA
    e27a826 View commit details
    Browse the repository at this point in the history

Commits on Aug 30, 2024

  1. Update impl

    @apaz-cli
    apaz-cli committed Aug 30, 2024
    Configuration menu
    Copy the full SHA
    77a0207 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'main' of https://github.com/lightning-ai/litgpt into ap… 

    …/batched_generate
    @apaz-cli
    apaz-cli committed Aug 30, 2024
    Configuration menu
    Copy the full SHA
    7cceb3a View commit details
    Browse the repository at this point in the history

  3. Time to test.

    @apaz-cli
    apaz-cli committed Aug 30, 2024
    Configuration menu
    Copy the full SHA
    8983e07 View commit details
    Browse the repository at this point in the history

  4. Finish impl. Still cleaning up.

    @apaz-cli
    apaz-cli committed Aug 30, 2024
    Configuration menu
    Copy the full SHA
    f29e11a View commit details
    Browse the repository at this point in the history

  5. Add comment.

    @apaz-cli
    apaz-cli committed Aug 30, 2024
    Configuration menu
    Copy the full SHA
    12266fa View commit details
    Browse the repository at this point in the history

  6. More comments

    @apaz-cli
    apaz-cli committed Aug 30, 2024
    Configuration menu
    Copy the full SHA
    9b967e0 View commit details
    Browse the repository at this point in the history

Commits on Sep 3, 2024

  1. Yielding remaining tokens is not unreachable.

    @apaz-cli
    apaz-cli committed Sep 3, 2024
    Configuration menu
    Copy the full SHA
    631649c View commit details
    Browse the repository at this point in the history

  2. max_token_lists

    @apaz-cli
    apaz-cli committed Sep 3, 2024
    Configuration menu
    Copy the full SHA
    0d6445c View commit details
    Browse the repository at this point in the history

  3. Add test.

    @apaz-cli
    apaz-cli committed Sep 3, 2024
    Configuration menu
    Copy the full SHA
    277b15a View commit details
    Browse the repository at this point in the history

  4. Added broken consistency test.

    @apaz-cli
    apaz-cli committed Sep 3, 2024
    Configuration menu
    Copy the full SHA
    39655b0 View commit details
    Browse the repository at this point in the history

  5. Bumped numpy version.

    @apaz-cli
    apaz-cli committed Sep 3, 2024
    Configuration menu
    Copy the full SHA
    df33526 View commit details
    Browse the repository at this point in the history

  6. Use deterministic algorithms.

    @apaz-cli
    apaz-cli committed Sep 3, 2024
    Configuration menu
    Copy the full SHA
    bf6675b View commit details
    Browse the repository at this point in the history

  7. Add debug prints.

    @apaz-cli
    apaz-cli committed Sep 3, 2024
    Configuration menu
    Copy the full SHA
    e7c901a View commit details
    Browse the repository at this point in the history

  8. Rewrite to grab our own sample cuts.

    @apaz-cli
    apaz-cli committed Sep 3, 2024
    Configuration menu
    Copy the full SHA
    8c9cb00 View commit details
    Browse the repository at this point in the history

  9. Final cleanup.

    @apaz-cli
    apaz-cli committed Sep 3, 2024
    Configuration menu
    Copy the full SHA
    e543020 View commit details
    Browse the repository at this point in the history

  10. Remove unused import.

    @apaz-cli
    apaz-cli committed Sep 3, 2024
    Configuration menu
    Copy the full SHA
    61dc3a2 View commit details
    Browse the repository at this point in the history

  11. Remove commented print statements.

    @apaz-cli
    apaz-cli committed Sep 3, 2024
    Configuration menu
    Copy the full SHA
    769266a View commit details
    Browse the repository at this point in the history

  12. Update consistency test, fix. Still broken though.

    @apaz-cli
    apaz-cli committed Sep 3, 2024
    Configuration menu
    Copy the full SHA
    23156c9 View commit details
    Browse the repository at this point in the history

Commits on Sep 5, 2024

  1. Added docstrings to generate_fn and batched_generate_fn().

    @apaz-cli
    apaz-cli committed Sep 5, 2024
    Configuration menu
    Copy the full SHA
    69ce5cc View commit details
    Browse the repository at this point in the history