Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add LLM Pipeline #137

Merged
merged 15 commits into from
Sep 30, 2024
Merged

Add LLM Pipeline #137

merged 15 commits into from
Sep 30, 2024

Commits on Sep 27, 2024

  1. runner: add llm-generate route and pipeline

    @kyriediculous
    kyriediculous committed Sep 27, 2024
    Configuration menu
    Copy the full SHA
    409ad51 View commit details
    Browse the repository at this point in the history

  2. add llama3.1 8B to downloads

    @kyriediculous
    kyriediculous committed Sep 27, 2024
    Configuration menu
    Copy the full SHA
    6e1ed29 View commit details
    Browse the repository at this point in the history

  3. worker: add llm-generate container management

    @kyriediculous
    kyriediculous committed Sep 27, 2024
    Configuration menu
    Copy the full SHA
    46f4515 View commit details
    Browse the repository at this point in the history

  4. llm: support streamed responses

    @kyriediculous
    kyriediculous committed Sep 27, 2024
    Configuration menu
    Copy the full SHA
    8e6dfc6 View commit details
    Browse the repository at this point in the history

  5. Load LLM model distributed over multiple GPUs

    @kyriediculous
    kyriediculous committed Sep 27, 2024
    Configuration menu
    Copy the full SHA
    f5f7d66 View commit details
    Browse the repository at this point in the history

  6. feat: support 8bit and fp16 for llm pipeline

    @kyriediculous
    kyriediculous committed Sep 27, 2024
    Configuration menu
    Copy the full SHA
    5df016b View commit details
    Browse the repository at this point in the history

  7. rename llm pipeline and route from 'llm-generate' to 'llm'

    @kyriediculous
    kyriediculous committed Sep 27, 2024
    Configuration menu
    Copy the full SHA
    d577b74 View commit details
    Browse the repository at this point in the history

  8. fix: move check_torch_cuda.py script to check versions to 'dev' folder

    @kyriediculous
    kyriediculous committed Sep 27, 2024
    Configuration menu
    Copy the full SHA
    653ff2b View commit details
    Browse the repository at this point in the history

  9. fix: return container on non-streaming response

    @kyriediculous
    kyriediculous committed Sep 27, 2024
    Configuration menu
    Copy the full SHA
    e488aa4 View commit details
    Browse the repository at this point in the history

  10. fix update naming convention 'Llm' -> 'LLM'

    @kyriediculous
    kyriediculous committed Sep 27, 2024
    Configuration menu
    Copy the full SHA
    53f326e View commit details
    Browse the repository at this point in the history

  11. fix: return container after stream completes

    @kyriediculous
    kyriediculous committed Sep 27, 2024
    Configuration menu
    Copy the full SHA
    22fbcf0 View commit details
    Browse the repository at this point in the history

  12. update naming conventions for SDK and go bindings

    @kyriediculous
    kyriediculous committed Sep 27, 2024
    Configuration menu
    Copy the full SHA
    e02ef21 View commit details
    Browse the repository at this point in the history

Commits on Sep 30, 2024

  1. rename LlmGenerateMultipartWriter -> LLMMultipartWriter

    @kyriediculous
    kyriediculous committed Sep 30, 2024
    Configuration menu
    Copy the full SHA
    39e082e View commit details
    Browse the repository at this point in the history

  2. openapi tags

    @kyriediculous
    kyriediculous committed Sep 30, 2024
    Configuration menu
    Copy the full SHA
    620f8b0 View commit details
    Browse the repository at this point in the history

  3. openapi bindings and spec with improved tags and description

    @kyriediculous
    kyriediculous committed Sep 30, 2024
    Configuration menu
    Copy the full SHA
    f7521c3 View commit details
    Browse the repository at this point in the history