Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat: Add OpenAI Rate limiting #1805

Merged
merged 24 commits into from
Nov 29, 2023
Merged

feat: Add OpenAI Rate limiting #1805

merged 24 commits into from
Nov 29, 2023

Commits on Nov 27, 2023

  1. Implement adaptive rate limiter for OpenAI

    @anticorrelator
    anticorrelator committed Nov 27, 2023
    Configuration menu
    Copy the full SHA
    b7711b6 View commit details
    Browse the repository at this point in the history

  2. Add adaptive rate limiter to Bedrock model

    @anticorrelator
    anticorrelator committed Nov 27, 2023
    Configuration menu
    Copy the full SHA
    48006c8 View commit details
    Browse the repository at this point in the history

  3. Use a sensible default maximum request rate

    @anticorrelator
    anticorrelator committed Nov 27, 2023
    Configuration menu
    Copy the full SHA
    cbc355b View commit details
    Browse the repository at this point in the history

  4. Ruff 🐶

    @anticorrelator
    anticorrelator committed Nov 27, 2023
    Configuration menu
    Copy the full SHA
    c35f111 View commit details
    Browse the repository at this point in the history

  5. Mark test as xfail after llama_index update

    @anticorrelator
    anticorrelator committed Nov 27, 2023
    Configuration menu
    Copy the full SHA
    ff885af View commit details
    Browse the repository at this point in the history

  6. Do not retry on rate limit errors with tenacity

    @anticorrelator
    anticorrelator committed Nov 27, 2023
    Configuration menu
    Copy the full SHA
    bd3d311 View commit details
    Browse the repository at this point in the history

  7. Merge remote-tracking branch 'origin' into dustin/implement-ratelimiter

    @anticorrelator
    anticorrelator committed Nov 27, 2023
    Configuration menu
    Copy the full SHA
    cc72439 View commit details
    Browse the repository at this point in the history

  8. Remove xfail after llama_index version lock

    @anticorrelator
    anticorrelator committed Nov 27, 2023
    Configuration menu
    Copy the full SHA
    19e0dff View commit details
    Browse the repository at this point in the history

  9. Use events and locks instead of nesting asyncio.run

    @anticorrelator
    anticorrelator committed Nov 27, 2023
    Configuration menu
    Copy the full SHA
    af535c7 View commit details
    Browse the repository at this point in the history

  10. Ensure that events are always set after rate limit handling

    @anticorrelator
    anticorrelator committed Nov 27, 2023
    Configuration menu
    Copy the full SHA
    cffcc72 View commit details
    Browse the repository at this point in the history

Commits on Nov 28, 2023

  1. Retry on httpx ReadTimeout errors

    @anticorrelator
    anticorrelator committed Nov 28, 2023
    Configuration menu
    Copy the full SHA
    61ff1d3 View commit details
    Browse the repository at this point in the history

  2. Update rate limiters with verbose generation info

    @anticorrelator
    anticorrelator committed Nov 28, 2023
    Configuration menu
    Copy the full SHA
    fee9743 View commit details
    Browse the repository at this point in the history

  3. Improve end of queue handling in AsyncExecutor

    @anticorrelator
    anticorrelator committed Nov 28, 2023
    Configuration menu
    Copy the full SHA
    e06c7aa View commit details
    Browse the repository at this point in the history

  4. Merge remote-tracking branch 'origin' into dustin/implement-ratelimiter

    @anticorrelator
    anticorrelator committed Nov 28, 2023
    Configuration menu
    Copy the full SHA
    ecbfcd2 View commit details
    Browse the repository at this point in the history

  5. improve types to remove the need for casts (#1817)

    @axiomofjoy
    axiomofjoy authored Nov 28, 2023
    Configuration menu
    Copy the full SHA
    01e203a View commit details
    Browse the repository at this point in the history

  6. Improve interrupt handling

    @anticorrelator
    anticorrelator committed Nov 28, 2023
    Configuration menu
    Copy the full SHA
    1f4abe1 View commit details
    Browse the repository at this point in the history

  7. Exit early from queue.join on termination events

    @anticorrelator
    anticorrelator committed Nov 28, 2023
    Configuration menu
    Copy the full SHA
    a2798c8 View commit details
    Browse the repository at this point in the history

  8. Properly cancel running tasks

    @anticorrelator
    anticorrelator committed Nov 28, 2023
    Configuration menu
    Copy the full SHA
    7b4d66e View commit details
    Browse the repository at this point in the history

  9. Add pytest-asyncio to hatch env

    @anticorrelator
    anticorrelator committed Nov 28, 2023
    Configuration menu
    Copy the full SHA
    9f3cd2e View commit details
    Browse the repository at this point in the history

  10. Do not await cancelled tasks

    @anticorrelator
    anticorrelator committed Nov 28, 2023
    Configuration menu
    Copy the full SHA
    ddc562c View commit details
    Browse the repository at this point in the history

  11. Improve task_done marking logic

    @anticorrelator
    anticorrelator committed Nov 28, 2023
    Configuration menu
    Copy the full SHA
    f9f2753 View commit details
    Browse the repository at this point in the history

Commits on Nov 29, 2023

  1. Merge remote-tracking branch 'origin' into dustin/implement-ratelimiter

    @anticorrelator
    anticorrelator committed Nov 29, 2023
    Configuration menu
    Copy the full SHA
    ef0cbda View commit details
    Browse the repository at this point in the history

  2. Merge branch 'main' into dustin/implement-ratelimiter

    @anticorrelator
    anticorrelator authored Nov 29, 2023
    Configuration menu
    Copy the full SHA
    5657ec7 View commit details
    Browse the repository at this point in the history

  3. Increase default concurrency

    @anticorrelator
    anticorrelator committed Nov 29, 2023
    Configuration menu
    Copy the full SHA
    eef2e55 View commit details
    Browse the repository at this point in the history