Token Rate Limiting lab #26
vieiraae
announced in
Announcements
Replies: 2 comments
-
|
Awesome article from @mattfeltonma explaining in detail how to handle rate limiting using the APIM policy: https://journeyofthegeek.com/2024/06/19/azure-openai-service-how-to-handle-rate-limiting/ |
Beta Was this translation helpful? Give feedback.
0 replies
-
|
The following repo dynamically adjust rate-limits applied to different workloads: https://github.com/Azure-Samples/apim-genai-gateway-toolkit/tree/main/capabilities/rate-limiting |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
Token Rate Limiting lab
Playground to try the token rate limiting policy to either a list of Azure OpenAI endpoints or mock servers.
Get started
Proceed by opening the Jupyter notebook, and follow the steps provided.
Add questions or share feedback bellow
Beta Was this translation helpful? Give feedback.
All reactions