Urgent: 429 Errors and Potential Rate Limiting on Model Access

[dnuanghsh]

System Info

Hi team,

From yesterday, we have been seeing 429 errors when trying to access different models.

Here's an example of the error we're seeing:

File "/usr/local/lib/python3.8/site-packages/requests/models.py", line 1024, in raise_for_status  
    raise HTTPError(http_error_msg, response=self)  
requests.exceptions.HTTPError: 429 Client Error: Too Many Requests for URL: https://huggingface.co/meta-llama/Meta-Llama-3-8B/resolve/main/config.json  

We suspect we are being rate-limited. We can share the IPs with you.

Could we look into resolving this issue?

Thank you so much!

cc @amyeroberts @philschmid @sgugger

Who can help?

@amyeroberts @philschmid @sgugger

Information

• The official example scripts
• My own modified scripts

Tasks

• An officially supported task in the examples folder (such as GLUE/SQuAD, ...)
• My own task or dataset (give details below)

Reproduction

We ran into the issues in our internal runs.

Expected behavior

Run without hitting the 429 errors.

[Rocketknight1]

Hi @dnuanghsh, can you tell us how many requests were being made simultaneously?

[dnuanghsh]

Hi @Rocketknight1, there was a sudden surge in the number of requests due to an integration test (reverted) on 11/19. Our normal workload never encountered this issue. Is it possible for us to follow a similar pattern as mentioned in #21129 to resolve the issue? Thank you so much!

[dnuanghsh]

Update: It looks like the rate limiting has been lifted.

Could you please share more information on the pattern of rate limiting? What typically causes rate limiting? Additionally, do you have any best practices for working with huggingface to avoid encountering this issue in the future? Thank you!

[xyg-coder]

@Rocketknight1 Do you have any suggestion to avoid this issue in the future? Like any business account to relieve this restrict? Thanks

[meganriley]

Hi @dnuanghsh glad to hear it's working for you now. Can you please email api-enterprise@huggingface.co with your IP address so that we can investigate why this happened?

@xyg-coder Feel free to email us at api-enterprise@huggingface.co as well, as well as any information about when you received the rate limiting error message/what you were doing.

[dnuanghsh]

@meganriley We have emailed you. Thank you!