504 Server Error: Gateway Time-out #2581

RichardErkhov · 2024-10-01T05:04:39Z

Describe the bug

getting 504 timeout, cant fix it, and I dont think it suppose to happen. I think if you raise the timeout limits (or allow user to do that), then it will be good, because the times gradually increase between first thousand and second thousand. How do I fix it? I guess the issue is getting gradually worse, since in August #2460 it worked, and I must say since around 25 of september this issue exists to that extent. Before that I could scan it full without the limit with little to no issues (504 still existed, just not for 15 hours straight and still ongoing)

Reproduction

from huggingface_hub import list_models

count = 0
for model in list_models(task="text-generation", tags=["safetensors",], limit=5000,):
count += 1
print(count, model.id, model.downloads)

Logs

...
1994 unionai/pythia-1b-deduped-finetune-alpaca-cleaned 12
1995 TheBloke/stable-vicuna-13B-GPTQ 68
1996 emozilla/pythia-1.4b-deduped-rp-420m-4k 7
1997 emozilla/pythia-1.4b-deduped-rp-280m-4k 7
1998 TheBloke/OpenAssistant-SFT-7-Llama-30B-GPTQ 603
1999 Neko-Institute-of-Science/pygmalion-7b 810
2000 crumb/gpt2023 753
Traceback (most recent call last):
  File "/root/miniconda3/envs/quant/lib/python3.10/site-packages/huggingface_hub/utils/_http.py", line 406, in hf_raise_for_status
    response.raise_for_status()
  File "/root/miniconda3/envs/quant/lib/python3.10/site-packages/requests/models.py", line 1024, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 504 Server Error: Gateway Time-out for url: https://huggingface.co/api/models?filter=text-generation&filter=safetensors&limit=5000&sort=trendingScore&cursor=eyIkb3IiOlt7InRyZW5kaW5nU2NvcmUiOjAsIl9pZCI6eyIkZ3QiOiI2NDRkZDEwOGZhOTRlOTNiMGVkMmRlMGQifX0seyJ0cmVuZGluZ1Njb3JlIjp7IiRsdCI6MH19LHsidHJlbmRpbmdTY29yZSI6bnVsbH1dfQ%3D%3D

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/root/miniconda3/envs/quant/lib/python3.10/site-packages/huggingface_hub/hf_api.py", line 1852, in list_models
    for item in items:
  File "/root/miniconda3/envs/quant/lib/python3.10/site-packages/huggingface_hub/utils/_pagination.py", line 46, in paginate
    hf_raise_for_status(r)
  File "/root/miniconda3/envs/quant/lib/python3.10/site-packages/huggingface_hub/utils/_http.py", line 477, in hf_raise_for_status
    raise _format(HfHubHTTPError, str(e), response) from e
huggingface_hub.errors.HfHubHTTPError: 504 Server Error: Gateway Time-out for url: https://huggingface.co/api/models?filter=text-generation&filter=safetensors&limit=5000&sort=trendingScore&cursor=eyIkb3IiOlt7InRyZW5kaW5nU2NvcmUiOjAsIl9pZCI6eyIkZ3QiOiI2NDRkZDEwOGZhOTRlOTNiMGVkMmRlMGQifX0seyJ0cmVuZGluZ1Njb3JlIjp7IiRsdCI6MH19LHsidHJlbmRpbmdTY29yZSI6bnVsbH1dfQ%3D%3D

<!DOCTYPE html>
<html class="" lang="en">
<head>
    <meta charset="utf-8" />
    <meta
            name="viewport"
            content="width=device-width, initial-scale=1.0, user-scalable=no"
    />
    <meta
            name="description"
            content="We're on a journey to advance and democratize artificial intelligence through open source and open science."
    />
    <meta property="fb:app_id" content="1321688464574422" />
    <meta name="twitter:card" content="summary_large_image" />
    <meta name="twitter:site" content="@huggingface" />
    <meta
            property="og:title"
            content="Hugging Face - The AI community building the future."
    />
    <meta property="og:type" content="website" />

    <title>Hugging Face - The AI community building the future.</title>
    <style>
        body {
            margin: 0;
        }

        main {
            background-color: white;
            min-height: 100vh;
            padding: 7rem 1rem 8rem 1rem;
            text-align: center;
            font-family: Source Sans Pro, ui-sans-serif, system-ui, -apple-system,
            BlinkMacSystemFont, Segoe UI, Roboto, Helvetica Neue, Arial, Noto Sans,
            sans-serif, Apple Color Emoji, Segoe UI Emoji, Segoe UI Symbol,
            Noto Color Emoji;
        }

        img {
            width: 6rem;
            height: 6rem;
            margin: 0 auto 1rem;
        }

        h1 {
            font-size: 3.75rem;
            line-height: 1;
            color: rgba(31, 41, 55, 1);
            font-weight: 700;
            box-sizing: border-box;
            margin: 0 auto;
        }

        p, a {
            color: rgba(107, 114, 128, 1);
            font-size: 1.125rem;
            line-height: 1.75rem;
            max-width: 28rem;
            box-sizing: border-box;
            margin: 0 auto;
        }

        .dark main {
            background-color: rgb(11, 15, 25);
        }
        .dark h1 {
            color: rgb(209, 213, 219);
        }
        .dark p, .dark a {
            color: rgb(156, 163, 175);
        }
    </style>
    <script>
        // On page load or when changing themes, best to add inline in `head` to avoid FOUC
        const key = "_tb_global_settings";
        let theme = window.matchMedia("(prefers-color-scheme: dark)").matches
            ? "dark"
            : "light";
        try {
            const storageTheme = JSON.parse(window.localStorage.getItem(key)).theme;
            if (storageTheme) {
                theme = storageTheme === "dark" ? "dark" : "light";
            }
        } catch (e) {}
        if (theme === "dark") {
            document.documentElement.classList.add("dark");
        } else {
            document.documentElement.classList.remove("dark");
        }
    </script>
</head>

<body>
<main>
    <img
            src="https://cdn-media.huggingface.co/assets/huggingface_logo.svg"
            alt=""
    />
    <div>
        <h1>504</h1>
        <p>Gateway Timeout</p>
    </div>
</main>
</body>
</html>

System info

- huggingface_hub version: 0.25.1
- Platform: Linux-5.15.0-122-generic-x86_64-with-glibc2.35
- Python version: 3.10.14
- Running in iPython ?: No
- Running in notebook ?: No
- Running in Google Colab ?: No
- Running in Google Colab Enterprise ?: No
- Token path ?: /root/.cache/huggingface/token
- Has saved token ?: True
- Who am I ?: RichardErkhov
- Configured git credential helpers: 
- FastAI: N/A
- Tensorflow: N/A
- Torch: 2.3.1
- Jinja2: 3.1.4
- Graphviz: N/A
- keras: N/A
- Pydot: N/A
- Pillow: 10.3.0
- hf_transfer: 0.1.6
- gradio: N/A
- tensorboard: N/A
- numpy: 1.26.4
- pydantic: N/A
- aiohttp: N/A
- ENDPOINT: https://huggingface.co
- HF_HUB_CACHE: /root/.cache/huggingface/hub
- HF_ASSETS_CACHE: /root/.cache/huggingface/assets
- HF_TOKEN_PATH: /root/.cache/huggingface/token
- HF_HUB_OFFLINE: False
- HF_HUB_DISABLE_TELEMETRY: False
- HF_HUB_DISABLE_PROGRESS_BARS: None
- HF_HUB_DISABLE_SYMLINKS_WARNING: False
- HF_HUB_DISABLE_EXPERIMENTAL_WARNING: False
- HF_HUB_DISABLE_IMPLICIT_TOKEN: False
- HF_HUB_ENABLE_HF_TRANSFER: False
- HF_HUB_ETAG_TIMEOUT: 10
- HF_HUB_DOWNLOAD_TIMEOUT: 10

hanouticelina · 2024-10-01T08:57:59Z

Hi @RichardErkhov, really sorry that you encountered this issue, we're currently experiencing problems with these types of requests. Our backend team is aware of the situation, but unfortunately, the fix isn't straightforward. We'll make sure to update you as soon as we have more info on this!
thanks for your understanding 🤗

RichardErkhov · 2024-10-01T08:59:28Z

Ok, thank you, waiting for solution. Any help I can provide ?

hanouticelina · 2024-10-01T09:36:13Z

thanks a lot for offering help! 🤗 you actually helped by reporting the issue :) we are able to reproduce the error, so I think we are good on this side. We will keep you updated.

Wauplin · 2025-01-31T09:09:27Z

Hi @RichardErkhov , we've done some optimizations server-side for the search API. You shouldn't be noticing 504 error anymore. Let us know if it's not the case 🤗 (x-linking private issue).

Btw, I've tested

from huggingface_hub import list_models


count = 0
for model in list_models(task="text-generation", tags=["safetensors"]):
    count += 1
    print(count, model.id, model.downloads)

which now works perfectly!

RichardErkhov added the bug Something isn't working label Oct 1, 2024

hanouticelina mentioned this issue Oct 1, 2024

Error formatting fails to capture request ID and includes raw HTML #2583

Closed

Wauplin closed this as completed Jan 31, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

504 Server Error: Gateway Time-out #2581

504 Server Error: Gateway Time-out #2581

RichardErkhov commented Oct 1, 2024

hanouticelina commented Oct 1, 2024

RichardErkhov commented Oct 1, 2024

hanouticelina commented Oct 1, 2024

Wauplin commented Jan 31, 2025

504 Server Error: Gateway Time-out #2581

504 Server Error: Gateway Time-out #2581

Comments

RichardErkhov commented Oct 1, 2024

Describe the bug

Reproduction

Logs

System info

hanouticelina commented Oct 1, 2024

RichardErkhov commented Oct 1, 2024

hanouticelina commented Oct 1, 2024

Wauplin commented Jan 31, 2025