userdata.get('HF_TOKEN') hangs forever on Google Colab

[prhbrt]

Describe the bug

from huggingface_hub import  snapshot_download
snapshot_download('SBB/eynollah-main-regions')

hangs indefinitely on Google Colab.

This is quick-fixed by

from huggingface_hub.utils import _runtime
_runtime._is_google_colab = False

Reproduction

Run the code in Google Colab.

Logs

No response

System info

(Only runs with the quick-fix:)

- huggingface_hub version: 0.20.1
- Platform: Linux-6.1.58+-x86_64-with-glibc2.35
- Python version: 3.10.12
- Running in iPython ?: Yes
- iPython shell: Shell
- Running in notebook ?: Yes
- Running in Google Colab ?: No
- Token path ?: /root/.cache/huggingface/token
- Has saved token ?: True
- Who am I ?: prhbrt
- Configured git credential helpers: 
- FastAI: 2.7.13
- Tensorflow: 2.15.0
- Torch: 2.1.0+cu121
- Jinja2: 3.1.2
- Graphviz: 0.20.1
- Pydot: 1.4.2
- Pillow: 9.4.0
- hf_transfer: N/A
- gradio: N/A
- tensorboard: N/A
- numpy: 1.23.5
- pydantic: 1.10.13
- aiohttp: 3.9.1
- ENDPOINT: https://huggingface.co
- HF_HUB_CACHE: /root/.cache/huggingface/hub
- HF_ASSETS_CACHE: /root/.cache/huggingface/assets
- HF_TOKEN_PATH: /root/.cache/huggingface/token
- HF_HUB_OFFLINE: False
- HF_HUB_DISABLE_TELEMETRY: False
- HF_HUB_DISABLE_PROGRESS_BARS: None
- HF_HUB_DISABLE_SYMLINKS_WARNING: False
- HF_HUB_DISABLE_EXPERIMENTAL_WARNING: False
- HF_HUB_DISABLE_IMPLICIT_TOKEN: False
- HF_HUB_ENABLE_HF_TRANSFER: False
- HF_HUB_ETAG_TIMEOUT: 10
- HF_HUB_DOWNLOAD_TIMEOUT: 10

[Wauplin]

Hi @prhbrt, thanks for reporting this issue. I'm sorry you are facing it. It might be due to the fact that userdata.get is not thread-safe (and snapshot_download uses multithreading). I cannot reproduce the issue you're describing using your Colab notebook so you might have ended up in a weird state. I have a few questions that might help investigating:

1. Do you have a secret called HF_TOKEN in your secrets vault in Google colab? (see on screenshot where to find it). If yes, is the secret enabled for this notebook? (e.g. "notebook access" ticked)
2. Do you have any secrets in the secrets vault?
3. Did you get a pop-up asking to share the secret at least once?
4. Could you stop the notebook you've shared, delete the runtime, restart and tell me if you end up in the same state?
5. Could you start a new notebook and simply run import huggingface_hub; huggingface_hub.whoami() ?
6. Finally, do you think you have any browser extension that could have blocked a pop-up? Which browser are you using?

Thanks in advance for your answers. I will try to investigate on my side and also contact the google colab team to check how we can fix/avoid this.

[prhbrt]

Do you have a secret called HF_TOKEN in your secrets vault in Google colab? (see on screenshot where to find it). If yes, is the secret enabled for this notebook? (e.g. "notebook access" ticked)

Yes

Do you have any secrets in the secrets vault?

None other than HF_TOKEN

Did you get a pop-up asking to share the secret at least once?

Yes

Could you stop the notebook you've shared, delete the runtime, restart and tell me if you end up in the same state?

I tried a few times, also deleting the runtime, and the issue persisted

Just checked different browsers, considering your other question, and 'now' (a few hours later) I can't reproduce the issue in any of them (Brave, Chrome, Firefox)

Could you start a new notebook and simply run import huggingface_hub; huggingface_hub.whoami() ?

Finally, do you think you have any browser extension that could have blocked a pop-up? Which browser are you using?

Brave.

Thanks in advance for your answers. I will try to investigate on my side and also contact the google colab team to check how we can fix/avoid this.

It's not a blocker for me, I'm only reporting in the hopes it helps others and improves the lib.

[Wauplin]

Thanks for taking the time @prhbrt! This is definitely helpful to know 👍
Glad to know it's not a blocker for you. I'll still try to work on a fix for everyone.

[iUnknownAdorn]

We are facing a similar issue, but without HF_TOKEN, but for downloading the model.

Here is a reproducible example

This tries to load a Whisper v3 model. This code was working till 29th Dec.

Clicking on the executor at bottom:

• This is what calls the threaded download: https://github.com/huggingface/huggingface_hub/blob/main/src/huggingface_hub/utils/_validators.py#L118
• This is what calls the downstream to download the model: https://github.com/huggingface/huggingface_hub/blob/main/src/huggingface_hub/_snapshot_download.py#L308

Unfortunately, this is a breaking change for our team. @Wauplin Do you think this is related?

[JhonDan1999]

I am facing the same issue here.
I think something is changing in the Google Colab environment because the same code was working perfectly fine last week for downloading the stable diffusion XL model. However, now it is stuck at downloading just 19 files, with 0% progress. I tried adding the Hugging Face API key tokens in the Colab settings, but it didn't work!

[Wauplin]

Thanks for reporting @iUnknownAdorn @JhonDan1999 and sorry for the inconvenience. I have been able to reproduce the issue and created PR #1953 to fix it. The plan is to make a hot-fix release once it's merged and make it the default in google colab. In the meantime, a temporary solution is to install a different version of huggingface_hub before executing any code. More details in #1953 (comment).

[Wauplin]

Update: huggingface_hub 0.20.2 is now the default version on Google Colab, meaning this issue should be fixed for everyone without having to upgrade (see googlecolab/colabtools#4272 (comment)).