Cannot change pre-trained-model download directory

[xingzhehe]

Describe the bug

I tried the very first example Text-to-Image generation with Stable Diffusion.

I have set every single variable I can find on Internet to the current path, but it keeps downloading model to ~/.cache. I have very limited home space, so it keeps telling me no space on the disk.

Reproduction

import torch
from diffusers import StableDiffusionPipeline
import os

os.environ['TORCH_HOME'] = os.path.abspath(os.getcwd())
os.environ['HF_HOME'] = os.path.abspath(os.getcwd())
os.environ['XDG_CACHE_HOME'] = os.path.abspath(os.getcwd())
os.environ['HUGGINGFACE_HUB_CACHE'] = os.path.abspath(os.getcwd())
os.environ['TRANSFORMERS_CACHE'] = os.path.abspath(os.getcwd())

pipe = StableDiffusionPipeline.from_pretrained("runwayml/stable-diffusion-v1-5", torch_dtype=torch.float16)
pipe = pipe.to("cuda")

prompt = "a photo of an astronaut riding a horse on mars"
image = pipe(prompt).images[0]

Logs

No response

System Info

I am using ubuntu 20.04, and the latest diffuser.

[patrickvonplaten]

Hey @xingzhehe,

Can you just try to use the cache_dir=... parameter to define your cache? See:

diffusers/src/diffusers/pipelines/pipeline_utils.py

Line 486 in 24d624a

cache_dir (`Union[str, os.PathLike]`, *optional*):

(sorry for some reason our docs don't display it atm - will check)

[stevhliu]

You can see the cache_dir parameter from the main version of the docs :)

[xingzhehe]

@patrickvonplaten @stevhliu

Thanks! but after I set cache_dir=".", it seems not to be able to find some config file:

Traceback (most recent call last):
File "XXXXXX/test_diffuser.py", line 13, in
pipe = StableDiffusionPipeline.from_pretrained("runwayml/stable-diffusion-v1-5", torch_dtype=torch.float16, cache_dir='.')
File "XXXXXX/anaconda3/envs/xxxx/lib/python3.9/site-packages/diffusers/pipelines/pipeline_utils.py", line 589, in from_pretrained
config_dict = cls.load_config(
File "XXXXXX/anaconda3/envs/xxxx/lib/python3.9/site-packages/diffusers/configuration_utils.py", line 380, in load_config
config_dict = cls._dict_from_json_file(config_file)
File "XXXXXX/anaconda3/envs/xxxx/lib/python3.9/site-packages/diffusers/configuration_utils.py", line 479, in _dict_from_json_file
with open(json_file, "r", encoding="utf-8") as reader:
FileNotFoundError: [Errno 2] No such file or directory: './models--runwayml--stable-diffusion-v1-5/snapshots/39593d5650112b4cc580433f6b0435385882d819/model_index.json'

[patrickvonplaten]

Hmm that's weird what happens if you set cache_dir="./temp" as a test?

[xingzhehe]

Hmm that's weird what happens if you set cache_dir="./temp" as a test?

It gives the same error.

[wrk226]

I encountered a similar situation and solved my problem by using symbolic link.
For example, assume you want to move the cache files to /home/temp folder，you can do:

mv ~/.cache/huggingface /home/temp/
ln -s /home/temp/huggingface ~/.cache/huggingface

Hope it helps.

[xingzhehe]

I encountered a similar situation and solved my problem by using symbolic link. For example, assume you want to move the cache files to /home/temp folder，you can do:

mv ~/.cache/huggingface /home/temp/
ln -s /home/temp/huggingface ~/.cache/huggingface

Hope it helps.

Yes it helps! Thank you so much! But I really think this is a bug and should be fixed.

[patrickvonplaten]

Hmm I really cannot reproduce this - @Wauplin do you have an idea maybe what could be going on here?

Also @xingzhehe, could you please post what error message you get when doing:

import torch
from diffusers import StableDiffusionPipeline

pipe = StableDiffusionPipeline.from_pretrained("runwayml/stable-diffusion-v1-5", torch_dtype=torch.float16, cache_dir="./cache_dir")

[Wauplin]

@xingzhehe sorry about that. It is related to a bug introduced by latest huggingface_hub release. I'm working on fixing it and will keep you updated once published. In the meantime you can either downgrade huggingface_hub to 0.12.1 or set an absolute path for cache_dir (e.g. /path/to/your/folder)

Related issue: huggingface/huggingface_hub#1388

[Wauplin]

Hi @xingzhehe, I made a patch release to fix this (see release notes).
So if you upgrade huggingface_hub to version 0.13.2, it should be your issue. You must delete the corrupted cache folder before. Please let me know if you have any related issue.

[xingzhehe]

Hi @xingzhehe, I made a patch release to fix this (see release notes). So if you upgrade huggingface_hub to version 0.13.2, it should be your issue. You must delete the corrupted cache folder before. Please let me know if you have any related issue.

Thanks! The issue has been fixed!

[SuperBruceJia]

Please simply use

# Export the TRANSFORMERS_CACHE, HF_HOME, and HF_DATASETS_CACHE paths
export TRANSFORMERS_CACHE=/projectnb/pnn/.cache
export HF_HOME=/projectnb/pnn/.cache
export HF_DATASETS_CACHE=/projectnb/pnn/.cache

or

import os

os.environ['TRANSFORMERS_CACHE'] = "/projectnb/pnn/.cache"
os.environ['HF_HOME'] = "/projectnb/pnn/.cache"
os.environ['HF_DATASETS_CACHE'] = "/projectnb/pnn/.cache"