Implemented CLI download functionality

[martinbrose]

Addresses #1105.

• CLI interface for downloading files #1105

[HuggingFaceDocBuilderDev]

The documentation is not available anymore as the PR was closed or merged.

[Wauplin]

Hi @martinbrose, thanks a ton for working on this PR! It is a quite expected feature that will drastically simplify how some users are downloading weights from the Hub. The goal is to kinda replace git clone-like commands by this one 🚀 🤗

I started to try the CLI locally and got opinionated on how the command should look like which lead to me making changes directly on your PR. I hope you didn't find it offensive :) Here is how the command looks like now:

# Download file
huggingface-cli download gpt2 config.json

# Download entire repo
huggingface-cli download fffiloni/zeroscope --repo-type=space --revision=refs/pr/78

# Download repo with filters
huggingface-cli download gpt2 --include="*.safetensors"

# Download with token
huggingface-cli download Wauplin/private-model --token=hf_***

# Download quietly (no progress bar, no warnings, only the returned path)
huggingface-cli download gpt2 config.json --quiet

EDIT: I have also added some content in the download guide to make sure it's documented somewhere.

[martinbrose]

Thanks @Wauplin,

I've reviewed your changes with interest and they made sense to me.
Of course, feel free to change how you see fit.

Cheers

[Wauplin]

@LysandreJik @julien-c Would you mind having a look to this PR as it adds a quite important command IMO (also cc @osanseviero). Might be interesting to showcase this command in the "Clone this model repository" pop-up on the Hub. Before doing so, we would need the --to-local-dir option to be implemented but that can be done in a future PR.

cc @stevhliu I have added a small section in the docs if you can have a look at it 🙏

EDIT: implemented --to-local-dir in 874b70e

[severo]

Do you think we could have a shorter command (huggingface-cli is long). Maybe hf-cli, or hf?

[Wauplin]

Do you think we could have a shorter command (huggingface-cli is long). Maybe hf-cli, or hf?

I'm definitely in favor of a simple hf download ...!
I often thought about it but never made the change. We would still keep both hf and huggingface-cli for backward compatibility. We can do it in a future PR (also I don't know how much @julien-c or @LysandreJik are opinionated on the subject 😄 )

[julien-c]

re shorter command name, i wouldn't recommend because:

• users have auto-complete in their shells?
• let's not fragment usage as huggingface-cli already exists and is widely documented
• there's even system packages (brew etc) with that name

[julien-c]

API-wise, looks great to me!

and i liked the detailed pre-spec in #1105 (comment) cc @Wauplin

I think this will be quite a useful feature!

[stevhliu]

Super cool! 🤩

[LysandreJik]

Played with it, works well! I like the API. Thanks for working on it @Wauplin