Token in env variables

[FrancescoSaverioZuppichini]

Why

Sometimes (tests, docker, ... etc) it's easier to pass the token required to use the hub in a env variable, something like:

export HUGGING_FACE_HUB_TOKEN=<YOUR_TOKEN>

This avoids the need of getting the token, creating a folder, and storing it inside

How

I've edited the logic inside hf_api.HFolder.get_token. Pasted below for convenience

    @classmethod
    def get_token(cls) -> Optional[str]:
        """
        Get token or None if not existent. A token can be also provided using env variables: `export HUGGING_FACE_HUB_TOKEN=<YOUR_TOKEN>`.
        """
        token: Optional[str] = os.environ.get("HUGGING_FACE_HUB_TOKEN")
        try:
            with open(cls.path_token, "r") as f:
                token = f.read()
        except FileNotFoundError:
            pass

        return token

Test it

I've added a few new line in the tests but even if I've mocked the os.environ dict the token is still None, I can use a little help 🙏

Thanks

Francesco

[LysandreJik]

Hey @FrancescoSaverioZuppichini, sorry for getting back so late on this - the issue with this approach is that it won't configure git properly as it won't put the token in the git-credential store, so it's only a partial fix.

Could you let me know of an issue you encountered with docker, so that we may take a look together and see the best resolution?

[FrancescoSaverioZuppichini]

Hi @LysandreJik , thank you for your reply. Apologies for the noob question, how is the token placed in the git-credential store?

To use the hub in docker, you have to create the folder and paste the token inside it. Something like:

mkdir <HF_TOKEN_FOLDER>
echo $HF_TOKEN > <HF_TOKEN_FOLDER>/token

This has to be done at build time you also (probably) need to define the token variable with the ARG keyword.

In my opinion, it would be cool to let the code directly set everything up if it sees the token in the correct env variable. What do you think?

[LysandreJik]

Sorry for taking so long to get back to this. I agree with you regarding the need, thanks for offering a PR! Only left a comment, but once that is taken care of, you can feel free to merge. I'm looking at how to enable that from git's perspective and will get back to you.

Thanks, @FrancescoSaverioZuppichini!

[LysandreJik]

Cool :)

[LysandreJik]

Actually, before merging, let me take a quick look at how we can handle that from git's side.

[LysandreJik]

Your proposition of toggling it in the credential store could be done by calling the following method: write_to_credential_store.

I think the environment variable will not change during the runtime, so setting it at the beginning would be the best. Would you like to try your hand at it?

Edit: one issue with this approach is that this will impact the global state of git. Even if the env variable is set once, every subsequent git operation that requires auth on the hf hub will use the info stored into the git credentials. I'm not sure if this is a big deal or not.

[adrinjalali]

I'm not sure if this should have much to do with the git credential storage @LysandreJik . I think it makes sense for people to be able to set the token as an environment variable taken from their secrets, and the code itself wouldn't have anything to do with the token and it would just read the env variable instead.

Am I missing something?

[LysandreJik]

Thinking more about it, you're correct @adrinjalali. Having it as an environment variable without any impact on the git-credentials logging system sounds good to me.

Apart from the try/except statement, should be good to go. Thanks!

[FrancescoSaverioZuppichini]

Tried to rebase it, something didn't work. Gonna open a new PR