-
Notifications
You must be signed in to change notification settings - Fork 28.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Model sharing doc #8498
Model sharing doc #8498
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -52,21 +52,21 @@ users to clone it and you (and your organization members) to push to it. First, | |
Go in a terminal and run the following command. It should be in the virtual environment where you installed 🤗 | ||
Transformers, since that command :obj:`transformers-cli` comes from the library. | ||
|
||
.. code-block:: | ||
.. code-block:: bash | ||
|
||
transformers-cli login | ||
|
||
|
||
Once you are logged in with your model hub credentials, you can start building your repositories. To create a repo: | ||
|
||
.. code-block:: | ||
.. code-block:: bash | ||
|
||
transformers-cli repo create your-model-name | ||
|
||
This creates a repo on the model hub, which can be cloned. You can then add/remove from that repo as you would with any | ||
other git repo. | ||
|
||
.. code-block:: | ||
.. code-block:: bash | ||
|
||
git clone https://huggingface.co/username/your-model-name | ||
|
||
|
@@ -159,24 +159,25 @@ Or, if you're using the Trainer API | |
.. code-block:: | ||
|
||
>>> trainer.save_model("path/to/awesome-name-you-picked") | ||
>>> tokenizer.save_pretrained("path/to/repo/clone/your-model-name") | ||
|
||
You can then add these files to the staging environment and verify that they have been correctly staged with the ``git | ||
status`` command: | ||
|
||
.. code-block:: | ||
.. code-block:: bash | ||
|
||
git add --all | ||
git status | ||
|
||
Finally, the files should be comitted: | ||
|
||
.. code-block:: | ||
.. code-block:: bash | ||
|
||
git commit -m "First version of the your-model-name model and tokenizer." | ||
|
||
And pushed to the remote: | ||
|
||
.. code-block:: | ||
.. code-block:: bash | ||
|
||
git push | ||
|
||
|
@@ -199,7 +200,7 @@ don't forget to link to its model card so that people can fully trace how your m | |
If you have never made a pull request to the 🤗 Transformers repo, look at the :doc:`contributing guide <contributing>` | ||
to see the steps to follow. | ||
|
||
.. Note:: | ||
.. note:: | ||
|
||
You can also send your model card in the folder you uploaded with the CLI by placing it in a `README.md` file | ||
inside `path/to/awesome-name-you-picked/`. | ||
|
@@ -225,3 +226,41 @@ You may specify a revision by using the ``revision`` flag in the ``from_pretrain | |
>>> "julien-c/EsperBERTo-small", | ||
>>> revision="v2.0.1" # tag name, or branch name, or commit hash | ||
>>> ) | ||
|
||
Workflow in a Colab notebook | ||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | ||
|
||
If you're in a Colab notebook (or similar) with no direct access to a terminal, here is the workflow you can use to | ||
upload your model. You can execute each one of them in a cell by adding a ! at the beginning. | ||
|
||
First you need to install `git-lfs` in the environment used by the notebook: | ||
|
||
.. code-block:: bash | ||
|
||
sudo apt-get install git-lfs | ||
|
||
Then you can use the :obj:`transformers-cli` to create your new repo: | ||
|
||
|
||
.. code-block:: bash | ||
|
||
transformers-cli login | ||
transformers-cli repo create your-model-name | ||
|
||
Once it's created, you can clone it and configure it (replace username by your username on huggingface.co): | ||
|
||
.. code-block:: bash | ||
|
||
git clone https://huggingface.co/username/your-model-name | ||
cd your-model-name | ||
git lfs install | ||
git config --global user.email "email@example.com" | ||
|
||
Once you've saved your model inside, you can add it and push it with usual git commands. Note that you have to replace | ||
`username:password` with your username and password to huggingface.co. | ||
|
||
.. code-block:: bash | ||
|
||
git add . | ||
git commit -m "Initial commit" | ||
git push https://username:password@huggingface.co/username/your-model-name | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. you can also use There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. cc @Pierrci There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I can add this but how does one get a token on huggingface.co? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Hm, I think we should not encourage the users to speciy a password or token directly in a notebook cell. Instead we could use the following solution: from getpass import getpass
token = getpass('Enter the HF password: ') There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Yeah, that's probably true @stefan-it There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Also feel free to chime in to #8520 @stefan-it ! |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
btw you only need to do this once, after you install git-lfs
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Are you sure? I've felt like I needed to do this in every repo where I wanted git-lfs. I thought it was the equivalent of doing a
git init
, but for git-lfs.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm not sure, on my setup I never have to re-do it, but not sure