Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

MNT deprecate name and organization in favor of repo_id #733

Merged
merged 18 commits into from
Mar 11, 2022
Merged

MNT deprecate name and organization in favor of repo_id #733

Show file tree
Hide file tree
Changes from 15 commits
Commits
Show all changes
18 commits
Select commit Hold shift + click to select a range
51f4505
MNT deprecate name and organization in favor of repo_id
adrinjalali Mar 2, 2022
8f2de8f
add tests
adrinjalali Mar 2, 2022
c51b89e
deprecate the other tho method args
adrinjalali Mar 4, 2022
c9a40cc
minor doc change
adrinjalali Mar 4, 2022
39adcfc
Merge remote-tracking branch 'upstream/main' into deprecate/1
adrinjalali Mar 4, 2022
a5bbcc8
reference the right lfsmultipartthresh
adrinjalali Mar 4, 2022
244af38
extra empty line
adrinjalali Mar 4, 2022
8c082f8
fix more tests
adrinjalali Mar 4, 2022
d3ccdbc
fix more tests
adrinjalali Mar 4, 2022
9c9df16
fix more tests
adrinjalali Mar 4, 2022
43232c7
fix more tests
adrinjalali Mar 4, 2022
b34630e
add reference for token
adrinjalali Mar 7, 2022
692e174
allow positional repo_id
adrinjalali Mar 9, 2022
e7b1d3a
fix org/name order
adrinjalali Mar 9, 2022
f55cae0
minor fix
adrinjalali Mar 9, 2022
4075dc7
Merge remote-tracking branch 'upstream/main' into deprecate/1
adrinjalali Mar 10, 2022
f8dc9de
fix bug, add test, improve readability
adrinjalali Mar 10, 2022
9dca781
Merge branch 'main' into deprecate/1
LysandreJik Mar 11, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/hub/adding-a-library.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -115,7 +115,7 @@ The `create_repo` method creates a repository on the Hub. Use the `name` paramet

```python
>>> from huggingface_hub import create_repo
>>> create_repo(name="test-model")
>>> create_repo(repo_id="test-model")
'https://huggingface.co/lysandre/test-model'
```

Expand Down
3 changes: 1 addition & 2 deletions docs/hub/adding-a-model.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -215,8 +215,7 @@ First we need to instantiate the `HfApi` class, which holds all of the magic:
Afterwards we can run the `create_repo` function, specifying a number of settings and options for our new repository:
```python
>>> api.create_repo(
>>> name = "dummy", # The name of our repository
>>> organization = None, # The namespace of the expected repository. Automatically grabs your logged-in profile name
>>> repo_id = "dummy", # The name of our repository, by default under your user
>>> private = False, # Whether the repo should be public or private
>>> repo_type = "model" # The type of repository, such as "model", "space", "dataset"
>>> )
Expand Down
4 changes: 2 additions & 2 deletions docs/hub/endpoints.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,8 +21,8 @@ We have open endpoints that you can use to retrieve information from the Hub as
| /api/spaces GET | Get information from all Spaces in the Hub. You can specify additional parameters to have more specific results. - `search`: Filter based on substrings for repos and their usernames, such as `resnet` or `microsoft` - `author`: Filter models by an author or organization, such as `huggingface` or `microsoft` - `filter`: Filter based on tags, such as `text-classification` or `spacy`. - `sort`: Property to use when sorting, such as `downloads` or `author`. - `direction`: Direction in which to sort, such as `-1` for descending, and anything else for ascending. - `limit`: Limit the number of models fetched. - `full`: Whether to fetch most model data, such as all tags, the files, etc. - `config`: Whether to also fetch the repo config. | `list_models()` | ```params= { "search":"search", "author":"author", "filter":"filter", "sort":"sort", "direction":"direction", "limit":"limit", "full":"full", "config":"config"}``` | |
| /api/spaces/{repo_id} /api/spaces/{repo_id}/revision/{revision} GET | Get all information for a specific model. | `model_info(repo_id, revision)` | ```headers = { "authorization" : "Bearer $token" }``` | |
| /api/metrics GET | Get information from all metrics in the Hub. | `list_metrics()` | | |
| /api/repos/create POST | Create a repository. It's a model repo by default. - type: Type of repo (datasets or spaces; model by default). - name: Name of repo. - organization: Name of organization. - - private: Whether the repo is private. | `create_repo()` | ```headers = { authorization : "Bearer $token" }``` ```json= {"type":"type", "name":"name", "organization":"organization", "private":"private"}``` | |
| /api/repos/delete DELETE | Delete a repository. It's a model repo by default. - type: Type of repo (datasets or spaces; model by default). - name: Name of repo. - organization: Name of organization. | `delete_repo()` | ```headers = { "authorization" : "Bearer $token" }``` ```json= {"type":"type", "name":"name", "organization":"organization"}``` | |
| /api/repos/create POST | Create a repository. It's a model repo by default. - type: Type of repo (datasets or spaces; model by default). - name: Name of repo. - organization: Name of organization. - - private: Whether the repo is private. | `create_repo()` | ```headers = { authorization : "Bearer $token" }``` ```json= {"type":"type", "repo_id":"repo_id", "private":"private"}``` | |
| /api/repos/delete DELETE | Delete a repository. It's a model repo by default. - type: Type of repo (datasets or spaces; model by default). - name: Name of repo. - organization: Name of organization. | `delete_repo()` | ```headers = { "authorization" : "Bearer $token" }``` ```json= {"type":"type", "repo_id":"repo_id"}``` | |
| /api/repos/{type}/{repo_id}/settings PUT | Update repo visibility. | `update_repo_visibility()` | ```headers = { "authorization" : "Bearer $token" }``` ```json= {"private":"private"}``` | |
| /api/repos/move POST | Move a repository (rename within same namespace or transfer from user to organization). | `move_repo()` | ```headers = { "authorization" : "Bearer $token" }``` ```json= {"fromRepo" : "namespace/repo_name", "toRepo" : "namespace2/repo_name2"}``` | |
| /api/{type}/{repo_id}/upload/{revision}/{path_in_repo} POST | Upload a file to a specific repository. | `upload_file()` | ```headers = { "authorization" : "Bearer $token" }``` ```"data"="bytestream"``` | |
Expand Down
6 changes: 3 additions & 3 deletions docs/hub/how-to-upstream.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -89,13 +89,13 @@ Pass the full repository ID to `delete_repo`. The full repository ID looks like
```python
>>> from huggingface_hub import get_full_repo_name, delete_repo
>>> name = get_full_repo_name(repo_name)
>>> delete_repo(name=name)
>>> delete_repo(repo_id=name)
```

Delete a dataset repository by adding the `repo_type` parameter:

```python
>>> delete_repo(name=REPO_NAME, repo_type="dataset")
>>> delete_repo(repo_id=REPO_NAME, repo_type="dataset")
```

### Change repository visibility
Expand Down Expand Up @@ -128,7 +128,7 @@ The `clone_from` parameter clones a repository from a Hugging Face model ID to a
Easily combine the `clone_from` parameter with `create_repo` to create and clone a repository:

```python
>>> repo_url = create_repo(name="repo_name")
>>> repo_url = create_repo(repo_id="repo_name")
>>> repo = Repository(local_dir="repo_local_path", clone_from=repo_url)
```

Expand Down
2 changes: 1 addition & 1 deletion docs/hub/tutorial-add-library.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -115,7 +115,7 @@ The `create_repo` method creates a repository on the Hub. Use the `name` paramet

```python
>>> from huggingface_hub import create_repo
>>> create_repo(name="test-model")
>>> create_repo(repo_id="test-model")
'https://huggingface.co/lysandre/test-model'
```

Expand Down
5 changes: 3 additions & 2 deletions src/huggingface_hub/commands/user.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -255,9 +255,10 @@ def run(self):
exit()
try:
url = self._api.create_repo(
self.args.name,
repo_id=self.args.name
if self.args.organization is None
else f"{self.args.organization}/{self.args.name}",
adrinjalali marked this conversation as resolved.
Show resolved Hide resolved
token=token,
organization=self.args.organization,
repo_type=self.args.type,
space_sdk=self.args.space_sdk,
)
Expand Down
119 changes: 97 additions & 22 deletions src/huggingface_hub/hf_api.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,28 @@
USERNAME_PLACEHOLDER = "hf_user"


# TODO: remove after deprecation period is over (v0.7)
def _validate_repo_id_deprecation(repo_id, name, organization):
"""Returns (name, organization) from the input."""
if repo_id and (name or organization):
raise ValueError(
"Only pass `repo_id` and leave deprecated `name` and "
"`organization` to be None."
)
elif name or organization:
warnings.warn(
"`name` and `organization` input arguments are deprecated and "
"will be removed in v0.7. Pass `repo_id` instead.",
FutureWarning,
)
else:
if "/" in repo_id:
name, organization = repo_id.split("/")
else:
name, organization = repo_id, None
adrinjalali marked this conversation as resolved.
Show resolved Hide resolved
return name, organization


def repo_type_and_id_from_hf_id(hf_id: str):
"""
Returns the repo type and ID from a huggingface.co URL linking to a repository
Expand Down Expand Up @@ -992,34 +1014,47 @@ def dataset_info(

def create_repo(
self,
name: str,
repo_id: str = None,
token: Optional[str] = None,
organization: Optional[str] = None,
private: Optional[bool] = None,
repo_type: Optional[str] = None,
exist_ok=False,
lfsmultipartthresh: Optional[int] = None,
space_sdk: Optional[str] = None,
name: str = None,
) -> str:
"""
HuggingFace git-based system, used for models, datasets, and spaces.
"""Create an empty repo on the HuggingFace Hub.

Args:
repo_id: A namespace (user or an organization) and a repo name
seperated by a ``/``.

Call HF API to create a whole repo.
.. versionadded: 0.4.0

Params:
private: Whether the model repo should be private (requires a paid huggingface.co account)
token: An authentication token [1]_.

repo_type: Set to :obj:`"dataset"` or :obj:`"space"` if uploading to a dataset or space, :obj:`None` or :obj:`"model"` if uploading to a model. Default is :obj:`None`.
private: Whether the model repo should be private.

exist_ok: Do not raise an error if repo already exists
repo_type: Set to :obj:`"dataset"` or :obj:`"space"` if uploading
to a dataset or space, :obj:`None` or :obj:`"model"` if
uploading to a model. Default is :obj:`None`.

lfsmultipartthresh: Optional: internal param for testing purposes.
exist_ok: If ``True``, do not raise an error if repo already
exists.

space_sdk: Choice of SDK to use if repo_type is "space". Can be "streamlit", "gradio", or "static".
space_sdk: Choice of SDK to use if repo_type is "space". Can be
"streamlit", "gradio", or "static".

Returns:
URL to the newly created repo.

References:
.. [1] https://huggingface.co/settings/tokens

Reference
"""
name, organization = _validate_repo_id_deprecation(repo_id, name, organization)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would add a check to ensure that at least repo_id or name is passed, since they are now both optional arguments:

>>> from huggingface_hub import create_repo
>>> create_repo()
Traceback (most recent call last):
  File "/usr/lib/python3.6/code.py", line 91, in runcode
    exec(code, self.locals)
  File "<input>", line 1, in <module>
  File "/home/lysandre/Workspaces/Python/huggingface_hub/src/huggingface_hub/hf_api.py", line 1056, in create_repo
    name, organization = _validate_repo_id_deprecation(repo_id, name, organization)
  File "/home/lysandre/Workspaces/Python/huggingface_hub/src/huggingface_hub/hf_api.py", line 68, in _validate_repo_id_deprecation
    if "/" in repo_id:
TypeError: argument of type 'NoneType' is not iterable

Something like the following

Suggested change
name, organization = _validate_repo_id_deprecation(repo_id, name, organization)
if name is None and repo_id is None:
raise ValueError("Please pass either `name` or `repo_id`")
name, organization = _validate_repo_id_deprecation(repo_id, name, organization)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1, but since name is deprecated, I would just ask to specify repo_id.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Indeed, good call!


path = f"{self.endpoint}/api/repos/create"
if token is None:
token = self._validate_or_retrieve_token()
Expand Down Expand Up @@ -1082,8 +1117,8 @@ def create_repo(
"Ignoring provided space_sdk because repo_type is not 'space'."
)

if lfsmultipartthresh is not None:
json["lfsmultipartthresh"] = lfsmultipartthresh
if getattr(self, "_lfsmultipartthresh", None):
json["lfsmultipartthresh"] = self._lfsmultipartthresh
osanseviero marked this conversation as resolved.
Show resolved Hide resolved
r = requests.post(
path,
headers={"authorization": f"Bearer {token}"},
Expand All @@ -1109,18 +1144,36 @@ def create_repo(

def delete_repo(
self,
name: str,
repo_id: str = None,
token: Optional[str] = None,
organization: Optional[str] = None,
repo_type: Optional[str] = None,
name: str = None,
):
"""
HuggingFace git-based system, used for models, datasets, and spaces.

Call HF API to delete a whole repo.
"""Delete a repo from the HuggingFace Hub.

CAUTION(this is irreversible).

Args:
repo_id: A namespace (user or an organization) and a repo name
seperated by a ``/``.

.. versionadded: 0.4.0

token: An authentication token [1]_.

repo_type: Set to :obj:`"dataset"` or :obj:`"space"` if uploading
to a dataset or space, :obj:`None` or :obj:`"model"` if
uploading to a model. Default is :obj:`None`.

Returns:
None

References:
.. [1] https://huggingface.co/settings/tokens
"""
name, organization = _validate_repo_id_deprecation(repo_id, name, organization)

path = f"{self.endpoint}/api/repos/delete"
if token is None:
token = self._validate_or_retrieve_token()
Expand Down Expand Up @@ -1177,18 +1230,40 @@ def delete_repo(

def update_repo_visibility(
self,
name: str,
private: bool,
repo_id: str = None,
private: bool = False,
token: Optional[str] = None,
organization: Optional[str] = None,
repo_type: Optional[str] = None,
name: str = None,
) -> Dict[str, bool]:
"""
Update the visibility setting of a repository.
"""Update the visibility setting of a repository.

Args:
repo_id: A namespace (user or an organization) and a repo name
seperated by a ``/``.

.. versionadded: 0.4.0
adrinjalali marked this conversation as resolved.
Show resolved Hide resolved

private: Whether the model repo should be private.

token: An authentication token [1]_.

repo_type: Set to :obj:`"dataset"` or :obj:`"space"` if uploading
to a dataset or space, :obj:`None` or :obj:`"model"` if
uploading to a model. Default is :obj:`None`.

Returns:
The HTTP response in json.

References:
.. [1] https://huggingface.co/settings/tokens
"""
if repo_type not in REPO_TYPES:
raise ValueError("Invalid repo type")

name, organization = _validate_repo_id_deprecation(repo_id, name, organization)

if token is None:
token = self._validate_or_retrieve_token()
elif not self._is_valid_token(token):
Expand Down
5 changes: 3 additions & 2 deletions src/huggingface_hub/hub_mixin.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -257,9 +257,10 @@ def push_to_hub(
if repo_url is None and not os.path.exists(repo_path_or_name):
repo_name = Path(repo_path_or_name).name
repo_url = HfApi(endpoint=api_endpoint).create_repo(
repo_name,
repo_id=repo_name
if organization is None
else f"{organization}/{repo_name}",
token=token,
organization=organization,
private=private,
repo_type=None,
exist_ok=True,
Expand Down
Loading