Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Document Collection API #973

Merged
merged 5 commits into from
Sep 28, 2023
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
142 changes: 125 additions & 17 deletions docs/hub/api.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,11 @@ We have open endpoints that you can use to retrieve information from the Hub as

The base URL for those endpoints below is `https://huggingface.co`. For example, to construct the `/api/models` call below, one can call the URL [https://huggingface.co/api/models](https://huggingface.co/api/models)

## GET /api/models
## Search API

Wauplin marked this conversation as resolved.
Show resolved Hide resolved
The following endpoints help get information about models, datasets, Spaces, and metrics stored on the Hub.
Wauplin marked this conversation as resolved.
Show resolved Hide resolved

### GET /api/models

Get information from all models in the Hub. The response is paginated, use the [`Link` header](https://docs.github.com/en/rest/guides/using-pagination-in-the-rest-api?apiVersion=2022-11-28#link-header) to get the following pages. You can specify additional parameters to have more specific results.
- `search`: Filter based on substrings for repos and their usernames, such as `resnet` or `microsoft`
Expand Down Expand Up @@ -33,19 +37,19 @@ params = {

This is equivalent to `huggingface_hub.list_models()`.

## GET /api/models/(repo_id) or /api/models/(repo_id)/revision/(revision)
### GET /api/models/(repo_id) or /api/models/(repo_id)/revision/(revision)

Get all information for a specific model.

This is equivalent to `huggingface_hub.model_info(repo_id, revision)`.

## GET /api/models-tags-by-type
### GET /api/models-tags-by-type

Gets all the available model tags hosted in the Hub.

This is equivalent to `huggingface_hub.get_model_tags()`.

## GET /api/datasets
### GET /api/datasets

Get information from all datasets in the Hub. The response is paginated, use the [`Link` header](https://docs.github.com/en/rest/guides/using-pagination-in-the-rest-api?apiVersion=2022-11-28#link-header) to get the following pages. You can specify additional parameters to have more specific results.
- `search`: Filter based on substrings for repos and their usernames, such as `pets` or `microsoft`
Expand Down Expand Up @@ -73,7 +77,7 @@ params = {

This is equivalent to `huggingface_hub.list_datasets()`.

## GET /api/datasets/(repo_id) or /api/datasets/(repo_id)/revision/(revision)
### GET /api/datasets/(repo_id) or /api/datasets/(repo_id)/revision/(revision)

Get all information for a specific dataset.

Expand All @@ -87,21 +91,21 @@ params = {"full": "full"}

This is equivalent to `huggingface_hub.dataset_info(repo_id, revision)`.

## GET /api/datasets/(repo_id)/parquet
### GET /api/datasets/(repo_id)/parquet

Get the list of auto-converted parquet files.

## GET /api/datasets/(repo_id)/parquet/(config)/(split)/(n).parquet
### GET /api/datasets/(repo_id)/parquet/(config)/(split)/(n).parquet

Get the nth shard of the auto-converted parquet files.

## GET /api/datasets-tags-by-type
### GET /api/datasets-tags-by-type

Gets all the available dataset tags hosted in the Hub.

This is equivalent to `huggingface_hub.get_dataset_tags()`.

## GET /api/spaces
### GET /api/spaces

Get information from all Spaces in the Hub. The response is paginated, use the [`Link` header](https://docs.github.com/en/rest/guides/using-pagination-in-the-rest-api?apiVersion=2022-11-28#link-header) to get the following pages. You can specify additional parameters to have more specific results.
- `search`: Filter based on substrings for repos and their usernames, such as `resnet` or `microsoft`
Expand Down Expand Up @@ -130,18 +134,21 @@ params = {

This is equivalent to `huggingface_hub.list_spaces()`.

## GET /api/spaces/(repo_id) or /api/spaces/(repo_id)/revision/(revision)
### GET /api/spaces/(repo_id) or /api/spaces/(repo_id)/revision/(revision)
Get all information for a specific model.

This is equivalent to `huggingface_hub.space_info(repo_id, revision)`.

## GET /api/metrics
### GET /api/metrics

Get information from all metrics in the Hub.

This is equivalent to `huggingface_hub.list_metrics()`.

## POST /api/repos/create
## Repo API

Wauplin marked this conversation as resolved.
Show resolved Hide resolved
The following endpoints manage repository settings like creating and deleting a repository.
### POST /api/repos/create

Create a repository. It's a model repo by default.

Expand All @@ -164,7 +171,7 @@ payload = {

This is equivalent to `huggingface_hub.create_repo()`.

## DELETE /api/repos/delete
### DELETE /api/repos/delete

Delete a repository. It's a model repo by default.

Expand All @@ -185,7 +192,7 @@ payload = {

This is equivalent to `huggingface_hub.delete_repo()`.

## PUT /api/repos/(type)/(repo_id)/settings
### PUT /api/repos/(repo_type)/(repo_id)/settings

Update repo visibility.

Expand All @@ -199,7 +206,7 @@ payload = {

This is equivalent to `huggingface_hub.update_repo_visibility()`.

## POST /api/repos/move
### POST /api/repos/move

Move a repository (rename within the same namespace or transfer from user to organization).

Expand All @@ -214,7 +221,11 @@ payload = {

This is equivalent to `huggingface_hub.move_repo()`.

## GET /api/whoami-v2
## User API

Wauplin marked this conversation as resolved.
Show resolved Hide resolved
The following endpoint gets information about a user.

### GET /api/whoami-v2
Wauplin marked this conversation as resolved.
Show resolved Hide resolved

Get username and organizations the user belongs to.

Expand All @@ -224,4 +235,101 @@ Payload:
headers = { "authorization" : "Bearer $token" }
```

This is equivalent to `huggingface_hub.whoami()`.
This is equivalent to `huggingface_hub.whoami()`.

## Collection API

Use Collections to group repositories from the Hub (Models, Datasets, Spaces and Papers) on a dedicated page.

You can learn more about it in the Collections [guide](./collections.md). Collections can also be managed using the Python client (see [guide](https://huggingface.co/docs/huggingface_hub/main/en/guides/collections)).

### POST /api/collections

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it's weird to me that you can create a collection with a first item (but not several items, i assume?) vs. creating a collection then adding an item from a differnt call.

(this comment is about the API design, not really about the documentation)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes I agree. I assume this mostly comes from the fact that on the Hub we can create a collection from a model page (which means creating it with a single item).

FYI, what I did in huggingface_hub is to not support creating a collection with an item directly. Users have to create collection and then add items to it.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ah ok, perfect if you did that in hfh then

Create a new collection on the Hub with a title, a description (optional) and a first item (optional). An item is defined by a type (`model`, `dataset`, `space` or `paper`) and an id (repo_id or paper_id on the Hub).

Payload:

```js
payload = {
"title": "My cool models",
"namespace": "username_or_org",
"description": "Here is a shortlist of models I've trained.",
"item" : {
"type": "model",
"id": "username/cool-model",
}
"private": false,

}
```

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
This is equivalent to `huggingface_hub.create_collection()`.
This is equivalent to [`huggingface_hub.create_collection`].

This is equivalent to `huggingface_hub.create_collection()`.

### GET /api/collections/(namespace)/(slug)-(id)

Return information about a collection.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
This is equivalent to `huggingface_hub.get_collection()`.
This is equivalent to [`huggingface_hub.get_collection`].

This is equivalent to `huggingface_hub.get_collection()`.

### PATCH /api/collections/(namespace)/(slug)-(id)

Update the metadata of a collection on the Hub. You can't add or modify the items of the collection with this method. All fields of the payload are optional.

Payload:

```js
payload = {
"title": "My cool models",
"description": "Here is a shortlist of models I've trained.",
"private": false,
"position": 0, // position of the collection on your profile
"theme": "green",
}
```

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
This is equivalent to `huggingface_hub.update_collection_metadata()`.
This is equivalent to [`huggingface_hub.update_collection_metadata`].

This is equivalent to `huggingface_hub.update_collection_metadata()`.

### DELETE /api/collections/(namespace)/(slug)-(id)

Return a collection. This is a non-revertible operation. A deleted collection cannot be restored.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
This is equivalent to `huggingface_hub.delete_collection()`.
This is equivalent to [`huggingface_hub.delete_collection`].

This is equivalent to `huggingface_hub.delete_collection()`.

### POST /api/collections/(namespace)/(slug)-(id)/item)

Add an item to a collection. An item is defined by a type (`model`, `dataset`, `space` or `paper`) and an id (repo_id or paper_id on the Hub). A note can also be attached to the item (optional).

Payload:

```js
payload = {
"item" : {
"type": "model",
"id": "username/cool-model",
}
"note": "Here is the model I trained on ...",
}
```

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
This is equivalent to `huggingface_hub.add_collection_item()`.
This is equivalent to [`huggingface_hub.add_collection_item`].

This is equivalent to `huggingface_hub.add_collection_item()`.

### PATCH /api/collections/(namespace)/(slug)-(id)/items/(item_id)

Update an item in a collection. You must know the item object id which is different from the repo_id/paper_id provided when adding the item to the collection. The `item_id` can be retrieved by fetching the collection.

You can update the note attached to the item or the position of the item in the collection. Both fields are optional.

```js
payload = {
"position": 0,
"note": "Here is the model I trained on ...",
}
```

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
This is equivalent to `huggingface_hub.update_collection_item()`.
This is equivalent to [`huggingface_hub.update_collection_item`].

This is equivalent to `huggingface_hub.update_collection_item()`.

### DELETE /api/collections/(namespace)/(slug)-(id)/items/(item_id)

Remove an item from a collection. You must know the item object id which is different from the repo_id/paper_id provided when adding the item to the collection. The `item_id` can be retrieved by fetching the collection.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
This is equivalent to `huggingface_hub.delete_collection_item()`.
This is equivalent to [`huggingface_hub.delete_collection_item`].

This is equivalent to `huggingface_hub.delete_collection_item()`.
Loading