Skip to content

Commit

Permalink
Update blog
Browse files Browse the repository at this point in the history
  • Loading branch information
RobbeSneyders committed Mar 6, 2024
1 parent 016b160 commit df6081c
Show file tree
Hide file tree
Showing 2 changed files with 13 additions and 5 deletions.
16 changes: 12 additions & 4 deletions docs/blog/posts/2024-03-06|Datacomp_index.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,12 +29,20 @@ why we built an index for the Datacomp-12M dataset. While it is a lot smaller th
should already enable a lot of use cases again, and can hopefully be the start towards building
indices for more and larger datasets.

You can access the index directly on the Hugging Face Hub
[here](https://huggingface.co/datasets/fondant-ai/datacomp-small-clip/blob/main/faiss), or read on
below on how to use it with Fondant.

<!-- more -->

The resulting embedded dataset and index have been published on the Hugging Face Hub
[here](https://huggingface.co/datasets/fondant-ai/datacomp-small-clip). The data repository is
structured as follows:
- [data/](https://huggingface.co/datasets/fondant-ai/datacomp-small-clip/viewer): The dataset
containing ids, urls, and CLIP embeddings
- [faiss](https://huggingface.co/datasets/fondant-ai/datacomp-small-clip/blob/main/faiss):
The faiss index
- [id_mapping](https://huggingface.co/datasets/fondant-ai/datacomp-small-clip/tree/main/id_mapping):
The mapping of the faiss ids to the original urls

Read on below on how to use it with Fondant.

## Using the index

### With Fondant
Expand Down
2 changes: 1 addition & 1 deletion docs/overrides/main.html
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
<p style="text-align: center">

We generated a CLIP index for the datacomp-12.8M dataset.
<a href="/blog/2024/03/05/building-a-datacomp-clip-index-with-fondant/"
<a href="/en/latest/blog/2024/03/05/building-a-datacomp-clip-index-with-fondant/"
style="color: white; text-decoration: underline">Learn how you can use it!</a>
</p>
{% endblock %}

0 comments on commit df6081c

Please sign in to comment.