Skip to content

Commit

Permalink
Firestore vector store docs (#18)
Browse files Browse the repository at this point in the history
* Firestore vector store docs

* indexing

* prettier

* feedback
  • Loading branch information
kevinthecheung authored May 3, 2024
1 parent 321b94d commit 5126b42
Show file tree
Hide file tree
Showing 5 changed files with 79 additions and 99 deletions.
2 changes: 0 additions & 2 deletions docs/_guides.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -67,8 +67,6 @@ toc:
path: /docs/genkit/plugins/pinecone
- title: pgvector (code template)
path: /docs/genkit/templates/pgvector
- title: Firestore vector store (code template)
path: /docs/genkit/templates/firestore-vector
- title: Firebase
path: /docs/genkit/plugins/firebase
- title: Google Cloud
Expand Down
1 change: 1 addition & 0 deletions docs/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -370,6 +370,7 @@ maintained by the Genkit team:
<td><code>firebase</code></td>
<td>
<b>Cloud deployment</b>: Cloud Functions, Firebase Authentication, App Check<br>
<b>Vector database</b>: Cloud Firestore vector store<br>
</td>
</tr>
</table>
Expand Down
78 changes: 77 additions & 1 deletion docs/plugins/firebase.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@

The Firebase plugin provides several integrations with Firebase services:

- Indexers and retrievers using Cloud Firestore vector store
- Trace storage using Cloud Firestore
- Flow deployment using Cloud Functions
- Authorization policies for Firebase Authentication users
Expand Down Expand Up @@ -58,7 +59,82 @@ Application Default Credentials. To specify your credentials:
This plugin provides several integrations with Firebase services, which you can
use together or individually.

### Cloud Firestore
### Cloud Firestore vector store

You can use Cloud Firestore as a vector store for RAG indexing and retrieval.

The `firebase` plugin provides a convenience function for defining Firestore
retrievers, `defineFirestoreRetriever()`:

```js
import { defineFirestoreRetriever } from '@genkit-ai/firebase';
import { initializeApp } from 'firebase-admin/app';
import { getFirestore } from 'firebase-admin/firestore';

const app = initializeApp();
const firestore = getFirestore(app);

const yourRetrieverRef = defineFirestoreRetriever({
name: 'yourRetriever',
firestore: getFirestore(app),
collection: 'yourCollection',
contentField: 'yourDataChunks',
vectorField: 'embedding',
embedder: textEmbeddingGecko,
distanceMeasure: 'COSINE', // 'EUCLIDEAN', 'DOT_PRODUCT', or 'COSINE' (default)
});
```

To use it, pass it to the `retrieve()` function:

```js
const docs = await retrieve({
retriever: yourRetrieverRef,
query: 'look for something',
config: { limit: 5 },
});
```

For indexing, use an embedding generator along with the Admin SDK:

```js
import { initializeApp } from 'firebase-admin';
import { getFirestore, FieldValue } from 'firebase-admin/firestore';
import { textEmbeddingGecko } from '@genkit-ai/vertexai';
import { embed } from '@genkit-ai/ai/embedder';

const app = initializeApp();
const firestore = getFirestore(app);

const indexConfig = {
collection: 'yourCollection',
contentField: 'yourDataChunks',
vectorField: 'embedding',
embedder: textEmbeddingGecko,
};

async function indexToFirestore(content) {
const embedding = await embed({
embedder: indexConfig.embedder,
content,
});
await firestore.collection(indexConfig.collection).add({
[indexConfig.vectorField]: FieldValue.vector(embedding),
[indexConfig.contentField]: content,
});
}
```

Firestore depends on indexes to provide fast and efficient querying on
collections. The prior example requires the `embedding` field to be indexed to
work. To do so, invoke the function and Firestore will throw an error with a
command to create an index. Execute that command and your index should be ready
to use.

See the [Retrieval-augmented generation](../rag.md) page for a general
discussion on indexers and retrievers.

### Cloud Firestore trace storage

You can use Cloud Firestore to store traces:

Expand Down
2 changes: 1 addition & 1 deletion docs/rag.md
Original file line number Diff line number Diff line change
Expand Up @@ -249,6 +249,7 @@ export const ragFlow = defineFlow(
Genkit provides indexer and retriever support through its plugin system. The
following plugins are officially supported:

- [Cloud Firestore vector store](plugins/firebase.md)
- [Chroma DB](plugins/chroma.md) vector database
- [Pinecone](plugins/pinecone.md) cloud vector database

Expand All @@ -257,7 +258,6 @@ code templates, which you can customize for your database configuration and
schema:

- PostgreSQL with [`pgvector`](templates/pgvector.md)
- [Firestore vector store](templates/firestore-vector.md)

Embedding model support is provided through the following plugins:

Expand Down
95 changes: 0 additions & 95 deletions docs/templates/firestore-vector.md

This file was deleted.

0 comments on commit 5126b42

Please sign in to comment.