Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Add support for metadata filtering and namespaces for the Upstash Vector component #3254

Merged
merged 3 commits into from
Aug 14, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 21 additions & 0 deletions docs/docs/Components/components-vector-stores.md
Original file line number Diff line number Diff line change
Expand Up @@ -472,6 +472,27 @@ Ensure the Supabase service key, URL, and table name are properly configured.
---


### Upstash Vector


`UpstashVector` searches a Upstash Vector Store for documents similar to the input. It has it's own embedding
model which can be used to search documents without needing an external embedding model.


**Parameters:**

- **Index URL:** The URL of the Upstash index.
- **Index Token:** The token for the Upstash index.
- **Text Key:** The key in the record to use as text.
- **Namespace:** The namespace name. A new namespace is created if not found. Leave empty for default namespace.
- **Search Query:** The search query.
- **Metadata Filter:** The metadata filter. Filters documents by metadata. Look at the [docs](https://upstash.com/docs/vector/features/filtering) for more information.
- **Embedding:** The embedding model used. To use Upstash's embeddings, don't provide an embedding.
- **Number of Results:** The number of results to return.

---


### Vectara {#b4e05230b62a47c792a89c5511af97ac}


Expand Down
35 changes: 32 additions & 3 deletions src/backend/base/langflow/components/vectorstores/Upstash.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,14 @@

from langflow.base.vectorstores.model import LCVectorStoreComponent
from langflow.helpers.data import docs_to_data
from langflow.io import HandleInput, IntInput, StrInput, SecretStrInput, DataInput, MultilineInput
from langflow.io import (
HandleInput,
IntInput,
StrInput,
SecretStrInput,
DataInput,
MultilineInput,
)
from langflow.schema import Data


Expand All @@ -16,9 +23,17 @@ class UpstashVectorStoreComponent(LCVectorStoreComponent):
icon = "Upstash"

inputs = [
StrInput(name="index_url", display_name="Index URL", info="The URL of the Upstash index.", required=True),
StrInput(
name="index_url",
display_name="Index URL",
info="The URL of the Upstash index.",
required=True,
),
SecretStrInput(
name="index_token", display_name="Index Token", info="The token for the Upstash index.", required=True
name="index_token",
display_name="Index Token",
info="The token for the Upstash index.",
required=True,
),
StrInput(
name="text_key",
Expand All @@ -27,7 +42,17 @@ class UpstashVectorStoreComponent(LCVectorStoreComponent):
value="text",
advanced=True,
),
StrInput(
name="namespace",
display_name="Namespace",
info="Leave empty for default namespace.",
),
MultilineInput(name="search_query", display_name="Search Query"),
MultilineInput(
name="metadata_filter",
display_name="Metadata Filter",
info="Filters documents by metadata. Look at the documentation for more information.",
),
DataInput(
name="ingest_data",
display_name="Ingest Data",
Expand Down Expand Up @@ -68,6 +93,7 @@ def _build_upstash(self) -> UpstashVectorStore:
text_key=self.text_key,
index_url=self.index_url,
index_token=self.index_token,
namespace=self.namespace,
)
upstash_vs.add_documents(documents)
else:
Expand All @@ -77,13 +103,15 @@ def _build_upstash(self) -> UpstashVectorStore:
text_key=self.text_key,
index_url=self.index_url,
index_token=self.index_token,
namespace=self.namespace,
)
else:
upstash_vs = UpstashVectorStore(
embedding=self.embedding or use_upstash_embedding,
text_key=self.text_key,
index_url=self.index_url,
index_token=self.index_token,
namespace=self.namespace,
)

return upstash_vs
Expand All @@ -95,6 +123,7 @@ def search_documents(self) -> List[Data]:
docs = vector_store.similarity_search(
query=self.search_query,
k=self.number_of_results,
filter=self.metadata_filter,
)

data = docs_to_data(docs)
Expand Down