You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs-website/docs/concepts/device-management.mdx
+5-6Lines changed: 5 additions & 6 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -24,8 +24,7 @@ Haystack’s device management is built on the following abstractions:
24
24
25
25
With the above abstractions, Haystack can fully address any supported device that’s part of your local machine and can support the usage of multiple devices at the same time. Every component that supports local inference will internally handle the conversion of these generic representations to their backend-specific representations.
26
26
27
-
:::info
28
-
Source Code
27
+
:::info Source Code
29
28
30
29
Find the full code for the abstractions above in the Haystack GitHub [repo](https://github.com/deepset-ai/haystack/blob/6a776e672fb69cc4ee42df9039066200f1baf24e/haystack/utils/device.py).
In such cases, ensure that the parameter precedence and selection behavior is clearly documented. In the case of `HuggingFaceLocalGenerator`, the device map passed through the `huggingface_pipeline_kwargs` parameter overrides the explicit `device` parameter and is documented as such.
122
+
In such cases, ensure that the parameter precedence and selection behavior is clearly documented. In the case of `HuggingFaceLocalGenerator`, the device map passed through the `huggingface_pipeline_kwargs` parameter overrides the explicit `device` parameter and is documented as such.
Copy file name to clipboardExpand all lines: docs-website/docs/concepts/document-store.mdx
+8-11Lines changed: 8 additions & 11 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -9,16 +9,14 @@ description: "You can think of the Document Store as a database that stores your
9
9
10
10
You can think of the Document Store as a database that stores your data and provides them to the Retriever at query time. Learn how to use Document Store in a pipeline or how to create your own.
11
11
12
-
Document Store is an object that stores your documents. In Haystack, a Document Store is different from a component, as it doesn’t have the `run()` method. You can think of it as an interface to your database – you put the information there, or you can look through it. This means that a Document Store is not a piece of a pipeline but rather a tool that the components of a pipeline have access to and can interact with.
12
+
Document Store is an object that stores your documents. In Haystack, a Document Store is different from a component, as it doesn't have the `run()` method. You can think of it as an interface to your database – you put the information there, or you can look through it. This means that a Document Store is not a piece of a pipeline but rather a tool that the components of a pipeline have access to and can interact with.
13
13
14
-
:::tip
15
-
Work with Retrievers
14
+
:::tip Work with Retrievers
16
15
17
16
The most common way to use a Document Store in Haystack is to fetch documents using a Retriever. A Document Store will often have a corresponding Retriever to get the most out of specific technologies. See more information in our [Retriever](../pipeline-components/retrievers.mdx) documentation.
18
17
:::
19
18
20
-
:::info
21
-
How to choose a Document Store?
19
+
:::note How to choose a Document Store?
22
20
23
21
To learn about different types of Document Stores and their strengths and disadvantages, head to the [Choosing a Document Store](document-store/choosing-a-document-store.mdx) page.
24
22
:::
@@ -40,7 +38,7 @@ See the installation and initialization details for each Document Store in the "
40
38
41
39
### Work with Documents
42
40
43
-
Convert your data into `Document` objects before writing them into a Document Store along with its metadata and document ID.
41
+
Convert your data into `Document` objects before writing them into a Document Store along with its metadata and document ID.
44
42
45
43
The ID field is mandatory, so if you don’t choose a specific ID yourself, Haystack will do its best to come up with a unique ID based on the document’s information and assign it automatically. However, since Haystack uses the document’s contents to create an ID, two identical documents might have identical IDs. Keep it in mind as you update your documents, as the ID will not be updated automatically.
46
44
@@ -61,14 +59,13 @@ To write documents into the `InMemoryDocumentStore`, simply call the `.write_doc
61
59
62
60
```python
63
61
document_store.write_documents([
64
-
Document(content="My name is Jean and I live in Paris."),
65
-
Document(content="My name is Mark and I live in Berlin."),
62
+
Document(content="My name is Jean and I live in Paris."),
63
+
Document(content="My name is Mark and I live in Berlin."),
66
64
Document(content="My name is Giorgio and I live in Rome.")
67
65
])
68
66
```
69
67
70
-
:::info
71
-
`DocumentWriter`
68
+
:::note `DocumentWriter`
72
69
73
70
See `DocumentWriter` component [docs](../pipeline-components/writers/documentwriter.mdx) to write your documents into a Document Store in a pipeline.
74
71
:::
@@ -100,4 +97,4 @@ The `init` function should indicate all the specifics for the chosen database or
100
97
101
98
We also recommend having a custom corresponding Retriever to get the most out of a specific Document Store.
102
99
103
-
See [Creating Custom Document Stores](document-store/creating-custom-document-stores.mdx) page for more details.
100
+
See [Creating Custom Document Stores](document-store/creating-custom-document-stores.mdx) page for more details.
Copy file name to clipboardExpand all lines: docs-website/docs/concepts/document-store/choosing-a-document-store.mdx
+4-3Lines changed: 4 additions & 3 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -56,9 +56,10 @@ Continue further down the article for a more complex explanation of the strength
56
56
57
57
Vector libraries are often included in the “vector database” category improperly, as they are limited to handling only vectors, are designed to work in-memory, and normally don’t have a clean way to store data on disk. Still, they are the way to go every time performance and speed are the top requirements for your AI application, as these libraries can use hardware resources very effectively.
58
58
59
-
> 🚧 In progress
60
-
>
61
-
> We are currently developing the support for vector libraries in Haystack.
59
+
:::warning In progress
60
+
61
+
We are currently developing the support for vector libraries in Haystack.
Copy file name to clipboardExpand all lines: docs-website/docs/concepts/pipelines.mdx
+1-2Lines changed: 1 addition & 2 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -100,8 +100,7 @@ Thanks to serialization, you can save and then load your pipelines. Serializatio
100
100
101
101
Haystack pipelines delegate the serialization to its components, so serializing a pipeline simply means serializing each component in the pipeline one after the other, along with their connections. The pipeline is serialized into a dictionary format, which acts as an intermediate format that you can then convert into the final format you want.
102
102
103
-
:::info
104
-
Serialization formats
103
+
:::info Serialization formats
105
104
106
105
Haystack only supports YAML format at this time. We'll be rolling out more formats gradually.
Copy file name to clipboardExpand all lines: docs-website/docs/concepts/pipelines/serialization.mdx
+6-7Lines changed: 6 additions & 7 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -9,10 +9,9 @@ description: "Save your pipelines into a custom format and explore the serializa
9
9
10
10
Save your pipelines into a custom format and explore the serialization options.
11
11
12
-
Serialization means converting a pipeline to a format that you can save on your disk and load later.
12
+
Serialization means converting a pipeline to a format that you can save on your disk and load later.
13
13
14
-
:::info
15
-
Serialization formats
14
+
:::info Serialization formats
16
15
17
16
Haystack 2.0 only supports YAML format at this time. We will be rolling out more formats gradually.
18
17
:::
@@ -28,7 +27,7 @@ pipe = Pipeline()
28
27
print(pipe.dumps())
29
28
30
29
## Prints:
31
-
##
30
+
##
32
31
## components: {}
33
32
## connections: []
34
33
## max_loops_allowed: 100
@@ -131,7 +130,7 @@ from haystack import component, default_from_dict, default_to_dict
131
130
classSetIntersector:
132
131
def__init__(self, intersect_with: set):
133
132
self.intersect_with = intersect_with
134
-
133
+
135
134
@component.output_types(result=set)
136
135
defrun(self, data: set):
137
136
return data.intersect(self.intersect_with)
@@ -151,7 +150,7 @@ class SetIntersector:
151
150
152
151
Once a pipeline is available in its dictionary format, the last step of serialization is to convert that dictionary into a format you can store or send over the wire. Haystack supports YAML out of the box, but if you need a different format, you can write a custom Marshaller.
153
152
154
-
A `Marshaller` is a Python class responsible for converting text to a dictionary and a dictionary to text according to a certain format. Marshallers must respect the `Marshaller`[protocol](https://github.com/deepset-ai/haystack/blob/main/haystack/marshal/protocol.py), providing the methods `marshal` and `unmarshal`.
153
+
A `Marshaller` is a Python class responsible for converting text to a dictionary and a dictionary to text according to a certain format. Marshallers must respect the `Marshaller`[protocol](https://github.com/deepset-ai/haystack/blob/main/haystack/marshal/protocol.py), providing the methods `marshal` and `unmarshal`.
155
154
156
155
This is the code for a custom TOML marshaller that relies on the `rtoml` library:
Copy file name to clipboardExpand all lines: docs-website/docs/development/deployment.mdx
+3-4Lines changed: 3 additions & 4 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -9,7 +9,7 @@ description: "Deploy your Haystack pipelines through various services such as Do
9
9
10
10
Deploy your Haystack pipelines through various services such as Docker, Kubernetes, Ray, or a variety of Serverless options.
11
11
12
-
As a framework, Haystack is typically integrated into a variety of applications and environments, and there is no single, specific deployment strategy to follow. However, it is very common to make Haystack pipelines accessible through a service that can be easily called from other software systems.
12
+
As a framework, Haystack is typically integrated into a variety of applications and environments, and there is no single, specific deployment strategy to follow. However, it is very common to make Haystack pipelines accessible through a service that can be easily called from other software systems.
13
13
14
14
These guides focus on tools and techniques that can be used to run Haystack pipelines in common scenarios. While these suggestions should not be considered the only way to do so, they should provide inspiration and the ability to customize them according to your needs.
15
15
@@ -25,12 +25,11 @@ Here are the currently available guides on Haystack pipeline deployment:
25
25
26
26
Haystack can be easily integrated into any HTTP application, but if you don’t have one, you can use Hayhooks, a ready-made application that serves Haystack pipelines as REST endpoints. We’ll be using Hayhooks throughout this guide to streamline the code examples. Refer to the Hayhooks [documentation](hayhooks.mdx) to get details about how to run the server and deploy your pipelines.
27
27
28
-
:::note
29
-
Looking to scale with confidence?
28
+
:::note Looking to scale with confidence?
30
29
31
30
If your team needs **enterprise-grade support, best practices, and deployment guidance** to run Haystack in production, check out **Haystack Enterprise**.
32
31
33
32
📜 [Learn more about Haystack Enterprise](https://haystack.deepset.ai/blog/announcing-haystack-enterprise)
34
33
35
34
👉 [Get in touch with our team](https://www.deepset.ai/products-and-services/haystack-enterprise)
Copy file name to clipboardExpand all lines: docs-website/docs/development/hayhooks.mdx
+4-5Lines changed: 4 additions & 5 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -9,8 +9,7 @@ description: "Hayhooks is a web application you can use to serve Haystack pipeli
9
9
10
10
Hayhooks is a web application you can use to serve Haystack pipelines through HTTP endpoints. This page provides an overview of the main features of Hayhooks.
11
11
12
-
:::info
13
-
Hayhooks GitHub
12
+
:::info Hayhooks GitHub
14
13
15
14
You can find the code and an in-depth explanation of the features in the [Hayhooks GitHub repository](https://github.com/deepset-ai/hayhooks).
16
15
:::
@@ -238,10 +237,10 @@ To deploy a pipeline without listing it as an MCP Tool, set `skip_mcp = True` in
Copy file name to clipboardExpand all lines: docs-website/docs/development/logging.mdx
+1-2Lines changed: 1 addition & 2 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -66,8 +66,7 @@ If Haystack detects a [structlog installation](https://www.structlog.org/en/stab
66
66
To make development a more pleasurable experience, Haystack uses [structlog’s `ConsoleRender`](https://www.structlog.org/en/stable/console-output.html) by default to render structured logs as a nicely aligned and colorful output:
67
67
<ClickableImagesrc="/img/e49a1f2-Screenshot_2024-02-27_at_16.13.51.png"alt="Python code snippet demonstrating basic logging setup with getLogger and a warning level log message output" />
68
68
69
-
:::tip
70
-
Rich Formatting
69
+
:::tip Rich Formatting
71
70
72
71
Install [_rich_](https://rich.readthedocs.io/en/stable/index.html) to beautify your logs even more!
Copy file name to clipboardExpand all lines: docs-website/docs/document-stores/azureaisearchdocumentstore.mdx
+3-4Lines changed: 3 additions & 4 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -16,7 +16,7 @@ A Document Store for storing and retrieval from Azure AI Search Index.
16
16
17
17
[Azure AI Search](https://learn.microsoft.com/en-us/azure/search/search-what-is-azure-search) is an enterprise-ready search and retrieval system to build RAG-based applications on Azure, with native LLM integrations.
18
18
19
-
`AzureAISearchDocumentStore` supports semantic reranking and metadata/content filtering. The Document Store is useful for various tasks such as generating knowledge base insights (catalog or document search), information discovery (data exploration), RAG, and automation.
19
+
`AzureAISearchDocumentStore` supports semantic reranking and metadata/content filtering. The Document Store is useful for various tasks such as generating knowledge base insights (catalog or document search), information discovery (data exploration), RAG, and automation.
Due to Azure search index latency, the document count returned in the example might be zero if executed immediately. To ensure accurate results, be mindful of this latency when retrieving documents from the search index.
53
52
:::
@@ -60,4 +59,4 @@ The Haystack Azure AI Search integration includes three Retriever components. Ea
60
59
61
60
-[`AzureAISearchEmbeddingRetriever`](../pipeline-components/retrievers/azureaisearchembeddingretriever.mdx): This Retriever accepts the embeddings of a single query as input and returns a list of matching documents. The query must be embedded beforehand, which can be done using an [Embedder](../pipeline-components/embedders.mdx) component.
62
61
-[`AzureAISearchBM25Retriever`](../pipeline-components/retrievers/azureaisearchbm25retriever.mdx): A keyword-based Retriever that retrieves documents matching a query from the Azure AI Search index.
63
-
-[`AzureAISearchHybridRetriever`](../pipeline-components/retrievers/azureaisearchhybridretriever.mdx): This Retriever combines embedding-based retrieval and keyword search to find matching documents in the search index to get more relevant results.
62
+
-[`AzureAISearchHybridRetriever`](../pipeline-components/retrievers/azureaisearchhybridretriever.mdx): This Retriever combines embedding-based retrieval and keyword search to find matching documents in the search index to get more relevant results.
0 commit comments