Discussion on AsyncPipeline and async-enabled components

[julian-risch]

This is the discussion board for the experimental AsyncPipeline class and async-enabled components.

This experiment allows individual components to opt into async support. Not all components benefit from being async-enabled - I/O-bound components are the most suitable candidates. It also provides a backward-compatible way to execute Haystack pipelines containing both async and non-async components.

🧑‍🍳 Try the notebook here

Note: Experimental features live in this repository for a fixed period of time. We don't guarantee that we will continue maintaining experimental features. But if they are successful, stable and if you like it, we will move the feature to the core Haystack package.

[bilgeyucel]

The notebook is published as cookbook now 👉 https://haystack.deepset.ai/cookbook/async_pipeline