Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

v1.6: Limit task batch size #2667

Merged
merged 3 commits into from
Jan 11, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions config/sidebar-learn.json
Original file line number Diff line number Diff line change
Expand Up @@ -408,6 +408,11 @@
"label": "Overview",
"slug": "overview"
},
{
"source": "learn/experimental/max_number_of_batched_tasks.mdx",
"label": "Limit number of batched tasks",
"slug": "max_number_of_batched_tasks"
},
{
"source": "learn/experimental/metrics.mdx",
"label": "Metrics",
Expand Down
26 changes: 26 additions & 0 deletions learn/experimental/max_number_of_batched_tasks.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
---
title: Limit task batch size — Meilisearch documentation
description: Use this experimental feature to limit the number of tasks Meilisearch performs in a single batch. May improve system stability in some cases.
---

# Limit task batch size

Use this experimental feature to limit the number of tasks Meilisearch performs in a single batch. May improve stability in systems handling a large queue of resource-intensive tasks.

To enable it, relaunch Meilisearch with the `--experimental-max-number-of-batched-tasks` command-line option, specifying the maximum amount of tasks you want to process in a single batch:

```sh
./meilisearch --experimental-max-number-of-batched-tasks 100
```

You may also activate this option with environment variables or in the configuration file by using `MEILI_EXPERIMENTAL_MAX_NUMBER_OF_BATCHED_TASKS`.

[Visit the GitHub discussion to learn how to use this feature](https://github.com/orgs/meilisearch/discussions/713).

Are you using this feature? Meilisearch wants to hear from you! Tell us about your experience in the [GitHub discussion](https://github.com/orgs/meilisearch/discussions/713). All feedback is useful and helps make Meilisearch better and easier to use.

<Capsule intent="warning">
This is an experimental feature. Experimental features are unstable: their API might significantly change and become incompatible between releases. Meilisearch does not recommend using experimental features in a production environment.

Meilisearch makes experimental features available expecting they will become stable in a future release. However, it is not possible to guarantee when and if this will happen.
</Capsule>
3 changes: 2 additions & 1 deletion learn/experimental/overview.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -44,8 +44,9 @@ Activating or deactivating experimental features this way does not require you t

| Name | Description | How to configure |
|----------------------------------------------------------------------------------|------------------------------------------------------------|---------------------------------------------------|
| [Limit task batch size](/learn/experimental/max_number_of_batched_tasks) | Limits number of tasks processed in a single batch | At launch with a CLI flag or environment variable |
| [Metrics API](/learn/experimental/metrics) | Exposes Prometheus-compatible analytics data | At launch with a CLI flag or environment variable, during runtime with the API route |
| [Puffin reports](/learn/experimental/puffin_reports) | Export profiler data to diagnose performance issues | During runtime with the API route |
| [Ranking score details](/learn/experimental/ranking_rule_score_details) | Exposes advanced result ranking score data | During runtime with the API route |
| [Reduce indexing memory usage](/learn/experimental/reduce-indexing-memory-usage) | Optimizes indexing performance | At launch with a CLI flag or environment variable |
| [Vector search](/learn/experimental/vector_search) | Allows Meilisearch to function as a vector embedding store | During runtime with the API route |
| [Puffin reports](/learn/experimental/puffin_reports) | Export profiler data to diagnose performance issues | During runtime with the API route |