Experimental feature: Update the size of the search queue

[irevoire]

Feature name	Enable when launching	Description
Customize the size of the search queue	./meilisearch --experimental-search-queue-size 10	Output logs as json

Meilisearch v1.8.0 (released on May 06, 2024) introduces a new experimental cli flag to control the number of search requests that can stay in a queue before processing.
The larger the queue is, the more RAM consumption it'll cause.
The default value is 1000. You might want to increase this number if you see very little RAM consumption on your server but receive HTTP 503 errors.
You might want to decrease this number if you notice that Meilisearch crashes during your search spike.

Usage

./meilisearch --experimental-search-queue-size 10

[flexchar]

Just so I understand correctly that ONCE those search requests are processed the system resources such as RAM would be released back?

[irevoire]

Hey, yes don't worry 😁

[doutatsu]

How would I pass this when using a Dockerfile?

ARG MEILI_MASTER_KEY
ARG MEILI_VERSION

FROM getmeili/meilisearch:v${MEILI_VERSION}

EXPOSE 7700

[doutatsu]

@curquiza I've followed what @irevoire suggested, but my RAM usage is the same - I would have assumed it to increase, as mentioned:

[irevoire]

Hey @doutatsu, yeah the search queue is not supposed to consume much RAM.
First of all, with a default value of 1000 meilisearch, it will only use around 7.8KiB of RAM at startup.
In my tests, if 1000 search requests are processed concurrently, the size could reach 20MiB, but that only happens if the queue is full.

Increasing the value to 8000 should increase your RAM consumption to 62KiB and ~150MiB if the queue gets full.

That’s for the meilisearch side, but then, if you don't receive any search requests, your OS may lie about the allocation and wait until you receive something before actually consuming any RAM.

---

Does that help?
Is it possible that you missed the allocation because you were not sending that many search requests?

[doutatsu]

@irevoire Ah I see, I was expecting going from 1000 to 2000 would double my existing average RAM usage - which is why I was looking for a huge spike. I am using this to try and see if it will help with timeouts, so I am going to keep observing for now in case I see any changes either in that or any RAM spikes.

For reference, the average throughput is about 0.38 rps atm

[irevoire]

I am using this to try and see if it will help with timeouts

Once the queue is full, meilisearch won't return a timeout but instead a 503, service unavailable. So it shouldn't be the case of your trouble.
Is there an issue with these timeouts? Maybe we can help you understand what's going on.
The first guess would be that they come from a reverse proxy in your infra 🤔

[doutatsu]

@irevoire It's extension of a long issue conversation in the rails integration - meilisearch/meilisearch-rails#355. I was told to try this experimental feature.

There is no reverse proxy, I am simply using the rails client to make requests to the server

[Strijdhagen]

It appears that this setting defaults to 0 (possibly, only after enabling experimental features).
I just got the error in my application using javascript SDK: MeiliSearchApiError:
Too many search requests running at the same time: 0. Retry after 10s.

Only after adding the experimental setting to my toml file did the error go away.

[irevoire]

Hey, I’m not sure to understand, can you share with me the way you enabled the feature?

[Nio-o]

I have same issue. After upgrading to 1.9.0 I have error "Too many search requests running at the same time: 0. Retry after 10s".

When I add argument --experimental-search-queue-size 1000 to entrypoint script (or config.toml) error disappears. It seems like default value is not 1000

Config with error (actually default config from previous versions)

db_path = "/var/lib/meilisearch/data"
env = "production"
http_addr = "<REDACTED>"
master_key = "<REDACTED>"
no_analytics = true
http_payload_size_limit = "100 MB"
log_level = "INFO"
dump_dir = "/var/lib/meilisearch/dumps"
ignore_missing_dump = false
ignore_dump_if_db_exists = false
schedule_snapshot = false
snapshot_dir = "/var/lib/meilisearch/snapshots"
ignore_missing_snapshot = false
ignore_snapshot_if_db_exists = false
ssl_require_auth = false
ssl_resumption = false
ssl_tickets = false
experimental_enable_metrics = false
experimental_reduce_indexing_memory_usage = false

Adding experimental_search_queue_size = 1000 at the end helps to avoid described issue.

Test example:

npx autocannon \
  -H 'accept: application/json' \
  -H 'authorization: Bearer <TOKEN>' \
  -H 'content-type: application/json' \
  -b '{"queries":[<ANY QUERIES HERE>]}' \
  -c 10 \
  -d 10 \
  -m POST \
  --renderStatusCodes \
  https://example.com

Tested on 1.9.0 and 1.10.2

[irevoire]

Hey @Nio-o, thanks for the second report.

I investigated a bit more and found the issue! It’ll be fixed in the next rc

Thanks a lot @Nio-o and @Strijdhagen for finding and reporting the bug 🎉