Slowness during the display

[curquiza]

As it was reported on our Slack community, some slowness is noticeable using the mini-dashboard.

This slowness is not present when only using instant-meilisearch (the dependency used by this repo).
As you can see:

Mini-dashboard:
https://user-images.githubusercontent.com/20380692/139108096-d4e8d7fc-03f8-4157-adc6-11697c02f239.mp4

Only instant-meilisearch
https://user-images.githubusercontent.com/20380692/139108334-2cd7f9fd-a200-46a2-a618-ced5a453a086.mp4

Also here are the processing time of Instant MeiliSearch

Can interest @bidoubiwa
Ask for @bidoubiwa for the dataset

[curquiza]

Checked with @bidoubiwa, some additional information that might interest the mini-dashboard:

• we are aware of some latencies (or feeling of latency) on the Instant-MeiliSearch side. We already know how to improve them, here are: Abort requests meilisearch-js-plugins#588 and Add option to re-fetch on every next-page and load more. meilisearch-js-plugins#569. Once they are fixed, this might help the mini-dashboard being faster! Should be done by @bidoubiwa after the strapi work!
• MeiliSearch is slower since v0.21.0 for some intensive searches because of the words criterion that gathers more candidates for each request. Ex: for the "coco titi" query, MeiliSearch now returns all candidates containing "coco" AND "titi" but also just "coco". Before v0.21.0 MeiliSearch only returns candidates containing "coco" AND "titi". This feature was widely asked, that's why it was added, but core team plans to add an option allowing the users to remove this behavior (beginning of 2022 probably). We'll let you know when it's released since it can interest the mini-dashboard.

However, these both points do not explain why there is a difference in latency between instant-meilisearch alone and the mini dashboard, unfortunately (cf videos in the issue description)

[entrptaher]

In my mini dashboard, just loading the index list causes the whole page to slow down. It's to be noted I have 1300 indexes as part of the experiment. Each index can have as much as 1000-150k documents, which is noticable in the screenshot. Some of the indexes have 0 documents.

[bidoubiwa]

Hey @entrptaher, I see a search request that has been done in the calls. How long do the stats and the search requests take? This would let us now which one of the routes are being stressed in your setup :)

[gmourier]

Hey @bidoubiwa 👋

Using the GET /indexes route could help since it can be paginated with offset/ limit since v0.28

[bidoubiwa]

I agree @gmourier ! I was wondering out of curiosity if it was stats that is causing slowness but we should definitely use GET /indexes. @mdubus what do you think?

[entrptaher]

The requests are fast, but when they render 1300 items, it gets slow. :(

[bidoubiwa]

I think the solution to that would be to lazyload the index list See example and more information here

[vaibhav135]

I would like to do this.

[bidoubiwa]

Feel free to do so @vaibhav135 :) Thanks 🙏

We prefer not assigning people to our issues because sometimes people ask to be assigned and never come back, which discourages the real volunteers contributors from opening a PR to fix this issue.
We will accept and merge the first PR that fixes correctly and well implements the issue following our contributing guidelines.

[vaibhav135]

@bidoubiwa a quick update on this one. I am almost done with it. I am using react-window with react-select. Just need to do little style change and I will be done. Btw I want to know if testing it with 1300 (datasets) is enough or I should go for more.

[bidoubiwa]

Hey @vaibhav135

Since this PR is very old, I'm not sure this is still an issue. In your PR, could you see a big difference in processing time?

[vaibhav135]

@bidoubiwa Yes definitely, I would say when you have like 1000 or more datasets loaded. The current dropdown is barely usable and super slow. I will attach the before and after once I am done.

[bidoubiwa]

Nice! Thanks a lot :) If you already see some significant improvements, you don't need to test with more. If you still would like to, you can use this one https://github.com/meilisearch/datasets/tree/main/datasets/movies

[vaibhav135]

@bidoubiwa , here's a quick before and after

Before
https://github.com/meilisearch/mini-dashboard/assets/39329182/acd1bc10-9fcd-4c3f-b221-c9c711562f1d

After
https://github.com/meilisearch/mini-dashboard/assets/39329182/1aa2f611-ab2a-4f07-8752-16a86401d359

[bidoubiwa]

Hey @vaibhav135 thanks, indeed it's way faster :)