Verify that the default engines are indeed working

[unixfox]

This instance https://search.stinpriza.org/ is sometimes at the top of the list because the default engine sometimes instantly timeout and thus gives a very low response time:

Checking if the default engines are working is probably the best way to avoid having broken instances at the top of the list.
I remembered there were an instance like this a few months ago on stats.searx.xyz and it was always at the top of the list because it would instantly return no results.

[dalf]

Related to searx/searx#1559 (itself related to searx/searx#1813)
Side note: searx-checker is not deployed (*), that's mean if some people use searx-docker, they have removed the searx-checker part (or I don't know ?).

Once searx/searx#1559 is implemented, we have to wait that ops update their instances.

Another way is to check the engines with searx-stats2, actually it's already the case for the wikipedia and google engines using this code:
https://github.com/dalf/searx-stats2/blob/94323278a3553f62941e46aa11f0f77a4bd39c6d/searxstats/fetcher/timing.py#L17-L19
https://github.com/dalf/searx-stats2/blob/94323278a3553f62941e46aa11f0f77a4bd39c6d/searxstats/fetcher/timing.py#L26-L36

This solution will take way more time than an embedded searx-checker to test all the engines (searx-stats2 has to avoid to trigger filtron).

---

(*) about searx-checker deployment:

• in https://searx.space/ , engines tab, you can see ❌ and ✔️ for https://a.searx.space : it is use https://a.searx.space/status results of searx-checker. It's the only instance to have searx-checker installed.
• for other instances, you can see 🟡 which is a hack: it uses the stats page ( https://a.searx.space/stats for example), to check if an engine has returned results. The stat page is fetched 4 times to try to get the stats of all the uwsgi workers (once again related to share data between uwsgi workers searx/searx#1813): https://github.com/dalf/searx-stats2/blob/94323278a3553f62941e46aa11f0f77a4bd39c6d/searxstats/fetcher/selfreport.py#L64-L65

[dalf]

Another way to deal with this issue: since Firefox already loads the index page, it can be use to send a query with the default engines. Then the performance API can give the response time.

Additional benefit: the code can checks the static resources on the result page.

[dalf]

Related to #21

[dalf]

See https://github.com/dalf/searx-stats2/pull/28

[dalf]

#28 is merged.
Things could be better, but instances at the top of the list are usable.
Feel free to reopen this issue is required.

[unixfox]

I'm seeing a same behavior now. Some instances on the top of searx.space returns most of the time no results (with default settings), it requires pressing enter again and again to have results.
I made a screenshot of the home page of searx.space and the instances in red have this issue:

I don't really know how to deal with that issue but it could be that by chance searx-stats2 check the instance at a time when it returns results but if it retried a second time it would have detected no results.

[dalf]

For information, searx-stats2 checks every 3 hours each instance with 3 requests:
https://github.com/dalf/searx-stats2/blob/46b51b8e45a1acff4b0293d83b9de84b6630ca97/searxstats/fetcher/timing.py#L139-L143
There is a delay between 120 and 160 seconds between each request.

I don't see a simple solution to improve the issue you are describing.

Some ideas:

• store statistics in a database, and display average of the last 24 hour / last week / last month: how many requests have failed for the last 24 hours for example.
• improve searx so searx-stats2 can rely on that instead of guessing:
  1. for now some engine fails without errors (bing for example): Engine code: describe which XPath can fail, which must not. searx/searx#1802 can improve that.
  2. then improve statistics: [enh] measure response time with more details. searx/searx#477 (especially the error statistics).
  3. then implement searx-checker: Embedded searx-checker searx/searx#1559
  4. [EDIT] Ultimately, searx can send a report to searx.space as soon as an engine starts to fail for sure (not in the spirit of the project)

If both ideas are implemented, searx-stats2 can have a better idea of which instance is reliable.