Load test: many requests

[JonasKellerer]

This is related to GenSpectrum/LAPIS#666

Try to find out if SILO has problems with many requests.

[corneliusroemer]

These are the error responses I get:

{
    "error": {
        "type": "about:blank",
        "title": "Internal Server Error",
        "status": 500,
        "detail": "Could not connect to silo: class java.io.IOException HTTP/1.1 header parser received no bytes"
    },
    "info": {
        "dataVersion": null,
        "requestId": "fd8424b4-b1f8-406c-9ee7-a8fc4adddf3f"
    }
}

[fengelniederhammer]

Some of our findings:

• POCO has a default of "max queued HTTP connections" of 64. When this limit is breached, POCO simply rejects the connections and returns a 500. This is the "Could not connect to silo: class java.io.IOException HTTP/1.1 header parser received no bytes" error that we see in LAPIS.
• We can increase this limit of max queued connections (e.g. to 10,000), then SILO will handle all requests within the limit, but response times eventually become very slow.
• When using HTTP 2, a lot more requests arrive simultaneously in SILO, thus this is a lot more likely to happen via HTTP 2. When using HTTP 1.1, requests are handled in almost serial order and we don't observe refused connections.

To mitigate the issue of refused connections, we set the maxQueuedHttpConnections as implemented in #301 to a quite high value, still using HTTP 2 in a proxy. This solved GenSpectrum/cov-spectrum-website#932 and GenSpectrum/LAPIS#666 (the pages load successfully, but load times may be quite high, up to a minute).

We will however implement a cache in LAPIS to speed up response times: GenSpectrum/LAPIS#137

[JonasKellerer]

We have now the new ticket for the cache.