WebSocket long-polling in a Kibana cluster may cause hard to diagnose errors

[chrisdavies]

In a clustered environment, with long-polling, each new long-poll request could theoretically hit a different Kibana server than the one previously used. This means that the server-side state for expression evaluation might get lost.

It's probably a pretty rare edge-case, but one worth bringing up and discussing.

[elasticmachine]

Pinging @elastic/kibana-canvas

[epixa]

Why do you think that would be rare? I would expect this to effectively break expressions on all multi-instance Kibana installs.

[chrisdavies]

I suspect that sticky sessions are probably the norm, so long-polling requests probably will normally reconnect to the same server. There was a discussion about this topic in the Canvas channel, and it sounds like Canvas is mostly stateless (e.g. when a long-poll request completes, the state will have been transferred back to the client). If I understand correctly, that means that most of the time, the client can connect to any server for the next long-poll request, because it will be sending the context along with it.

The case where this might cause a problem more frequently is if socket.io tries to multiplex requests over the same long-polling connection. I suspect it would try to do this, otherwise, it'd use up all of the browser's available connections pretty fast. If it's multiplexing, then we'll see this issue more frequently.

So you may be right. It may not be rare, but it's hard to say without concrete testing of the scenario.

[epixa]

We've actively discouraged people from using sticky sessions in the past, and we certainly don't encourage people to use them. In fact, we have features in the product (e.g. security.encryptionKey) that exist specifically to ensure folks do not need to use sticky sessions. I don't believe Cloud uses them at all.

So at the least, we must assume sticky sessions are not in play.

[stacey-gammon]

Websockets have been removed!

[Randy-312]

With what version will the webSockets be removed?

[cqliu1]

With what version will the webSockets be removed?

It will be removed in 6.7.0.

[Randy-312]

Great.. Next week?
Deciding how much time to spend solving this problem.. although if it IS something else, i should keep working on it.

I wasn't able to find materials around canvas requiring sticky sessions, but this socket.io issue does appear to confirm that is still required.

So, with this fix, will we still need to have Sticky Sessions in a Load Balancer between Kibana an ES?
.. any ECE insights for this?