Support multiple Elasticsearch nodes and sniffing

[arioch]

With Kibana2 it was possible to have multiple ElasticSearch nodes specified in the config file. I'm trying to accomplish the same with Kibana3, but I can't seem to get it done.

Is this supported at all? Or did I make an error in the snippet below?

var config = new Settings(
  {
    elasticsearch: [ 'http://10.0.1.2:9200', 'http://10.0.1.2:9200' ],
  }
);

[rashidkpc]

round robin is not currently supported. Support is planned

[arioch]

Thanks for the prompt answer.
I'll have to continue using Kibana2 for now.

[rashidkpc]

You could also run a non-data elasticsearch node on whatever you're running Kibana on, and point Kibana at that. That way that node will join the cluster and much more effectively load balance for you.

[arioch]

Behind a reverse proxy I presume?
Forwarding requests from the browser to the non-data node to wherever it's running.

[Ziaunys]

I use a load balancer as a proxy between my ES interface nodes. Each ES node serves Kibana 3 for redundancy. It works pretty well since I only have two.

[deviantony]

+1 for that !

[phani546]

Hi All,

Can i get small description about your procedure mentioned regarding load balancer for elastic search and kibana.

Thanks in advance

phani

[arioch]

Should be fixed in #2368, closing.

[jimmyjones2]

#2368 is closed and points back here!

[arioch]

My bad, it wasn't fixed yet.
Reopening.

[tylerjl]

👍 interest for this; would be useful to avoid 1) added complexity of a load balancer or 2) overhead of running ES on the same host.

[krinkere]

+1 I am running ELK cluster with multiple Logstash Shippers and Logstash Forwarders sending their logs to multiple Redis brokers and multiple Logstash Indexers, all of that gets indexed into ES cluster... Everything is configured so it wont have any single point of failure... except Kibana. Please fix!

[monotek]

+1

[crudbug]

👍

[alaendle]

👍 Also like to vote for that - might be very useful in advanced scenarios.

[ESamir]

+1

[jakommo]

👍

[arioch]

We've been voting for 2.5 years now.
It's quite clear that it's not coming any time soon.

However check comments 3-5 for an alternative.

[vainkop]

+1

[folprates]

+1

[ooizj]

+1

[ksemaev]

+1

[suppix]

+1

[kadzu]

+1

[Alsheh]

+1

[shalenkoff]

any updates?

[TraderApophis]

+1

[mitcham]

+1

[damianconnolly]

+1 Feels "SPOFy"

[hellogbx]

+1

[DaveyDevOps]

+1

[AndyBoat]

+1

[Aodio]

+1

[DaveyDevOps]

Came across this blog today, see Kibana High Availability.

Maybe thinking about this wrong. With this feature (issue #214) our one Kibana could withstand loss of local ES coordinator node, but we would still have the one Kibana as point of failure.

Multiple "Kibani", each connected to a local coordinator would provide better availability.

[fbaligand]

You think right.
The better would be a http load balancer in front of two kibana instances, that are connected to at least two elasticsearch coordinator nodes.

[praseodym]

It makes a lot more sense to simply put a load balancer like HAProxy in front of Elasticsearch. This scales fine with a large number of Elasticsearch nodes, whereas running a large number of Kibana instances is usually a waste of resources.

As an added bonus, you can point Logstash and Beats monitoring endpoints to that losdbalancer as well, because the monitoring functionality doesn’t support multiple endpoints either.

[zerkms]

@praseodym then it's a haproxy instance that is a spof

[praseodym]

@zerkms there’s always the traditional HAProxy+Keepalived pair for a high available loadbalancing.

Juist to be clear: I want Kibana to support multiple ES servers as much as anyone else here, but working around that by running multiple Kibana instances and a load balancer in front just isn’t the best solution.

[PMDubuc]

It makes a lot more sense to simply put a load balancer like HAProxy in front of Elasticsearch. This scales fine with a large number of Elasticsearch nodes, whereas running a large number of Kibana instances is usually a waste of resources.

As an added bonus, you can point Logstash and Beats monitoring endpoints to that losdbalancer as well, because the monitoring functionality doesn’t support multiple endpoints either.

Both Logstash and Beats (Filebeat at least) do support multiple endpoints. Kibana doesn't.

[jbudz]

Phase 1 of this is merged and targeted for 6.6 with #21928.

elasticsearch.url is deprecated and replaced with elasticsearch.hosts. There's a few configuration options for sniffing. This does not include Console, we want those requests to be deterministic - link.

We have settings updated in the 6.6 docs now and we'll have production setup docs updated in the coming weeks. Let me know if there's any questions or concerns.

[fbaligand]

Hi @jbudz

First, thanks for this great news!
Very happy this will be released in 6.6!

One question:
When you say “This does not include Console”, you talk only about sniffing? I mean: “elasticsearch.hosts” feature is available for Console app?

My concern is: when I do a cluster rolling upgrade/restart, I wish that Kibana is still up and can continue to query elasticsearch cluster, given that there is still at least one es node started.

[jbudz]

Hey @fbaligand, you got it. Kibana applications with the exception of console will continue operating transparently by accessing the next node if one goes down. Console for the time being will proxy to the first host in elasticsearch.hosts. With console being a direct, lower level access point we want something closer to a dropdown with node names.

[fbaligand]

OK, it's interesting to be able to choose a node to send a query through console.
So if understand well, in next Kibana version, Console will always interact with first node.
But it is planned in a future version to have a combobox (or something like that) to choose the node to query? Is it correct?