Servers should be able to join a cluster

[pauldix]

Servers should be able to join a cluster. Here's a rough outline of how it should work, but please ask questions.

When a server starts up, if it doesn't have a metastore (on disk), it should do one of the following:

• If join was given as a command line argument, it should attempt to join the cluster at that URL
• If no join was specified, it should create a new cluster of just itself
• if join was specified, but they have the metastore on disk, ignore the join

Join

Here's how join should work. The URL supplied should be a server (or a load balancer that points to many InfluxDB servers). The point is that it should be able to join a cluster when pointed to any server in that cluster.

The first 3 servers to join the cluster should run the meta service (Raft). Any servers that join after that should just get a copy of the metastore. They should keep the cached version on disk.

TODO

• Incremental joins
• Caching metastore
• Promoting server to run consensus service
• Demoting server so it drops out of the consensus group and stops running the service

[btashton]

I really wonder if you are reinventing the wheel here. What advantage does this give over something like etcd and a discovery URL? And if you dont want to use etcd, perhaps it is a good model. I know I was looking at using it to generate the peers list for the config file.

[pauldix]

@btashton that's the role that Raft is playing for us. It's just integrated into the InfluxDB cluster. What this gives us is a simpler deployment story and less things to manage.

Yes, it's reinventing the wheel, but that's what incremental progress in software is all about. Improving things just a little bit along the way. And this gives us an improvement in usability for our users.

[rynbrd]

Based on the description it sounds possible that this would allow a new cluster to be deployed automatically in AWS via an AutoScaling group and ELB. The tricky part is when the first node comes online and is added to the ELB. If passed a cluster URL pointing to the ELB it will have no available backends to connect to. As long as the node creates a new cluster from itself when it's unable to contact the cluster or if the only node in the cluster is itself then things will work out.

We do something similar with our CoreOS/etcd clusters in AWS, though they require a discovery service be running.

[beckettsean]

@jwilder can we close this?