Add auto cluster support

[Gsantomaggio]

Hi,

This PR adds support for RabbitMQ Autocluster .
It supports backends as etcd and Consul.

Create a cluster is very easy, for example:

docker run --name=rabbit1  -e AUTOCLUSTER_TYPE=etcd -e ETCD_HOST=172.18.0.1   -e RABBITMQ_ERLANG_COOKIE='ilovebeam'  rabbitmq-autocluster  

docker run -d --name=rabbit2 --link=rabbit1 -e AUTOCLUSTER_TYPE=etcd -e ETCD_HOST=172.18.0.1 -e RABBITMQ_ERLANG_COOKIE='ilovebeam' rabbitmq-autocluster

I think that it is useful for Docker.

What do you think?

[yosifkit]

We try to keep the images as close to upstream as possible and since this plugin is not from upstream I would be hesitant to create an image based upon it. It would be fairly simple for users to create their own image FROM rabbitmq to add this and other plugins they need; using automated builds and repository links would keep the image up to date. Or they could use the image provided by the author of the plugin: https://hub.docker.com/r/aweber/rabbitmq-autocluster/

[michaelklishin]

@yosifkit that plugin is actually very useful to many. We even borrowed a few ideas from it into RabbitMQ 3.7.0 :) So if there's an exception to your rule, I'd say it should be this plugin.

@Gsantomaggio is a RabbitMQ contributor, by the way.

[michaelklishin]

It's a good point that Gavin Roy publishes some kind of Docker image for the plugin already.

[Gsantomaggio]

@yosifkit that makes sense.

I knew about rabbitmq-autocluster image, but I'd wanted to move it here because I think that it is very useful for Docker see Consul and etcd installed by default on CoreOS). Just to avoid to have too much "official images".

Btw I understand your point of view, and feel free to close the PR.

[tianon]

I'm still leaning towards -1 on including this explicitly here, but since @michaelklishin thinks it's a valid exception, I think we should continue to consider it. 👍

I'm also going to cc @gmr to see if he's got any additional thoughts, concerns, or requests with regards to this that he's willing to share with us. 😇 🙏

[ghost]

We used autoclusterer briefly in Kolla and it worked well, though we landed on rabbitmq-clusterer. For our particular usage rabbitmq-clusterer worked better.

I would love to see some clustering support land in the official image. To that end I have another option with a PR for rabbitmq-clusterer[0]. But more than anything, I would love to have a way to extend this rabbitmq image to allow for clustering without having to fork the image and modify or replace the entrypoint script.

[0] #131

[michaelklishin]

Please note that as of the future 3.7.0 release core rabbitmq-autocluster abstractions will be in the core. That plugin will primarily become a collection of backends that are not included into the core because of 3rd party dependencies (e.g. etcd2, Consul, Kubernetes).

Of course, the 3.6.x version will be around for a while, in fact, we have plans to contribute to that branch because it could use some love in the short term.

[Gsantomaggio]

In agreement with the maintainer the aweber plugin has been forked to the official RabbitMQ repository

At this point we have two ways:

1. Close this PR, and (who is interested) use this docker image
2. Leave it open and update it with the RabbitMQ release ( when will be available )

wdyt ? @yosifkit @tianon @michaelklishin

[michaelklishin]

As of RabbitMQ 3.7.0 (unreleased), most key ideas from rabbitmq-autocluster will be available in the core. I guess the right thing to do for 3.6.x is for the maintainers of rabbitmq-autocluster (currently team RabbitMQ) to maintain a Docker image that includes that plugin.