node type is being ignored when joining a cluster

[scalp42]

Hi @jjasghar

Using v4.5.1 and RabbitMQ 3.5.7, I set up two nodes "ready" to cluster:

• rabbitmq-i-fef72824

root@rabbitmq-i-fef72824  ~ # rabbitmqctl cluster_status
Cluster status of node 'rabbit@ec2-54-200-235-67.us-west-2.compute.amazonaws.com' ...
[{nodes,[{disc,['rabbit@ec2-54-200-235-67.us-west-2.compute.amazonaws.com']}]},
 {running_nodes,['rabbit@ec2-54-200-235-67.us-west-2.compute.amazonaws.com']},
 {cluster_name,<<"rabbit@ip-10-52-42-243.us-west-2.compute.internal">>},
 {partitions,[]}]

• rabbitmq-i-65c8f9a1

root@rabbitmq-i-65c8f9a1 ~ # rabbitmqctl cluster_status
Cluster status of node 'rabbit@ec2-54-201-251-170.us-west-2.compute.amazonaws.com' ...
[{nodes,[{disc,['rabbit@ec2-54-201-251-170.us-west-2.compute.amazonaws.com']}]},
 {running_nodes,['rabbit@ec2-54-201-251-170.us-west-2.compute.amazonaws.com']},
 {cluster_name,<<"rabbit@ip-10-52-32-132.us-west-2.compute.internal">>},
 {partitions,[]}]

If I join one to the other by hand with without specifying --ram, I get the desired disc type:

root@rabbitmq-i-fef72824 ~ # rabbitmqctl stop_app ; rabbitmqctl join_cluster rabbit@ec2-54-201-251-170.us-west-2.compute.amazonaws.com

Stopping node 'rabbit@ec2-54-200-235-67.us-west-2.compute.amazonaws.com' ...
Clustering node 'rabbit@ec2-54-200-235-67.us-west-2.compute.amazonaws.com' with 'rabbit@ec2-54-201-251-170.us-west-2.compute.amazonaws.com' ...

root@rabbitmq-i-fef72824 ~ # rabbitmqctl start_app
Starting node 'rabbit@ec2-54-200-235-67.us-west-2.compute.amazonaws.com' ...


root@rabbitmq-i-fef72824 ~ # rabbitmqctl cluster_status
Cluster status of node 'rabbit@ec2-54-200-235-67.us-west-2.compute.amazonaws.com' ...
[{nodes,[{disc,['rabbit@ec2-54-200-235-67.us-west-2.compute.amazonaws.com',
                'rabbit@ec2-54-201-251-170.us-west-2.compute.amazonaws.com']}]},
 {running_nodes,['rabbit@ec2-54-201-251-170.us-west-2.compute.amazonaws.com',
                 'rabbit@ec2-54-200-235-67.us-west-2.compute.amazonaws.com']},
 {cluster_name,<<"rabbit@ip-10-52-32-132.us-west-2.compute.internal">>},
 {partitions,[]}]

So far so good, both nodes show up as disc type.

Now if I reset both nodes:

root@rabbitmq-i-fef72824 ~ # service rabbitmq-server stop ; pkill -KILL -U rabbitmq ;  rm -fr /rabbitmq/mnesia/* /var/lib/rabbitmq/mnesia/ /var/lib/rabbitmq/erl_crash.dump /var/log/rabbitmq/* ; service rabbitmq-server restart
 * Stopping message broker rabbitmq-server                                                                                                                                                                                                                               [ OK ]
 * Restarting message broker rabbitmq-server                                                                                                                                                                                                                             [ OK ]

root@rabbitmq-i-fef72824 [gz-dev] ~ # rabbitmqctl cluster_status
Cluster status of node 'rabbit@ec2-54-200-235-67.us-west-2.compute.amazonaws.com' ...
[{nodes,[{disc,['rabbit@ec2-54-200-235-67.us-west-2.compute.amazonaws.com']}]},
 {running_nodes,['rabbit@ec2-54-200-235-67.us-west-2.compute.amazonaws.com']},
 {cluster_name,<<"rabbit@ip-10-52-42-243.us-west-2.compute.internal">>},
 {partitions,[]}]

And I try to port the exact same commands using the LWRP:

json = %q|[{"name":"rabbit@ec2-54-201-251-170.us-west-2.compute.amazonaws.com","type":"disc"}]|

rabbitmq_cluster json do
  action :join
  ignore_failure true
end

Run Chef:

[2015-12-16T01:14:12+00:00] INFO: Processing rabbitmq_cluster[[{"name":"rabbit@ec2-54-201-251-170.us-west-2.compute.amazonaws.com","type":"disc"}]] action join (gz-rabbitmq::join line 5)
[2015-12-16T01:14:12+00:00] INFO: [rabbitmq_cluster] Action join ...
[2015-12-16T01:14:14+00:00] INFO: [rabbitmq_cluster] Clustering node 'rabbit@ec2-54-200-235-67.us-west-2.compute.amazonaws.com' with 'rabbit@ec2-54-201-251-170.us-west-2.compute.amazonaws.com' ...

[2015-12-16T01:14:15+00:00] INFO: [rabbitmq_cluster] Node rabbit@ec2-54-200-235-67.us-west-2.compute.amazonaws.com joined in rabbit@ec2-54-201-251-170.us-west-2.compute.amazonaws.com
[2015-12-16T01:14:15+00:00] INFO: [{nodes,[{disc,['rabbit@ec2-54-201-251-170.us-west-2.compute.amazonaws.com']},{ram,['rabbit@ec2-54-200-235-67.us-west-2.compute.amazonaws.com']}]},{running_nodes,['rabbit@ec2-54-201-251-170.us-west-2.compute.amazonaws.com','rabbit@ec2-54-200-235-67.us-west-2.compute.amazonaws.com']},{cluster_name,<<"rabbit@ip-10-52-42-243.us-west-2.compute.internal">>},{partitions,[]}]

You can see that the node joins the cluster as a ram node instead of disc even though is specified:

root@rabbitmq-i-fef72824 ~ # rabbitmqctl cluster_status
Cluster status of node 'rabbit@ec2-54-200-235-67.us-west-2.compute.amazonaws.com' ...
[{nodes,[{disc,['rabbit@ec2-54-201-251-170.us-west-2.compute.amazonaws.com']},
         {ram,['rabbit@ec2-54-200-235-67.us-west-2.compute.amazonaws.com']}]},
 {running_nodes,['rabbit@ec2-54-201-251-170.us-west-2.compute.amazonaws.com',
                 'rabbit@ec2-54-200-235-67.us-west-2.compute.amazonaws.com']},
 {cluster_name,<<"rabbit@ip-10-52-42-243.us-west-2.compute.internal">>},
 {partitions,[]}]

Any ideas?

cc @michaelklishin maybe

[scalp42]

Found the culprit: https://github.com/jjasghar/rabbitmq/blob/master/providers/cluster.rb#L143