Spring Websockets Broker relay supporting active/standby broker

[kmandalas]

In order to support Broker Cluster setups and according to Connecting to a Broker we need to configure a supplier of addresses, instead of a fixed host and port.

However if we have an active/standby setup or in other words High Availability (HA) and Failover cluster (example: https://activemq.apache.org/components/artemis/documentation/1.0.0/ha.html) where only one of the broker instances is active at any time, is this approach still applicable? How will the ReactorNettyTcpClient behave in case of a Broker cluster node failure in general? It is mentioned that the STOMP broker relay always connects, and reconnects as needed if connectivity is lost, to the same host and port. In case of multiple addresses or an active/passive setup, if the TCP connection fails then an attempt is made against the next available node in the list?

@rstoyanchev Adding to the above questions just to note that AbstractWebSocketMessageBrokerConfigurer is Deprecated since 4.0.1 so perhaps an update in the reference docs is needed along with a more thorough explanation on what can be supported out of the box VS the custom implementation needed in production environments where connecting to a cluster is usually expected.

[rstoyanchev]

@kmandalas the sample in the docs is accurate but could use an update to take into account deprecations:

@Configuration
@EnableWebSocketMessageBroker
public class WebSocketConfig implements WebSocketMessageBrokerConfigurer {

	// ...

	@Override
	public void configureMessageBroker(MessageBrokerRegistry registry) {
		registry.enableStompBrokerRelay("/queue/", "/topic/").setTcpClient(createTcpClient());
		registry.setApplicationDestinationPrefixes("/app");
	}

	private ReactorNettyTcpClient<byte[]> createTcpClient() {
		return new ReactorNettyTcpClient<>(
				client -> client.remoteAddress(() -> {
					// return the address to connect to per subscribe
				}),
				new StompReactorNettyCodec());
	}
}

I'm not sure how this relates to an ActiveMQ HA and Failover cluster but in the above example you configure the client with a Supplier<SocketAddress> which is used on every connect. You have to somehow decide what address to provide on each call of Supplier#get().

[kmandalas]

@rstoyanchev thanks for the feedback. Regarding ArtemisMQ HA, there is a "master" node and "backup" nodes. Lets assume 1 backup node for simplicity. Only if the master nodes becomes unavailable, connection to backup node should be made. If the master becomes available again, it takes over. I think its clear that custom implementation is needed, so this answers my question.

[kmandalas]

@jbertram would you mind to advise on this just what would be a proper approach based on your experience with ActiveMQ Artemis? I need to implement this for my current project production usage, just checking if later it would be meaningful to contribute something as a more generic feature (although is difficult since at this context we need to be Broker independent). However maybe some generic policies would be meaningful if could be provided, for example: a RoundRobinPolicy or ActivePassivePolicy?

[jbertram]

I'm not sure you'd need something specific to ActiveMQ Artemis. Ideally you'd use a "sticky" first-available implementation where you supply a list of potential addresses and then you "stick" to the first one that works and keep using that until it no longer works. Once you get a failure then you go back and start from scratch trying each address in the list until you find one that works. This kind of implementation should work for any active/passive broker setup. I think implementing a handful of generic policies (this being one of them) is a good idea.

In any event, the STOMP specification doesn't define semantics for any kind of failover of clustering so it's really up to the client and broker implementations to sort this stuff out.

[kmandalas]

Thanks for your input @jbertram

@rstoyanchev do you think that there is room for opening a discussion for such an Enhancement issue?

[rstoyanchev]

We can continue the discussion here. What enhancement do you have in mind?

[kmandalas]

@rstoyanchev to provide default implementations for a set of "policies" like: RoundRobin, ActivePassive (and perhaps a couple of more generic ones) where a list of broker URLs/ports are provided in properties, along with the selected policy. Configuring a client with a Supplier to remain as a backup if none of the out-of-the-box policies fit.

[rstoyanchev]

@kmandalas I think this could be helpful. If you come up with some useful hierarchy like that feel free to suggest it with a PR. I would add that this will all be on the level of Reactor Netty APIs and such would belong better in Reactor Netty.