HA/Failover support with Artemis

[JoshBargar]

I am trying to use spring-boot-starter-artemis in my application so that I can utilize auto-configuration, as well as an embedded broker with testing, although I am unable to set up high availability/failover with auto-configuration for non-test code. The only method I have been successful with is creating my own ConnectionFactory, which then prevents me from using an embedded broker via application.properties because it seems that auto-configuration is then disabled. Our application previously used ActiveMQ which allowed us to define the failover in the broker URL. Does anyone have any idea how to achieve this with Artemis?

[snicoll]

I am not sure I understand what you're trying to do. Do I understand you want to use HA with an embedded broker?

The only method I have been successful with is creating my own ConnectionFactory

What does that look like exactly? And what benefits such setup brings?

I can see that the auto-configuration could benefit from reusing an existing ConnectionFactory though that would make things quite inconsistent if said connection factory wasn't targeting the embedded instance. Setup is in ArtemisEmbeddedServerConfiguration.

[JoshBargar]

I am not sure I understand what you're trying to do. Do I understand you want to use HA with an embedded broker?

The only method I have been successful with is creating my own ConnectionFactory

What does that look like exactly? And what benefits such setup brings?

I can see that the auto-configuration could benefit from reusing an existing ConnectionFactory though that would make things quite inconsistent if said connection factory wasn't targeting the embedded instance. Setup is in ArtemisEmbeddedServerConfiguration.

I want to achieve 2 things:

• Enable Artemis via auto configuration WITH HA/failover
• Be able to use said auto-configuration to swap out an embedded broker for use with integration tests (so that I don't have to create any test classes)

So, to me, it seems that I can either use auto-configuration without HA, or manually configure with HA and then create a test connection factory for my integration test classes.

Would I be able to utilize ArtemisConfigurationCustomizer alongside auto-configuration to add a TransportConfiguration for failover?

[snicoll]

Thanks for the feedback. Can you share the code you have to enable failover? This will help figuring out how far we are to support this one way or the other.

[JoshBargar]

Thanks for the feedback. Can you share the code you have to enable failover? This will help figuring out how far we are to support this one way or the other.

Here are my config classes:

@Configuration
@EnableJms
public class InterchangeConfig {

	@Bean
	public DefaultJmsListenerContainerFactory containerFactory(ConnectionFactory connectionFactory,
			JmsErrorHandler jmsErrorHandler, MappingJackson2MessageConverter messageConverter,
			ObjectMapper objectMapper) {
		DefaultJmsListenerContainerFactory factory = new DefaultJmsListenerContainerFactory();
		
		objectMapper.setSerializationInclusion(Include.ALWAYS);
		messageConverter.setObjectMapper(objectMapper);

		factory.setConnectionFactory(connectionFactory);
		factory.setErrorHandler(jmsErrorHandler);
		factory.setSubscriptionDurable(true);
		factory.setSubscriptionShared(true);
		factory.setDestinationResolver(new DynamicDestinationResolver());
		factory.setMessageConverter(messageConverter);

		return factory;
	}

}

Config to add failover:

@Profile("!test")
@Configuration
public class ArtemisConfig implements ArtemisConfigurationCustomizer {
    @Autowired
    private JmsBrokerProperties jmsBrokerProperties;

    @Override
    public void customize(org.apache.activemq.artemis.core.config.Configuration configuration) {
        configuration.addAcceptorConfiguration(transportConfiguration()));
    }

    private TransportConfiguration transportConfiguration() {
        String connectorFactoryFqcn = NettyConnectorFactory.class.getName();
        Map<String, Object> backupTransportParameters = new HashMap<>(2);

	backupTransportParameters.put("host", jmsBrokerProperties.getBackupHostname());
	backupTransportParameters.put("port", jmsBrokerProperties.getBackupPort().toString());

	return new TransportConfiguration(connectorFactoryFqcn,
				backupTransportParameters);
    }
}

Properties file:

spring.artemis.mode=native
spring.artemis.host=localhost
spring.artemis.port=61616
spring.artemis.user=admin
spring.artemis.password=admin

artemis.failover.hostName=localhost
artemis.failover.port=81818

Test Properties file:

spring.artemis.mode=embedded
spring.artemis.host=localhost
spring.artemis.port=61616
spring.artemis.user=admin
spring.artemis.password=admin

[snicoll]

So it looks like you've been able to set it up quite nicely in the end. We could add support for HA in Spring Boot itself but we haven't seen a lot of demand for it and your current code is quite concise. There is no test slice for JMS and therefore no easy way to "force"/replace the mode if it has been set by the user. If we consider adding one, this should be done in a separate issue.

[snicoll]

We discussed this one at the last meeting and decided we didn't want to extend our JMS support in that direction. The customizer does its job quite nicely anyway IMO.

Thanks for the suggestion in any case.

[kmandalas]

@snicoll IMO failover transport must be supported. The Failover Transport and Transport Options are pretty common options for real Production usage and expected to be supported according to https://activemq.apache.org/failover-transport-reference.html
In general Cluster support is an absolute must either master-slave or load-balanced clusters.

However, for simple ActiveMQ I think that
spring.activemq.broker-url= tcp://localhost:61616
can be replaced by:
spring.activemq.broker-url = failover:(tcp://localhost:61616,tcp://localhost:61617,tcp://localhost:61618,tcp://localhost:61619) .

For Artemis, isn't there this capability? I guess related also to Open Issue: #10739

[jbertram]

@kmandalas, yes Artemis has this capability as well, although the syntax and functionality is a bit different. You'd simply specify ha=true on the URL and the client would automatically receive topology updates from the broker about the nodes in the cluster. Of course, since you can't actually configure the URL that isn't an option. You already linked the corresponding issue for that.

[kmandalas]

@jbertram In the JavaDoc of ArtemisConfigurationCustomizer it says:

Callback interface that can be implemented by beans wishing to customize the Artemis JMS server Configuration before it is used by an auto-configured EmbeddedActiveMQ instance.

So this approach above, will it work with External Broker in Production environment? If an active Broker node becomes inactive and the passive takes over is this approach valid?

[jbertram]

@kmandalas unless I'm missing something I don't see how a callback for configuring an embedded broker will help with an external broker. The two use-cases are quite different.

[kmandalas]

@jbertram please check code snippet in #17623 (comment)

@Profile("!test")
@Configuration
public class ArtemisConfig implements ArtemisConfigurationCustomizer {
..
.
.
}

Notice the @Profile("!test") usage: seems like a workaround to be able to add artemis.failover.hostName and artemis.failover.port not for testing but for actual usage and in combination with auto-configuration to be able to cover testing as well.

My question is: does Spring support Artemis Cluster (HA/Failover)? With or without auto-configuration? Otherwise it practically cannot be used in any production use case.

[snicoll]

does Spring support Artemis Cluster (HA/Failover)? With or without auto-configuration?

@kmandalas you've already answered that question yourself and @jbertram gave you several hints already. If Artemis has the capability (it has), then there's nothing stopping you from configuring that for your Spring applications. Once #10739 is implemented, then you can specify the url using configuration only and let the auto-configuration drives things for you.

If you have more questions, please follow-up on StackOverflow, as mentioned in the guidelines for contributing, we prefer to use GitHub issues only for bugs and enhancements.