Configuration property for Cassandra's control queries timeout not working - PT2S error on Startup.

[rajeevshukla]

Caused by: com.datastax.oss.driver.api.core.DriverTimeoutException: query 'SELECT * FROM system_schema.columns' timed out after PT2S
	at com.datastax.oss.driver.api.core.DriverTimeoutException.copy(DriverTimeoutException.java:34) ~[java-driver-core-4.10.0.jar:na]
	at com.datastax.oss.driver.internal.core.util.concurrent.CompletableFutures.getUninterruptibly(CompletableFutures.java:149) ~[java-driver-core-4.10.0.jar:na]
	at com.datastax.oss.driver.api.core.session.Session.refreshSchema(Session.java:140) ~[java-driver-core-4.10.0.jar:na]
	at org.springframework.data.cassandra.config.CqlSessionFactoryBean.afterPropertiesSet(CqlSessionFactoryBean.java:460) ~[spring-data-cassandra-3.2.0-M5.jar:3.2.0-M5]
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeInitMethods(AbstractAutowireCapableBeanFactory.java:1845) ~[spring-beans-5.3.5.jar:5.3.5]
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1782) ~[spring-beans-5.3.5.jar:5.3.5]

I am using spring boot 2.5.0.M3.

Below are my properties -

spring.data.cassandra.contact-points=xxxx
spring.data.cassandra.username=xxx
spring.data.cassandra.password=xxx
spring.data.cassandra.keyspace-name=xxxx
spring.data.cassandra.port=9042
spring.data.cassandra.schema-action=NONE
spring.data.cassandra.local-datacenter=xxx
spring.data.cassandra.connection.connect-timeout=PT10S
spring.data.cassandra.connection.init-query-timeout=PT20S
spring.data.cassandra.request.timeout=PT10S
spring.data.cassandra.controlconnection.timeout=PT10S

What I have seen that these properties are not getting exposed to the DataStax driver. Tried everything to expose it using programmatically but that is not working.

Below is my pom.xml

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
	xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
	xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
	<modelVersion>4.0.0</modelVersion>
	<parent>
		<groupId>org.springframework.boot</groupId>
		<artifactId>spring-boot-starter-parent</artifactId>
		<version>2.5.0-M3</version>
		<relativePath /> <!-- lookup parent from repository -->
	</parent>
	<groupId>com.example</groupId>
	<artifactId>spring-cassandra-testing</artifactId>
	<version>0.0.1-SNAPSHOT</version>
	<name>spring-cassandra-testing</name>
	<description>Demo project for Spring Boot</description>
	<properties>
		<java.version>1.8</java.version>
	</properties>
	<dependencies>
		<dependency>
			<groupId>org.springframework.boot</groupId>
			<artifactId>spring-boot-starter-data-cassandra</artifactId>
		</dependency>

		<dependency>
			<groupId>org.springframework.boot</groupId>
			<artifactId>spring-boot-starter-test</artifactId>
			<scope>test</scope>
		</dependency>
	</dependencies>

	<build>
		<plugins>
			<plugin>
				<groupId>org.springframework.boot</groupId>
				<artifactId>spring-boot-maven-plugin</artifactId>
			</plugin>
		</plugins>
	</build>
	<repositories>
		<repository>
			<id>spring-milestones</id>
			<name>Spring Milestones</name>
			<url>https://repo.spring.io/milestone</url>
		</repository>
	</repositories>
	<pluginRepositories>
		<pluginRepository>
			<id>spring-milestones</id>
			<name>Spring Milestones</name>
			<url>https://repo.spring.io/milestone</url>
		</pluginRepository>
	</pluginRepositories>

</project>

As per the issue raised here this issue should be fixed in 2.5.0.M1.

Complete error :

 .   ____          _            __ _ _
 /\\ / ___'_ __ _ _(_)_ __  __ _ \ \ \ \
( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \
 \\/  ___)| |_)| | | | | || (_| |  ) ) ) )
  '  |____| .__|_| |_|_| |_\__, | / / / /
 =========|_|==============|___/=/_/_/_/
 :: Spring Boot ::             (v2.5.0-M3)

2021-04-26 15:29:44.882  INFO 17796 --- [           main] c.e.d.SpringCassandraTestingApplication  : Starting SpringCassandraTestingApplication using Java 1.8.0_275 on W17127605 with PID 17796 (C:\Users\xxx\xxxx-workspace\spring-cassandra-testing\target\classes started by ek678013 in C:\Users\ek678013\xxxx-workspace\spring-cassandra-testing)
2021-04-26 15:29:44.886  INFO 17796 --- [           main] c.e.d.SpringCassandraTestingApplication  : No active profile set, falling back to default profiles: default
2021-04-26 15:29:45.320  INFO 17796 --- [           main] .s.d.r.c.RepositoryConfigurationDelegate : Bootstrapping Spring Data Cassandra repositories in DEFAULT mode.
2021-04-26 15:29:45.338  INFO 17796 --- [           main] .s.d.r.c.RepositoryConfigurationDelegate : Finished Spring Data repository scanning in 10 ms. Found 0 Cassandra repository interfaces.
2021-04-26 15:29:45.456  INFO 17796 --- [           main] .s.d.r.c.RepositoryConfigurationDelegate : Bootstrapping Spring Data Reactive Cassandra repositories in DEFAULT mode.
2021-04-26 15:29:45.459  INFO 17796 --- [           main] .s.d.r.c.RepositoryConfigurationDelegate : Finished Spring Data repository scanning in 2 ms. Found 0 Reactive Cassandra repository interfaces.
2021-04-26 15:29:45.464  INFO 17796 --- [           main] .s.d.r.c.RepositoryConfigurationDelegate : Bootstrapping Spring Data Cassandra repositories in DEFAULT mode.
2021-04-26 15:29:45.467  INFO 17796 --- [           main] .s.d.r.c.RepositoryConfigurationDelegate : Finished Spring Data repository scanning in 2 ms. Found 0 Cassandra repository interfaces.
Driving confg loader Builder customizer....
inside session Builder configurer..
Loading Custom resource...
2021-04-26 15:29:46.095  INFO 17796 --- [           main] c.d.o.d.i.core.DefaultMavenCoordinates   : DataStax Java driver for Apache Cassandra(R) (com.datastax.oss:java-driver-core) version 4.10.0
2021-04-26 15:29:46.247  INFO 17796 --- [           main] c.d.o.d.i.c.c.InternalDriverContext      : Could not register Graph extensions; this is normal if Tinkerpop was explicitly excluded from classpath
2021-04-26 15:29:46.627  INFO 17796 --- [     s0-admin-0] c.d.oss.driver.internal.core.time.Clock  : Using native clock for microsecond precision
2021-04-26 15:29:51.511  WARN 17796 --- [        s0-io-2] c.d.o.d.i.core.session.DefaultSession    : [s0] Unexpected error while refreshing schema during initialization, keeping previous version (CompletionException: com.datastax.oss.driver.api.core.DriverTimeoutException: query 'SELECT * FROM system_schema.columns' timed out after PT2S)
2021-04-26 15:29:52.181  INFO 17796 --- [           main] c.d.o.d.i.c.c.InternalDriverContext      : Could not register Graph extensions; this is normal if Tinkerpop was explicitly excluded from classpath
2021-04-26 15:29:52.183  INFO 17796 --- [     s1-admin-0] c.d.oss.driver.internal.core.time.Clock  : Using native clock for microsecond precision
2021-04-26 15:29:55.936  WARN 17796 --- [        s1-io-2] c.d.o.d.i.core.session.DefaultSession    : [s1] Unexpected error while refreshing schema during initialization, keeping previous version (CompletionException: com.datastax.oss.driver.api.core.DriverTimeoutException: query 'SELECT * FROM system_schema.columns' timed out after PT2S)
2021-04-26 15:30:13.279  WARN 17796 --- [           main] s.c.a.AnnotationConfigApplicationContext : Exception encountered during context initialization - cancelling refresh attempt: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'cassandraConverter' defined in class path resource [com/example/demo/CassandraConfig.class]: Bean instantiation via factory method failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [org.springframework.data.cassandra.core.convert.CassandraConverter]: Factory method 'cassandraConverter' threw exception; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'cassandraSession' defined in class path resource [com/example/demo/CassandraConfig.class]: Invocation of init method failed; nested exception is com.datastax.oss.driver.api.core.DriverTimeoutException: query 'SELECT * FROM system_schema.tables' timed out after PT2S
2021-04-26 15:30:13.303  INFO 17796 --- [           main] ConditionEvaluationReportLoggingListener : 

Error starting ApplicationContext. To display the conditions report re-run your application with 'debug' enabled.
2021-04-26 15:30:13.339 ERROR 17796 --- [           main] o.s.boot.SpringApplication               : Application run failed

org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'cassandraConverter' defined in class path resource [com/example/demo/CassandraConfig.class]: Bean instantiation via factory method failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [org.springframework.data.cassandra.core.convert.CassandraConverter]: Factory method 'cassandraConverter' threw exception; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'cassandraSession' defined in class path resource [com/example/demo/CassandraConfig.class]: Invocation of init method failed; nested exception is com.datastax.oss.driver.api.core.DriverTimeoutException: query 'SELECT * FROM system_schema.tables' timed out after PT2S
	at org.springframework.beans.factory.support.ConstructorResolver.instantiate(ConstructorResolver.java:658) ~[spring-beans-5.3.5.jar:5.3.5]
	at org.springframework.beans.factory.support.ConstructorResolver.instantiateUsingFactoryMethod(ConstructorResolver.java:486) ~[spring-beans-5.3.5.jar:5.3.5]
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.instantiateUsingFactoryMethod(AbstractAutowireCapableBeanFactory.java:1334) ~[spring-beans-5.3.5.jar:5.3.5]
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:1177) ~[spring-beans-5.3.5.jar:5.3.5]
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:564) ~[spring-beans-5.3.5.jar:5.3.5]
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:524) ~[spring-beans-5.3.5.jar:5.3.5]
	at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:335) ~[spring-beans-5.3.5.jar:5.3.5]
	at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:234) ~[spring-beans-5.3.5.jar:5.3.5]
	at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:333) ~[spring-beans-5.3.5.jar:5.3.5]
	at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:208) ~[spring-beans-5.3.5.jar:5.3.5]
	at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:944) ~[spring-beans-5.3.5.jar:5.3.5]
	at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:918) ~[spring-context-5.3.5.jar:5.3.5]
	at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:583) ~[spring-context-5.3.5.jar:5.3.5]
	at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:761) [spring-boot-2.5.0-M3.jar:2.5.0-M3]
	at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:430) [spring-boot-2.5.0-M3.jar:2.5.0-M3]
	at org.springframework.boot.SpringApplication.run(SpringApplication.java:328) [spring-boot-2.5.0-M3.jar:2.5.0-M3]
	at org.springframework.boot.SpringApplication.run(SpringApplication.java:1325) [spring-boot-2.5.0-M3.jar:2.5.0-M3]
	at org.springframework.boot.SpringApplication.run(SpringApplication.java:1314) [spring-boot-2.5.0-M3.jar:2.5.0-M3]
	at com.example.demo.SpringCassandraTestingApplication.main(SpringCassandraTestingApplication.java:11) [classes/:na]
Caused by: org.springframework.beans.BeanInstantiationException: Failed to instantiate [org.springframework.data.cassandra.core.convert.CassandraConverter]: Factory method 'cassandraConverter' threw exception; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'cassandraSession' defined in class path resource [com/example/demo/CassandraConfig.class]: Invocation of init method failed; nested exception is com.datastax.oss.driver.api.core.DriverTimeoutException: query 'SELECT * FROM system_schema.tables' timed out after PT2S
	at org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:185) ~[spring-beans-5.3.5.jar:5.3.5]
	at org.springframework.beans.factory.support.ConstructorResolver.instantiate(ConstructorResolver.java:653) ~[spring-beans-5.3.5.jar:5.3.5]
	... 18 common frames omitted
Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'cassandraSession' defined in class path resource [com/example/demo/CassandraConfig.class]: Invocation of init method failed; nested exception is com.datastax.oss.driver.api.core.DriverTimeoutException: query 'SELECT * FROM system_schema.tables' timed out after PT2S
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1786) ~[spring-beans-5.3.5.jar:5.3.5]
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:602) ~[spring-beans-5.3.5.jar:5.3.5]
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:524) ~[spring-beans-5.3.5.jar:5.3.5]
	at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:335) ~[spring-beans-5.3.5.jar:5.3.5]
	at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:234) ~[spring-beans-5.3.5.jar:5.3.5]
	at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:333) ~[spring-beans-5.3.5.jar:5.3.5]
	at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:233) ~[spring-beans-5.3.5.jar:5.3.5]
	at org.springframework.beans.factory.support.DefaultListableBeanFactory.resolveNamedBean(DefaultListableBeanFactory.java:1273) ~[spring-beans-5.3.5.jar:5.3.5]
	at org.springframework.beans.factory.support.DefaultListableBeanFactory.resolveNamedBean(DefaultListableBeanFactory.java:1234) ~[spring-beans-5.3.5.jar:5.3.5]
	at org.springframework.beans.factory.support.DefaultListableBeanFactory.resolveBean(DefaultListableBeanFactory.java:494) ~[spring-beans-5.3.5.jar:5.3.5]
	at org.springframework.beans.factory.support.DefaultListableBeanFactory.getBean(DefaultListableBeanFactory.java:349) ~[spring-beans-5.3.5.jar:5.3.5]
	at org.springframework.beans.factory.support.DefaultListableBeanFactory.getBean(DefaultListableBeanFactory.java:342) ~[spring-beans-5.3.5.jar:5.3.5]
	at org.springframework.data.cassandra.config.AbstractSessionConfiguration.requireBeanOfType(AbstractSessionConfiguration.java:103) ~[spring-data-cassandra-3.2.0-M5.jar:3.2.0-M5]
	at org.springframework.data.cassandra.config.AbstractSessionConfiguration.getRequiredSession(AbstractSessionConfiguration.java:204) ~[spring-data-cassandra-3.2.0-M5.jar:3.2.0-M5]
	at org.springframework.data.cassandra.config.AbstractCassandraConfiguration.cassandraConverter(AbstractCassandraConfiguration.java:73) ~[spring-data-cassandra-3.2.0-M5.jar:3.2.0-M5]
	at com.example.demo.CassandraConfig$$EnhancerBySpringCGLIB$$e63abe7b.CGLIB$cassandraConverter$18(<generated>) ~[classes/:na]
	at com.example.demo.CassandraConfig$$EnhancerBySpringCGLIB$$e63abe7b$$FastClassBySpringCGLIB$$c4eb5fb6.invoke(<generated>) ~[classes/:na]
	at org.springframework.cglib.proxy.MethodProxy.invokeSuper(MethodProxy.java:244) ~[spring-core-5.3.5.jar:5.3.5]
	at org.springframework.context.annotation.ConfigurationClassEnhancer$BeanMethodInterceptor.intercept(ConfigurationClassEnhancer.java:331) ~[spring-context-5.3.5.jar:5.3.5]
	at com.example.demo.CassandraConfig$$EnhancerBySpringCGLIB$$e63abe7b.cassandraConverter(<generated>) ~[classes/:na]
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_275]
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_275]
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_275]
	at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_275]
	at org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:154) ~[spring-beans-5.3.5.jar:5.3.5]
	... 19 common frames omitted
Caused by: com.datastax.oss.driver.api.core.DriverTimeoutException: query 'SELECT * FROM system_schema.tables' timed out after PT2S
	at com.datastax.oss.driver.api.core.DriverTimeoutException.copy(DriverTimeoutException.java:34) ~[java-driver-core-4.10.0.jar:na]
	at com.datastax.oss.driver.internal.core.util.concurrent.CompletableFutures.getUninterruptibly(CompletableFutures.java:149) ~[java-driver-core-4.10.0.jar:na]
	at com.datastax.oss.driver.api.core.session.Session.refreshSchema(Session.java:140) ~[java-driver-core-4.10.0.jar:na]
	at org.springframework.data.cassandra.config.CqlSessionFactoryBean.afterPropertiesSet(CqlSessionFactoryBean.java:460) ~[spring-data-cassandra-3.2.0-M5.jar:3.2.0-M5]
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeInitMethods(AbstractAutowireCapableBeanFactory.java:1845) ~[spring-beans-5.3.5.jar:5.3.5]
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1782) ~[spring-beans-5.3.5.jar:5.3.5]
	... 43 common frames omitted

Thanks,

[wilkinsona]

There are already a number of people trying to help you on Stack Overflow. To avoid duplicating effort and wasting people’s time, I’m going to close this one for now. We can re-open it if the Stack Overflow question identifies a bug.

I think it would be worth updating your sample that you’ve linked to in your question so that it’s only trying to configure Cassandra in one way. It would also be useful if it used something like Testcontainers to make Cassandra available as the sample does not work in its current form.

[Aelentel]

i'm sorry to barge in, but that issue is still relevant as in Spring Cassandra 3.2.2
the timeout settings are still not honored even if they exist.
namely "spring.data.cassandra.request.timeout"

it's a bit cumbersome to control request timeout when you are developping from a slow VPN and that no query execute under 2s.

the current solution i have is using System.setProperty("datastax-java-driver.basic.request.timeout", "5 seconds"); at startup time.
however this should be available and working from spring configuration phase and not by using the driver own configuration mechanism.

[wilkinsona]

@Aelentel This issue was about the control query timeout and we never established what, if anything, the problem may have been. spring.data.cassandra.request.timeout is a separate property that maps to a separate configuration setting in Cassandra. We rely on it in our own tests so we're as certain as we can be that it works. Perhaps you have defined a Cassandra-related bean that has caused the auto-configuration to back off. Unfortunately, it's impossible to tell from what you have described thus far.

If you would like us to spend some more time investigating, please open a new issue and spend some time providing a complete yet minimal sample that reproduces the problem. You can share it with us by pushing it to a separate repository on GitHub or by zipping it up and attaching it to the new issue.

[Aelentel]

@wilkinsona thanks for the answer.
i think you nailed it by "Perhaps you have defined a Cassandra-related bean that has caused the auto-configuration to back off." time to review the code and fix my own mess, sorry about that.

[yevsh]

@Aelentel

@wilkinsona thanks for the answer. i think you nailed it by "Perhaps you have defined a Cassandra-related bean that has caused the auto-configuration to back off." time to review the code and fix my own mess, sorry about that.

Did you find the issue? I have same problem

in my case, i see that setter setTimeout is called

in
org.springframework.boot.autoconfigure.cassandra.CassandraProperties.Request

but doesn't affect.
(only -Ddatastax-java-driver.basic.request.timeout works)

[Aelentel]

@yevsh Yes that was indeed the fact that i defined a Cassandra Bean so basically most of the initialization was done by the Bean i defined and not by Spring Boot autoconfigure mechanism.

what we did in the end was this, note that the timeout values are in this example coded directly, please don't do that :)

public class CassandraConfiguration extends AbstractCassandraConfiguration
{
 // snip of most of the overrides
 	@Override
	protected DriverConfigLoaderBuilderConfigurer getDriverConfigLoaderBuilderConfigurer()
	{
		return config ->
			config
				.withString(DefaultDriverOption.METADATA_SCHEMA_REQUEST_TIMEOUT, "30s")
				.withString(DefaultDriverOption.CONTROL_CONNECTION_TIMEOUT, "10s")
				.withString(DefaultDriverOption.REQUEST_TIMEOUT, "30s")
				.build();
	}
}

[panoet]

@Aelentel

@wilkinsona thanks for the answer. i think you nailed it by "Perhaps you have defined a Cassandra-related bean that has caused the auto-configuration to back off." time to review the code and fix my own mess, sorry about that.

Did you find the issue? I have same problem

in my case, i see that setter setTimeout is called

in org.springframework.boot.autoconfigure.cassandra.CassandraProperties.Request

but doesn't affect. (only -Ddatastax-java-driver.basic.request.timeout works)

-Ddatastax-java-driver.basic.request.timeout worked for me

[FrommyMind]

Spring boot 3.4.2
spring-data-cassandra 4.4.3

spring.cassandra.request.timeout: 10s works for me.