RDS Cluster Support

[maciejwalkowiak]

From spring-cloud-aws created by davidsloan-productlab: spring-attic/spring-cloud-aws#356

Instead of an RDS instance I'm trying to use an RDS Aurora cluster.

I can use an instance within the cluster using
@EnableRdsInstance

However when trying to use the cluster itself I see an exception with this as the cause:
No database instance with id:'my-cluster-name' found. Please specify a valid db instance

Looking into the code, I can see that in AmazonRdsDataSourceFactoryBean, only instances are queried using AmazonRDS.describeDBInstances

It would be good to either amend this to support a cluster, or introduce a new annotation
@EnableRdsCluster

What do you think the chances are for RDS cluster support in Spring Cloud AWS?

[maciejwalkowiak]

Much needed one..

People are now going for serverless RDS, so cluster connection establishment is really required.

[maciejwalkowiak]

Vlad's post is interesting and certainly helps for day-to-day running. But it's of little help to Spring projects that are looking to reap the benefits of Aurora clusters for failover. The current provision of a retry interceptor isn't really good enough. Services should be able to connect to a cluster, the SDK should be aware of which instances are write and read-only, and should direct requests accordingly.

Roll on 3.0.0.M1. Hopefully that addresses the failover issue.

[maciejwalkowiak]

@aravindparappil46 currently you have to do it manually - look at Vlad's post for inspiration: https://vladmihalcea.com/read-write-read-only-transaction-routing-spring/

[maciejwalkowiak]

This has also been reported here: This issue has also been reported here: spring-attic/spring-cloud-aws#302

[maciejwalkowiak]

I've done it for now using reader/writer datasources + AOP .
basic idea is here: https://fable.sh/blog/splitting-read-and-write-operations-in-spring-boot/

Looking forward to 3.0.0.M1 milestone!

[maciejwalkowiak]

Do we know if this is on the road map at all?

[maciejwalkowiak]

Looking forward to the 3.0.0.M1 milestone!
Definitely need this feature to be able to connect to an Aurora RDS Cluster (through annotation and through the application.properties file) for Spring & Spring Boot projects.

Meanwhile, if anyone knows a workaround for connecting to an Aurora cluster in Spring Boot and enabling read replica support, please post here!

[maciejwalkowiak]

unable to connect to an Aurora RDS Cluster with read replica (through annotation and through the application.properties file) for spring boot project using spring cloud
Any news Please !

[dlamblin]

This is unexpected and leads an RDS Aurora MySQL user to consider just preferring a jdbc url to the cluster, which works normally, finds the writeable instance and handles the read/write failover afaiks; unlike the specific RDS support provided here.

[BerndFarkaDyna]

@maciejwalkowiak:

is there any update on this issue:

for me currently it looks like this is severe blocker using RDS in combination with spring boot:

the functionality to utilize the read replicas is simple broken in combination with spring-cloud-aws-jdbc?

or am I missing something?

[Lucas3oo]

Also most people want to select the connection pool themselves. Spring cloud AWS kind of hard code it to Tomcat's connection pool.

[maciejwalkowiak]

@dlamblin for the failover scenario, I would also recommend just using plain JDBC url. Read replica support is the interesting feature of Spring Cloud AWS.

@BerndFarkaDyna read replicas do work, just not with Aurora.

@Lucas3oo very much agreed.

[fletchgqc]

I'm trying to switch from RDS Postgresql to Aurora Serverless and I found that it doesn't work with Spring Cloud AWS.

Am I correct in surmising that the reason it doesn't work is that Aurora Serverless only provides a cluster (instead of instance) URL? And therefore, in order to use Aurora Serverless, this issue needs to be fixed?

If so, I think that as a very first step, this should be addressed in the documentation. Perhaps where it says "The minimum configuration parameters are a unique id for the data source, a valid db-instance-identifier attribute that points to a valid Amazon RDS database instance." we should add "Aurora Serverless is currently not supported, because it provides a cluster URL and Spring Cloud AWS does not yet support cluster endpoints."

[maciejwalkowiak]

@fletchgqc both traditional Aurora and Aurora Serverless aren't supported.

[fletchgqc]

@maciejwalkowiak Ahh OK, thanks for the info. Maybe that should be mentioned in the docs? Because Aurora is a part of RDS, and RDS is partly supported.

[maciejwalkowiak]

@fletchgqc you're right #172

[Lucas3oo]

To add some info in general: You can setup Aurora as a cluster with only one instance in it. Then the "endpoint" will be the "writer" endpoint, But the specific "reader" endpoint will in fact be the same hostname in case of a one instance cluster.
So it is perfectly OK to use an Aurora cluster like that from Spring. But Since the failover is done using DNS names you should set the DNS cache TTL in JRE to absolute minimum. Default is cache forever.

[sr-nu]

Yes having seamless Reader Writer support for Aurora similar to RDS is really important. Please prioritize or provide a workaround.

[nickcaballero]

This kind of works out of the box if you use the MariaDB driver in Aurora mode. When using a readOnly transaction, the driver will choose one of the reader connections instead of using the writer.

https://mariadb.com/kb/en/failover-and-high-availability-with-mariadb-connector-j/#masterslave-connection-selection

[sr-nu]

Not working for Postgres as expected. All queries directed to Writer.

…

On Fri, 1 Oct 2021, 01:45 Nick Caballero, ***@***.***> wrote: This kind of works out of the box if you use the MariaDB driver in Aurora mode. When using a readOnly transaction, the driver will choose one of the reader connections instead of using the writer. https://mariadb.com/kb/en/failover-and-high-availability-with-mariadb-connector-j/#masterslave-connection-selection — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#50 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAA75PXZIVPCZLFNAG65P2LUESO2DANCNFSM4XLLFDNA> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.

[hemju]

+1 to the feature request

[maciejwalkowiak]

Replaced by #322