provider/aws: Add AWS RDS Read Replica

[catsby]

My proposal for adding support to for read replicas to resource_aws_db, by way of adding a replicate_source_db attribute, referencing the identifier of the source database.

Caveats:

• Replicas inherit many of their attributes from the primary database (username, password, engine). Many of these are required when creating a database instance, but not when creating a replica. The trouble there is if being a replica is just a flag on a resource, that resource still needs those required attributes, even if they're ignored (inherited) from the source. Until Terraform has a ValidateFunc or RequiredIf concept(s), we're stuck repeating ourselves.
• Replicas can be promoted to full, standalone databases. Terraform can support this by sending a PromoteReadReplica API call when the replicate_source_db attribute is removed in the configuration. This may be a surprise to some users, since we can't notify them that "removing this attribute makes a full read/write database" during plan.
• If you use the example config below and let TF do the inheritance of attributes, you may get in a weird state when you promote. Manual managing of the config file will be required if you need to separate the two database more.

Example config:

resource "aws_db_instance" "primary" {
  identifier = "tf-primary-db"
  allocated_storage = 5
  engine = "mysql"
  engine_version = "5.6.19b"
  instance_class = "db.t1.micro"
  name = "baz"
  password = "fofofofxx"
  username = "foo"
  multi_az = false
  apply_immediately = true
  backup_retention_period = 1
  tags {
    Name = "tf-primary-db"
  }
  parameter_group_name = "default.mysql5.6"
}

resource "aws_db_instance" "replica" {
  identifier = "tf-replica-db"
  replicate_source_db = "${aws_db_instance.primary.identifier}"
  allocated_storage = "${aws_db_instance.primary.allocated_storage}"
  engine = "${aws_db_instance.primary.engine}"
  engine_version = "${aws_db_instance.primary.engine_version}"
  instance_class = "${aws_db_instance.primary.instance_class}"
  password = "${aws_db_instance.primary.password}"
  username = "${aws_db_instance.primary.username}"
  tags {
    Name = "tf-replica-db"
  }
}

In practice, users may want to manually duplicate all those fields. In the implementation, we'll have to ignore most of them that don't apply.

The fields that are applicable to replica creation:

AutoMinorVersionUpgrade *bool 
AvailabilityZone *string
DBInstanceClass *string
DBInstanceIdentifier *string
DBSubnetGroupName *string
IOPS *int64
OptionGroupName *string
Port *int64
PubliclyAccessible *bool
SourceDBInstanceIdentifier *string
StorageType *string
Tags []*Tag

Relevant docs:

• DB Instance Replication
• Working with PostgreSQL and MySQL Read Replicas
• API Docs for create

Things:

• Docs (mostly)
• Example config
• Implementation
• Acceptance tests

I welcome thoughts on this approach

[catsby]

Implementation is (mostly) done. You can create, destroy, and promote a replica. To promote, simply remove the replicate_source_db attribute in the config, that will send a PromoteReadReplica API call and sever the replication link.

The acceptance test is incomplete, but I wanted to see if anyone had feedback on implementation / usage.

[catsby]

cc @radeksimko, @ryanking as at least 2 people who I know requested this in some fashion 😄

[ryanking]

Interface lgtm.

[radeksimko]

Overall looks good, I have not been looking at the API docs yet, but since we gonna be able to build a M-S-S RDS cluster, it would be nice to test typical scenarios, like engine upgrade.

I know that RDS does some clever things like "upgrade each slave separately first", then upgrade master, while promoting one of the slaves to master = no downtime caused. I'm not sure if all this happens under the hood automatically, I haven't tried that yet, but it's a typical scenario that should work well.

[catsby]

I know that RDS does some clever things like "upgrade each slave separately first", then upgrade master, while promoting one of the slaves to master = no downtime caused.

Wow, is this process documented somewhere? I didn't read the docs entirely so I may have just missed it. Seems like promoting a follower would be a bad thing, since it has different address. Unless you're talking about their HA setup (multi_az), which is a bit different; they'll swap out DNS internally in that case.

What are you thinking I need to do here to accommodate?

[radeksimko]

What are you thinking I need to do here to accommodate?

Like I mentioned, I did not actually experienced this, I've just read it somewhere, can't remember where, sorry...

What I was rather trying to suggest is to battle test this in situations, when you upgrade certain things. Just make sure that the most basic expectable upgrade paths are working.

[catsby]

Rebased on master and added an acceptance test.
I apologize in advance for time spent running that acceptance test... creating primary, backup primary, create replica, test things, then destroy primary, destroy replica ... it takes a long time.

@phinze please review when you get a chance.

[radeksimko]

Oops, typo here - relicate_source => replicate_source

[radeksimko]

Overall LGTM. 👍

make testacc TEST=github.com/hashicorp/terraform/builtin/providers/aws TESTARGS='-run=DBInstance' 2>/dev/null
go generate ./...
TF_ACC=1 go test github.com/hashicorp/terraform/builtin/providers/aws -v -run=DBInstance -timeout 45m
=== RUN TestAccAWSDBInstance
--- PASS: TestAccAWSDBInstance (618.04s)
=== RUN TestAccAWSDBInstanceReplica
--- PASS: TestAccAWSDBInstanceReplica (1620.66s)
PASS
ok      github.com/hashicorp/terraform/builtin/providers/aws    2238.710s

yet I haven't actually tried building a replicated RDS outside of the test suite.

This resource should make decent amount of people happier. 😄

[phinze]

It's replicate_source_db in code and replicate_source here.

[phinze]

One wee docs thing, but otherwise LGTM!

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.