Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Multi-AZ support for RDS clusters #23684

Merged
merged 38 commits into from
Mar 16, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
38 commits
Select commit Hold shift + click to select a range
923fd86
rds: Fix some acceptance tests
YakDriver Mar 7, 2022
8a96c73
Add constants for instance classes
YakDriver Mar 8, 2022
8f43566
rds/cluster: Debugging
YakDriver Mar 8, 2022
5071312
rds/instance: Modernize tests
YakDriver Mar 8, 2022
c8c9186
rds/option_group_test: Make version dynamic
YakDriver Mar 8, 2022
3f97227
rds/ds/orderable: Make version dynamic
YakDriver Mar 8, 2022
6425207
rds/snapshot: Make version dynamic
YakDriver Mar 8, 2022
377d439
snapshot/tests: Dynamic version
YakDriver Mar 8, 2022
1f748cf
Additional test fixes
YakDriver Mar 8, 2022
b12939f
Continue skipping unfixed test
YakDriver Mar 8, 2022
d54c884
Flint and steel
YakDriver Mar 8, 2022
cb0944d
More flint
YakDriver Mar 8, 2022
c587562
Fix additional tests
YakDriver Mar 10, 2022
7a2dea7
rds: Fix additional issues
YakDriver Mar 11, 2022
a585794
Update changelog
YakDriver Mar 11, 2022
f158a60
Add session to client
YakDriver Mar 11, 2022
c7f22d7
Glint in your eye
YakDriver Mar 11, 2022
b0a7915
Remove bad comment
YakDriver Mar 11, 2022
d445f14
#23475 - Add `mysql` and `postgres` to validEngine func to support Mu…
bschaatsbergen Mar 6, 2022
3b3aecc
#23475 - Add required parameters: storage_type, allocated_storage, io…
bschaatsbergen Mar 6, 2022
7523b4d
Add acceptance tests for storage_type, allocated_storage, iops and db…
bschaatsbergen Mar 6, 2022
00a4655
#23475 - Add `mysql` and `postgres` to validEngine func to support Mu…
bschaatsbergen Mar 6, 2022
b9d67e0
#23475 - Add required parameters: storage_type, allocated_storage, io…
bschaatsbergen Mar 6, 2022
38d2cdd
Add acceptance tests for storage_type, allocated_storage, iops and db…
bschaatsbergen Mar 6, 2022
c274ff2
Create 23533.txt
bschaatsbergen Mar 15, 2022
cdca52c
Set iops to minimum value in acceptance tests
bschaatsbergen Mar 15, 2022
e868d0c
Merge branch 'add-multi-az-rds-engine-types' of https://github.com/bs…
bschaatsbergen Mar 15, 2022
d8de3ce
Update changelog txt
bschaatsbergen Mar 15, 2022
8de533a
Set `allocated_storage` to Computed as default response yields a `1` …
bschaatsbergen Mar 15, 2022
c259b4a
Touch up acceptance tests, get rid of AZs
bschaatsbergen Mar 15, 2022
cdb08cf
Add Multi-AZ RDS cluster documentation
bschaatsbergen Mar 15, 2022
c60f530
Update rds_cluster.html.markdown
bschaatsbergen Mar 15, 2022
2d7e1fe
Set expected iops to 1000
bschaatsbergen Mar 15, 2022
c5806a6
Lint Multi-AZ acceptance tests
bschaatsbergen Mar 15, 2022
0c53ba8
Point out which attributes are required to set for multi-az rds clust…
bschaatsbergen Mar 15, 2022
f4bc47f
r/aws_rds_cluster: Fix terrafmt errors.
ewbankkit Mar 16, 2022
7b652d2
Tweak CHANGELOG entry.
ewbankkit Mar 16, 2022
fd38a00
Fix terrafmt website errors.
ewbankkit Mar 16, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .changelog/23684.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
```release-note:enhancement
resource/aws_rds_cluster: Add `db_cluster_instance_class`, `allocated_storage`, `storage_type`, and `iops` arguments to support [Multi-AZ deployments for MySQL & PostgreSQL](https://aws.amazon.com/blogs/aws/amazon-rds-multi-az-db-cluster/)
```
63 changes: 63 additions & 0 deletions internal/service/rds/cluster.go
Original file line number Diff line number Diff line change
Expand Up @@ -157,6 +157,11 @@ func ResourceCluster() *schema.Resource {
Computed: true,
},

"db_cluster_instance_class": {
Type: schema.TypeString,
Optional: true,
},

"engine": {
Type: schema.TypeString,
Optional: true,
Expand Down Expand Up @@ -231,6 +236,23 @@ func ResourceCluster() *schema.Resource {
},
},

"allocated_storage": {
Type: schema.TypeInt,
Optional: true,
Computed: true,
},

"storage_type": {
Type: schema.TypeString,
Optional: true,
ForceNew: true,
},

"iops": {
Type: schema.TypeInt,
Optional: true,
},

"storage_encrypted": {
Type: schema.TypeBool,
Optional: true,
Expand Down Expand Up @@ -870,6 +892,10 @@ func resourceClusterCreate(d *schema.ResourceData, meta interface{}) error {
createOpts.DBClusterParameterGroupName = aws.String(attr.(string))
}

if attr, ok := d.GetOk("db_cluster_instance_class"); ok {
createOpts.DBClusterInstanceClass = aws.String(attr.(string))
}

if attr, ok := d.GetOk("engine_version"); ok {
createOpts.EngineVersion = aws.String(attr.(string))
}
Expand Down Expand Up @@ -922,6 +948,18 @@ func resourceClusterCreate(d *schema.ResourceData, meta interface{}) error {
createOpts.ReplicationSourceIdentifier = aws.String(attr.(string))
}

if attr, ok := d.GetOkExists("allocated_storage"); ok {
createOpts.AllocatedStorage = aws.Int64(int64(attr.(int)))
}

if attr, ok := d.GetOkExists("storage_type"); ok {
createOpts.StorageType = aws.String(attr.(string))
}

if attr, ok := d.GetOkExists("iops"); ok {
createOpts.Iops = aws.Int64(int64(attr.(int)))
}

if attr, ok := d.GetOkExists("storage_encrypted"); ok {
createOpts.StorageEncrypted = aws.Bool(attr.(bool))
}
Expand Down Expand Up @@ -1075,6 +1113,7 @@ func resourceClusterRead(d *schema.ResourceData, meta interface{}) error {
}

d.Set("endpoint", dbc.Endpoint)
d.Set("db_cluster_instance_class", dbc.DBClusterInstanceClass)
d.Set("engine_mode", dbc.EngineMode)
d.Set("engine", dbc.Engine)
d.Set("hosted_zone_id", dbc.HostedZoneId)
Expand Down Expand Up @@ -1102,7 +1141,11 @@ func resourceClusterRead(d *schema.ResourceData, meta interface{}) error {
return fmt.Errorf("error setting scaling_configuration: %s", err)
}

d.Set("allocated_storage", dbc.AllocatedStorage)
d.Set("storage_type", dbc.StorageType)
d.Set("iops", dbc.Iops)
d.Set("storage_encrypted", dbc.StorageEncrypted)

d.Set("enable_http_endpoint", dbc.HttpEndpointEnabled)

var vpcg []string
Expand Down Expand Up @@ -1181,6 +1224,11 @@ func resourceClusterUpdate(d *schema.ResourceData, meta interface{}) error {
requestUpdate = true
}

if d.HasChange("db_cluster_instance_class") {
req.EngineVersion = aws.String(d.Get("db_cluster_instance_class").(string))
requestUpdate = true
}

if d.HasChange("engine_version") {
req.EngineVersion = aws.String(d.Get("engine_version").(string))
requestUpdate = true
Expand All @@ -1200,6 +1248,21 @@ func resourceClusterUpdate(d *schema.ResourceData, meta interface{}) error {
requestUpdate = true
}

if d.HasChange("storage_type") {
req.StorageType = aws.String(d.Get("storage_type").(string))
requestUpdate = true
}

if d.HasChange("allocated_storage") {
req.AllocatedStorage = aws.Int64(int64(d.Get("allocated_storage").(int)))
requestUpdate = true
}

if d.HasChange("iops") {
req.Iops = aws.Int64(int64(d.Get("iops").(int)))
requestUpdate = true
}

if d.HasChange("preferred_backup_window") {
req.PreferredBackupWindow = aws.String(d.Get("preferred_backup_window").(string))
requestUpdate = true
Expand Down
156 changes: 156 additions & 0 deletions internal/service/rds/cluster_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -308,6 +308,94 @@ func TestAccRDSCluster_availabilityZones(t *testing.T) {
})
}

func TestAccRDSCluster_storageType(t *testing.T) {
var dbCluster rds.DBCluster
rName := sdkacctest.RandomWithPrefix(acctest.ResourcePrefix)
resourceName := "aws_rds_cluster.test"

resource.ParallelTest(t, resource.TestCase{
PreCheck: func() { acctest.PreCheck(t) },
ErrorCheck: acctest.ErrorCheck(t, rds.EndpointsID),
Providers: acctest.Providers,
CheckDestroy: testAccCheckClusterDestroy,
Steps: []resource.TestStep{
{
Config: testAccClusterConfig_StorageType(rName),
Check: resource.ComposeTestCheckFunc(
testAccCheckClusterExists(resourceName, &dbCluster),
resource.TestCheckResourceAttr(resourceName, "storage_type", "io1"),
),
},
},
})
}

func TestAccRDSCluster_allocatedStorage(t *testing.T) {
var dbCluster rds.DBCluster
rName := sdkacctest.RandomWithPrefix(acctest.ResourcePrefix)
resourceName := "aws_rds_cluster.test"

resource.ParallelTest(t, resource.TestCase{
PreCheck: func() { acctest.PreCheck(t) },
ErrorCheck: acctest.ErrorCheck(t, rds.EndpointsID),
Providers: acctest.Providers,
CheckDestroy: testAccCheckClusterDestroy,
Steps: []resource.TestStep{
{
Config: testAccClusterConfig_AllocatedStorage(rName),
Check: resource.ComposeTestCheckFunc(
testAccCheckClusterExists(resourceName, &dbCluster),
resource.TestCheckResourceAttr(resourceName, "allocated_storage", "100"),
),
},
},
})
}

func TestAccRDSCluster_iops(t *testing.T) {
var dbCluster rds.DBCluster
rName := sdkacctest.RandomWithPrefix(acctest.ResourcePrefix)
resourceName := "aws_rds_cluster.test"

resource.ParallelTest(t, resource.TestCase{
PreCheck: func() { acctest.PreCheck(t) },
ErrorCheck: acctest.ErrorCheck(t, rds.EndpointsID),
Providers: acctest.Providers,
CheckDestroy: testAccCheckClusterDestroy,
Steps: []resource.TestStep{
{
Config: testAccClusterConfig_Iops(rName),
Check: resource.ComposeTestCheckFunc(
testAccCheckClusterExists(resourceName, &dbCluster),
resource.TestCheckResourceAttr(resourceName, "iops", "1000"),
),
},
},
})
}

func TestAccRDSCluster_dbClusterInstanceClass(t *testing.T) {
var dbCluster rds.DBCluster
rName := sdkacctest.RandomWithPrefix(acctest.ResourcePrefix)
resourceName := "aws_rds_cluster.test"

resource.ParallelTest(t, resource.TestCase{
PreCheck: func() { acctest.PreCheck(t) },
ErrorCheck: acctest.ErrorCheck(t, rds.EndpointsID),
Providers: acctest.Providers,
CheckDestroy: testAccCheckClusterDestroy,
Steps: []resource.TestStep{
{
Config: testAccClusterConfig_DbClusterInstanceClass(rName),
Check: resource.ComposeTestCheckFunc(
testAccCheckClusterExists(resourceName, &dbCluster),
resource.TestCheckResourceAttr(resourceName, "db_cluster_instance_class", "db.r6gd.xlarge"),
),
},
},
})
}

func TestAccRDSCluster_backtrackWindow(t *testing.T) {
var dbCluster rds.DBCluster
resourceName := "aws_rds_cluster.test"
Expand Down Expand Up @@ -2251,6 +2339,74 @@ resource "aws_rds_cluster" "test" {
`, rName)
}

func testAccClusterConfig_StorageType(rName string) string {
return fmt.Sprintf(`
resource "aws_rds_cluster" "test" {
apply_immediately = true
cluster_identifier = %[1]q
db_cluster_instance_class = "db.r6gd.xlarge"
engine = "mysql"
storage_type = "io1"
allocated_storage = 100
iops = 1000
master_password = "mustbeeightcharaters"
master_username = "test"
skip_final_snapshot = true
}
`, rName)
}

func testAccClusterConfig_AllocatedStorage(rName string) string {
return fmt.Sprintf(`
resource "aws_rds_cluster" "test" {
apply_immediately = true
cluster_identifier = %[1]q
db_cluster_instance_class = "db.r6gd.xlarge"
engine = "mysql"
storage_type = "io1"
allocated_storage = 100
iops = 1000
master_password = "mustbeeightcharaters"
master_username = "test"
skip_final_snapshot = true
}
`, rName)
}

func testAccClusterConfig_Iops(rName string) string {
return fmt.Sprintf(`
resource "aws_rds_cluster" "test" {
apply_immediately = true
cluster_identifier = %[1]q
db_cluster_instance_class = "db.r6gd.xlarge"
engine = "mysql"
storage_type = "io1"
allocated_storage = 100
iops = 1000
master_password = "mustbeeightcharaters"
master_username = "test"
skip_final_snapshot = true
}
`, rName)
}

func testAccClusterConfig_DbClusterInstanceClass(rName string) string {
return fmt.Sprintf(`
resource "aws_rds_cluster" "test" {
apply_immediately = true
cluster_identifier = %[1]q
db_cluster_instance_class = "db.r6gd.xlarge"
engine = "mysql"
storage_type = "io1"
allocated_storage = 100
iops = 1000
master_password = "mustbeeightcharaters"
master_username = "test"
skip_final_snapshot = true
}
`, rName)
}

func testAccClusterConfig_BacktrackWindow(backtrackWindow int) string {
return fmt.Sprintf(`
resource "aws_rds_cluster" "test" {
Expand Down
2 changes: 2 additions & 0 deletions internal/service/rds/validate.go
Original file line number Diff line number Diff line change
Expand Up @@ -148,6 +148,8 @@ func validEngine() schema.SchemaValidateFunc {
"aurora",
"aurora-mysql",
"aurora-postgresql",
"postgres",
"mysql",
}, false)
}

Expand Down
26 changes: 25 additions & 1 deletion website/docs/r/rds_cluster.html.markdown
Original file line number Diff line number Diff line change
Expand Up @@ -95,6 +95,26 @@ resource "aws_rds_cluster" "example" {
}
```

### RDS Multi-AZ Cluster

-> More information about RDS Multi-AZ Clusters can be found in the [RDS User Guide](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/multi-az-db-clusters-concepts.html).

To create a Multi-AZ RDS cluster, you must additionally specify the `engine`, `storage_type`, `allocated_storage`, `iops` and `db_cluster_instance_class` attributes.

```terraform
resource "aws_rds_cluster" "example" {
cluster_identifier = "example"
availability_zones = ["us-west-2a", "us-west-2b", "us-west-2c"]
engine = "mysql"
db_cluster_instance_class = "db.r6gd.xlarge"
storage_type = "io1"
allocated_storage = 100
iops = 1000
master_username = "test"
master_password = "mustbeeightcharaters"
}
```

## Argument Reference

For more detailed documentation about each argument, refer to
Expand All @@ -120,9 +140,10 @@ The following arguments are supported:
* `deletion_protection` - (Optional) If the DB instance should have deletion protection enabled. The database can't be deleted when this value is set to `true`. The default is `false`.
* `enable_http_endpoint` - (Optional) Enable HTTP endpoint (data API). Only valid when `engine_mode` is set to `serverless`.
* `enabled_cloudwatch_logs_exports` - (Optional) Set of log types to export to cloudwatch. If omitted, no logs will be exported. The following log types are supported: `audit`, `error`, `general`, `slowquery`, `postgresql` (PostgreSQL).
* `engine` - (Optional) The name of the database engine to be used for this DB cluster. Defaults to `aurora`. Valid Values: `aurora`, `aurora-mysql`, `aurora-postgresql`
* `engine` - (Optional) The name of the database engine to be used for this DB cluster. Defaults to `aurora`. Valid Values: `aurora`, `aurora-mysql`, `aurora-postgresql`, `mysql`, `postgres`. (Note that `mysql` and `postgres` are Multi-AZ RDS clusters).
* `engine_mode` - (Optional) The database engine mode. Valid values: `global` (only valid for Aurora MySQL 1.21 and earlier), `multimaster`, `parallelquery`, `provisioned`, `serverless`. Defaults to: `provisioned`. See the [RDS User Guide](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/aurora-serverless.html) for limitations when using `serverless`.
* `engine_version` - (Optional) The database engine version. Updating this argument results in an outage. See the [Aurora MySQL](https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/AuroraMySQL.Updates.html) and [Aurora Postgres](https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/AuroraPostgreSQL.Updates.html) documentation for your configured engine to determine this value. For example with Aurora MySQL 2, a potential value for this argument is `5.7.mysql_aurora.2.03.2`. The value can contain a partial version where supported by the API. The actual engine version used is returned in the attribute `engine_version_actual`, [defined below](#engine_version_actual).
* `db_cluster_instance_class` - (Optional) The compute and memory capacity of each DB instance in the Multi-AZ DB cluster, for example db.m6g.xlarge. Not all DB instance classes are available in all AWS Regions, or for all database engines. For the full list of DB instance classes and availability for your engine, see [DB instance class](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/Concepts.DBInstanceClass.html) in the Amazon RDS User Guide. (This setting is required to create a Multi-AZ DB cluster).
* `final_snapshot_identifier` - (Optional) The name of your final DB snapshot when this DB cluster is deleted. If omitted, no final snapshot will be made.
* `global_cluster_identifier` - (Optional) The global cluster identifier specified on [`aws_rds_global_cluster`](/docs/providers/aws/r/rds_global_cluster.html).
* `enable_global_write_forwarding` - (Optional) Whether cluster should forward writes to an associated global cluster. Applied to secondary clusters to enable them to forward writes to an [`aws_rds_global_cluster`](/docs/providers/aws/r/rds_global_cluster.html)'s primary cluster. See the [Aurora Userguide documentation](https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/aurora-global-database-write-forwarding.html) for more information.
Expand All @@ -140,6 +161,9 @@ The following arguments are supported:
* `skip_final_snapshot` - (Optional) Determines whether a final DB snapshot is created before the DB cluster is deleted. If true is specified, no DB snapshot is created. If false is specified, a DB snapshot is created before the DB cluster is deleted, using the value from `final_snapshot_identifier`. Default is `false`.
* `snapshot_identifier` - (Optional) Specifies whether or not to create this cluster from a snapshot. You can use either the name or ARN when specifying a DB cluster snapshot, or the ARN when specifying a DB snapshot.
* `source_region` - (Optional) The source region for an encrypted replica DB cluster.
* `allocated_storage` - (Optional) The amount of storage in gibibytes (GiB) to allocate to each DB instance in the Multi-AZ DB cluster. (This setting is required to create a Multi-AZ DB cluster).
* `storage_type` - (Optional) Specifies the storage type to be associated with the DB cluster. (This setting is required to create a Multi-AZ DB cluster). Valid values: `io1`, Default: `io1`.
* `iops` - (Optional) The amount of Provisioned IOPS (input/output operations per second) to be initially allocated for each DB instance in the Multi-AZ DB cluster. For information about valid Iops values, see [Amazon RDS Provisioned IOPS storage to improve performance](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/CHAP_Storage.html#USER_PIOPS) in the Amazon RDS User Guide. (This setting is required to create a Multi-AZ DB cluster). Must be a multiple between .5 and 50 of the storage amount for the DB cluster.
* `storage_encrypted` - (Optional) Specifies whether the DB cluster is encrypted. The default is `false` for `provisioned` `engine_mode` and `true` for `serverless` `engine_mode`. When restoring an unencrypted `snapshot_identifier`, the `kms_key_id` argument must be provided to encrypt the restored cluster. Terraform will only perform drift detection if a configuration value is provided.
* `tags` - (Optional) A map of tags to assign to the DB cluster. If configured with a provider [`default_tags` configuration block](/docs/providers/aws/index.html#default_tags-configuration-block) present, tags with matching keys will overwrite those defined at the provider-level.
* `vpc_security_group_ids` - (Optional) List of VPC security groups to associate with the Cluster
Expand Down