[Bug]: Cannot create Global Aurora (MySQL) cluster based upon snapshot

[rosstimson]

Terraform Core Version

1.3.7

AWS Provider Version

4.52.0

Affected Resource(s)

• aws_db_cluster_snapshot (data)
• aws_rds_global_cluster
• aws_rds_cluster
• aws_rds_cluster_instance

Expected Behavior

A MySQL Global Aurora cluster should be created with a primary cluster and writer instance based upon a snapshot, it should look something like this:

Note that this correct looking cluster was created using the exact same code just with the snapshot_identifier = data.aws_db_cluster_snapshot.sre_5136_to_be_upgraded.id part commented out.

Actual Behavior

A MySQL Global Aurora cluster is created but the primary cluster and instance are not added to it, they are created separately as a standalone regional cluster.

Relevant Error/Panic Output Snippet

No response

Terraform Configuration Files

# Create Initial Cluster to be Copied Next
# -----------------------------------------------------------------------------

resource "aws_rds_global_cluster" "sre_5136" {
  global_cluster_identifier = "sre-5136-global-test"
  engine                    = "aurora"
  engine_version            = "5.6.mysql_aurora.1.22.4"
  storage_encrypted         = true
}

resource "aws_rds_cluster" "sre_5136_primary" {
  engine                    = aws_rds_global_cluster.sre_5136.engine
  engine_version            = aws_rds_global_cluster.sre_5136.engine_version
  cluster_identifier        = "sre-5136-test-primary-cluster"
  master_username           = "example_user"
  master_password           = "password"
  database_name             = "example_db"
  global_cluster_identifier = aws_rds_global_cluster.sre_5136.id
  db_subnet_group_name      = "default"
}

resource "aws_rds_cluster_instance" "sre_5136_primary" {
  engine               = aws_rds_global_cluster.sre_5136.engine
  engine_version       = aws_rds_global_cluster.sre_5136.engine_version
  identifier           = "sre-5136-test-primary-cluster-instance"
  cluster_identifier   = aws_rds_cluster.sre_5136_primary.id
  instance_class       = "db.r4.large"
  db_subnet_group_name = "default"
}


# Try Creating a New Upgraded Cluster Based on Snapshot From Above Cluster
# -----------------------------------------------------------------------------

# Grab the most recent snapshot from the RDS cluster above.
data "aws_db_cluster_snapshot" "sre_5136_to_be_upgraded" {
  db_cluster_identifier = aws_rds_cluster.sre_5136_primary.id
  most_recent           = true
}

# Upgrade to the latest engine version supported by the snapshot.
resource "aws_rds_global_cluster" "sre_5136_upgraded" {
  global_cluster_identifier = "sre-5136-global-test-upgraded"
  engine                    = "aurora"
  engine_version            = "5.6.mysql_aurora.1.22.4"
  storage_encrypted         = true
}

# Use the snapshot of the above database to create a new dev database.
resource "aws_rds_cluster" "sre_5136_upgraded" {
  engine                    = aws_rds_global_cluster.sre_5136_upgraded.engine
  engine_version            = aws_rds_global_cluster.sre_5136_upgraded.engine_version
  cluster_identifier        = "sre-5136-upgraded-cluster"
  
  # Commented out as not needed when using a snapshot but needing when testing creation not based on a snapshot.
  # master_username           = "example_user"
  # master_password           = "password"
  # database_name             = "example_db"
  
  global_cluster_identifier = aws_rds_global_cluster.sre_5136_upgraded.id
  db_subnet_group_name      = "default"

  snapshot_identifier = data.aws_db_cluster_snapshot.sre_5136_to_be_upgraded.id

  lifecycle {
    ignore_changes = [snapshot_identifier]
  }
}

resource "aws_rds_cluster_instance" "sre_5136_upgraded" {
  engine               = aws_rds_global_cluster.sre_5136_upgraded.engine
  engine_version       = aws_rds_global_cluster.sre_5136_upgraded.engine_version
  identifier           = "sre-5136-upgraded-test-primary-cluster-instance"
  cluster_identifier   = aws_rds_cluster.sre_5136_upgraded.id
  instance_class       = "db.r5.large"
  db_subnet_group_name = "default"
}

Steps to Reproduce

1. Create the first Global Aurora and snapshot it and create a manual snaphshot for testing.
2. Add the second Global Aurora and terraform apply.

I was then double checking what should happen by creating the second cluster from scratch by uncommenting the master_username, master_password, and database_name and commenting out this section:

  snapshot_identifier = data.aws_db_cluster_snapshot.sre_5136_to_be_upgraded.id

  lifecycle {
    ignore_changes = [snapshot_identifier]
  }

The Global cluster is built correctly then.

Debug Output

No response

Panic Output

No response

Important Factoids

No response

References

• Old issue: Not able to launch global RDS with existing snapshot #9210
• Docs: https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/db_cluster_snapshot

Would you like to implement a fix?

None

[github-actions]

Community Note

Voting for Prioritization

• Please vote on this issue by adding a 👍 reaction to the original post to help the community and maintainers prioritize this request.
• Please see our prioritization guide for information on how we prioritize.
• Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request.

Volunteering to Work on This Issue

• If you are interested in working on this issue, please leave a comment.
• If this would be your first contribution, please review the contribution guide.

[ewbankkit]

User Guide for Aurora.

[jar-b]

As the AWS documentation linked above indicates, it is not currently possible to restore a database from snapshot AND join to an existing global cluster in a single operation. You can however use a cluster restored from snapshot as the starting point for a new global cluster.

You can restore a snapshot of an Aurora DB cluster or from an Amazon RDS DB instance to use as the starting point for your Aurora global database. You restore the snapshot and create a new Aurora provisioned DB cluster at the same time. You then add another AWS Region to the restored DB cluster, thus turning it into an Aurora global database. Any Aurora DB cluster that you create using a snapshot in this way becomes the primary cluster of your Aurora global database

---

Here is a modified version of the original configuration which inverts the dependency flow so the global cluster is sourced from the cluster restored from snapshot:

Show/Hide Configuration

terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 4.0"
    }
  }
}

# Configure the AWS Provider
provider "aws" {}

# Create Initial Cluster to be Copied Next
# -----------------------------------------------------------------------------
resource "aws_rds_global_cluster" "example" {
  global_cluster_identifier = "jb-test-global-cluster"
  engine                    = "aurora"
  engine_version            = "5.6.mysql_aurora.1.22.4"
  storage_encrypted         = true
}

resource "aws_rds_cluster" "example_primary" {
  engine                    = aws_rds_global_cluster.example.engine
  engine_version            = aws_rds_global_cluster.example.engine_version
  cluster_identifier        = "jb-test-primary-cluster"
  master_username           = "example_user"
  master_password           = "password"
  database_name             = "example_db"
  global_cluster_identifier = aws_rds_global_cluster.example.id
  db_subnet_group_name      = "default"

  skip_final_snapshot = true
}

resource "aws_rds_cluster_instance" "example_primary" {
  engine               = aws_rds_global_cluster.example.engine
  engine_version       = aws_rds_global_cluster.example.engine_version
  identifier           = "jb-test-primary-cluster-instance"
  cluster_identifier   = aws_rds_cluster.example_primary.id
  instance_class       = "db.r4.large"
  db_subnet_group_name = "default"
}

resource "aws_db_cluster_snapshot" "example_primary" {
  db_cluster_identifier          = aws_rds_cluster.example_primary.id
  db_cluster_snapshot_identifier = "jb-test-primary-cluster-snapshot"
}

# Try Creating a New Upgraded Cluster Based on Snapshot From Above Cluster
# -----------------------------------------------------------------------------

# Grab the most recent snapshot from the RDS cluster above.
data "aws_db_cluster_snapshot" "example_to_be_upgraded" {
  depends_on = [aws_db_cluster_snapshot.example_primary]

  db_cluster_identifier = aws_rds_cluster.example_primary.id
  most_recent           = true
}

# Use the snapshot of the above database to create a new dev database.
resource "aws_rds_cluster" "example_upgraded_primary" {
  # Because the global cluster is sourced from this cluster, the initial 
  # engine and engine_version values are defined here and automatically
  # inherited by the global cluster.
  engine             = "aurora"
  engine_version     = "5.6.mysql_aurora.1.22.4"
  cluster_identifier = "jb-test-upgraded-primary-cluster"

  db_subnet_group_name = "default"
  snapshot_identifier  = data.aws_db_cluster_snapshot.example_to_be_upgraded.id

  lifecycle {
    ignore_changes = [
      snapshot_identifier,
      # NOTE: Using this DB Cluster to create a Global Cluster, the
      # global_cluster_identifier attribute will become populated and
      # Terraform will begin showing it as a difference. Do not configure:
      # global_cluster_identifier = aws_rds_global_cluster.example.id
      # as it creates a circular reference. Use ignore_changes instead.
      global_cluster_identifier,
    ]
  }

  skip_final_snapshot = true
}

# Global cluster is now sourced from the existing cluster restored from snapshot
resource "aws_rds_global_cluster" "example_upgraded" {
  global_cluster_identifier    = "jb-test-upgraded-global-cluster"
  source_db_cluster_identifier = aws_rds_cluster.example_upgraded_primary.arn
  force_destroy                = true
}

resource "aws_rds_cluster_instance" "example_upgraded_primary" {
  engine               = aws_rds_cluster.example_upgraded_primary.engine
  engine_version       = aws_rds_cluster.example_upgraded_primary.engine_version
  identifier           = "jb-test-upgraded-primary-cluster-instance"
  cluster_identifier   = aws_rds_cluster.example_upgraded_primary.id
  instance_class       = "db.r5.large"
  db_subnet_group_name = "default"
}

This results in the intended relationship between resources. See the aws_rds_global_cluster documentation for additional background.

[jar-b]

To avoid misleading outcomes, #30158 will also add a plan time check to prevent snapshot_identifier and global_cluster_identifier from being set in the same configuration.

[github-actions]

This functionality has been released in v4.60.0 of the Terraform AWS Provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

For further feature requests or bug reports with this functionality, please create a new GitHub issue following the template. Thank you!

[github-actions]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.