Elasticache Global Datastore (aws_elasticache_global_replication_group) modify shards (resharding)

[bnckobe]

I am trying to do is using global datastore for cross region replication. I have created Redis primary and secondary cluster in 2 regions with multi shards (elasticache_replication_group with cluster mode). They are control by global datastore. I need to scale up/down (resharding) when needed. Problem is shard is control by global datastore after created. I am not able to make the change on either primary or secondary. And for now aws_elasticache_global_replication_group is not providing any argument to do so.

Could anybody tell me is it anyway I can achieve that currently please? Or aws_elasticache_global_replication_group need to be enhanced to handle that? Appreciate it!

[justinretzolk]

Hey @bnckobe 👋 Thank you for taking the time to raise this! It looks like this may be more of a question than a bug report. If I'm correct in that interpretation, you may have better luck asking in the AWS Provider Discuss Forum. If I'm incorrect in that assumption, and you feel this may be a bug, can you update the issue description to include the information requested in the bug template?

[bnckobe]

---

name: 🐛 Bug Report
about: If something isn't working as expected 🤔.

---

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Terraform CLI and Terraform AWS Provider Version

❯ tf --version
Terraform v1.1.4
on darwin_amd64
+ provider registry.terraform.io/hashicorp/aws v3.71.0
+ provider registry.terraform.io/hashicorp/random v3.1.0
+ provider registry.terraform.io/hashicorp/vault v2.24.1
+ 

Affected Resource(s)

aws_elasticache_global_replication_group
aws_elasticache_replication_group

Terraform Configuration Files

Please include all Terraform configurations required to reproduce the bug. Bug reports without a functional reproduction may be closed without investigation.

# Please replace vpc-xxxxx and sg-xxxx 

provider "aws" {
  region = "ca-central-1"
}

provider "aws" {
  alias  = "dr_aws"
  region = "eu-west-1"
}

data "aws_subnet_ids" "all" {
  vpc_id = "vpc-xxxxxxxxxxxxxxxxxxxx"
}

locals {
  vpc_name = "vpc-xxxxxxxxxxxxxxxxxxxx"
}

resource "aws_elasticache_subnet_group" "this" {
  name        = "redis-cluster-dr-subnet"
  description = "Elasticache subnet group for redis-cluster-dr"
  subnet_ids  = data.aws_subnet_ids.all.ids
}


resource "aws_elasticache_replication_group" "redis" {
  replication_group_id          = format("%.20s", "redis-cluster-dr")
  replication_group_description = "Terraform-managed ElastiCache replication group for redis-cluster-dr-${local.vpc_name}"
  node_type                     = "cache.m5.large"
  automatic_failover_enabled    = true
  multi_az_enabled              = true
  auto_minor_version_upgrade    = true
  engine                        = "redis"
  at_rest_encryption_enabled    = true
  transit_encryption_enabled    = true
  engine_version                = "6.x"
  port                          = 6379
  parameter_group_name          = "default.redis6.x.cluster.on"
  subnet_group_name             = aws_elasticache_subnet_group.this.id
  security_group_ids            = ["sg-xxxxxxxxxxxxxxxxxx"]
  apply_immediately             = true

cluster_mode {
      replicas_per_node_group = 1
      num_node_groups         = 2
  }
}

resource "aws_elasticache_global_replication_group" "global_store" {
  global_replication_group_id_suffix = format("%.20s", "redis-cluster-dr")
  primary_replication_group_id       = aws_elasticache_replication_group.redis.id
  depends_on = [
    aws_elasticache_replication_group.redis,
  ]
}

data "aws_subnet_ids" "dr_subnets" {
  provider = aws.dr_aws
  vpc_id   ="vpc-xxxxxxxxxxxxxxxxxx"
}

resource "aws_elasticache_subnet_group" "secondary_subnet" {
  provider    = aws.dr_aws
  name        = "redis-cluster-dr-secondary-subnet"
  description = "Elasticache subnet group for redis-cluster-dr"
  subnet_ids  = data.aws_subnet_ids.dr_subnets.ids
}

resource "aws_elasticache_replication_group" "secondary" {
  provider                      = aws.dr_aws
  replication_group_id          = "${format("%.20s", "redis-cluster-dr")}-secondary"
  replication_group_description = "Terraform-managed ElastiCache replication group for redis-cluster-dr-vpc-xxxxxxxxxxxxxxxxxx"
  global_replication_group_id   = aws_elasticache_global_replication_group.global_store.global_replication_group_id

  subnet_group_name  = aws_elasticache_subnet_group.secondary_subnet.id
  security_group_ids = ["sg-xxxxxxxxxxxxxxxxxx"]

  multi_az_enabled           = true
  automatic_failover_enabled = true

 cluster_mode {
      replicas_per_node_group = 1
  }

  depends_on = [
    aws_elasticache_replication_group.redis,
    aws_elasticache_global_replication_group.global_store,
  ]

}

Debug Output

Panic Output

Expected Behavior

1. After first terraform apply one global datastore will be created with 2 redis cluster in differetn regions. And everything is fine.
2. Make change to shard # (num_node_groups) and apply. Expect that shard # of both redis cluster will be changed.

Actual Behavior

Get the error shows below after changing shard #.

aws_elasticache_replication_group.redis: Modifying... [id=redis-cluster-dr]
╷
│ Error: error modifying ElastiCache Replication Group (redis-cluster-dr) shard configuration: error modifying ElastiCache Replication Group shard configuration: InvalidParameterValue: Cluster [redis-cluster-dr] is part of a global cluster [bxodz-redis-cluster-dr]. Request rejected.
│       status code: 400, request id: fe7615a8-e53c-4708-b79b-92750db72ca5
│ 
│   with aws_elasticache_replication_group.redis,
│   on main.tf line 24, in resource "aws_elasticache_replication_group" "redis":
│   24: resource "aws_elasticache_replication_group" "redis" {
│ 

Steps to Reproduce

1. terraform apply
2. change num_node_groups to any number different than the first apply
3. terraform apply

Important Factoids

When using global datastore, the cluster is controlled by it. The cluster can NOT be modified by itself but through the global datastore. But aws_elasticache_global_replication_group has no argument to do so.

References

• #0000

[bnckobe]

Thanks a lot @justinretzolk for your reply. TBH I don't think this is question or bug but feature request or enhancement. Any way I did what you asked for. I created post from the forum also filled up the bug report form. Let me know if I missed anything please. Thanks again for your help.

[github-actions]

This functionality has been released in v4.37.0 of the Terraform AWS Provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

For further feature requests or bug reports with this functionality, please create a new GitHub issue following the template. Thank you!

[github-actions]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.