provider aws v2.16.0 fails to create a new replica node for elasticache cluster with identifier of 20 characters

[balusarakesh]

environment:

Terraform v0.11.13

terragrunt version v0.17.4

aws-cli/1.16.184 Python/2.7.14 Darwin/17.7.0 botocore/1.12.174

TF code:

resource "aws_elasticache_replication_group" "ts" {

  replication_group_id = "${substr("${var.datacenter}-ts-redis-cluster", 0, 
                          min(length("${var.datacenter}-ts-redis-cluster"), 20))}"
  replication_group_description = "${var.datacenter}-ts-redis-cluster"
  automatic_failover_enabled = true
  ...
  ...
  ...
  number_cache_clusters = "3"
}

we are updating the number of replica nodes to 3 and when we apply the above code we see the following error

Error: Error applying plan:

1 error(s) occurred:

* aws_elasticache_replication_group.ts: 1 error(s) occurred:

* aws_elasticache_replication_group.ts: error creating Elasticache Cache Cluster (adding replica): InvalidParameterValue: The parameter CacheClusterIdentifier is not a valid identifier because it is longer than 20 characters.
	status code: 400, request id: c6c5e6f6-975a-11e9-8181-474c4619931c

• With the existing terraform provider aws version the cluster id is limited to only 20 characters.
• We have a couple of existing replica nodes with more than 20 characters in length
• The cluster is created by hand and we are importing it to terraform
• We tried to run the command aws elasticache increase-replica-count --replication-group-id speyside-ts-redis-cl --new-replica-count 2 --apply-immediately with awscli version <1.2 and we got the same error. When we updated awscli version to 1.16.184 it is working fine.

[schainks]

This issue can be worked around with AWS CLI, but obviously means it happens outside of terraform state management.

Try something like:

aws elasticache increase-replica-count --replication-group-id <YOUR GROUP ID> --new-replica-count <INTEGER> --region <REGION> --apply-immediately

Elasticache should spin up replicas in a different AZ than the primary.

The fix for this would be if terraform provider can copy whatever API call the AWS CLI is doing to modify clusters that were created before this identifier string length limit was imposed. I do not expect AWS to raise this limit anytime soon, but they clearly are allowing users to work around it, and terraform should also allow for this.

[bflad]

The Elasticache service now supports cluster identifiers up to 50 characters. We will support this new limit via: #9941

[schainks]

What about modifying existing clusters that were named before the limit went into place?

[bflad]

The above has been merged and will release in version 2.28.0 of the Terraform AWS Provider, Thursday. 👍

What about modifying existing clusters that were named before the limit went into place?

The AWS Announcement does not provide any detail into this. From Terraform's perspective, we will allow and/or passthrough any configured value (under the new 50 character limit) to the Elasticache API.

[ghost]

This has been released in version 2.28.0 of the Terraform AWS provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

For further feature requests or bug reports with this functionality, please create a new GitHub issue following the template for triage. Thanks!

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thanks!