AWS has changed Elasticache Redis versioning again

[philomory]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Terraform CLI and Terraform AWS Provider Version

Terraform v1.0.7
on windows_amd64
+ provider registry.terraform.io/hashicorp/aws v3.70.0

Affected Resource(s)

• aws_elasticache_replication_group

(This probably also impacts aws_elasticache_cluster when used with Redis, but I haven't personally verified it)

Terraform Configuration Files

Please include all Terraform configurations required to reproduce the bug. Bug reports without a functional reproduction may be closed without investigation.

terraform {
  backend "local" { }

  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 3.70"
    }
  }
}

provider "aws" {
  region  = "us-east-1"
}

resource "aws_elasticache_replication_group" "example1" {
  replication_group_id          = "tf-rep-group-1"
  replication_group_description = "test description"
  engine_version                = "6.0"
  node_type                     = "cache.m4.large"
  parameter_group_name          = "default.redis6.x"
}

resource "aws_elasticache_replication_group" "example2" {
  replication_group_id          = "tf-rep-group-2"
  replication_group_description = "test description"
  engine_version                = "6.2"
  node_type                     = "cache.m4.large"
  parameter_group_name          = "default.redis6.x"
}

Debug Output

https://gist.github.com/philomory/eeead53871f26c6a3825e36c17a862af

Expected Behavior

Create two Redis clusters, one at 6.0 and one at 6.2

Actual Behavior

An error is displayed, stating engine_version: Redis versions must match <major>.x when using version 6 or higher, or <major>.<minor>.<bug-fix>.

Steps to Reproduce

1. terraform apply

Important Factoids

Previously, Amazon had announced the following for Redis versions 6 and higher:

Beginning with Redis 6, ElastiCache for Redis will offer a single version for each Redis OSS major release, rather than offering multiple minor versions. Versionless engine support is designed to minimize confusion and ambiguity on having to choose from multiple minor versions. ElastiCache for Redis will also automatically keep your cache cluster up to date on the selected major version (Redis 6 and above). By providing the updated engine version, it ensures improved performance and enhanced security.

You specify the engine version by using 6.x. ElastiCache for Redis will automatically invoke the preferred minor version of Redis 6 that is available.

(This was as opposed to with versions 5.x and lower, where you could specify full version numbers).

However, at some point in the recent past (probably coinciding with the 2021-11-23 announcement of support for Redis 6.2, but I'm not certain), the documentation was changed to state:

Beginning with Redis 6.0, ElastiCache for Redis will offer a single version for each Redis OSS minor release, rather than offering multiple patch versions.

Beginning with Redis 6.0, you can also opt-in to the next auto minor version upgrade by setting the AutoMinorVersionUpgrade parameter to yes, and ElastiCache for Redis will manage the minor version upgrade, through self-service updates. This will be handled through standard customer-notification channels via a self-service update campaign. For more information, see Self-service updates in Amazon ElastiCache.

The output of aws elasticache describe-cache-engine-versions now confirms that there are engine versions "6.0" and "6.2" now, but not any engine version identified as "6.x" (although oddly attempting to create a cluster with engine_version "6.x" does still work, and results in a cluster using version 6.2.5).

References

• Elasticache Redis 6.2 support announcement
• Elasticache Redis supported engine versions
• Elasticache Redis uprading versions
• Elasticache redis 6.x support #15625, the issue initially resulting in implementation of support for 6.x as an engine_version and removal of support for e.g. 6.0.
• elasticache: Adds Redis 6.x support #18920, the pull request that resulted from the above-mentioned issue

[manelpb]

Yes Please!!! This is basically blocking us to upgrade to version 6.0

[ktham]

So what needs to change?

Does that Elasticache API now want us to specify 6.0.x and 6.2.x rather than 6.x for engine_version now for creating an elasticache cluster? Note that the engine_version parameter for creating the cluster is not necessarily the same value returned by the Describe API.

The output of aws elasticache describe-cache-engine-versions now confirms that there are engine versions "6.0" and "6.2" now, but not any engine version identified as "6.x" (although oddly attempting to create a cluster with engine_version "6.x" does still work, and results in a cluster using version 6.2.5).

That has always been the case for the 6.x clusters. The Describe API always return the actual version number that is running (i.e. 5.0.5, 6.0.0, etc...), not necessarily the engine_version parameter specified in the create API call. (i.e. "6.x").

[ktham]

I've been looking at the docs, indeed, it seems like with the 6.2 release, it seems like AWS wants us to use "6.0" and "6.2" in the engine_version field 🤦 I'm guessing AWS is still supporting "6.x" for now but silently wants us to migrate with that documentation change.

[ktham]

Yes Please!!! This is basically blocking us to upgrade to version 6.0

@manelpb - Seems like you can still specify "6.x" in the engine_version field and still be fine and create the latest available 6.x cluster (i.e. 6.2.5). That was the advertised behavior for a long time now. (i.e. not being able to specify the minor version and create the latest 6.x one).

[manelpb]

Yes Please!!! This is basically blocking us to upgrade to version 6.0

@manelpb - Seems like you can still specify "6.x" in the engine_version field and still be fine and create the latest available 6.x cluster (i.e. 6.2.5). That was the advertised behavior for a long time now. (i.e. not being able to specify the minor version and create the latest 6.x one).

@ktham Yes, but that would force us to migrate to 6.2 instead of 6.0. Even though they don't have any significant breaking change, there was indeed a change that could affect the way they calculate key expiration, as described here:

Command behavior changes:
* EXISTS should not alter LRU (#8016)
  In Redis 5.0 and 6.0 it would have touched the LRU/LFU of the key.

https://raw.githubusercontent.com/redis/redis/6.2/00-RELEASENOTES

We don't want to migrate right from 5.0 to 6.2 without making sure it won't affect all of our servers, which might take some time.

[ktham]

@ktham Yes, but that would force us to migrate to 6.2 instead of 6.0. Even though they don't have any significant breaking change, there was indeed a change that could affect the way they calculate key expiration

Ah, I see. cc @gdavison, there might be some tweaks needed to #18920 to support setting engine_version to "6.2" and "6.0" in addition to "6.x" now that AWS wants us to specify the minor version in addition to the major version.

I think we'll want to:

• Update the regular expression used for validation here to allow "MAJOR.MINOR" where minor can be an "x" or a numeric value:

  terraform-provider-aws/internal/service/elasticache/validation.go

  Line 17 in 2d0b6c1

  redisVersionPostV6RegexpRaw = `([6-9]|[[:digit:]]{2})\.x`

• And update the wording here:

  terraform-provider-aws/internal/service/elasticache/validation.go

  Line 36 in 2d0b6c1

  errors = append(errors, fmt.Errorf("%s: Redis versions must match <major>.x when using version 6 or higher, or <major>.<minor>.<bug-fix>", k))

• And add the relevant test cases

[philomory]

The output of aws elasticache describe-cache-engine-versions now confirms that there are engine versions "6.0" and "6.2" now, but not any engine version identified as "6.x" (although oddly attempting to create a cluster with engine_version "6.x" does still work, and results in a cluster using version 6.2.5).

That has always been the case for the 6.x clusters. The Describe API always return the actual version number that is running (i.e. 5.0.5, 6.0.0, etc...), not necessarily the engine_version parameter specified in the create API call. (i.e. "6.x").

Note that I'm not talking about the API call used to describe the version that a particular cluster is running, this API call returns a list of versions (for both Redis and Memcached) that are supported by the API. I'm pretty sure you're thinking of describe-cache-clusters, not describe-cache-engine-versions.

describe-cache-clusters says my cluster has "EngineVersion": "6.2.5"; that's the actual running version. But describe-cache-engine-versions lists "EngineVersion": "6.2", not "EngineVersion": "6.2.5", because "6.2" a valid string to pass to the API.

Among other things, DescribeCacheEngineVersions is the API call that the Elasticache Console uses to populate the "Engine version compatibility" dropdown in the "Create cluster" interface. The drop-down used to have "6.x" as an option, because "6.x" used to be returned by the API; now it has "6.0" and "6.2" as options, because that's what the API returns now.

[ktham]

I'm pretty sure you're thinking of describe-cache-clusters, not describe-cache-engine-versions.

Ah, indeed. Sorry, I didn't read your PR description carefully initially

[manelpb]

Looks like we have a pull request open with a change to fix this issue. Has anyone looked at it?

[MatiasNielsen]

Using engine_version "6.x" seems to work.

[bschaatsbergen]

Picking this one up as the PR seems to be lying around

[KMahankali]

Do we have any ETA when will be the provider this will be available to support 6.0 and 6.2 for engine_version?

[bschaatsbergen]

Do we have any ETA when will be the provider this will be available to support 6.0 and 6.2 for engine_version?

Just finished the PR (need to run tests), up to the reviewers, as ElastiCache is a core service you might get lucky and it becomes part of 4.6.0 (upcoming release).

[manelpb]

Thank you @bschaatsbergen

[bschaatsbergen]

I had to jump between a few hoops in order to make this work as friendly as possible without breaking anything. I removed the 6.x support as upgrading your resource from 6.0 to 6.x is not possible, the ModifyCacheCluster API call seems to accept the 6.x but ignore it and your cluster will stay on 6.0. The regex pattern now no longer accepts 6.x, it only accepts <major>.<minor>, meaning you're in charge of either supplying 6.0 or 6.2

Being able to specify 6.0 and wanting to upgrade would be pretty straight-forward:

resource "aws_elasticache_cluster" "example" {
  cluster_id           = "cluster-example"
  engine               = "redis"
  node_type            = "cache.t3.medium"
  num_cache_nodes      = 1
  engine_version       = "6.0" <--- simply change to 6.2 if you want to upgrade
  port                 = 6379
  apply_immediately = true
}

  # aws_elasticache_cluster.example will be updated in-place
  ~ resource "aws_elasticache_cluster" "example" {
      ~ engine_version           = "6.0" -> "6.2"
        id                       = "cluster-example"
        tags                     = {}
        # (19 unchanged attributes hidden)
    }

Thoughts?

[manelpb]

As long we understand that this would cause a breaking change and we add a note about it, I think it should be fine

[bschaatsbergen]

Hey all, as you can see by the above the fix is merged and will be part of 4.12.0 🎉

[manelpb]

Hey all, as you can see by the above the fix is merged and will be part of 4.12.0 🎉

Thank you so much!!

[github-actions]

This functionality has been released in v4.12.0 of the Terraform AWS Provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

For further feature requests or bug reports with this functionality, please create a new GitHub issue following the template. Thank you!

[github-actions]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.