RDS resource not resolving KMS Alias on subsequent plans

[hashibot]

This issue was originally opened by @wv-cfromm as hashicorp/terraform#15883. It was migrated here as a result of the provider split. The original body of the issue is below.

---

If I create an RDS instance and reference a KMS key via Alias ARN, the instance is created and properly refers to the Alias' target. However, on subsequent plans, this forces perpetual recreation, as the ARN in state (the alias arn) does not match the ARN on the instance (the key arn).

Refreshing state should resolve the Alias before comparing to existing instance. This would allow Aliases to be usable on RDS resources.

[trung]

I hit the same issue

data "aws_kms_alias" "kms_key" {
  name = "alias/${var.kms_key_alias}"
}
resource "aws_db_instance" "db" {
...
  kms_key_id                 = "${data.aws_kms_alias.kms_key.arn}"
...
}

After tf apply successfully, do tf plan and it results:

-/+ aws_db_instance.db (new resource required)
...
      kms_key_id:                        "arn:aws:kms:xxx:yyy:key/50534cd0-92f0-46c3-bb1b-f1c055a80813" => "arn:aws:kms:xxx:yyy:alias/foo" (forces new resource)       
...

[cfromm1911]

Referencing the provider-split question of providing code, I believe trung's example is sufficient. Its pretty cut/dry. Referencing an Alias creates an instance that is properly tied to Key, but the Alias is seen as a difference by subsequent plans.

[mgladdish]

I just lost an entire day to this. Very frustrating!

It essentially means we can't use pre-existing keys (referenced via data "aws_kms_alias" ...) with rds resources.

[bflad]

You can use the aws_kms_alias data source, but you have to assemble its key ARN manually as a workaround. arn:aws:kms:REGION:ACCOUNT:key/${data.aws_kms_alias.NAME.target_key_id}

[trung]

ListAliases API used by aws_kms_alias returns ARN of key-alias format (e.g.: arn:aws:kms:xxx:yyy:alias/foo) and not key-identifier format (e.g.: arn:aws:kms:xxx:yyy:key/50534cd0-92f0-46c3-bb1b-f1c055a80813). See ARN Syntax Format for KMS

CreateDBInstance API used by aws_db_instance resource accepts kms_key_id which can be key-alias or key-identifier ARN. However, DescribeDBInstances API used by aws_db_instance resource when reading back only returns kms_key_id of key-identifier ARN. Therefore causing the diff.

I think the fix could be to expose key-identifier ARN in aws_kms_alias data source. In the mean time, a sensible workaround would be to use replace() interpolation function to translate the ARN in key-alias to key-identifier format.

E.g:

resource "aws_db_instance" "db" {
...
  kms_key_id                 = "${replace(data.aws_kms_alias.kms_key.arn, data.aws_kms_alias.name, format("key/%s", data.aws_kms_alias.target_key_id))}"
...
}

Note: I haven't tried it myself.

Another potential issue with aws_kms_alias is that the API only pulls max of 100 aliases so there's a chance that an alias would not be found.

Updated: fixed the replacement string

[trung]

Added @tombuildsstuff to review the issue

[trung]

Related to #2009

[Constantin07]

+1 facing the same issue.

[ssala7]

+1 kms_key_id is a replacement call to RDS. Please include support

[bmacauley-reward]

nightmare...lost a day on this issue

[fractos]

@bmacauley-reward heh, fancy seeing you here :)

hit this too, major pain.

[bflad]

Hi everyone 👋 Since this issue was raised awhile ago, both the aws_kms_alias resource and data source now support a target_key_arn attribute, which is compatible with all kms_key_id arguments across all Terraform AWS Provider resources. This is the recommended path for all configurations experiencing this issue.

For some additional context, the maintainers do not intend to directly support KMS Alias ARNs due to how the AWS APIs always return the KMS Key ID. Since Terraform performs drift detection with configured values against the value stored in the Terraform state when refreshed by the resource, this difference would always show unless we introduced additional cross-service API calls to every resource that supports this parameter. This API call would occur every Terraform plan for every resource and could dramatically affect account-level KMS rate limiting along with permissions issues for restrictive environments. Given these operational concerns, we prefer to recommend the target_key_arn attribute.

If the documentation around this can be improved, please reach out.

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thanks!