aws_secretsmanager_secret does not rebuild deleted secret

[zarlant]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Terraform Version

Terraform v0.11.7

• provider.aws v1.25.0

Affected Resource(s)

• aws_secretsmanager_secret

Terraform Configuration Files

resource "aws_secretsmanager_secret" "testsecret" {
  name = "mycool_secret"
}

Expected Behavior

Terraform should look to see if the secret exists and create it.

Actual Behavior

Terraform sees the secret id in the state file and does not verify that the secret still exists in AWS

Steps to Reproduce

1. terraform apply
2. Manually delete secret using the CLI, or some other means
3. terraform plan or terraform apply
4. Terraform thinks there is nothing to do

Important Factoids

This only seems to happen if the secret is deleted outside of terraform.

[bflad]

Relevant Secrets Manager documentation: https://docs.aws.amazon.com/secretsmanager/latest/userguide/manage_delete-restore-secret.html

Deleting a secret only marks it as deprecated and does not immediately fully delete it. We should probably add a documentation note about this in the resource documentation.

We might be able to mark these deprecated secrets for recreation in the resource read function, but the creation in that scenario I believe will always fail if the secret name is the same. We'll need to check the user experience of that (e.g. it returns a helpful error when a deprecated secret already exists) or if we should return our own error about deprecated secrets instead.

A workaround is to create a different secret in the deprecation period.

[jangeja]

If terraform checks to see if the key is scheduled for deletion before trying to create, this error would be resolved. This would be very helpful especially for people first starting out with terraform.

[hleb-rubanau]

This only seems to happen if the secret is deleted outside of terraform.

I believe it also happens in case of "terraform destroy" followed by "terraform apply".

Maybe appropriate action would be to use "undelete" API call instead of "create" if secret is scheduled for deletion.

This logic may not necessary be default. However, having a flag on terraform secret resource (enabling automatical "undelete" behaviour) would be very nice thing to have.

Otherwise, terraform runs are not 100% idempotent. (I.e. I cannot wipe my testing infrastructure in the end of the day to just recreate it next morning).

[hleb-rubanau]

Uh, got it. My suggestion won't work out.

Terraform has no idea of secret existence because after deletion secret's ARN is no more in the terraform state. So, prior to any planning secret should be imported into the state.

Any ideas to automate this process should not be considered neither suggested -- as implicit import will break the fundamental concepts of Terraform (it may lead to the situation where Alice and Bob compete for the control over same resource just because of using same name).

I solved the problem with little manual overhead, fairly acceptable:

1. Manually "cancelled deletion" for affected secrets in the AWS management console
2. Performed "terraform import" for affected resources

Maybe step 1 was not even necessary, but I do not have time to check it now.

[bflad]

Pull request submitted with support of the new ForceDeleteWithoutRecovery parameter for DeleteSecret: #5583

[bflad]

In version 1.33.0 of the AWS provider, likely releasing later today, the aws_secretsmanager_secret resource will support:

• Setting recovery_window_in_days to 0, which will "immediately" delete the secret (technically its an asynchronous process started in the background by the Secrets Manager service).
• On creation it will retry on InvalidRequestException: You can’t perform this operation on the secret because it was deleted. errors waiting for the background process to complete, which allows for recreating deleted secrets using the method above.

Please note for updating existing aws_secretsmanager_secret resources: updating the recovery_window_in_days argument must have terraform apply ran before any sort of deletion operation (e.g. terraform destroy, count = 0) for it to take effect.

[bflad]

This has been released in version 1.33.0 of the AWS provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thanks!