aws_dynamodb_table - with ttl disabled, can't "terraform apply" twice

[ghost]

This issue was originally opened by @nunoperalta as hashicorp/terraform#22942. It was migrated here as a result of the provider split. The original body of the issue is below.

---

Terraform Version

Terraform v0.12.9
+ provider.aws v2.30.0

Terraform Configuration Files

resource "aws_dynamodb_table" "dynamodb-testdb" {
  name           = "TestDb"
  billing_mode   = "PAY_PER_REQUEST"
  hash_key       = "PriKey"
  range_key      = "Quantity"

  attribute {
    name = "PriKey"
    type = "S"
  }

  attribute {
    name = "Quantity"
    type = "N"
  }

  ttl {
    attribute_name = "TimeToExist"
    enabled        = false
  }
}

Expected Behavior

When doing "terraform apply" twice, there should be no changes to make.

Actual Behavior

First "terraform apply" will create the DynamoDB table.
However, second time, there will be a change at:

      ~ ttl {
          + attribute_name = "TimeToExist"
            enabled        = false
        }

If I confirm the change, I get this crash:

Error: error updating DynamoDB Table (TestDb) time to live: error updating DynamoDB Table (TestDb) Time To Live: ValidationException: TimeToLive is already disabled
status code: 400, request id: XXXXXX

If I remove the "attribute_name", I get this error:

The argument "attribute_name" is required, but no definition was found.

[inge4pres]

Just to give more context here and to #3463

I just found out talking with AWS that the DynamoDB TTL API is batched, meaning that there is no way without to set atomically AttributeName and Enabled. Running the same operation through AWS CLI will result in the same 400 BadRequest errors because that's just how Dynamo DB TTL API works.

It is normal for DynamoDB API to reply with an error when trying to disable an already-disabeld TTL; and it's normal to not be able to enable/disable in a short period of time, you would get this error form CLI

➜  ~ aws dynamodb update-time-to-live --table-name my-table --time-to-live-specification "Enabled=true, AttributeName=UpdateTime"

An error occurred (ValidationException) when calling the UpdateTimeToLive operation: Time to live has been modified multiple times within a fixed interval

This is a PITA for Terraform because it means that when operating through the DynamoDB TTL settings we should have a polling or a time-bound control that we can actually perform an atomic operation.
I'll chat again with AWS soon and hopefully prepare a pull request with a proposed fix...

[inge4pres]

As per this doc page

It can take up to one hour for the change to fully process. Any additional UpdateTimeToLive calls for the same table during this one hour duration result in a ValidationException. 

so we can't really have this state reconciliation loop in Terraform...

[MattMcKeithen]

The "ValidationException: TimeToLive is already disabled" error is quite annoying if you are trying to modularize this.
As a work around, consider using a dynamic block.

credit: https://www.reddit.com/r/Terraform/comments/d1va2o/terrfaorm_support_null_block/

locals {
  ttl = (var.ttl_enable == true ? [
    {
      ttl_enable = var.ttl_enable
      ttl_attribute : var.ttl_attribute
    }
  ] : [])
}

...

  dynamic "ttl" {
    for_each = local.ttl
    content {
      enabled        = local.ttl[0].ttl_enable
      attribute_name = local.ttl[0].ttl_attribute
    }
  }

[ORDEP81]

Curious, was there an actual fix to this?

[justinretzolk]

Hey y'all 👋 Thank you for taking the time to file this issue and for the continued discussion! Given that there's been a number of AWS provider releases since it was initially filed, can anyone confirm whether you're still experiencing this behavior?

[poolieweb]

Still happening.

[bgshacklett]

Perhaps the documentation should be updated so that this isn't the default example.

https://github.com/hashicorp/terraform-provider-aws/blob/main/website/docs/r/dynamodb_table.html.markdown?plain=1#L44-L47

[obctpc]

The issue is still happening.
I looked at the tfstate and the attribute name is not set if it is disabled.

[DevSusu]

Still happening here ✋

[jakekdodd]

Still happening with provider version 4.10.0

[vineetsharma883]

still happening in 4.9.0

[dany74q]

^ +1

[sumeetninawe]

Still happening..

[jonnymccullagh]

still happening

[drewmobile]

still happening

[grantjoy]

still happening

[amirkav]

Still happening

[mmontero-twilio]

Still happening

[SupornoChaudhury]

I am still getting this error. And the only workaround seems to be the one suggested by @MattMcKeithen of using dynamic block. (not tried it yet)
Does anyone use any other workaround? It is of utmost importance for us to keep the TTL disabled.

[CyrilDevOps]

Sadly a dynamic block doesn't work,
with a block like

dynamic "ttl" {
    for_each = var.ttl_enabled ? [1] : []
    content {
      attribute_name = var.ttl_attribute
      enabled        = var.ttl_enabled
    }
  }

If you enable ttl it work, but then if you disable it, the block disappear and terraform doesn't see you want a enabled=false and it doesn't report any change and AWS keep the ttl settings.

I see that as a bug.

• allow enable = false and no attribute_name, and/or
• enforce enable = false when the ttl block isn't present.

[sathishdv]

Still happening

[shasi24]

Still happening

[alfredo-gil]

Still happening

[dth138]

still happening

[kennethjmyers]

This isn't really a "fix" but here is what I did to get it to not error (you probably only need step 3 but this is how I got there):

1. Removed the dynamo definitions from tfstate
2. Run terraform import for the tables I had just removed from tfstate. In the newly imported definitions the definition looked as follows:

   "ttl": [
             {
               "attribute_name": "",
               "enabled": false
             }
           ]
   

   This is different because the example I copied from documentation was as follows:

   ttl {
     attribute_name = "TimeToExist"
     enabled        = false
   }
   

3. So in my new definition I just converted "TimeToExist" to an empty string "" and when I did terraform plan it no longer wanted to make changes.

Sorry if someone already said this above but seems like the issue comes from "disabled" ttl not needing or using the supplied attribute name, but the resource definition requires an attribute name and many people probably just copy the example which gives one. So when you try to apply an update, it sees a "new" attribute definition it didn't use the first time. The documentation example should really be fixed to reflect this and the documentation should explain this quirk.

[rg-lesmills]

@kennethjmyers I have tried setting attribute_name to an empty string as suggested, but the apply gets rejected by AWS with a validation error:

Error: updating Amazon DynamoDB Table (ronnies-component-RonnieDB-sandbox): updating Time To Live: InvalidParameter: 1 validation error(s) found.
│ - minimum field size of 1, UpdateTimeToLiveInput.TimeToLiveSpecification.AttributeName.

Looking at the plan, it seems like terraform is taking a blank string in the table config and turning that into null:

      ~ ttl {
          - attribute_name = "ttl_field" -> null
          ~ enabled        = true -> false
        }

I'm using the typescript CDK @cdktf/provider-aws module 16.03 which is provider version 5.8.0

Steps to reproduce:

1. create a table with ttl.enabled = true and ttl.attribute_name whatever
2. set ttl.enabled = false and tt.attribute_name = ''
3. terraform apply results in minimum field size of 1 validation error as above

In order to actually disable the TTL and have terraform stop trying to re-add attribute_name as in the OP of this issue, I need to follow these steps:

1. create a table with ttl.enabled = true and ttl.attribute_name whatever
2. set ttl.enabled = false and leave ttl.attribute_name unchanged
3. terraform apply results in correct update to DynamoDB table
4. now set ttl.attribute_name to empty string
5. subsequent terraform plan/apply shows no changes required, as desired

[vgw-chriskruger]

This problem has existed for years on this thread alone and this is the new issue for this problem. This is just one of dozens of open bugs annoying my team with terraform at this juncture.

[ek2020]

Still happening, Curious, was there an actual fix to this? opened - Sep 30, 2019 - Now( Dec 26, 2023 ).

[pscosta]

Still happening , sadly.

[github-actions]

Warning

This issue has been closed, meaning that any additional comments are hard for our team to see. Please assume that the maintainers will not see them.

Ongoing conversations amongst community members are welcome, however, the issue will be locked after 30 days. Moving conversations to another venue, such as the AWS Provider forum, is recommended. If you have additional concerns, please open a new issue, referencing this one where needed.

[github-actions]

This functionality has been released in v5.55.0 of the Terraform AWS Provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

For further feature requests or bug reports with this functionality, please create a new GitHub issue following the template. Thank you!

[github-actions]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.