Perpetual diff in aws_elasticsearch_domain when advanced_options are set

[knyar]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Terraform CLI and Terraform AWS Provider Version

Terraform v0.14.11
+ provider registry.terraform.io/hashicorp/archive v2.2.0
+ provider registry.terraform.io/hashicorp/aws v3.63.0
+ provider registry.terraform.io/hashicorp/random v3.1.0
+ provider registry.terraform.io/hashicorp/template v2.2.0

Affected Resource(s)

• aws_elasticsearch_domain

Terraform Configuration Files

resource "aws_elasticsearch_domain" "es_domain" {
  domain_name           = "foo.example.com"
  elasticsearch_version = "7.9"

  advanced_options = {
    "rest.action.multi.allow_explicit_index" = "true"
  }
...
}

Debug Output

https://gist.github.com/knyar/72754a7d819434d24b6f96dfe6840652

Expected Behavior

Terraform plan shows no diffs when there has been no changes to Terraform configuration or actual AWS resource.

Actual Behavior

Terraform plan shows the following change on every run:

  # [resource name] will be updated in-place
  ~ resource "aws_elasticsearch_domain" "es_domain" {
      ~ advanced_options      = {
          - "override_main_response_version"         = "false" -> null
            # (1 unchanged element hidden)
        }
        id                    = "[resource arn]"
        # (8 unchanged attributes hidden)

Two points to note:

• If I manually add an override_main_response_version=false to advanced_options, no diff is displayed (as expected).
• If I remove the advanced_options block from my configuration, no diff is displayed (a bit unexpected).

Steps to Reproduce

The diff seems to have appeared out of nowhere with no changes to configuration or terraform versions. I suspect it might be caused by AWS API changes related to OpenSearch.

References

• This issue was previously reported in data "aws_iam_policy_document" and other resources are always read during apply #21164

[autero1]

Now getting this with provider version 3.6.0:

Error: ValidationException: Unrecognized advanced option 'override_main_response_version' passed in advancedOptions

[Gowiem]

I'm running into the same issue with the same issue as @autero1. I'm utilizing v3.63.0 of the AWS provider. @autero1 were you able to find a workaround for the provider thinking that advanced option is unrecognized?

[autero1]

I'm running into the same issue with the same issue as @autero1. I'm utilizing v3.63.0 of the AWS provider. @autero1 were you able to find a workaround for the provider thinking that advanced option is unrecognized?

Could not figure it out, ended up ignoring with

  lifecycle {
    ignore_changes = [advanced_options]
  }

which is far from perfect 😞

[Gowiem]

@autero1 Yeah... I would do that if I wasn't consuming this from a module, but that isn't the case. Damn, what sucks is that this doesn't look likely to get fixed soon 😞

[YakDriver]

I'm seeing the same behavior in the acceptance tests:

        Terraform used the selected providers to generate the following execution
        plan. Resource actions are indicated with the following symbols:
          ~ update in-place
        
        Terraform will perform the following actions:
        
          # aws_elasticsearch_domain.test will be updated in-place
          ~ resource "aws_elasticsearch_domain" "test" {
              ~ advanced_options      = {
                  - "override_main_response_version"         = "false" -> null
                  - "rest.action.multi.allow_explicit_index" = "true" -> null
                    # (1 unchanged element hidden)
                }
                id                    = "arn:aws:es:us-west-2:12345678901:domain/tf-acc-test-1541726374890865"
                # (7 unchanged attributes hidden)
                # (10 unchanged blocks hidden)
            }
        
        Plan: 0 to add, 1 to change, 0 to destroy.

[github-actions]

This functionality has been released in v3.70.0 of the Terraform AWS Provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

For further feature requests or bug reports with this functionality, please create a new GitHub issue following the template. Thank you!

[github-actions]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.