Module recreates all settings on each terraform plan/apply

[aknysh]

terraform-aws-elastic-beanstalk-environment recreates all settings on each terraform plan/apply

    setting.1039973377.name:               "InstancePort" => "InstancePort"
    setting.1039973377.namespace:      "aws:elb:listener:22" => "aws:elb:listener:22"
    setting.1039973377.resource:           "" => ""
    setting.1039973377.value:                "22" => "22"
    setting.1119692372.name:               "" => "ListenerEnabled"
    setting.1119692372.namespace:      "" => "aws:elbv2:listener:443"
    setting.1119692372.resource:           "" => ""
    setting.1119692372.value:                "" => "false"
    setting.1136119684.name:               "RootVolumeSize" => "RootVolumeSize"
    setting.1136119684.namespace:     "aws:autoscaling:launchconfiguration" => "aws:autoscaling:launchconfiguration"
    setting.1136119684.resource:           "" => ""
    setting.1136119684.value:              "8" => "8"
    setting.1201312680.name:             "ListenerEnabled" => "ListenerEnabled"
    setting.1201312680.namespace:   "aws:elb:listener:443" => "aws:elb:listener:443"
    setting.1201312680.resource:         "" => ""
    setting.1201312680.value:             "false" => "false"

This feature/bug was present for years and is still not fixed:

hashicorp/terraform#6729
hashicorp/terraform-provider-aws#901
hashicorp/terraform#6257
hashicorp/terraform-provider-aws#280
hashicorp/terraform#11056
hashicorp/terraform-provider-aws#461

(tested some ideas from the links above, nothing worked 100%)

The only possible solution is to add this:

lifecycle {
    ignore_changes = ["setting"]
}

but it’s a hack since it will not update the env if you update any of the settings.

Regarding terraform-aws-elastic-beanstalk-environment recreating the settings all the time, here what’s probably happening:

• Terraform sends all settings to AWS, but some of them are not relevant to the environment you are deploying
• Elastic Beanstalk accepts all settings, applies the relevant ones, and throws away the rest
• Next time Terraform asks about the settings, Elastic Beanstalk returns a subset of the values and probably in different order
• Terraform can’t decide/calculate if the settings are the same - they sure look different (and would require an advanced algorithm to determine if they are the same)
• Terraform assigns new ID to the entire array of settings and tries to recreate all of them
• Elastic Beanstalk accepts the settings, applies the relevant ones, and throws away the rest - the cycle repeats

What’s a possible solution?
Introduce var.settings (list of maps) to be able to provide all the required settings from outside of the module.
It might work, but in practice would be very difficult to know all the needed settings and tedious to implement.

[osterman]

Excellent write up

[tigrannajaryan]

As a minimum would be possible to at least not print the key/value pairs which are unchanged? It would significantly reduce the amount of noise and make it easier to understand if there is anything really different in the "settings".
In my attempts to do terraform apply with unchanged EB settings the vast majority of them are printed to the console with exact same before => after values.
So for example I get outputs like this:

...
      setting.1610162273.name:      "BatchSizeType" => "BatchSizeType"
      setting.1610162273.namespace: "aws:elasticbeanstalk:command" => "aws:elasticbeanstalk:command"
      setting.1610162273.resource:  "" => ""
      setting.1610162273.value:     "Fixed" => "Fixed"
      setting.1683651254.name:      "Application Healthcheck URL" => "Application Healthcheck URL"
      setting.1683651254.namespace: "aws:elasticbeanstalk:application" => "aws:elasticbeanstalk:application"
      setting.1683651254.resource:  "" => ""
      setting.1683651254.value:     "/healthcheck" => "/healthcheck"
      setting.2175068570.name:      "BatchSize" => "BatchSize"
      setting.2175068570.namespace: "aws:elasticbeanstalk:command" => "aws:elasticbeanstalk:command"
      setting.2175068570.resource:  "" => ""
      setting.2175068570.value:     "1" => "1"

...

Is there a reason Terraform prints these unchanged settings?

[osterman]

No, the plan output is determined by terraform and cannot be silenced. We have no control over it. Needs to be fixed in the provider.

[tigrannajaryan]

So I guess what I am asking for would be a Terraform feature, right? (I am completely new to Terraform , just started using yesterday).

[aknysh]

@tigrannajaryan
yes, looks like the issue is in the TF AWS provider https://github.com/terraform-providers/terraform-provider-aws

[michael-kutsch]

Some of the settings used here do not apply to every Beanstalk setup. E.g. when using an ALB (and not the classic ones) all aws:elb:*-settings will be reapplied every time. Those settings are not valid for an ALB or NLB (which have both aws:elbv2:*-settings only).

Besides that, Beanstalk/AWS sometimes accepts slightly different values when it comes to whitespaces. In this module, this setting will cause a reset of all settings, because of "wrong" whitespacing:

setting { # setting changes every time on apply
    namespace = "aws:autoscaling:launchconfiguration"
    name      = "SSHSourceRestriction"
    value     = "tcp, 22, 22, ${var.ssh_source_restriction}" // <= whitespaces
  }

This will fix it (in this case):

  setting { # no unecessary change on apply
    namespace = "aws:autoscaling:launchconfiguration"
    name      = "SSHSourceRestriction"
    value     = "tcp,22,22,${var.ssh_source_restriction}" // <= no whitespaces
  }

What I just did, was to remove every setting which is not applicable for a v2-loadbalancer and checking every setting on whitespace-"errors". Voila: no changes anymore.

I hope that the beanstalk setting will become an own resource, that should fix some other flaws with it. Knowing that whitespace-thingy and the reapplication of mismatched settings fixed it for me.

@aknysh : can it be that you created a EBS setup with a classic loadbalancer? This would be same case I described but vice versa. If you add a v1-ELB, all the v2-settings are reapplied every time.

[michael-kutsch]

Oh, another thing... seems to be a bug in terraform to me:

settings with lists may be sent unsorted to the AWS API:

  setting { 
    namespace = "aws:ec2:vpc"
    name      = "Subnets"
    value     = "${join(",", var.private_subnets)}" // order of values might change
  }

Just sort the input:

  setting {
    namespace = "aws:ec2:vpc"
    name      = "Subnets"
    value     = "${join(",", sort(var.private_subnets))}" // every time the same
  }

[michael-kutsch]

For the module, it would make sense to apply the settings as a map from the outside.

With this, we could eliminate those settings which are not applicable for the setup (e.g. passing aws:elb:* settings for a v2-load balancer). Maybe with some locals and an if/else-approach?

As soon as I have time, I'll try this out and create a PR for this for further discussion.

[aknysh]

@DrFunk-n-stein thanks!
nice findings.
yes, unsorted settings, whitespaces in values, settings not relevant to the environment you are creating, etc. - all case EB to recreate all settings.
As mentioned above and as you pointed out, providing a map of settings from outside would solve that issue.
Not sure how easy the module would be to use (users would have to know and find out all settings they need).
Please open a PR, we'll review it promptly.
And thanks for the help!

[michael-kutsch]

narf. We have to wait for 0.12.:
hashicorp/terraform#7034

[michael-kutsch]

I was making my mind about this, maybe we can do a workaround and create several resources of an environment equipped with count. Those could be dependant on the type of load balancer - this way we could handle separate configurations. Downside: it's bloats up the module and there might be a fix for the list-of-maps-issue in 0.12

[jdn-za]

This is something that I have been meaning to try for a while, which won't solve the problem at all but might at the least make it easier to read

https://github.com/coinbase/terraform-landscape

[adjavaherian]

it looks like some of these problems can be solved by using an app template. https://www.terraform.io/docs/providers/aws/r/elastic_beanstalk_configuration_template.html. worked for me.

[eriknaslund]

@adjavaherian - thanks a bunch for your tip about the configuration template! It did the trick for me as well.

There's one problem with this approach though. Updating the elastic_beanstalk_configuration_template will not cause the elastic_beanstalk_environment to update.

[lyenliang]

it looks like some of these problems can be solved by using an app template. https://www.terraform.io/docs/providers/aws/r/elastic_beanstalk_configuration_template.html. worked for me.

The next time I update the setting in aws_elastic_beanstalk_configuration_template, and run terraform plan, I see a lot of settings again:

      setting.399529023.name:       "MinSize" => "MinSize"
      setting.399529023.namespace:  "aws:autoscaling:asg" => "aws:autoscaling:asg"
      setting.399529023.resource:   "" => ""
      setting.399529023.value:      "2" => "2"
      setting.4044472833.name:      "Xms" => "Xms"
      setting.4044472833.namespace: "aws:elasticbeanstalk:container:tomcat:jvmoptions" => "aws:elasticbeanstalk:container:tomcat:jvmoptions"
      setting.4044472833.resource:  "" => ""
      setting.4044472833.value:     "1024m" => "1024m"
      setting.4107956248.name:      "Application Healthcheck URL" => "Application Healthcheck URL"
      setting.4107956248.namespace: "aws:elasticbeanstalk:application" => "aws:elasticbeanstalk:application"
      setting.4107956248.resource:  "" => ""
      setting.4107956248.value:     "/" => "/"

[mshivanna]

this is a known bug and we are kind of stuck on this. planning to write custom comparison

[Sytten]

I was playing with a custom diff in the resource, but I could not detect an abnormal comparison of settings using the test TestAccAWSBeanstalkEnv_settings_update. I am really unsure that the problem is really... The function diff.HasChange("setting") returns false when applying the same configuration which makes no sense considering it does return a diff. I am missing something.

[boomshadow]

@aknysh You had mentioned at the beginning that we could hackishly override the changes:

The only possible solution is to add this:

lifecycle {
ignore_changes = ["setting"]
}

but it’s a hack since it will not update the env if you update any of the settings.

I am interested in adding that option but I get an error "The block type name "lifecycle" is reserved for use by Terraform in a future version."

Did you get that working, and if so, where did you make the code change?

Edit

Never mind, I see that this is a terraform 0.12.x issue; they removed the ability to set lifecycle within modules 😢

[helvalius]

Would be nice to have a fix/workaround here as seeing the changes is quite annoying. Seems that you can't configure lifecycle in terraform 0.12.x as @boomshadow also described.

[informeto]

This is still not fixed completely. I've added a comment on the merged fix that still has potential for the problem.
b809467#r39525364

[kartikrao]

@helvalius @maximmi - This is still happening on v0.22

[helvalius]

Yea the fix from @informeto helps and there is a new issue with the parallel merge of the SpotOn support, for me those variables are still having the issue. I'll see to update that in my pull request or someone else could do a fix for those.

[sseveringhaus-newclassrooms]

I just wanted to link to this issue: hashicorp/terraform-provider-aws#1471

...as it seems to be where there is some relatively recent discussion regarding the Terraform provider. There is also a workaround that is quite effective: Just add resource = "" to each setting.

[informeto]

For me, switching to AWS provider v3 solved the problem of reapplication of settings. I was earlier on 2.70.0 and once I switched to 3.22.0, this issue was gone.

[florian0410]

Using aws provider with version 3.22 and terraform 0.14.7 and still having the issue using a Classic load balancer only:

  # module.elastic_beanstalk_environment.aws_elastic_beanstalk_environment.default will be updated in-place
  ~ resource "aws_elastic_beanstalk_environment" "default" {
        id                     = "e-vzua8aifyy"
        name                   = "envir"
        tags                   = {
            "Environment" = "envir"
        }
        # (16 unchanged attributes hidden)

      + setting {
          + name      = "HealthCheckPath"
          + namespace = "aws:elasticbeanstalk:environment:process:default"
          + value     = "/"
        }
      + setting {
          + name      = "InstancePort"
          + namespace = "aws:elb:listener"
          + value     = "80"
        }
      + setting {
          + name      = "ListenerEnabled"
          + namespace = "aws:elb:listener"
          + value     = "true"
        }
      + setting {
          + name      = "ListenerProtocol"
          + namespace = "aws:elb:listener"
          + value     = "HTTP"
        }
      + setting {
          + name      = "Port"
          + namespace = "aws:elasticbeanstalk:environment:process:default"
          + value     = "80"
        }
      + setting {
          + name      = "Protocol"
          + namespace = "aws:elasticbeanstalk:environment:process:default"
          + value     = "HTTP"
        }
        # (62 unchanged blocks hidden)
    }

My configs are default for these params.

[lukas085]

Is this issue still not resolved? it's been two years since last @florian0410 post and I have the same problem. I'm using classic loadbalancer. My Terraform version is 1.1.9 with aws v4.58.0 provider. Module version v0.49.0

[peterdudfield]

I'm still getting this issue

Interestingly I dont get it here, but I do have the issue here
but I dont really understand the differentce

terraform version 1.7.2, aws version ~> 5.0