aws db parameter group ... converts keys and values to lower case and fails 'apply' due to aws_db_parameter_group changes

[ebgc]

Hi there,

Thank you for opening an issue. Please note that we try to keep the Terraform issue tracker reserved for bug reports and feature requests. For general usage questions, please see: https://www.terraform.io/community.html.

Terraform Version

mac os x 10.12
Terraform v0.7.4

Affected Resource(s)

• resource_aws_db_parameter_group
• aws_db_instance

Terraform Configuration Files

resource "aws_db_instance" "default" {
  allocated_storage = "${var.db_storage}"
  allow_major_version_upgrade = false
  apply_immediately = false
  auto_minor_version_upgrade = false
  backup_retention_period = "${var.backup_retention_period}"
  backup_window = "${var.backup_window}"
  copy_tags_to_snapshot = "${var.copy_tags_to_snapshot}"
  db_subnet_group_name = "${aws_db_subnet_group.rds.id}"
  engine = "${var.db_engine}"
  engine_version = "${var.db_version}"
  final_snapshot_identifier = "${var.env_type}-${var.env_name}-${var.env_number}-${var.service}-${var.date}"
  identifier = "${var.env_type}-${var.env_name}-${var.env_number}-${var.service}-${var.db_engine}"
  instance_class = "${var.db_instance}"
  maintenance_window = "${var.maintenance_window}"
  multi_az = "${var.multi_az}"
  name = "${var.db_name}"
  parameter_group_name = "${aws_db_parameter_group.default.id}"
  password = "${var.db_password}"
  publicly_accessible = "${var.db_public_accessible}"
  skip_final_snapshot = "${var.skip_final_snapshot}"
  snapshot_identifier = "${var.db_snapshot_id}"
  storage_type = "${var.db_storage_type}"
  username = "${var.db_username}"
  vpc_security_group_ids = ["${aws_security_group.rds.id}"]

  lifecycle {
    ignore_changes = ["final_snapshot_identifier"]
  }

  tags {
    Name = "${var.env_type}-${var.env_name}-${var.env_number}-${var.service}-${var.db_engine}"
    Description = "${var.env_type}-${var.env_name}-${var.env_number}-${var.service}-${var.db_engine}"
    Environment = "${var.environment}"
  }
}

resource "aws_db_parameter_group" "default" {
  name = "${var.env_type}-${var.env_name}-${var.env_number}-${var.service}-${data.template_file.db_family_no_dot.rendered}-pg"
  description = "${var.env_type}-${var.env_name}-${var.env_number}-${var.service}-${data.template_file.db_family_no_dot.rendered}-pg"
  family = "${data.template_file.db_family_dot.rendered}"

  # tf ignores them : timezone, client_encoding
  parameter {
    name = "lc_messages"
    value = "en_US.utf8"
    apply_method = "immediate"
  }

  parameter {
    name = "lc_monetary"
    value = "en_US.utf8"
    apply_method = "immediate"
  }

  parameter {
    name = "lc_numeric"
    value = "en_US.utf8"
    apply_method = "immediate"
  }

  parameter {
    name = "lc_time"
    value = "en_US.utf8"
    apply_method = "immediate"
  }

  tags {
    Name = "${var.env_type}-${var.env_name}-${var.env_number}-${var.service}-${data.template_file.db_family_no_dot.rendered}-pg"
    Description = "${var.env_type}-${var.env_name}-${var.env_number}-${var.service}-${data.template_file.db_family_no_dot.rendered}-pg"
    Environment = "${var.environment}"
  }
  lifecycle {
    ignore_changes = ["name", "description", "family", "tags"]
  }
}

Debug Output

1 error(s) occurred:

• aws_db_parameter_group.default: diffs didn't match during apply. This is a bug with Terraform and should be reported as a GitHub Issue.

Please include the following information in your report:

Terraform Version: 0.7.4
Resource ID: aws_db_parameter_group.default
Mismatch reason: attribute mismatch: parameter.1848500544.value
Diff One (usually from plan): *terraform.InstanceDiff{mu:sync.Mutex{state:0, sema:0x0}, Attributes:map[string]*terraform.ResourceAttrDiff{"parameter.1848500544.value":*terraform.ResourceAttrDiff{Old:"en_us.utf8", New:"en_US.utf8", NewComputed:false, NewRemoved:false, NewExtra:interface {}(nil), RequiresNew:false, Sensitive:false, Type:0x0}, "parameter.2080321556.value":*terraform.ResourceAttrDiff{Old:"en_us.utf8", New:"en_US.utf8", NewComputed:false, NewRemoved:false, NewExtra:interface {}(nil), RequiresNew:false, Sensitive:false, Type:0x0}, "parameter.4193353794.value":*terraform.ResourceAttrDiff{Old:"en_us.utf8", New:"en_US.utf8", NewComputed:false, NewRemoved:false, NewExtra:interface {}(nil), RequiresNew:false, Sensitive:false, Type:0x0}, "parameter.3444491795.value":*terraform.ResourceAttrDiff{Old:"en_us.utf8", New:"en_US.utf8", NewComputed:false, NewRemoved:false, NewExtra:interface {}(nil), RequiresNew:false, Sensitive:false, Type:0x0}}, Destroy:false, DestroyTainted:false}
Diff Two (usually from apply): *terraform.InstanceDiff{mu:sync.Mutex{state:0, sema:0x0}, Attributes:map[string]*terraform.ResourceAttrDiff(nil), Destroy:false, DestroyTainted:false}

Panic Output

n.a.

Expected Behavior

take what aws api returns without modification (dont use flattenParameters)
save what tf file defines at tfstate file without modification (dont use flattenParameters)

Actual Behavior

all keys and values will be down-cased on tfstate file even though on initial run they are set on aws as defined in tf file

Steps to Reproduce

have an aws_db_instance (postgres, 9.4.5) with resource_aws_db_parameter_group (postgres9.4) which sets 'lc_messages' to 'en_US.utf8', or 'timezone' to 'UTF'

even though we set 'en_US.utf8' in tf file (terraform during 'apply' will set 'en_US.utf8' on aws rds resource) - tfstate file will have 'en_us.utf8' set.
next run of terraform 'apply' will read resource from aws and convert it to lower case 'en_us.utf8' (see below 'flattenParameters' function) which will not fail compared to state file value (match). but because there is a diff with tf file (remember we set 'en_US.utf8') value it will try to modify the resource.
here another bug comes in play where terraform complains that db parameter group has a diff and reports: '* aws_db_parameter_group.default: diffs didn't match during apply.'

Important Factoids

---- terraform/builtin/providers/aws/resource_aws_db_parameter_group.go ----
func resourceAwsDbParameterGroupRead
...
d.Set("parameter", flattenParameters(describeParametersResp.Parameters))
...
---- terraform/builtin/providers/aws/resource_aws_db_parameter_group.go ----
func flattenParameters
...
r["name"] = strings.ToLower(_i.ParameterName)
...
r["value"] = strings.ToLower(_i.ParameterValue)
....
---- terraform/builtin/providers/aws/structure.go ----

---- terraform/builtin/providers/aws/structure.go ----

References

[svenwltr]

It is the opposite from what they wanted to achieve here: #1225

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

[mikegreen]

Appears this is still an issue, and also with rds_cluster_parameter_group