-
Notifications
You must be signed in to change notification settings - Fork 9.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Modify aws_db_instance and delete aws_db_parameter_group breaks #6448
Comments
Hi @jrobison-sb 👋 Thanks for reporting this and sorry for the trouble. Can you confirm a few pieces of information here?
Without seeing the debug log here its not possible to tell definitively, but my initial hunch is that the operations are happening in order, but that the parameter group change is requiring an instance reboot still, something which the If you would like to see the debug log yourself or provide it for reference in this issue via a Gist, documentation can be found here: https://www.terraform.io/docs/internals/debugging.html Please let us know about the items above and hopefully we can dig into this further, thanks. |
@bflad Thanks for your fast response. I'll answer each of your questions below.
I'm using
Yes.
No, the RDS instance is unchanged, and still using the old parameter group. Since the RDS instance wasn't changed, there is no As for a debug log, my module has hundreds (or more) of resources, and a full debug log is 128,000 lines, so I'm hesitant to publicly post that. If it's highly necessary, let me know, and maybe I can creatively grep or parse through it somehow. |
I'm experiencing what may be the same problem. It looks like the modification simply does not happen in my case. After an apply, the parameter group used by my instance is still the original one: If you remove the deletion of your old group, does the modification actually happen at all? |
I'm having the same issue when trying to upgrade from Postgres 10.6 to 11.1 on RDS. As with @liamg-form3 the parameter group remains the original one. My code doesn't explicitly delete the parameter group, the only change was to upgrade the engine version and use a new This workaround has worked fine on several instances with the same issue:
Not very elegant but quite easy to do, and some downtime is required anyway with RDS when upgrading Postgres. |
I believe the key here is:
After calling |
We got stuck trying to change the parameter group (PG) of our Postgres DB. The error msg: Assuming we want to change from PG A -> B our workaround solution for this was.
Hope this helps someone in the future. |
This requires a change in name for each (using `name_prefix`), which will likely recreate all parameter groups. This is intended to work around an issue where AWS won't allow the parameter group to be recreated while RDS are using it. See these issues for reference: * hashicorp/terraform-provider-aws#6448 * hashicorp/terraform-provider-aws#6049 * hashicorp/terraform-provider-aws#2402 * hashicorp/terraform-provider-aws#1571 * hashicorp/terraform-provider-aws#526
This requires a change in name for each (using `name_prefix`), which will likely recreate all parameter groups. This is intended to work around an issue where AWS won't allow the parameter group to be recreated while RDS are using it. See these issues for reference: * hashicorp/terraform-provider-aws#6448 * hashicorp/terraform-provider-aws#6049 * hashicorp/terraform-provider-aws#2402 * hashicorp/terraform-provider-aws#1571 * hashicorp/terraform-provider-aws#526
I also encountered this while trying to change the name of a db param group Plan:
Action: Action: Action:
Impact: Worked. Once the group was no longer actively assigned to a database, terraform could rename the custom xxxxx-postgres-11 to yyyy-postgres-11 database group. TF then swapped the default.postgres.11 group for the yyyy-postgres-11 group by applying the change immediately. Suggestion: |
The same issue 2 years after. No RDS cleanup/destroy possible: Error: InvalidDBParameterGroupState: One or more database instances are still members of this parameter group ambari-hdf-peterz, so the group cannot be deleted Error: error deleting RDS Cluster (ambari-hdf-peterz): DBClusterSnapshotAlreadyExistsFault: Cannot create the cluster snapshot because one with the identifier ambari-hdf-final-snapshot already exists. Error: Error deleting DB parameter group: InvalidDBParameterGroupState: One or more database instances are still members of this parameter group xxxxx, so the group cannot be deleted [terragrunt] 2020/06/19 12:37:01 Hit multiple errors: |
Hi guys, I agree with the previous comment. This causes problems in the automation and use of pipelines. In my case, when using Jenkins. Removing or modifying RDS is impossible in those tasks that pursue terraforms. Unfortunately, I can not offer a solution to the problem in the form of code, I can only assume that this option will work:
Here is our code: resource "aws_db_parameter_group" "pg" { parameter { I understand that there are workarounds using AWS console, but agree that this is not a solution to this problem. Reaction to destroy: Error deleting DB parameter group: InvalidDBParameterGroupState: One or more database instances are still members of this parameter group paramg, so the group cannot be deleted Then the pipeline will not move. UPD
I hope it will be useful to someone. |
This is preventing me to upgrade a SQL Server 2017 to 2019 on RDS. |
Can't move the postgres engine version on RDS because of this :( |
You want to use this snippet on the
This will create the new DB parameter group, update the DB, and finally delete the old parameter group. |
maybe there is no additional check that instances are updated too (Available state) |
Along with |
|
I came across this today but the underlying root cause was different from the ones listed above. We tried upgrading an RDS PostgreSQL instance from 11.13 to 12.10 and the modification was printed out as completed:
However, checking the TF state for this resource, it is still on version 11.13:
Upon visiting
And upon checking the logs that is under this:
So it's a tad unexpected that a failed/incomplete upgrade operation reported as successful ( But yeah, posting this here just in case other peeps are running into the same situation and scratch your head repeatedly at the (slightly) wrong places. |
The following situations were tested with a minimal configuration using Postgres and a custom parameter group with
In each case, the To allow time for feedback on the reproduction and documentation changes, we're going to leave this issue open. If no objections are received we will close this as completed in the near future. A more detailed breakdown of the minimal reproduction is included below: ConfigShowParameter group swap (original issue)terraform {
required_providers {
aws = {
source = "hashicorp/aws"
version = "~> 4.0"
}
}
}
provider "aws" {}
#resource "aws_db_parameter_group" "repro" {
#name_prefix = "jb-test"
#family = "postgres13"
#parameter {
#name = "log_connections"
#value = "1"
#}
#lifecycle {
#create_before_destroy = true
#}
#}
resource "aws_db_parameter_group" "repro-new" {
name_prefix = "jb-test-new"
family = "postgres13"
parameter {
name = "log_connections"
value = "1"
}
lifecycle {
create_before_destroy = true
}
}
resource "aws_db_instance" "repro" {
identifier = "jb-test"
instance_class = "db.t3.micro"
allocated_storage = 10
engine = "postgres"
engine_version = "13.7"
username = "jb"
password = "testtest"
parameter_group_name = aws_db_parameter_group.repro-new.name # was: aws_db_parameter_group.repro.name
apply_immediately = true
allow_major_version_upgrade = true
skip_final_snapshot = true
backup_retention_period = 1
}
Engine version upgrade (subsequent comments)terraform {
required_providers {
aws = {
source = "hashicorp/aws"
version = "~> 4.0"
}
}
}
provider "aws" {}
resource "aws_db_parameter_group" "repro" {
name_prefix = "jb-test" # "jb-test-nameupdate"
family = "postgres13" # "postgres12"
parameter {
name = "log_connections"
value = "1"
}
lifecycle {
create_before_destroy = true
}
}
resource "aws_db_instance" "repro" {
identifier = "jb-test"
instance_class = "db.t3.micro"
allocated_storage = 10
engine = "postgres"
engine_version = "13.7" # "12.7", "12.8"
username = "jb"
password = "testtest"
parameter_group_name = aws_db_parameter_group.repro.name
apply_immediately = true
allow_major_version_upgrade = true
skip_final_snapshot = true
backup_retention_period = 1
}
Reproduction DetailsShow$ terraform -v
Terraform v1.3.3
on darwin_arm64
+ provider registry.terraform.io/hashicorp/aws v4.36.1 Parameter group replacement with new group
Parameter group name_prefix change
Minor version engine upgrade
Major version engine upgrade
|
Closing given no further response to reproductions and documentation changes. |
Error: Error creating DB Cluster Parameter Group: DBParameterGroupAlreadyExists: Parameter group **** already exists when i am using lifecycle { |
@Kristin0 if AWS won't let you have two parameter groups with the same name, then you either need to use |
I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues. |
Community Note
Terraform Version
Affected Resource(s)
Terraform Configuration Files
If I have a running RDS instance which is using a DB parameter group, and I want to modify the DB instance to use a different parameter group, and delete the old parameter group, I'll get a TF plan which looks like this:
Expected Behavior
Modify the RDS instance, then delete the parameter group which is now unused.
Actual Behavior
It tries to delete the parameter group first, which fails because the parameter group is still in use.
If it would have done the
modify
action first on the DB instance, it would then be able to do thedestroy
action on the now unused parameter group.Steps to Reproduce
Run a plan which plans to modify a DB instance to change the parameter group to some other parameter group, and which also plans to delete the now unused parameter group.
The text was updated successfully, but these errors were encountered: