Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

FinalSnapshotIdentifier is required when a final snapshot is required #4910

Closed
ghost opened this issue Jun 21, 2018 · 14 comments
Closed

FinalSnapshotIdentifier is required when a final snapshot is required #4910

ghost opened this issue Jun 21, 2018 · 14 comments
Labels
service/rds Issues and PRs that pertain to the rds service. stale Old or inactive issues managed by automation, if no further action taken these will get closed. upstream-terraform Addresses functionality related to the Terraform core binary.

Comments

@ghost
Copy link

ghost commented Jun 21, 2018

This issue was originally opened by @ujjwalGargFS as hashicorp/terraform#18285. It was migrated here as a result of the provider split. The original body of the issue is below.


Terraform Version

Terraform v0.11.7
+ provider.aws v1.22.0

Terraform Configuration Files

resource "aws_rds_cluster" "aurora-cluster-ci" {
  cluster_identifier        = "aurora-cluster-ci"
  engine                    = "aurora-mysql"
  availability_zones        = ["us-east-1a", "us-east-1b", "us-east-1c"]
  database_name             = "${var.rds_dbname}"
  master_username           = "${var.rds_username}"
  master_password           = "${var.rds_password}"
  backup_retention_period   = 5
  engine_version            = "5.7.16"
  preferred_backup_window   = "07:00-09:00"
  apply_immediately         = true
  final_snapshot_identifier = "ci-aurora-cluster-backup"
  skip_final_snapshot       = true
}

Debug Output

1 error(s) occurred:

* aws_rds_cluster.aurora-cluster-ci (destroy): 1 error(s) occurred:

* aws_rds_cluster.aurora-cluster-ci: RDS Cluster FinalSnapshotIdentifier is required when a final snapshot is required

Crash Output

Expected Behavior

"aurora-cluster-ci" should get destroyed

Actual Behavior

Steps to Reproduce

Additional Context

References

@bflad
Copy link
Contributor

bflad commented Jun 21, 2018

Hi @ujjwalGargFS 👋 Sorry you ran into unexpected behavior here.

Was the skip_final_snapshot attribute changed at the same time as the resource deletion? If so, does it work if you apply that change first, then try deleting the resource?

@bflad bflad added waiting-response Maintainers are waiting on response from community or contributor. service/rds Issues and PRs that pertain to the rds service. labels Jun 21, 2018
@avengers009
Copy link

This error occurs, when we don't include this "final_snapshot_identifier" when you build.

delete it manually, and try build with "final_snapshot_identifier = foo" and destroy, it works for me.

@avengers009
Copy link

image

@radeksimko radeksimko added the upstream-terraform Addresses functionality related to the Terraform core binary. label Jun 21, 2018
@lkd
Copy link

lkd commented Aug 14, 2018

Given the sharp edges in RDS w.r.t. snapshots, maybe final_snapshot_identifier should just be required, with a note that you have to fill it in even if skip_final_snapshot is ever set true ?

@Jonathan-7
Copy link

Somewhat related to this. In the documentation, it says that when final_snapshot_identifier is omitted no final snapshot will be made. However, that is not the case since I get the same issue on delete when I omit it.

@bflad bflad removed the waiting-response Maintainers are waiting on response from community or contributor. label Oct 9, 2018
@dohoangkhiem
Copy link
Contributor

During the creation of RDS instance, if any kind of error happened (for example: failed to set the engine version), the attribute skip_final_snapshot is not written in the terraform state, which will cause annoying issue when running terraform destroy. It's really a critical issue for our automation script as we want all partial-update resources must be cleaned up in case of failures.

@futtetennista
Copy link

The workaround that worked in my case was to set the skip_final_snapshot argument to true - one can set this after creating the DB instance - apply the change and then delete the instance.

@HebertCL
Copy link

I just came across this issue and although I find it relatively simple to workaround just like @futtetennista mentioned, I think it could be easily avoided. According to aws_db_instance documentation the valueskip_final_snapshot is by default set to false, which means a snapshot will be created, therefore it will require a name. Couple of improvement ideas come to my mind: 1) Make skip_final_snapshot a required value. Even though it seems confusing the value is optional but mandatory when it comes to destroy the resource, I believe it has more to do with how AWS is designed to work rather than how Terraform works. 2) Document this behavior in the resource documentation and explain that AWS is expecting this values to be present when the resources will be deleted.

@edmundcraske
Copy link
Contributor

Just hit this - is it possible to make it so 'skip_final_snapshot' can be changed as part of a destroy, and that it doesn't need applying first? Or is this just intended/expected behaviour?

@efernandes-dev-ops
Copy link

So an easy way I got around this was to set 'skip_final_snapshot' = true and apply the change to the rds cluster. Once the change applied, I then deleted the rds cluster and added the updated changes I wanted to make to the cluster.

@chuleh
Copy link

chuleh commented Jul 27, 2019

In my case I had to manually edit the .tfstate file and set "skip_final_snapshot" to true. Then it worked.

fffonion added a commit to fffonion/terraform-provider-aws that referenced this issue Aug 22, 2019
The description of `final_snapshot_identifier` doesn't feel correct to me.
When `skip_final_snapshot` is by default set to false and `final_snapshot_identifier`
is omitted, the provider errored with 
> RDS Cluster FinalSnapshotIdentifier is required when a final snapshot is require

Related: hashicorp#4910
paulrigor added a commit to paulrigor/terraform-provider-aws that referenced this issue Sep 6, 2019
* provider: Ensure needs-triage issue labeling step in GitHub Actions v2 is conditional on opened action

The payload is slightly different in v2, so actions/github@v1.0.0 fails with the action flag and does not return a neutral status. Instead, we will pre-filter the step via if.

References:

- https://help.github.com/en/articles/workflow-syntax-for-github-actions#jobsjob_idif
- https://github.com/actions/github/blob/4f0213ed102c4b518a7ce8bc4e6268bdd710770e/entrypoint.js#L73-L85

* resource/aws_lb_target_group_attachment: Prevent potential panic and update test function names for codebase conventions

Output from acceptance testing:

```
--- PASS: TestAccAWSLBTargetGroupAttachment_lambda (31.23s)
--- PASS: TestAccAWSLBTargetGroupAttachment_disappears (91.09s)
--- PASS: TestAccAWSLBTargetGroupAttachment_ipAddress (102.80s)
--- PASS: TestAccAWSLBTargetGroupAttachment_basic (103.02s)
--- PASS: TestAccAWSLBTargetGroupAttachment_withoutPort (123.06s)
--- PASS: TestAccAWSLBTargetGroupAttachment_BackwardsCompatibility (123.37s)
```

* Update CHANGELOG for hashicorp#9610

* Added CW log export value for aurora-postgresql

* tests/resource/aws_rds_cluster_instance: Ensure covering acceptance tests for performance_insights_enabled on Aurora MySQL 1/2 and Postgresql

Reference: hashicorp#9227
Reference: hashicorp#9635

Output from acceptance testing:

```
--- PASS: TestAccAWSRDSClusterInstance_PerformanceInsightsEnabled_AuroraMysql1 (629.12s)
--- PASS: TestAccAWSRDSClusterInstance_namePrefix (677.80s)
--- PASS: TestAccAWSRDSClusterInstance_PerformanceInsightsEnabled_AuroraMysql2 (679.39s)
--- PASS: TestAccAWSRDSClusterInstance_PerformanceInsightsEnabled_AuroraPostgresql (680.27s)
--- PASS: TestAccAWSRDSClusterInstance_CopyTagsToSnapshot (699.01s)
--- PASS: TestAccAWSRDSClusterInstance_generatedName (712.49s)
--- PASS: TestAccAWSRDSClusterInstance_az (722.51s)
--- PASS: TestAccAWSRDSClusterInstance_disappears (763.18s)
--- PASS: TestAccAWSRDSClusterInstance_withInstanceEnhancedMonitor (873.08s)
--- PASS: TestAccAWSRDSClusterInstance_kmsKey (1062.91s)
--- PASS: TestAccAWSRDSClusterInstance_PubliclyAccessible (1247.95s)
--- PASS: TestAccAWSRDSClusterInstance_basic (1423.02s)
```

* Update CHANGELOG for hashicorp#9635

* resource/aws_db_instance: Only send performance_insights_kms_key_id on update if configured

Reference: hashicorp#8792
Reference: hashicorp#9399
Reference: hashicorp#9406

Previously before code update:

```
--- PASS: TestAccAWSRDSDBInstance_PerformanceInsightsRetentionPeriod (667.32s)
--- PASS: TestAccAWSRDSDBInstance_PerformanceInsightsEnabled_EnabledToDisabled (733.03s)
--- PASS: TestAccAWSRDSDBInstance_PerformanceInsightsKmsKeyId (862.56s)
--- FAIL: TestAccAWSRDSDBInstance_PerformanceInsightsEnabled_DisabledToEnabled (5686.66s)
    testing.go:568: Step 2 error: errors during apply:

        Error: Error modifying DB Instance tf-acc-test-5551941564373291595: InternalFailure:
        	status code: 500, request id: 9ad8b0ff-c5a2-43d0-a531-60aa047b8570
```

Output from acceptance testing:

```
--- PASS: TestAccAWSRDSDBInstance_PerformanceInsightsEnabled_EnabledToDisabled (680.88s)
--- PASS: TestAccAWSRDSDBInstance_PerformanceInsightsEnabled_DisabledToEnabled (680.95s)
--- PASS: TestAccAWSRDSDBInstance_PerformanceInsightsRetentionPeriod (685.18s)
--- PASS: TestAccAWSRDSDBInstance_PerformanceInsightsKmsKeyId (851.11s)
```

* resource/aws_rds_cluster_instance: Ensure performance_insights_kms_key_id updates include performance_insights_enabled

Reference: hashicorp#3015

This update ensures the correct RDS API error is shown when attempting to modify the Performance Insights KMS Key ID (we use `ExpectError` in the testing so this passes):

```
--- FAIL: TestAccAWSRDSClusterInstance_PerformanceInsightsKmsKeyId_DefaultKeyToCustomKey (691.76s)
    testing.go:568: Step 2 error: errors during apply:

        Error: Error modifying DB Instance tf-acc-test-2133090011528324987: InvalidParameterCombination: You cannot change your Performance Insights KMS key
```

Previously before code update:

```
--- PASS: TestAccAWSRDSClusterInstance_PerformanceInsightsEnabled_AuroraMysql2 (587.54s)
--- PASS: TestAccAWSRDSClusterInstance_PerformanceInsightsEnabled_AuroraMysql1 (594.85s)
--- PASS: TestAccAWSRDSClusterInstance_PerformanceInsightsKmsKeyId_CustomKey (660.70s)
--- FAIL: TestAccAWSRDSClusterInstance_PerformanceInsightsKmsKeyId_DefaultKeyToCustomKey (667.49s)
    testing.go:568: Step 2 error: errors during apply:

        Error: Error modifying DB Instance tf-acc-test-4757962708939555369: InvalidParameterCombination: Can not set PerformanceInsightsKMSKeyId without EnablePerformanceInsights
        	status code: 400, request id: c64b9f56-1b56-420c-9575-c5cfed87270b

          on /var/folders/v0/_d108fkx1pbbg4_sh864_7740000gn/T/tf-test833219777/main.tf line 15:
          (source code not available)

--- PASS: TestAccAWSRDSClusterInstance_PerformanceInsightsEnabled_AuroraPostgresql (699.46s)
```

Output from acceptance testing:

```
--- PASS: TestAccAWSRDSClusterInstance_PerformanceInsightsKmsKeyId_AuroraMysql2_DefaultKeyToCustomKey (630.56s)
--- PASS: TestAccAWSRDSClusterInstance_PerformanceInsightsEnabled_AuroraMysql1 (649.52s)
--- PASS: TestAccAWSRDSClusterInstance_generatedName (651.77s)
--- PASS: TestAccAWSRDSClusterInstance_PerformanceInsightsKmsKeyId_AuroraMysql1_DefaultKeyToCustomKey (652.94s)
--- PASS: TestAccAWSRDSClusterInstance_PerformanceInsightsEnabled_AuroraPostgresql (670.18s)
--- PASS: TestAccAWSRDSClusterInstance_PerformanceInsightsKmsKeyId_CustomKey (671.61s)
--- PASS: TestAccAWSRDSClusterInstance_PerformanceInsightsKmsKeyId_AuroraPostgresql_DefaultKeyToCustomKey (672.19s)
--- PASS: TestAccAWSRDSClusterInstance_PubliclyAccessible (683.28s)
--- PASS: TestAccAWSRDSClusterInstance_PerformanceInsightsEnabled_AuroraMysql2 (690.78s)
--- PASS: TestAccAWSRDSClusterInstance_disappears (730.05s)
--- PASS: TestAccAWSRDSClusterInstance_CopyTagsToSnapshot (732.08s)
--- PASS: TestAccAWSRDSClusterInstance_az (747.28s)
--- PASS: TestAccAWSRDSClusterInstance_namePrefix (764.03s)
--- PASS: TestAccAWSRDSClusterInstance_kmsKey (792.98s)
--- PASS: TestAccAWSRDSClusterInstance_withInstanceEnhancedMonitor (822.65s)
--- PASS: TestAccAWSRDSClusterInstance_basic (1516.75s)
```

* resource/aws_db_instance: Ensure monitoring attributes are always written to state and retry ModifyDBInstance on IAM eventual consistency error

References:

- hashicorp#315
- hashicorp#2188
- hashicorp#5559

Previously before code updates:

```
--- FAIL: TestAccAWSDBInstance_MonitoringRoleArn_RemovedToEnabled (430.95s)
    testing.go:568: Step 2 error: errors during apply:

        Error: Error modifying DB Instance tf-acc-test-1165998526456666486: InvalidParameterValue: IAM role ARN value is invalid or does not include the required permissions for: ENHANCED_MONITORING
        	status code: 400, request id: 524f599d-3870-48b3-843e-28885ae3f75c

          on /var/folders/v0/_d108fkx1pbbg4_sh864_7740000gn/T/tf-test908490254/main.tf line 29:
          (source code not available)

--- PASS: TestAccAWSDBInstance_MonitoringRoleInterval (565.47s)
--- PASS: TestAccAWSDBInstance_MonitoringRoleArn_EnabledToDisabled (593.87s)
--- PASS: TestAccAWSDBInstance_MonitoringRoleArn_EnabledToRemoved (626.12s)
```

Output from acceptance testing:

```
--- PASS: TestAccAWSDBInstance_MonitoringRoleArn_EnabledToDisabled (587.93s)
--- PASS: TestAccAWSDBInstance_MonitoringRoleArn_RemovedToEnabled (614.89s)
--- PASS: TestAccAWSDBInstance_MonitoringRoleArn_EnabledToRemoved (656.13s)
--- PASS: TestAccAWSDBInstance_MonitoringRoleInterval (702.57s)
```

* tests/resource/aws_db_instance: Ensure monitoring_interval testing covers disabling and re-enabling

Output from acceptance testing:

```
--- PASS: TestAccAWSDBInstance_MonitoringInterval (978.85s)
```

* resource/aws_rds_cluster_instance: Ensure monitoring attributes are always written to state

Reference: hashicorp#315

Output from acceptance testing:

```
--- PASS: TestAccAWSRDSClusterInstance_PerformanceInsightsEnabled_AuroraMysql2 (588.69s)
--- PASS: TestAccAWSRDSClusterInstance_PerformanceInsightsEnabled_AuroraMysql1 (640.98s)
--- PASS: TestAccAWSRDSClusterInstance_PerformanceInsightsEnabled_AuroraPostgresql (661.39s)
--- PASS: TestAccAWSRDSClusterInstance_namePrefix (674.32s)
--- PASS: TestAccAWSRDSClusterInstance_CopyTagsToSnapshot (703.51s)
--- PASS: TestAccAWSRDSClusterInstance_PubliclyAccessible (714.73s)
--- PASS: TestAccAWSRDSClusterInstance_disappears (735.39s)
--- PASS: TestAccAWSRDSClusterInstance_generatedName (774.99s)
--- PASS: TestAccAWSRDSClusterInstance_MonitoringRoleArn_EnabledToRemoved (834.26s)
--- PASS: TestAccAWSRDSClusterInstance_MonitoringRoleArn_RemovedToEnabled (876.44s)
--- PASS: TestAccAWSRDSClusterInstance_az (877.93s)
--- PASS: TestAccAWSRDSClusterInstance_MonitoringRoleArn_EnabledToDisabled (912.63s)
--- PASS: TestAccAWSRDSClusterInstance_MonitoringInterval (1037.61s)
--- PASS: TestAccAWSRDSClusterInstance_kmsKey (1205.30s)
--- PASS: TestAccAWSRDSClusterInstance_basic (1301.24s)
```

* Update CHANGELOG.md

* Final retries for gamelift resources

* Final retry after timeout creating gluecrawler

* Final guard duty retry

* r/ram_resource_share_accepter: New resource

* Add arguments, delete to disassociate

* Update module aws/aws-sdk-go to v1.23.0

* New Resources: aws_fsx_lustre_file_system and aws_fsx_windows_file_system

Reference: hashicorp#7035
Reference: hashicorp#7074

Builds on hashicorp#7074 with the following changes:

- Split into two separate resources (`aws_fsx_lustre_file_system` and `aws_fsx_windows_file_system`) to simplify practitioner configuration (e.g. lose the required configuration blocks), ease maintenance for resource logic and single file system arguments (e.g. `kms_key_id` only for Windows), and enhance validations (e.g. `storage_capacity` differences between file systems)
- Add covering acceptance testing with updates/force new for all arguments
- Better alignment with API naming for some arguments (e.g. changing `capacity` into `storage_capacity` to remove ambiguity with `throughput_capacity`)
- Implement [resource Customizable Timeouts](https://www.terraform.io/docs/extend/resources/retries-and-customizable-timeouts.html)
- Implement `skip_final_backup` argument for Windows
- Implement `network_interface_ids`, `owner_id`, and `vpc_id` attributes
- Augment resource documentation with additional notes found in [API reference](https://docs.aws.amazon.com/fsx/latest/APIReference/) and clarify `security_group_ids` import behavior

Output from acceptance testing:

```
--- PASS: TestAccAWSFsxLustreFileSystem_Tags (458.62s)
--- PASS: TestAccAWSFsxLustreFileSystem_basic (526.79s)
--- PASS: TestAccAWSFsxLustreFileSystem_disappears (564.58s)
--- PASS: TestAccAWSFsxLustreFileSystem_WeeklyMaintenanceStartTime (697.83s)
--- PASS: TestAccAWSFsxLustreFileSystem_StorageCapacity (940.13s)
--- PASS: TestAccAWSFsxLustreFileSystem_SecurityGroupIds (1074.08s)
--- PASS: TestAccAWSFsxLustreFileSystem_ImportedFileChunkSize (1276.18s)
--- PASS: TestAccAWSFsxLustreFileSystem_ImportPath (1322.13s)
--- PASS: TestAccAWSFsxLustreFileSystem_ExportPath (1338.75s)

--- PASS: TestAccAWSFsxWindowsFileSystem_Tags (2256.64s)
--- PASS: TestAccAWSFsxWindowsFileSystem_WeeklyMaintenanceStartTime (2346.44s)
--- PASS: TestAccAWSFsxWindowsFileSystem_basic (2404.35s)
--- PASS: TestAccAWSFsxWindowsFileSystem_disappears (2412.10s)
--- PASS: TestAccAWSFsxWindowsFileSystem_AutomaticBackupRetentionDays (2514.61s)
--- PASS: TestAccAWSFsxWindowsFileSystem_DailyAutomaticBackupStartTime (2849.43s)
--- PASS: TestAccAWSFsxWindowsFileSystem_SelfManagedActiveDirectory (2850.64s)
--- PASS: TestAccAWSFsxWindowsFileSystem_ThroughputCapacity (3292.44s)
--- PASS: TestAccAWSFsxWindowsFileSystem_SecurityGroupIds (3609.37s)
--- PASS: TestAccAWSFsxWindowsFileSystem_KmsKeyId (3977.10s)
--- PASS: TestAccAWSFsxWindowsFileSystem_StorageCapacity (4026.67s)
--- PASS: TestAccAWSFsxWindowsFileSystem_CopyTagsToBackups (4389.57s)
```

* Add docs

* Add acceptance test, remove comments, fix minor issue

* Adjust for better code

* Remove the invitation ARN as argument

* Update CHANGELOG.md

* Update website/docs/r/fsx_lustre_file_system.html.markdown

Co-Authored-By: Ryn Daniels <ryn@hashicorp.com>

* Add test sweepers for fsx filesystems

* Update CHANGELOG for hashicorp#7074 and hashicorp#9761

* Final retries for ELB resources

* Final retries for IAM resources

* Final retry after timeout deleting inspector assessment target

* Add AWS Lake Formation service client.
Run 'go mod tidy'.

* Update CHANGELOG for hashicorp#9745

* resource/aws_ram_resource_share_accepter: Minor adjustments and documentation enhancements for initial release

Reference: hashicorp#8259

Output from acceptance testing:

```
--- PASS: TestAccAwsRamResourceShareAccepter_basic (31.57s)
```

* Update CHANGELOG for hashicorp#8259

* provider: Remove Route 53 endpoint workaround in AWS GovCloud (US)

The endpoint information is now correctly included in the AWS Go SDK as of [v1.22.3](aws/aws-sdk-go@1f4898f). The AWS China endpoint information for Route 53 is still not present in the AWS Go SDK.

Output from acceptance testing in AWS GovCloud (US):

```
--- PASS: TestAccAWSRoute53Zone_VPC_Single (66.92s)
--- PASS: TestAccAWSRoute53Zone_VPC_Multiple (96.71s)
--- PASS: TestAccAWSRoute53Zone_VPC_Updates (173.89s)
```

* resource/aws_ram_resource_share_accepter: go fmt

* Final retries after internet gateway timeouts

* Final retry deleting iot types

* Final retry after timeout creating launch config

* tests/data-source/aws_organizations_organization: Add Organizations PreCheck

For consistency with other Organizations testing and to prevent the following failure:

```
       --- FAIL: TestAccAWSOrganizations/Organization/DataSource (3.55s)
            testing.go:568: Step 0 error: errors during apply:

                Error: Error creating organization: AlreadyInOrganizationException: The AWS account is already a member of an organization.
```

Acceptance testing from Organizations member account:

```
        --- SKIP: TestAccAWSOrganizations/Organization/DataSource (1.57s)
            provider_test.go:247: skipping tests; this AWS account must not be an existing member of an AWS Organization
```

Acceptance testing from standalone account:

```
        --- PASS: TestAccAWSOrganizations/Organization/DataSource (29.70s)
```

* Final retries for LB resources

* v2.24.0

* Cleanup after v2.24.0 release

* Update CHANGELOG for hashicorp#9747

* Update CHANGELOG for hashicorp#9748

* Update CHANGELOG for hashicorp#9740

* initial commit

* changes based on feedback: DiffSuppresFunc instead of Computed + Updated Acceptance test

* go fmt

* resource/aws_cloudwatch_log_subscription_filter: Use Default with distribution attribute, refactor tests to ensure all arguments and Kinesis Data Firehose/Kinesis Stream destinations are tested

Reference: https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/SubscriptionFilters.html

Output from acceptance testing (the test failure is a new bug discovery and will be fixed in a subsequent fix):

```
--- PASS: TestAccAWSCloudwatchLogSubscriptionFilter_basic (40.61s)
--- PASS: TestAccAWSCloudwatchLogSubscriptionFilter_disappears (41.09s)
--- PASS: TestAccAWSCloudwatchLogSubscriptionFilter_disappears_LogGroup (42.32s)
--- PASS: TestAccAWSCloudwatchLogSubscriptionFilter_Distribution (51.33s)
--- PASS: TestAccAWSCloudwatchLogSubscriptionFilter_DestinationArn_KinesisStream (108.80s)
--- FAIL: TestAccAWSCloudwatchLogSubscriptionFilter_RoleArn (114.10s)
    testing.go:568: Step 2 error: errors during apply:

        Error: Error updating SubscriptionFilter (tf-acc-test-3057491702311743141) for LogGroup (tf-acc-test-3057491702311743141), message: "Could not deliver test message to specified Kinesis stream. Check if the given kinesis stream is in ACTIVE state.", code: "InvalidParameterException"

          on /var/folders/v0/_d108fkx1pbbg4_sh864_7740000gn/T/tf-test472807723/main.tf line 98:
          (source code not available)

--- PASS: TestAccAWSCloudwatchLogSubscriptionFilter_DestinationArn_KinesisDataFirehose (126.02s)
```

* Update CHANGELOG for hashicorp#9265

* Update CHANGELOG.md

* Final retries for msk cluster

* Add Personailze service

* Run `make fmt`

* Final retries for redshift resources

* Final retries for routes

* Final sagemaker retries

* Security group retries

* Update CHANGELOG.md

* tests/resource/aws_rds_cluster_instance: Fix minor typo in master_password arguments

Reference: https://github.com/terraform-providers/terraform-provider-aws/pull/9746/files#r314539776

* Update CHANGELOG for hashicorp#9710

* Update module terraform-providers/terraform-provider-tls to v2.1.0

* Final retries for SQS resources

* Final retry for transfer server

* Retries and retry removal for opsworks resources

* Fix retiring_principal argument name for aws_kms_grant resource

* Final retry for wafregional acl association

* Final tag retries

* `status` is not an attribute in the schema (hashicorp#9823)

See https://github.com/terraform-providers/terraform-provider-aws/blob/00909998d919faf5494ab8f6b38241deb1957d99/aws/resource_aws_docdb_cluster.go#L33

* resource/aws_kinesis_firehose_delivery_stream: Address PR hashicorp#9103 feedback

Reference: hashicorp#9103

Mainly reverting some of the changes to processing_configuration/error_output_prefix and adding DiffSuppressFunc to handle the processing_configuration configuration block attribute.

Output from acceptance testing:

```
--- PASS: TestAccAWSKinesisFirehoseDeliveryStream_ExtendedS3_ErrorOutputPrefix (151.81s)
--- PASS: TestAccAWSKinesisFirehoseDeliveryStream_missingProcessingConfiguration (158.88s)
--- PASS: TestAccAWSKinesisFirehoseDeliveryStream_ExtendedS3_DataFormatConversionConfiguration_OrcSerDe_Empty (168.93s)
--- PASS: TestAccAWSKinesisFirehoseDeliveryStream_ExtendedS3_ExternalUpdate (169.36s)
--- PASS: TestAccAWSKinesisFirehoseDeliveryStream_SplunkConfigUpdates (170.83s)
--- PASS: TestAccAWSKinesisFirehoseDeliveryStream_s3KinesisStreamSource (136.10s)
--- PASS: TestAccAWSKinesisFirehoseDeliveryStream_ExtendedS3_DataFormatConversionConfiguration_Deserializer_Update (179.46s)
--- PASS: TestAccAWSKinesisFirehoseDeliveryStream_ExtendedS3Updates (180.75s)
--- PASS: TestAccAWSKinesisFirehoseDeliveryStream_importBasic (187.90s)
--- PASS: TestAccAWSKinesisFirehoseDeliveryStream_ExtendedS3_DataFormatConversionConfiguration_Enabled (190.90s)
--- PASS: TestAccAWSKinesisFirehoseDeliveryStream_ExtendedS3_DataFormatConversionConfiguration_ParquetSerDe_Empty (192.30s)
--- PASS: TestAccAWSKinesisFirehoseDeliveryStream_ExtendedS3_DataFormatConversionConfiguration_Serializer_Update (196.54s)
--- PASS: TestAccAWSKinesisFirehoseDeliveryStream_ExtendedS3basic (200.98s)
--- PASS: TestAccAWSKinesisFirehoseDeliveryStream_ExtendedS3_DataFormatConversionConfiguration_OpenXJsonSerDe_Empty (202.37s)
--- PASS: TestAccAWSKinesisFirehoseDeliveryStream_ExtendedS3_DataFormatConversionConfiguration_HiveJsonSerDe_Empty (222.59s)
--- PASS: TestAccAWSKinesisFirehoseDeliveryStream_ExtendedS3KmsKeyArn (226.21s)
--- PASS: TestAccAWSKinesisFirehoseDeliveryStream_s3WithCloudwatchLogging (189.02s)
--- PASS: TestAccAWSKinesisFirehoseDeliveryStream_s3basic (136.60s)
--- PASS: TestAccAWSKinesisFirehoseDeliveryStream_s3ConfigUpdates (303.77s)
--- PASS: TestAccAWSKinesisFirehoseDeliveryStream_s3basicWithTags (184.97s)
--- PASS: TestAccAWSKinesisFirehoseDeliveryStream_RedshiftConfigUpdates (808.04s)
--- PASS: TestAccAWSKinesisFirehoseDeliveryStream_ElasticsearchConfigUpdates (970.67s)
```

* Update CHANGELOG for hashicorp#9103

* Update CHANGELOG for hashicorp#9442

* Update CHANGELOG.md

* Update aws/resource_aws_internet_gateway.go

Co-Authored-By: Brian Flad <bflad417@gmail.com>

* Update aws/resource_aws_msk_cluster.go

Co-Authored-By: Brian Flad <bflad417@gmail.com>

* Update aws/resource_aws_security_group.go

Co-Authored-By: Brian Flad <bflad417@gmail.com>

* Update aws/resource_aws_security_group.go

Co-Authored-By: Brian Flad <bflad417@gmail.com>

* Update aws/resource_aws_sqs_queue_policy.go

Co-Authored-By: Brian Flad <bflad417@gmail.com>

* Update aws/resource_aws_sqs_queue_policy.go

Co-Authored-By: Brian Flad <bflad417@gmail.com>

* Update CHANGELOG.md

* Lint for the tautological linting gods

* Final ACL retries

* Update CHANGELOG.md

* WAF and wafregional token final retries (hashicorp#9826)

* update CHANGELOG for hashicorp#9826

* s3_bucket_object: Fix ETAG changes not forcing new

* Add default setting for aws_cloudwatch_event_target ecs_target's task_count (hashicorp#9773)

* Add default setting for aws_cloudwatch_event_target ecs_target's task_count.

* Remove forcenew from etag on s3_bucket_object

* Final retries for s3 timeouts

* Update CHANGELOG.md

* Update Changelog for hashicorp#9773

* Update db_instance.html.markdown

The description of `final_snapshot_identifier` doesn't feel correct to me.
When `skip_final_snapshot` is by default set to false and `final_snapshot_identifier`
is omitted, the provider errored with 
> RDS Cluster FinalSnapshotIdentifier is required when a final snapshot is require

Related: hashicorp#4910

* v2.25.0

* Cleanup after v2.25.0 release

* Fixes EC2 capacity reservation in state not found

* Support Kinesis Data Firehose server-side encryption.

* Update aws/resource_aws_kinesis_firehose_delivery_stream.go

Co-Authored-By: Brian Flad <bflad417@gmail.com>

* Update aws/resource_aws_kinesis_firehose_delivery_stream.go

Co-Authored-By: Brian Flad <bflad417@gmail.com>

* Update aws/resource_aws_kinesis_firehose_delivery_stream.go

Co-Authored-By: Brian Flad <bflad417@gmail.com>

* Run 'make fmt'.

* Review: Remove customizable timeouts.

* Review: Replace 'CustomizeDiff' with 'DiffSuppressFunc'.

* Review: Revert to separate 'waiter' functions.

* Review: Separate test configurations for SSE.

* Comment out r/aws_kinesis_firehose_delivery_stream acceptance tests that fail.

* Add AWS Amplify service client.

* Add Amazon AppStream service client.

* Use isAWSErr in EC2 Capacity Reservation read

* Update CHANGELOG.md

* Adding an ElasticsearchDomain data source to the aws provider

* Added VPC Options to ElasticsearchDomain datasource

* Aligning elasticsearch datasource attributes to resource attributes

* Adding an extra test case with an advanced domain configuration

* Documented the additional attributes

* Adding link to aws_elasticsearch_domain datasource docs page

* feedback from review and corrections to make test pass

* removed log_type check because there seems to be odd behaviour with the resource

* changing zone_awareness_config type to TypeList

* documented the zone_awareness_config attribute in the cluster_config block

* Import testing for vpn gateway

* Added IoT Analytics Service

* Update CHANGELOG.md

* Add AWS IoTEvents service

* Update docs with iotevents service

* Update erb and order attributes in documentation

* Import test refactors for vpc resources

* Multiple load_balancer blocks

Add a note that multiple load_balancer blocks are supported with minimum aws provider. This wasn't clear and was only through seeing an old issue that it was possible. With recent support for multiple target groups for ecs service, I think this will be something people will try and use more.

* update CHANGELOG for hashicorp#1867

* Update module aws/aws-sdk-go to v1.23.12

* v2.26.0

* Cleanup after v2.26.0 release

* r/aws_s3_bucket: Include any system tags that Terraform ignores when setting S3 bucket tags (hashicorp#7342)

* Update Changelog for hashicorp#7342

* Update website/docs/r/ecs_service.html.markdown

Co-Authored-By: Brian Flad <bflad417@gmail.com>

* resource/aws_kinesis_firehose_delivery_stream: Remove Computed from server_side_encryption attribute

So Terraform always performs drift detection.

Output from acceptance testing:

```
--- PASS: TestAccAWSKinesisFirehoseDeliveryStream_ElasticsearchConfigUpdates (708.34s)
--- PASS: TestAccAWSKinesisFirehoseDeliveryStream_ExtendedS3_DataFormatConversionConfiguration_Deserializer_Update (204.92s)
--- PASS: TestAccAWSKinesisFirehoseDeliveryStream_ExtendedS3_DataFormatConversionConfiguration_Enabled (188.93s)
--- PASS: TestAccAWSKinesisFirehoseDeliveryStream_ExtendedS3_DataFormatConversionConfiguration_HiveJsonSerDe_Empty (166.30s)
--- PASS: TestAccAWSKinesisFirehoseDeliveryStream_ExtendedS3_DataFormatConversionConfiguration_OpenXJsonSerDe_Empty (154.77s)
--- PASS: TestAccAWSKinesisFirehoseDeliveryStream_ExtendedS3_DataFormatConversionConfiguration_OrcSerDe_Empty (180.87s)
--- PASS: TestAccAWSKinesisFirehoseDeliveryStream_ExtendedS3_DataFormatConversionConfiguration_ParquetSerDe_Empty (165.68s)
--- PASS: TestAccAWSKinesisFirehoseDeliveryStream_ExtendedS3_DataFormatConversionConfiguration_Serializer_Update (152.61s)
--- PASS: TestAccAWSKinesisFirehoseDeliveryStream_ExtendedS3_ErrorOutputPrefix (177.03s)
--- PASS: TestAccAWSKinesisFirehoseDeliveryStream_ExtendedS3_ExternalUpdate (152.21s)
--- PASS: TestAccAWSKinesisFirehoseDeliveryStream_ExtendedS3basic (162.62s)
--- PASS: TestAccAWSKinesisFirehoseDeliveryStream_ExtendedS3KmsKeyArn (225.11s)
--- PASS: TestAccAWSKinesisFirehoseDeliveryStream_ExtendedS3Updates (157.35s)
--- PASS: TestAccAWSKinesisFirehoseDeliveryStream_importBasic (130.87s)
--- PASS: TestAccAWSKinesisFirehoseDeliveryStream_missingProcessingConfiguration (199.02s)
--- PASS: TestAccAWSKinesisFirehoseDeliveryStream_RedshiftConfigUpdates (714.21s)
--- PASS: TestAccAWSKinesisFirehoseDeliveryStream_s3basic (114.90s)
--- PASS: TestAccAWSKinesisFirehoseDeliveryStream_s3basicWithSSE (372.95s)
--- PASS: TestAccAWSKinesisFirehoseDeliveryStream_s3basicWithTags (194.45s)
--- PASS: TestAccAWSKinesisFirehoseDeliveryStream_s3ConfigUpdates (287.14s)
--- PASS: TestAccAWSKinesisFirehoseDeliveryStream_s3KinesisStreamSource (136.86s)
--- PASS: TestAccAWSKinesisFirehoseDeliveryStream_s3WithCloudwatchLogging (122.61s)
--- PASS: TestAccAWSKinesisFirehoseDeliveryStream_SplunkConfigUpdates (195.68s)
```

* Add iotevents service to endpointServiceNames

* deps: github.com/aws/aws-sdk-go/service/appstream@v1.23.12

* Update CHANGELOG for hashicorp#6523

* Remove duplicate nested lists for Global Accelerator resources

* remove status attribute from docdb_cluster_instance doc (hashicorp#9921)

* tests/service/s3: Check S3 Bucket location before attempting other S3 operations in sweepers

To prevent errors in the sweepers such as:

```
2019/08/30 12:23:14 [ERR] error running (aws_s3_bucket_object): error listing S3 Bucket (tf-acc-test-6376821381526517605) Objects: AccessDenied: Access Denied
```

And

```
2019/08/30 11:38:01 [INFO] Skipping S3 Bucket (tf-test-bucket-destination-2585329347318042546): AuthorizationHeaderMalformed: The authorization header is malformed; the region 'us-east-1' is wrong; expecting 'eu-west-1'
```

Output from sweeper:

```console
$ go test ./aws -v -sweep=us-east-1,us-west-2 -sweep-run=aws_s3_bucket -timeout 10h
...
2019/08/30 12:01:08 Sweeper Tests ran:
	- aws_s3_bucket_object
	- aws_s3_bucket
```

* Update module aws/aws-sdk-go to v1.23.13

* Update module hashicorp/terraform to v0.12.7

* resource/ecs_cluster: Add ability to enable ECS Cluster Insights

Fixes: hashicorp#9294

* Fix test names and ignore broken import

* Import test refactor for subnets

* Add the service name in the list of customizable endpoints

* Import acceptance test refactor for SSM resources

* Acceptance test import refactor for SQS queue policy

* Acceptance test import refactor for simpledb domain

* Acceptance test import refactor for SFN activity

* Acceptance test import refactor for SES template

* Acceptance test import refactor for service resources

* service/cloudhsmv2: Implement sweeper

To prevent sweeper errors such as:

```
2019/09/04 02:12:15 [ERR] error running (aws_vpc): Error deleting Subnet (subnet-064c5a7cad5a28b9e): DependencyViolation: The subnet 'subnet-064c5a7cad5a28b9e' has dependencies and cannot be deleted.
```

Output from sweeper in AWS Commercial:

```console
$ go test ./aws -v -sweep=us-east-1,us-west-2 -sweep-run=aws_cloudhsm_v2_cluster -timeout 10h
2019/09/04 10:27:24 [DEBUG] Running Sweepers for region (us-east-1):
...
2019/09/04 10:27:25 [INFO] Deleting CloudHSMv2 Cluster (cluster-wporekks46b) HSM: hsm-y4tvi5ukedh
...
2019/09/04 10:29:06 [INFO] Deleting CloudHSMv2 Cluster: cluster-wporekks46b
...
2019/09/04 10:30:57 Sweeper Tests ran:
  - aws_cloudhsm_v2_cluster
2019/09/04 10:30:57 [DEBUG] Running Sweepers for region (us-west-2):
...
2019/09/04 10:30:59 [INFO] Deleting CloudHSMv2 Cluster: cluster-w56d2uzszjf
...
2019/09/04 10:31:30 [INFO] Deleting CloudHSMv2 Cluster: cluster-zrvllautsay
...
2019/09/04 10:32:01 [INFO] Deleting CloudHSMv2 Cluster: cluster-chxrr77wb2v
...
2019/09/04 10:32:32 Sweeper Tests ran:
  - aws_cloudhsm_v2_cluster
```

Output from sweeper in AWS GovCloud (US):

```console
$ go test ./aws -v -sweep=us-gov-west-1 -sweep-run=aws_cloudhsm_v2_cluster -timeout 10h
2019/09/04 10:28:52 [DEBUG] Running Sweepers for region (us-gov-west-1):
...
2019/09/04 10:28:54 Sweeper Tests ran:
  - aws_cloudhsm_v2_cluster
```

Output from acceptance testing:

```
--- PASS: TestAccAWSCloudHsm2Cluster_basic (285.06s)

--- PASS: TestAccAWSCloudHsm2Hsm_basic (898.88s)
```

* service/ecs: Minor adjustments to finish cluster setting implementation

Output from acceptance testing:

```
--- PASS: TestAccAWSEcsCluster_disappears (9.90s)
--- PASS: TestAccAWSEcsCluster_basic (13.35s)
--- PASS: TestAccAWSEcsCluster_containerInsights (25.56s)
--- PASS: TestAccAWSEcsCluster_Tags (27.62s)

--- PASS: TestAccAWSEcsDataSource_ecsClusterContainerInsights (14.16s)
--- PASS: TestAccAWSEcsDataSource_ecsCluster (14.25s)
```

* Update CHANGELOG for hashicorp#9720

* deps: Update github.com/aws/aws-sdk-go@v1.23.15

Updated via:

```
$ go get github.com/aws/aws-sdk-go@v1.23.15
$ go mod tidy
$ go mod vendor
```

* Update CHANGELOG for hashicorp#9998

* Add created_add attribute to lightsail instance documentation

* Update website/docs/r/lightsail_instance.html.markdown

Co-Authored-By: Ryn Daniels <ryn@hashicorp.com>

* v2.27.0

* Cleanup after v2.27.0 release

* switch to terraform.NewResourceConfigRaw
@WafflesMcDuff
Copy link

Simpler to set the final_snapshot_identifier, apply the change, then run the destroy

@github-actions
Copy link

Marking this issue as stale due to inactivity. This helps our maintainers find and focus on the active issues. If this issue receives no comments in the next 30 days it will automatically be closed. Maintainers can also remove the stale label.

If this issue was automatically closed and you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thank you!

@github-actions github-actions bot added the stale Old or inactive issues managed by automation, if no further action taken these will get closed. label Sep 15, 2022
@github-actions github-actions bot closed this as not planned Won't fix, can't repro, duplicate, stale Oct 15, 2022
@github-actions
Copy link

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Nov 15, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
service/rds Issues and PRs that pertain to the rds service. stale Old or inactive issues managed by automation, if no further action taken these will get closed. upstream-terraform Addresses functionality related to the Terraform core binary.
Projects
None yet
Development

No branches or pull requests