resource_aws_kinesis_firehose_delivery_stream: Avoid invalid state on refresh (cont.) #9103
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
added 4 commits
June 24, 2019 10:35
… refresh When refreshing the state of a aws_kinesis_firehose_delivery_stream resource, the AWS SDK may have more that one way to describe the same configuration. In the case of ProcessingConfiguration, this can lead to perpetual diffs and for DataFormatConversionConfiguration it can lead to invalid state that causes later refresh and plan operations to crash. We have solved these two situations here by detecting the alternative representation of the "disabled" configuration, and disambiguated the terraform state data.
…s to put config with less chance of producing extra diffs.
…for external modification of destination
ghost
added
size/M
|
Any word on if this will get merged? |
bflad
reviewed
Aug 20, 2019
There was a problem hiding this comment.
Hey @staffan-einarsson 👋 Thank you so much for this contribution and sorry for the delayed review. We wanted to double check some of the API and Terraform behaviors before accepting this. Overall this will get merged in with the fix for data_format_conversion_configuration while processing_configuration will be handled with a more typical DiffSuppressFunc fix on the configuration block attribute. Thank you for the great covering acceptance test. 🚀
Output from acceptance testing:
--- PASS: TestAccAWSKinesisFirehoseDeliveryStream_ExtendedS3_ErrorOutputPrefix (151.81s)
--- PASS: TestAccAWSKinesisFirehoseDeliveryStream_missingProcessingConfiguration (158.88s)
--- PASS: TestAccAWSKinesisFirehoseDeliveryStream_ExtendedS3_DataFormatConversionConfiguration_OrcSerDe_Empty (168.93s)
--- PASS: TestAccAWSKinesisFirehoseDeliveryStream_ExtendedS3_ExternalUpdate (169.36s)
--- PASS: TestAccAWSKinesisFirehoseDeliveryStream_SplunkConfigUpdates (170.83s)
--- PASS: TestAccAWSKinesisFirehoseDeliveryStream_s3KinesisStreamSource (136.10s)
--- PASS: TestAccAWSKinesisFirehoseDeliveryStream_ExtendedS3_DataFormatConversionConfiguration_Deserializer_Update (179.46s)
--- PASS: TestAccAWSKinesisFirehoseDeliveryStream_ExtendedS3Updates (180.75s)
--- PASS: TestAccAWSKinesisFirehoseDeliveryStream_importBasic (187.90s)
--- PASS: TestAccAWSKinesisFirehoseDeliveryStream_ExtendedS3_DataFormatConversionConfiguration_Enabled (190.90s)
--- PASS: TestAccAWSKinesisFirehoseDeliveryStream_ExtendedS3_DataFormatConversionConfiguration_ParquetSerDe_Empty (192.30s)
--- PASS: TestAccAWSKinesisFirehoseDeliveryStream_ExtendedS3_DataFormatConversionConfiguration_Serializer_Update (196.54s)
--- PASS: TestAccAWSKinesisFirehoseDeliveryStream_ExtendedS3basic (200.98s)
--- PASS: TestAccAWSKinesisFirehoseDeliveryStream_ExtendedS3_DataFormatConversionConfiguration_OpenXJsonSerDe_Empty (202.37s)
--- PASS: TestAccAWSKinesisFirehoseDeliveryStream_ExtendedS3_DataFormatConversionConfiguration_HiveJsonSerDe_Empty (222.59s)
--- PASS: TestAccAWSKinesisFirehoseDeliveryStream_ExtendedS3KmsKeyArn (226.21s)
--- PASS: TestAccAWSKinesisFirehoseDeliveryStream_s3WithCloudwatchLogging (189.02s)
--- PASS: TestAccAWSKinesisFirehoseDeliveryStream_s3basic (136.60s)
--- PASS: TestAccAWSKinesisFirehoseDeliveryStream_s3ConfigUpdates (303.77s)
--- PASS: TestAccAWSKinesisFirehoseDeliveryStream_s3basicWithTags (184.97s)
--- PASS: TestAccAWSKinesisFirehoseDeliveryStream_RedshiftConfigUpdates (808.04s)
--- PASS: TestAccAWSKinesisFirehoseDeliveryStream_ElasticsearchConfigUpdates (970.67s)
| // 1. With a nil value | ||
| // 2. With enabled set to false and nil for ALL the config sections. | ||
| // These two cases are equivalent so we use the same state description, to avoid diffs. | ||
| if !enabled && len(ifc) == 0 && len(ofc) == 0 && len(sc) == 0 { |
There was a problem hiding this comment.
After various testing and investigation, even though we prefer to not introduce logic like this since it changes the behavior of the configuration when its returned from the API, this seems to be one of the few options we have for now since the configuration block enabled attribute has Default: true. 👍
| // 2. With enabled set to false and nil for ALL the config sections. | ||
| // These two cases are equivalent so we use the same state description, to avoid diffs. | ||
| if !enabled && len(ifc) == 0 && len(ofc) == 0 && len(sc) == 0 { | ||
| log.Printf("Found ambiguous AWS response") |
There was a problem hiding this comment.
Extraneous log line can be removed
| // 1. With a nil value | ||
| // 2. With an empty processor list and enabled set to false. | ||
| // These are equivalent so we use the same state description, to avoid diffs. | ||
| if !enabled && len(pc.Processors) == 0 { |
There was a problem hiding this comment.
Typically, if the configuration block contains a nested "enabled" or similar attribute that we appropriately handle the default case, we prefer to actually pass through the API response into the state as-is (e.g. 1 list item and enabled = false) and then introduce DiffSuppressFunc: suppressMissingOptionalConfigurationBlock, on the configuration block attribute. This means the Terraform resource can accept a configuration with enabled = false and no processors without showing a difference, while also ignoring the configuration block if all the underlying attribute defaults match.
This change was good to revert back to how it was. If for some reason this does need to be changed, we should add a covering acceptance test to show the errant behavior in a future change request. 👍
| configuration.ErrorOutputPrefix = aws.String(v.(string)) | ||
| } else { | ||
| // It is possible to just pass nil here, but this seems to be the |
There was a problem hiding this comment.
We haven't found an AWS API that has required this additional logic so we would prefer to leave this in its original form. If there is some errant behavior with the original logic, let's add a covering acceptance test to show the errant behavior in a future change request. 👍
| configuration.ErrorOutputPrefix = aws.String(v.(string)) | ||
| } else { | ||
| // It is possible to just pass nil here, but this seems to be the |
| @@ -339,6 +339,60 @@ func TestAccAWSKinesisFirehoseDeliveryStream_ExtendedS3_DataFormatConversionConf | |||
| }) | |||
| } | |||
|
|
|||
| func TestAccAWSKinesisFirehoseDeliveryStream_ExtendedS3_ExternalUpdate(t *testing.T) { | |||
There was a problem hiding this comment.
Thank you very much for adding this acceptance test! 💯
| @@ -1501,7 +1533,11 @@ func updateExtendedS3Config(d *schema.ResourceData) *firehose.ExtendedS3Destinat | |||
|
|
|||
| func expandFirehoseDataFormatConversionConfiguration(l []interface{}) *firehose.DataFormatConversionConfiguration { | |||
| if len(l) == 0 || l[0] == nil { | |||
| return nil | |||
| // It is possible to just pass nil here, but this seems to be the | |||
There was a problem hiding this comment.
We typically do not need to normalize information when sending requests to the API. This doesn't appear to hurt anything here though, so will leave this in for now.
bflad
added a commit
that referenced
this pull request
Aug 20, 2019
Reference: #9103 Mainly reverting some of the changes to processing_configuration/error_output_prefix and adding DiffSuppressFunc to handle the processing_configuration configuration block attribute. Output from acceptance testing: ``` --- PASS: TestAccAWSKinesisFirehoseDeliveryStream_ExtendedS3_ErrorOutputPrefix (151.81s) --- PASS: TestAccAWSKinesisFirehoseDeliveryStream_missingProcessingConfiguration (158.88s) --- PASS: TestAccAWSKinesisFirehoseDeliveryStream_ExtendedS3_DataFormatConversionConfiguration_OrcSerDe_Empty (168.93s) --- PASS: TestAccAWSKinesisFirehoseDeliveryStream_ExtendedS3_ExternalUpdate (169.36s) --- PASS: TestAccAWSKinesisFirehoseDeliveryStream_SplunkConfigUpdates (170.83s) --- PASS: TestAccAWSKinesisFirehoseDeliveryStream_s3KinesisStreamSource (136.10s) --- PASS: TestAccAWSKinesisFirehoseDeliveryStream_ExtendedS3_DataFormatConversionConfiguration_Deserializer_Update (179.46s) --- PASS: TestAccAWSKinesisFirehoseDeliveryStream_ExtendedS3Updates (180.75s) --- PASS: TestAccAWSKinesisFirehoseDeliveryStream_importBasic (187.90s) --- PASS: TestAccAWSKinesisFirehoseDeliveryStream_ExtendedS3_DataFormatConversionConfiguration_Enabled (190.90s) --- PASS: TestAccAWSKinesisFirehoseDeliveryStream_ExtendedS3_DataFormatConversionConfiguration_ParquetSerDe_Empty (192.30s) --- PASS: TestAccAWSKinesisFirehoseDeliveryStream_ExtendedS3_DataFormatConversionConfiguration_Serializer_Update (196.54s) --- PASS: TestAccAWSKinesisFirehoseDeliveryStream_ExtendedS3basic (200.98s) --- PASS: TestAccAWSKinesisFirehoseDeliveryStream_ExtendedS3_DataFormatConversionConfiguration_OpenXJsonSerDe_Empty (202.37s) --- PASS: TestAccAWSKinesisFirehoseDeliveryStream_ExtendedS3_DataFormatConversionConfiguration_HiveJsonSerDe_Empty (222.59s) --- PASS: TestAccAWSKinesisFirehoseDeliveryStream_ExtendedS3KmsKeyArn (226.21s) --- PASS: TestAccAWSKinesisFirehoseDeliveryStream_s3WithCloudwatchLogging (189.02s) --- PASS: TestAccAWSKinesisFirehoseDeliveryStream_s3basic (136.60s) --- PASS: TestAccAWSKinesisFirehoseDeliveryStream_s3ConfigUpdates (303.77s) --- PASS: TestAccAWSKinesisFirehoseDeliveryStream_s3basicWithTags (184.97s) --- PASS: TestAccAWSKinesisFirehoseDeliveryStream_RedshiftConfigUpdates (808.04s) --- PASS: TestAccAWSKinesisFirehoseDeliveryStream_ElasticsearchConfigUpdates (970.67s) ```
bflad
added a commit
that referenced
this pull request
Aug 20, 2019
|
This has been released in version 2.25.0 of the Terraform AWS provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading. For further feature requests or bug reports with this functionality, please create a new GitHub issue following the template for triage. Thanks! |
paulrigor
added a commit
to paulrigor/terraform-provider-aws
that referenced
this pull request
Sep 6, 2019
* provider: Ensure needs-triage issue labeling step in GitHub Actions v2 is conditional on opened action The payload is slightly different in v2, so actions/github@v1.0.0 fails with the action flag and does not return a neutral status. Instead, we will pre-filter the step via if. References: - https://help.github.com/en/articles/workflow-syntax-for-github-actions#jobsjob_idif - https://github.com/actions/github/blob/4f0213ed102c4b518a7ce8bc4e6268bdd710770e/entrypoint.js#L73-L85 * resource/aws_lb_target_group_attachment: Prevent potential panic and update test function names for codebase conventions Output from acceptance testing: ``` --- PASS: TestAccAWSLBTargetGroupAttachment_lambda (31.23s) --- PASS: TestAccAWSLBTargetGroupAttachment_disappears (91.09s) --- PASS: TestAccAWSLBTargetGroupAttachment_ipAddress (102.80s) --- PASS: TestAccAWSLBTargetGroupAttachment_basic (103.02s) --- PASS: TestAccAWSLBTargetGroupAttachment_withoutPort (123.06s) --- PASS: TestAccAWSLBTargetGroupAttachment_BackwardsCompatibility (123.37s) ``` * Update CHANGELOG for hashicorp#9610 * Added CW log export value for aurora-postgresql * tests/resource/aws_rds_cluster_instance: Ensure covering acceptance tests for performance_insights_enabled on Aurora MySQL 1/2 and Postgresql Reference: hashicorp#9227 Reference: hashicorp#9635 Output from acceptance testing: ``` --- PASS: TestAccAWSRDSClusterInstance_PerformanceInsightsEnabled_AuroraMysql1 (629.12s) --- PASS: TestAccAWSRDSClusterInstance_namePrefix (677.80s) --- PASS: TestAccAWSRDSClusterInstance_PerformanceInsightsEnabled_AuroraMysql2 (679.39s) --- PASS: TestAccAWSRDSClusterInstance_PerformanceInsightsEnabled_AuroraPostgresql (680.27s) --- PASS: TestAccAWSRDSClusterInstance_CopyTagsToSnapshot (699.01s) --- PASS: TestAccAWSRDSClusterInstance_generatedName (712.49s) --- PASS: TestAccAWSRDSClusterInstance_az (722.51s) --- PASS: TestAccAWSRDSClusterInstance_disappears (763.18s) --- PASS: TestAccAWSRDSClusterInstance_withInstanceEnhancedMonitor (873.08s) --- PASS: TestAccAWSRDSClusterInstance_kmsKey (1062.91s) --- PASS: TestAccAWSRDSClusterInstance_PubliclyAccessible (1247.95s) --- PASS: TestAccAWSRDSClusterInstance_basic (1423.02s) ``` * Update CHANGELOG for hashicorp#9635 * resource/aws_db_instance: Only send performance_insights_kms_key_id on update if configured Reference: hashicorp#8792 Reference: hashicorp#9399 Reference: hashicorp#9406 Previously before code update: ``` --- PASS: TestAccAWSRDSDBInstance_PerformanceInsightsRetentionPeriod (667.32s) --- PASS: TestAccAWSRDSDBInstance_PerformanceInsightsEnabled_EnabledToDisabled (733.03s) --- PASS: TestAccAWSRDSDBInstance_PerformanceInsightsKmsKeyId (862.56s) --- FAIL: TestAccAWSRDSDBInstance_PerformanceInsightsEnabled_DisabledToEnabled (5686.66s) testing.go:568: Step 2 error: errors during apply: Error: Error modifying DB Instance tf-acc-test-5551941564373291595: InternalFailure: status code: 500, request id: 9ad8b0ff-c5a2-43d0-a531-60aa047b8570 ``` Output from acceptance testing: ``` --- PASS: TestAccAWSRDSDBInstance_PerformanceInsightsEnabled_EnabledToDisabled (680.88s) --- PASS: TestAccAWSRDSDBInstance_PerformanceInsightsEnabled_DisabledToEnabled (680.95s) --- PASS: TestAccAWSRDSDBInstance_PerformanceInsightsRetentionPeriod (685.18s) --- PASS: TestAccAWSRDSDBInstance_PerformanceInsightsKmsKeyId (851.11s) ``` * resource/aws_rds_cluster_instance: Ensure performance_insights_kms_key_id updates include performance_insights_enabled Reference: hashicorp#3015 This update ensures the correct RDS API error is shown when attempting to modify the Performance Insights KMS Key ID (we use `ExpectError` in the testing so this passes): ``` --- FAIL: TestAccAWSRDSClusterInstance_PerformanceInsightsKmsKeyId_DefaultKeyToCustomKey (691.76s) testing.go:568: Step 2 error: errors during apply: Error: Error modifying DB Instance tf-acc-test-2133090011528324987: InvalidParameterCombination: You cannot change your Performance Insights KMS key ``` Previously before code update: ``` --- PASS: TestAccAWSRDSClusterInstance_PerformanceInsightsEnabled_AuroraMysql2 (587.54s) --- PASS: TestAccAWSRDSClusterInstance_PerformanceInsightsEnabled_AuroraMysql1 (594.85s) --- PASS: TestAccAWSRDSClusterInstance_PerformanceInsightsKmsKeyId_CustomKey (660.70s) --- FAIL: TestAccAWSRDSClusterInstance_PerformanceInsightsKmsKeyId_DefaultKeyToCustomKey (667.49s) testing.go:568: Step 2 error: errors during apply: Error: Error modifying DB Instance tf-acc-test-4757962708939555369: InvalidParameterCombination: Can not set PerformanceInsightsKMSKeyId without EnablePerformanceInsights status code: 400, request id: c64b9f56-1b56-420c-9575-c5cfed87270b on /var/folders/v0/_d108fkx1pbbg4_sh864_7740000gn/T/tf-test833219777/main.tf line 15: (source code not available) --- PASS: TestAccAWSRDSClusterInstance_PerformanceInsightsEnabled_AuroraPostgresql (699.46s) ``` Output from acceptance testing: ``` --- PASS: TestAccAWSRDSClusterInstance_PerformanceInsightsKmsKeyId_AuroraMysql2_DefaultKeyToCustomKey (630.56s) --- PASS: TestAccAWSRDSClusterInstance_PerformanceInsightsEnabled_AuroraMysql1 (649.52s) --- PASS: TestAccAWSRDSClusterInstance_generatedName (651.77s) --- PASS: TestAccAWSRDSClusterInstance_PerformanceInsightsKmsKeyId_AuroraMysql1_DefaultKeyToCustomKey (652.94s) --- PASS: TestAccAWSRDSClusterInstance_PerformanceInsightsEnabled_AuroraPostgresql (670.18s) --- PASS: TestAccAWSRDSClusterInstance_PerformanceInsightsKmsKeyId_CustomKey (671.61s) --- PASS: TestAccAWSRDSClusterInstance_PerformanceInsightsKmsKeyId_AuroraPostgresql_DefaultKeyToCustomKey (672.19s) --- PASS: TestAccAWSRDSClusterInstance_PubliclyAccessible (683.28s) --- PASS: TestAccAWSRDSClusterInstance_PerformanceInsightsEnabled_AuroraMysql2 (690.78s) --- PASS: TestAccAWSRDSClusterInstance_disappears (730.05s) --- PASS: TestAccAWSRDSClusterInstance_CopyTagsToSnapshot (732.08s) --- PASS: TestAccAWSRDSClusterInstance_az (747.28s) --- PASS: TestAccAWSRDSClusterInstance_namePrefix (764.03s) --- PASS: TestAccAWSRDSClusterInstance_kmsKey (792.98s) --- PASS: TestAccAWSRDSClusterInstance_withInstanceEnhancedMonitor (822.65s) --- PASS: TestAccAWSRDSClusterInstance_basic (1516.75s) ``` * resource/aws_db_instance: Ensure monitoring attributes are always written to state and retry ModifyDBInstance on IAM eventual consistency error References: - hashicorp#315 - hashicorp#2188 - hashicorp#5559 Previously before code updates: ``` --- FAIL: TestAccAWSDBInstance_MonitoringRoleArn_RemovedToEnabled (430.95s) testing.go:568: Step 2 error: errors during apply: Error: Error modifying DB Instance tf-acc-test-1165998526456666486: InvalidParameterValue: IAM role ARN value is invalid or does not include the required permissions for: ENHANCED_MONITORING status code: 400, request id: 524f599d-3870-48b3-843e-28885ae3f75c on /var/folders/v0/_d108fkx1pbbg4_sh864_7740000gn/T/tf-test908490254/main.tf line 29: (source code not available) --- PASS: TestAccAWSDBInstance_MonitoringRoleInterval (565.47s) --- PASS: TestAccAWSDBInstance_MonitoringRoleArn_EnabledToDisabled (593.87s) --- PASS: TestAccAWSDBInstance_MonitoringRoleArn_EnabledToRemoved (626.12s) ``` Output from acceptance testing: ``` --- PASS: TestAccAWSDBInstance_MonitoringRoleArn_EnabledToDisabled (587.93s) --- PASS: TestAccAWSDBInstance_MonitoringRoleArn_RemovedToEnabled (614.89s) --- PASS: TestAccAWSDBInstance_MonitoringRoleArn_EnabledToRemoved (656.13s) --- PASS: TestAccAWSDBInstance_MonitoringRoleInterval (702.57s) ``` * tests/resource/aws_db_instance: Ensure monitoring_interval testing covers disabling and re-enabling Output from acceptance testing: ``` --- PASS: TestAccAWSDBInstance_MonitoringInterval (978.85s) ``` * resource/aws_rds_cluster_instance: Ensure monitoring attributes are always written to state Reference: hashicorp#315 Output from acceptance testing: ``` --- PASS: TestAccAWSRDSClusterInstance_PerformanceInsightsEnabled_AuroraMysql2 (588.69s) --- PASS: TestAccAWSRDSClusterInstance_PerformanceInsightsEnabled_AuroraMysql1 (640.98s) --- PASS: TestAccAWSRDSClusterInstance_PerformanceInsightsEnabled_AuroraPostgresql (661.39s) --- PASS: TestAccAWSRDSClusterInstance_namePrefix (674.32s) --- PASS: TestAccAWSRDSClusterInstance_CopyTagsToSnapshot (703.51s) --- PASS: TestAccAWSRDSClusterInstance_PubliclyAccessible (714.73s) --- PASS: TestAccAWSRDSClusterInstance_disappears (735.39s) --- PASS: TestAccAWSRDSClusterInstance_generatedName (774.99s) --- PASS: TestAccAWSRDSClusterInstance_MonitoringRoleArn_EnabledToRemoved (834.26s) --- PASS: TestAccAWSRDSClusterInstance_MonitoringRoleArn_RemovedToEnabled (876.44s) --- PASS: TestAccAWSRDSClusterInstance_az (877.93s) --- PASS: TestAccAWSRDSClusterInstance_MonitoringRoleArn_EnabledToDisabled (912.63s) --- PASS: TestAccAWSRDSClusterInstance_MonitoringInterval (1037.61s) --- PASS: TestAccAWSRDSClusterInstance_kmsKey (1205.30s) --- PASS: TestAccAWSRDSClusterInstance_basic (1301.24s) ``` * Update CHANGELOG.md * Final retries for gamelift resources * Final retry after timeout creating gluecrawler * Final guard duty retry * r/ram_resource_share_accepter: New resource * Add arguments, delete to disassociate * Update module aws/aws-sdk-go to v1.23.0 * New Resources: aws_fsx_lustre_file_system and aws_fsx_windows_file_system Reference: hashicorp#7035 Reference: hashicorp#7074 Builds on hashicorp#7074 with the following changes: - Split into two separate resources (`aws_fsx_lustre_file_system` and `aws_fsx_windows_file_system`) to simplify practitioner configuration (e.g. lose the required configuration blocks), ease maintenance for resource logic and single file system arguments (e.g. `kms_key_id` only for Windows), and enhance validations (e.g. `storage_capacity` differences between file systems) - Add covering acceptance testing with updates/force new for all arguments - Better alignment with API naming for some arguments (e.g. changing `capacity` into `storage_capacity` to remove ambiguity with `throughput_capacity`) - Implement [resource Customizable Timeouts](https://www.terraform.io/docs/extend/resources/retries-and-customizable-timeouts.html) - Implement `skip_final_backup` argument for Windows - Implement `network_interface_ids`, `owner_id`, and `vpc_id` attributes - Augment resource documentation with additional notes found in [API reference](https://docs.aws.amazon.com/fsx/latest/APIReference/) and clarify `security_group_ids` import behavior Output from acceptance testing: ``` --- PASS: TestAccAWSFsxLustreFileSystem_Tags (458.62s) --- PASS: TestAccAWSFsxLustreFileSystem_basic (526.79s) --- PASS: TestAccAWSFsxLustreFileSystem_disappears (564.58s) --- PASS: TestAccAWSFsxLustreFileSystem_WeeklyMaintenanceStartTime (697.83s) --- PASS: TestAccAWSFsxLustreFileSystem_StorageCapacity (940.13s) --- PASS: TestAccAWSFsxLustreFileSystem_SecurityGroupIds (1074.08s) --- PASS: TestAccAWSFsxLustreFileSystem_ImportedFileChunkSize (1276.18s) --- PASS: TestAccAWSFsxLustreFileSystem_ImportPath (1322.13s) --- PASS: TestAccAWSFsxLustreFileSystem_ExportPath (1338.75s) --- PASS: TestAccAWSFsxWindowsFileSystem_Tags (2256.64s) --- PASS: TestAccAWSFsxWindowsFileSystem_WeeklyMaintenanceStartTime (2346.44s) --- PASS: TestAccAWSFsxWindowsFileSystem_basic (2404.35s) --- PASS: TestAccAWSFsxWindowsFileSystem_disappears (2412.10s) --- PASS: TestAccAWSFsxWindowsFileSystem_AutomaticBackupRetentionDays (2514.61s) --- PASS: TestAccAWSFsxWindowsFileSystem_DailyAutomaticBackupStartTime (2849.43s) --- PASS: TestAccAWSFsxWindowsFileSystem_SelfManagedActiveDirectory (2850.64s) --- PASS: TestAccAWSFsxWindowsFileSystem_ThroughputCapacity (3292.44s) --- PASS: TestAccAWSFsxWindowsFileSystem_SecurityGroupIds (3609.37s) --- PASS: TestAccAWSFsxWindowsFileSystem_KmsKeyId (3977.10s) --- PASS: TestAccAWSFsxWindowsFileSystem_StorageCapacity (4026.67s) --- PASS: TestAccAWSFsxWindowsFileSystem_CopyTagsToBackups (4389.57s) ``` * Add docs * Add acceptance test, remove comments, fix minor issue * Adjust for better code * Remove the invitation ARN as argument * Update CHANGELOG.md * Update website/docs/r/fsx_lustre_file_system.html.markdown Co-Authored-By: Ryn Daniels <ryn@hashicorp.com> * Add test sweepers for fsx filesystems * Update CHANGELOG for hashicorp#7074 and hashicorp#9761 * Final retries for ELB resources * Final retries for IAM resources * Final retry after timeout deleting inspector assessment target * Add AWS Lake Formation service client. Run 'go mod tidy'. * Update CHANGELOG for hashicorp#9745 * resource/aws_ram_resource_share_accepter: Minor adjustments and documentation enhancements for initial release Reference: hashicorp#8259 Output from acceptance testing: ``` --- PASS: TestAccAwsRamResourceShareAccepter_basic (31.57s) ``` * Update CHANGELOG for hashicorp#8259 * provider: Remove Route 53 endpoint workaround in AWS GovCloud (US) The endpoint information is now correctly included in the AWS Go SDK as of [v1.22.3](aws/aws-sdk-go@1f4898f). The AWS China endpoint information for Route 53 is still not present in the AWS Go SDK. Output from acceptance testing in AWS GovCloud (US): ``` --- PASS: TestAccAWSRoute53Zone_VPC_Single (66.92s) --- PASS: TestAccAWSRoute53Zone_VPC_Multiple (96.71s) --- PASS: TestAccAWSRoute53Zone_VPC_Updates (173.89s) ``` * resource/aws_ram_resource_share_accepter: go fmt * Final retries after internet gateway timeouts * Final retry deleting iot types * Final retry after timeout creating launch config * tests/data-source/aws_organizations_organization: Add Organizations PreCheck For consistency with other Organizations testing and to prevent the following failure: ``` --- FAIL: TestAccAWSOrganizations/Organization/DataSource (3.55s) testing.go:568: Step 0 error: errors during apply: Error: Error creating organization: AlreadyInOrganizationException: The AWS account is already a member of an organization. ``` Acceptance testing from Organizations member account: ``` --- SKIP: TestAccAWSOrganizations/Organization/DataSource (1.57s) provider_test.go:247: skipping tests; this AWS account must not be an existing member of an AWS Organization ``` Acceptance testing from standalone account: ``` --- PASS: TestAccAWSOrganizations/Organization/DataSource (29.70s) ``` * Final retries for LB resources * v2.24.0 * Cleanup after v2.24.0 release * Update CHANGELOG for hashicorp#9747 * Update CHANGELOG for hashicorp#9748 * Update CHANGELOG for hashicorp#9740 * initial commit * changes based on feedback: DiffSuppresFunc instead of Computed + Updated Acceptance test * go fmt * resource/aws_cloudwatch_log_subscription_filter: Use Default with distribution attribute, refactor tests to ensure all arguments and Kinesis Data Firehose/Kinesis Stream destinations are tested Reference: https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/SubscriptionFilters.html Output from acceptance testing (the test failure is a new bug discovery and will be fixed in a subsequent fix): ``` --- PASS: TestAccAWSCloudwatchLogSubscriptionFilter_basic (40.61s) --- PASS: TestAccAWSCloudwatchLogSubscriptionFilter_disappears (41.09s) --- PASS: TestAccAWSCloudwatchLogSubscriptionFilter_disappears_LogGroup (42.32s) --- PASS: TestAccAWSCloudwatchLogSubscriptionFilter_Distribution (51.33s) --- PASS: TestAccAWSCloudwatchLogSubscriptionFilter_DestinationArn_KinesisStream (108.80s) --- FAIL: TestAccAWSCloudwatchLogSubscriptionFilter_RoleArn (114.10s) testing.go:568: Step 2 error: errors during apply: Error: Error updating SubscriptionFilter (tf-acc-test-3057491702311743141) for LogGroup (tf-acc-test-3057491702311743141), message: "Could not deliver test message to specified Kinesis stream. Check if the given kinesis stream is in ACTIVE state.", code: "InvalidParameterException" on /var/folders/v0/_d108fkx1pbbg4_sh864_7740000gn/T/tf-test472807723/main.tf line 98: (source code not available) --- PASS: TestAccAWSCloudwatchLogSubscriptionFilter_DestinationArn_KinesisDataFirehose (126.02s) ``` * Update CHANGELOG for hashicorp#9265 * Update CHANGELOG.md * Final retries for msk cluster * Add Personailze service * Run `make fmt` * Final retries for redshift resources * Final retries for routes * Final sagemaker retries * Security group retries * Update CHANGELOG.md * tests/resource/aws_rds_cluster_instance: Fix minor typo in master_password arguments Reference: https://github.com/terraform-providers/terraform-provider-aws/pull/9746/files#r314539776 * Update CHANGELOG for hashicorp#9710 * Update module terraform-providers/terraform-provider-tls to v2.1.0 * Final retries for SQS resources * Final retry for transfer server * Retries and retry removal for opsworks resources * Fix retiring_principal argument name for aws_kms_grant resource * Final retry for wafregional acl association * Final tag retries * `status` is not an attribute in the schema (hashicorp#9823) See https://github.com/terraform-providers/terraform-provider-aws/blob/00909998d919faf5494ab8f6b38241deb1957d99/aws/resource_aws_docdb_cluster.go#L33 * resource/aws_kinesis_firehose_delivery_stream: Address PR hashicorp#9103 feedback Reference: hashicorp#9103 Mainly reverting some of the changes to processing_configuration/error_output_prefix and adding DiffSuppressFunc to handle the processing_configuration configuration block attribute. Output from acceptance testing: ``` --- PASS: TestAccAWSKinesisFirehoseDeliveryStream_ExtendedS3_ErrorOutputPrefix (151.81s) --- PASS: TestAccAWSKinesisFirehoseDeliveryStream_missingProcessingConfiguration (158.88s) --- PASS: TestAccAWSKinesisFirehoseDeliveryStream_ExtendedS3_DataFormatConversionConfiguration_OrcSerDe_Empty (168.93s) --- PASS: TestAccAWSKinesisFirehoseDeliveryStream_ExtendedS3_ExternalUpdate (169.36s) --- PASS: TestAccAWSKinesisFirehoseDeliveryStream_SplunkConfigUpdates (170.83s) --- PASS: TestAccAWSKinesisFirehoseDeliveryStream_s3KinesisStreamSource (136.10s) --- PASS: TestAccAWSKinesisFirehoseDeliveryStream_ExtendedS3_DataFormatConversionConfiguration_Deserializer_Update (179.46s) --- PASS: TestAccAWSKinesisFirehoseDeliveryStream_ExtendedS3Updates (180.75s) --- PASS: TestAccAWSKinesisFirehoseDeliveryStream_importBasic (187.90s) --- PASS: TestAccAWSKinesisFirehoseDeliveryStream_ExtendedS3_DataFormatConversionConfiguration_Enabled (190.90s) --- PASS: TestAccAWSKinesisFirehoseDeliveryStream_ExtendedS3_DataFormatConversionConfiguration_ParquetSerDe_Empty (192.30s) --- PASS: TestAccAWSKinesisFirehoseDeliveryStream_ExtendedS3_DataFormatConversionConfiguration_Serializer_Update (196.54s) --- PASS: TestAccAWSKinesisFirehoseDeliveryStream_ExtendedS3basic (200.98s) --- PASS: TestAccAWSKinesisFirehoseDeliveryStream_ExtendedS3_DataFormatConversionConfiguration_OpenXJsonSerDe_Empty (202.37s) --- PASS: TestAccAWSKinesisFirehoseDeliveryStream_ExtendedS3_DataFormatConversionConfiguration_HiveJsonSerDe_Empty (222.59s) --- PASS: TestAccAWSKinesisFirehoseDeliveryStream_ExtendedS3KmsKeyArn (226.21s) --- PASS: TestAccAWSKinesisFirehoseDeliveryStream_s3WithCloudwatchLogging (189.02s) --- PASS: TestAccAWSKinesisFirehoseDeliveryStream_s3basic (136.60s) --- PASS: TestAccAWSKinesisFirehoseDeliveryStream_s3ConfigUpdates (303.77s) --- PASS: TestAccAWSKinesisFirehoseDeliveryStream_s3basicWithTags (184.97s) --- PASS: TestAccAWSKinesisFirehoseDeliveryStream_RedshiftConfigUpdates (808.04s) --- PASS: TestAccAWSKinesisFirehoseDeliveryStream_ElasticsearchConfigUpdates (970.67s) ``` * Update CHANGELOG for hashicorp#9103 * Update CHANGELOG for hashicorp#9442 * Update CHANGELOG.md * Update aws/resource_aws_internet_gateway.go Co-Authored-By: Brian Flad <bflad417@gmail.com> * Update aws/resource_aws_msk_cluster.go Co-Authored-By: Brian Flad <bflad417@gmail.com> * Update aws/resource_aws_security_group.go Co-Authored-By: Brian Flad <bflad417@gmail.com> * Update aws/resource_aws_security_group.go Co-Authored-By: Brian Flad <bflad417@gmail.com> * Update aws/resource_aws_sqs_queue_policy.go Co-Authored-By: Brian Flad <bflad417@gmail.com> * Update aws/resource_aws_sqs_queue_policy.go Co-Authored-By: Brian Flad <bflad417@gmail.com> * Update CHANGELOG.md * Lint for the tautological linting gods * Final ACL retries * Update CHANGELOG.md * WAF and wafregional token final retries (hashicorp#9826) * update CHANGELOG for hashicorp#9826 * s3_bucket_object: Fix ETAG changes not forcing new * Add default setting for aws_cloudwatch_event_target ecs_target's task_count (hashicorp#9773) * Add default setting for aws_cloudwatch_event_target ecs_target's task_count. * Remove forcenew from etag on s3_bucket_object * Final retries for s3 timeouts * Update CHANGELOG.md * Update Changelog for hashicorp#9773 * Update db_instance.html.markdown The description of `final_snapshot_identifier` doesn't feel correct to me. When `skip_final_snapshot` is by default set to false and `final_snapshot_identifier` is omitted, the provider errored with > RDS Cluster FinalSnapshotIdentifier is required when a final snapshot is require Related: hashicorp#4910 * v2.25.0 * Cleanup after v2.25.0 release * Fixes EC2 capacity reservation in state not found * Support Kinesis Data Firehose server-side encryption. * Update aws/resource_aws_kinesis_firehose_delivery_stream.go Co-Authored-By: Brian Flad <bflad417@gmail.com> * Update aws/resource_aws_kinesis_firehose_delivery_stream.go Co-Authored-By: Brian Flad <bflad417@gmail.com> * Update aws/resource_aws_kinesis_firehose_delivery_stream.go Co-Authored-By: Brian Flad <bflad417@gmail.com> * Run 'make fmt'. * Review: Remove customizable timeouts. * Review: Replace 'CustomizeDiff' with 'DiffSuppressFunc'. * Review: Revert to separate 'waiter' functions. * Review: Separate test configurations for SSE. * Comment out r/aws_kinesis_firehose_delivery_stream acceptance tests that fail. * Add AWS Amplify service client. * Add Amazon AppStream service client. * Use isAWSErr in EC2 Capacity Reservation read * Update CHANGELOG.md * Adding an ElasticsearchDomain data source to the aws provider * Added VPC Options to ElasticsearchDomain datasource * Aligning elasticsearch datasource attributes to resource attributes * Adding an extra test case with an advanced domain configuration * Documented the additional attributes * Adding link to aws_elasticsearch_domain datasource docs page * feedback from review and corrections to make test pass * removed log_type check because there seems to be odd behaviour with the resource * changing zone_awareness_config type to TypeList * documented the zone_awareness_config attribute in the cluster_config block * Import testing for vpn gateway * Added IoT Analytics Service * Update CHANGELOG.md * Add AWS IoTEvents service * Update docs with iotevents service * Update erb and order attributes in documentation * Import test refactors for vpc resources * Multiple load_balancer blocks Add a note that multiple load_balancer blocks are supported with minimum aws provider. This wasn't clear and was only through seeing an old issue that it was possible. With recent support for multiple target groups for ecs service, I think this will be something people will try and use more. * update CHANGELOG for hashicorp#1867 * Update module aws/aws-sdk-go to v1.23.12 * v2.26.0 * Cleanup after v2.26.0 release * r/aws_s3_bucket: Include any system tags that Terraform ignores when setting S3 bucket tags (hashicorp#7342) * Update Changelog for hashicorp#7342 * Update website/docs/r/ecs_service.html.markdown Co-Authored-By: Brian Flad <bflad417@gmail.com> * resource/aws_kinesis_firehose_delivery_stream: Remove Computed from server_side_encryption attribute So Terraform always performs drift detection. Output from acceptance testing: ``` --- PASS: TestAccAWSKinesisFirehoseDeliveryStream_ElasticsearchConfigUpdates (708.34s) --- PASS: TestAccAWSKinesisFirehoseDeliveryStream_ExtendedS3_DataFormatConversionConfiguration_Deserializer_Update (204.92s) --- PASS: TestAccAWSKinesisFirehoseDeliveryStream_ExtendedS3_DataFormatConversionConfiguration_Enabled (188.93s) --- PASS: TestAccAWSKinesisFirehoseDeliveryStream_ExtendedS3_DataFormatConversionConfiguration_HiveJsonSerDe_Empty (166.30s) --- PASS: TestAccAWSKinesisFirehoseDeliveryStream_ExtendedS3_DataFormatConversionConfiguration_OpenXJsonSerDe_Empty (154.77s) --- PASS: TestAccAWSKinesisFirehoseDeliveryStream_ExtendedS3_DataFormatConversionConfiguration_OrcSerDe_Empty (180.87s) --- PASS: TestAccAWSKinesisFirehoseDeliveryStream_ExtendedS3_DataFormatConversionConfiguration_ParquetSerDe_Empty (165.68s) --- PASS: TestAccAWSKinesisFirehoseDeliveryStream_ExtendedS3_DataFormatConversionConfiguration_Serializer_Update (152.61s) --- PASS: TestAccAWSKinesisFirehoseDeliveryStream_ExtendedS3_ErrorOutputPrefix (177.03s) --- PASS: TestAccAWSKinesisFirehoseDeliveryStream_ExtendedS3_ExternalUpdate (152.21s) --- PASS: TestAccAWSKinesisFirehoseDeliveryStream_ExtendedS3basic (162.62s) --- PASS: TestAccAWSKinesisFirehoseDeliveryStream_ExtendedS3KmsKeyArn (225.11s) --- PASS: TestAccAWSKinesisFirehoseDeliveryStream_ExtendedS3Updates (157.35s) --- PASS: TestAccAWSKinesisFirehoseDeliveryStream_importBasic (130.87s) --- PASS: TestAccAWSKinesisFirehoseDeliveryStream_missingProcessingConfiguration (199.02s) --- PASS: TestAccAWSKinesisFirehoseDeliveryStream_RedshiftConfigUpdates (714.21s) --- PASS: TestAccAWSKinesisFirehoseDeliveryStream_s3basic (114.90s) --- PASS: TestAccAWSKinesisFirehoseDeliveryStream_s3basicWithSSE (372.95s) --- PASS: TestAccAWSKinesisFirehoseDeliveryStream_s3basicWithTags (194.45s) --- PASS: TestAccAWSKinesisFirehoseDeliveryStream_s3ConfigUpdates (287.14s) --- PASS: TestAccAWSKinesisFirehoseDeliveryStream_s3KinesisStreamSource (136.86s) --- PASS: TestAccAWSKinesisFirehoseDeliveryStream_s3WithCloudwatchLogging (122.61s) --- PASS: TestAccAWSKinesisFirehoseDeliveryStream_SplunkConfigUpdates (195.68s) ``` * Add iotevents service to endpointServiceNames * deps: github.com/aws/aws-sdk-go/service/appstream@v1.23.12 * Update CHANGELOG for hashicorp#6523 * Remove duplicate nested lists for Global Accelerator resources * remove status attribute from docdb_cluster_instance doc (hashicorp#9921) * tests/service/s3: Check S3 Bucket location before attempting other S3 operations in sweepers To prevent errors in the sweepers such as: ``` 2019/08/30 12:23:14 [ERR] error running (aws_s3_bucket_object): error listing S3 Bucket (tf-acc-test-6376821381526517605) Objects: AccessDenied: Access Denied ``` And ``` 2019/08/30 11:38:01 [INFO] Skipping S3 Bucket (tf-test-bucket-destination-2585329347318042546): AuthorizationHeaderMalformed: The authorization header is malformed; the region 'us-east-1' is wrong; expecting 'eu-west-1' ``` Output from sweeper: ```console $ go test ./aws -v -sweep=us-east-1,us-west-2 -sweep-run=aws_s3_bucket -timeout 10h ... 2019/08/30 12:01:08 Sweeper Tests ran: - aws_s3_bucket_object - aws_s3_bucket ``` * Update module aws/aws-sdk-go to v1.23.13 * Update module hashicorp/terraform to v0.12.7 * resource/ecs_cluster: Add ability to enable ECS Cluster Insights Fixes: hashicorp#9294 * Fix test names and ignore broken import * Import test refactor for subnets * Add the service name in the list of customizable endpoints * Import acceptance test refactor for SSM resources * Acceptance test import refactor for SQS queue policy * Acceptance test import refactor for simpledb domain * Acceptance test import refactor for SFN activity * Acceptance test import refactor for SES template * Acceptance test import refactor for service resources * service/cloudhsmv2: Implement sweeper To prevent sweeper errors such as: ``` 2019/09/04 02:12:15 [ERR] error running (aws_vpc): Error deleting Subnet (subnet-064c5a7cad5a28b9e): DependencyViolation: The subnet 'subnet-064c5a7cad5a28b9e' has dependencies and cannot be deleted. ``` Output from sweeper in AWS Commercial: ```console $ go test ./aws -v -sweep=us-east-1,us-west-2 -sweep-run=aws_cloudhsm_v2_cluster -timeout 10h 2019/09/04 10:27:24 [DEBUG] Running Sweepers for region (us-east-1): ... 2019/09/04 10:27:25 [INFO] Deleting CloudHSMv2 Cluster (cluster-wporekks46b) HSM: hsm-y4tvi5ukedh ... 2019/09/04 10:29:06 [INFO] Deleting CloudHSMv2 Cluster: cluster-wporekks46b ... 2019/09/04 10:30:57 Sweeper Tests ran: - aws_cloudhsm_v2_cluster 2019/09/04 10:30:57 [DEBUG] Running Sweepers for region (us-west-2): ... 2019/09/04 10:30:59 [INFO] Deleting CloudHSMv2 Cluster: cluster-w56d2uzszjf ... 2019/09/04 10:31:30 [INFO] Deleting CloudHSMv2 Cluster: cluster-zrvllautsay ... 2019/09/04 10:32:01 [INFO] Deleting CloudHSMv2 Cluster: cluster-chxrr77wb2v ... 2019/09/04 10:32:32 Sweeper Tests ran: - aws_cloudhsm_v2_cluster ``` Output from sweeper in AWS GovCloud (US): ```console $ go test ./aws -v -sweep=us-gov-west-1 -sweep-run=aws_cloudhsm_v2_cluster -timeout 10h 2019/09/04 10:28:52 [DEBUG] Running Sweepers for region (us-gov-west-1): ... 2019/09/04 10:28:54 Sweeper Tests ran: - aws_cloudhsm_v2_cluster ``` Output from acceptance testing: ``` --- PASS: TestAccAWSCloudHsm2Cluster_basic (285.06s) --- PASS: TestAccAWSCloudHsm2Hsm_basic (898.88s) ``` * service/ecs: Minor adjustments to finish cluster setting implementation Output from acceptance testing: ``` --- PASS: TestAccAWSEcsCluster_disappears (9.90s) --- PASS: TestAccAWSEcsCluster_basic (13.35s) --- PASS: TestAccAWSEcsCluster_containerInsights (25.56s) --- PASS: TestAccAWSEcsCluster_Tags (27.62s) --- PASS: TestAccAWSEcsDataSource_ecsClusterContainerInsights (14.16s) --- PASS: TestAccAWSEcsDataSource_ecsCluster (14.25s) ``` * Update CHANGELOG for hashicorp#9720 * deps: Update github.com/aws/aws-sdk-go@v1.23.15 Updated via: ``` $ go get github.com/aws/aws-sdk-go@v1.23.15 $ go mod tidy $ go mod vendor ``` * Update CHANGELOG for hashicorp#9998 * Add created_add attribute to lightsail instance documentation * Update website/docs/r/lightsail_instance.html.markdown Co-Authored-By: Ryn Daniels <ryn@hashicorp.com> * v2.27.0 * Cleanup after v2.27.0 release * switch to terraform.NewResourceConfigRaw
|
I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues. If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thanks! |
ghost
locked and limited conversation to collaborators
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Continued from #9016 due to a mess up on my part.
When refreshing the state of a aws_kinesis_firehose_delivery_stream
resource, the AWS SDK may have more that one way to describe the same
configuration. In the case of ProcessingConfiguration, this can lead to
perpetual diffs and for DataFormatConversionConfiguration it can lead to
invalid state that causes later refresh and plan operations to crash.
We have solved these two situations here by detecting the alternative
representation of the "disabled" configuration, and disambiguated the
terraform state data.
Community Note
Fixes #8982
Fixes #6053
Fixes #4392
Release note for CHANGELOG: