Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Timeout retry when deleting EMR cluster #9053

Merged
merged 2 commits into from
Jun 20, 2019
Merged

Timeout retry when deleting EMR cluster #9053

merged 2 commits into from
Jun 20, 2019

Conversation

ryndaniels
Copy link
Contributor

Community Note

  • Please vote on this pull request by adding a 👍 reaction to the original pull request comment to help the community and maintainers prioritize this request
  • Please do not leave "+1" comments, they generate extra noise for pull request followers and do not help prioritize the request

Related #7873

Release note for CHANGELOG:

BUG FIXES:
* resource/aws_emr_cluster: Final retry after timeout error when deleting EMR cluster

Output from acceptance testing:

make testacc TESTARGS="-run=TestAccAWSEMRCluster"
==> Checking that code complies with gofmt requirements...
TF_ACC=1 go test ./... -v -parallel 20 -run=TestAccAWSEMRCluster -timeout 120m
?       github.com/terraform-providers/terraform-provider-aws   [no test files]
=== RUN   TestAccAWSEMRCluster_basic
=== PAUSE TestAccAWSEMRCluster_basic
=== RUN   TestAccAWSEMRCluster_additionalInfo
=== PAUSE TestAccAWSEMRCluster_additionalInfo
=== RUN   TestAccAWSEMRCluster_configurationsJson
=== PAUSE TestAccAWSEMRCluster_configurationsJson
=== RUN   TestAccAWSEMRCluster_CoreInstanceGroup_AutoscalingPolicy
=== PAUSE TestAccAWSEMRCluster_CoreInstanceGroup_AutoscalingPolicy
=== RUN   TestAccAWSEMRCluster_CoreInstanceGroup_BidPrice
=== PAUSE TestAccAWSEMRCluster_CoreInstanceGroup_BidPrice
=== RUN   TestAccAWSEMRCluster_CoreInstanceGroup_InstanceCount
=== PAUSE TestAccAWSEMRCluster_CoreInstanceGroup_InstanceCount
=== RUN   TestAccAWSEMRCluster_CoreInstanceGroup_InstanceType
=== PAUSE TestAccAWSEMRCluster_CoreInstanceGroup_InstanceType
=== RUN   TestAccAWSEMRCluster_CoreInstanceGroup_Migration_CoreInstanceType
=== PAUSE TestAccAWSEMRCluster_CoreInstanceGroup_Migration_CoreInstanceType
=== RUN   TestAccAWSEMRCluster_CoreInstanceGroup_Migration_InstanceGroup
=== PAUSE TestAccAWSEMRCluster_CoreInstanceGroup_Migration_InstanceGroup
=== RUN   TestAccAWSEMRCluster_CoreInstanceGroup_Name
=== PAUSE TestAccAWSEMRCluster_CoreInstanceGroup_Name
=== RUN   TestAccAWSEMRCluster_instance_group
=== PAUSE TestAccAWSEMRCluster_instance_group
=== RUN   TestAccAWSEMRCluster_instance_group_names
=== PAUSE TestAccAWSEMRCluster_instance_group_names
=== RUN   TestAccAWSEMRCluster_instance_group_update
=== PAUSE TestAccAWSEMRCluster_instance_group_update
=== RUN   TestAccAWSEMRCluster_instance_group_EBSVolumeType_st1
=== PAUSE TestAccAWSEMRCluster_instance_group_EBSVolumeType_st1
=== RUN   TestAccAWSEMRCluster_updateAutoScalingPolicy
=== PAUSE TestAccAWSEMRCluster_updateAutoScalingPolicy
=== RUN   TestAccAWSEMRCluster_Kerberos_ClusterDedicatedKdc
=== PAUSE TestAccAWSEMRCluster_Kerberos_ClusterDedicatedKdc
=== RUN   TestAccAWSEMRCluster_MasterInstanceGroup_BidPrice
=== PAUSE TestAccAWSEMRCluster_MasterInstanceGroup_BidPrice
=== RUN   TestAccAWSEMRCluster_MasterInstanceGroup_InstanceType
=== PAUSE TestAccAWSEMRCluster_MasterInstanceGroup_InstanceType
=== RUN   TestAccAWSEMRCluster_MasterInstanceGroup_Migration_InstanceGroup
=== PAUSE TestAccAWSEMRCluster_MasterInstanceGroup_Migration_InstanceGroup
=== RUN   TestAccAWSEMRCluster_MasterInstanceGroup_Migration_MasterInstanceType
=== PAUSE TestAccAWSEMRCluster_MasterInstanceGroup_Migration_MasterInstanceType
=== RUN   TestAccAWSEMRCluster_MasterInstanceGroup_Name
=== PAUSE TestAccAWSEMRCluster_MasterInstanceGroup_Name
=== RUN   TestAccAWSEMRCluster_security_config
=== PAUSE TestAccAWSEMRCluster_security_config
=== RUN   TestAccAWSEMRCluster_Step_Basic
=== PAUSE TestAccAWSEMRCluster_Step_Basic
=== RUN   TestAccAWSEMRCluster_Step_ConfigMode
=== PAUSE TestAccAWSEMRCluster_Step_ConfigMode
=== RUN   TestAccAWSEMRCluster_Step_Multiple
=== PAUSE TestAccAWSEMRCluster_Step_Multiple
=== RUN   TestAccAWSEMRCluster_bootstrap_ordering
=== PAUSE TestAccAWSEMRCluster_bootstrap_ordering
=== RUN   TestAccAWSEMRCluster_terminationProtected
=== PAUSE TestAccAWSEMRCluster_terminationProtected
=== RUN   TestAccAWSEMRCluster_keepJob
=== PAUSE TestAccAWSEMRCluster_keepJob
=== RUN   TestAccAWSEMRCluster_visibleToAllUsers
=== PAUSE TestAccAWSEMRCluster_visibleToAllUsers
=== RUN   TestAccAWSEMRCluster_s3Logging
=== PAUSE TestAccAWSEMRCluster_s3Logging
=== RUN   TestAccAWSEMRCluster_tags
=== PAUSE TestAccAWSEMRCluster_tags
=== RUN   TestAccAWSEMRCluster_root_volume_size
=== PAUSE TestAccAWSEMRCluster_root_volume_size
=== RUN   TestAccAWSEMRCluster_custom_ami_id
=== PAUSE TestAccAWSEMRCluster_custom_ami_id
=== CONT  TestAccAWSEMRCluster_basic
=== CONT  TestAccAWSEMRCluster_custom_ami_id
=== CONT  TestAccAWSEMRCluster_Step_ConfigMode
=== CONT  TestAccAWSEMRCluster_MasterInstanceGroup_BidPrice
=== CONT  TestAccAWSEMRCluster_root_volume_size
=== CONT  TestAccAWSEMRCluster_tags
=== CONT  TestAccAWSEMRCluster_s3Logging
=== CONT  TestAccAWSEMRCluster_Step_Basic
=== CONT  TestAccAWSEMRCluster_security_config
=== CONT  TestAccAWSEMRCluster_MasterInstanceGroup_Name
=== CONT  TestAccAWSEMRCluster_MasterInstanceGroup_Migration_MasterInstanceType
=== CONT  TestAccAWSEMRCluster_MasterInstanceGroup_Migration_InstanceGroup
=== CONT  TestAccAWSEMRCluster_Kerberos_ClusterDedicatedKdc
=== CONT  TestAccAWSEMRCluster_updateAutoScalingPolicy
=== CONT  TestAccAWSEMRCluster_instance_group_EBSVolumeType_st1
=== CONT  TestAccAWSEMRCluster_instance_group_update
=== CONT  TestAccAWSEMRCluster_instance_group_names
=== CONT  TestAccAWSEMRCluster_instance_group
=== CONT  TestAccAWSEMRCluster_CoreInstanceGroup_Name
=== CONT  TestAccAWSEMRCluster_MasterInstanceGroup_InstanceType
--- FAIL: TestAccAWSEMRCluster_basic (150.33s)
    testing.go:568: Step 0 error: errors during apply:
        
        Error: Error waiting for EMR Cluster state to be "WAITING" or "RUNNING": TERMINATING: VALIDATION_ERROR: The requested instance type c4.large is not supported in the requested availability zone. Learn more at https://docs.aws.amazon.com/console/elasticmapreduce/ERROR_noinstancetype
        
          on /var/folders/pd/swwl85ks1nvbfy2pht84q2m40000gq/T/tf-test144138385/main.tf line 2:
          (source code not available)
        
        
=== CONT  TestAccAWSEMRCluster_CoreInstanceGroup_Migration_InstanceGroup
--- PASS: TestAccAWSEMRCluster_security_config (456.37s)
=== CONT  TestAccAWSEMRCluster_CoreInstanceGroup_Migration_CoreInstanceType
--- PASS: TestAccAWSEMRCluster_Step_Basic (494.05s)
=== CONT  TestAccAWSEMRCluster_terminationProtected
--- PASS: TestAccAWSEMRCluster_instance_group_EBSVolumeType_st1 (547.80s)
=== CONT  TestAccAWSEMRCluster_CoreInstanceGroup_InstanceType
--- PASS: TestAccAWSEMRCluster_instance_group_update (551.19s)
=== CONT  TestAccAWSEMRCluster_visibleToAllUsers
--- PASS: TestAccAWSEMRCluster_MasterInstanceGroup_Migration_InstanceGroup (571.31s)
=== CONT  TestAccAWSEMRCluster_CoreInstanceGroup_InstanceCount
--- PASS: TestAccAWSEMRCluster_updateAutoScalingPolicy (594.57s)
=== CONT  TestAccAWSEMRCluster_keepJob
--- FAIL: TestAccAWSEMRCluster_terminationProtected (104.08s)
    testing.go:568: Step 0 error: errors during apply:
        
        Error: Error waiting for EMR Cluster state to be "WAITING" or "RUNNING": TERMINATED_WITH_ERRORS: VALIDATION_ERROR: The requested instance type c4.large is not supported in the requested availability zone. Learn more at https://docs.aws.amazon.com/console/elasticmapreduce/ERROR_noinstancetype
        
          on /var/folders/pd/swwl85ks1nvbfy2pht84q2m40000gq/T/tf-test212915209/main.tf line 2:
          (source code not available)
        
        
=== CONT  TestAccAWSEMRCluster_CoreInstanceGroup_BidPrice
--- PASS: TestAccAWSEMRCluster_instance_group_names (607.68s)
=== CONT  TestAccAWSEMRCluster_CoreInstanceGroup_AutoscalingPolicy
--- PASS: TestAccAWSEMRCluster_MasterInstanceGroup_Migration_MasterInstanceType (609.21s)
=== CONT  TestAccAWSEMRCluster_bootstrap_ordering
--- PASS: TestAccAWSEMRCluster_custom_ami_id (611.40s)
=== CONT  TestAccAWSEMRCluster_configurationsJson
--- PASS: TestAccAWSEMRCluster_s3Logging (651.91s)
=== CONT  TestAccAWSEMRCluster_Step_Multiple
--- PASS: TestAccAWSEMRCluster_CoreInstanceGroup_Migration_InstanceGroup (513.35s)
=== CONT  TestAccAWSEMRCluster_additionalInfo
--- PASS: TestAccAWSEMRCluster_instance_group (670.99s)
--- PASS: TestAccAWSEMRCluster_Kerberos_ClusterDedicatedKdc (697.30s)
--- FAIL: TestAccAWSEMRCluster_bootstrap_ordering (91.83s)
    testing.go:568: Step 0 error: errors during apply:
        
        Error: Error waiting for EMR Cluster state to be "WAITING" or "RUNNING": TERMINATING: VALIDATION_ERROR: The requested instance type c4.large is not supported in the requested availability zone. Learn more at https://docs.aws.amazon.com/console/elasticmapreduce/ERROR_noinstancetype
        
          on /var/folders/pd/swwl85ks1nvbfy2pht84q2m40000gq/T/tf-test464554276/main.tf line 2:
          (source code not available)
        
        
--- FAIL: TestAccAWSEMRCluster_additionalInfo (120.22s)
    testing.go:568: Step 0 error: errors during apply:
        
        Error: Error waiting for EMR Cluster state to be "WAITING" or "RUNNING": TERMINATING: VALIDATION_ERROR: The requested instance type c4.large is not supported in the requested availability zone. Learn more at https://docs.aws.amazon.com/console/elasticmapreduce/ERROR_noinstancetype
        
          on /var/folders/pd/swwl85ks1nvbfy2pht84q2m40000gq/T/tf-test351706333/main.tf line 2:
          (source code not available)
        
        
--- PASS: TestAccAWSEMRCluster_MasterInstanceGroup_InstanceType (847.93s)
--- PASS: TestAccAWSEMRCluster_tags (882.94s)
--- PASS: TestAccAWSEMRCluster_root_volume_size (931.75s)
--- PASS: TestAccAWSEMRCluster_MasterInstanceGroup_Name (966.34s)
--- PASS: TestAccAWSEMRCluster_keepJob (403.05s)
--- PASS: TestAccAWSEMRCluster_configurationsJson (400.06s)
--- PASS: TestAccAWSEMRCluster_CoreInstanceGroup_Migration_CoreInstanceType (562.73s)
--- PASS: TestAccAWSEMRCluster_MasterInstanceGroup_BidPrice (1044.69s)
--- PASS: TestAccAWSEMRCluster_Step_ConfigMode (1107.78s)
--- PASS: TestAccAWSEMRCluster_Step_Multiple (475.20s)
--- PASS: TestAccAWSEMRCluster_CoreInstanceGroup_Name (1135.51s)
--- PASS: TestAccAWSEMRCluster_CoreInstanceGroup_AutoscalingPolicy (595.05s)
--- PASS: TestAccAWSEMRCluster_visibleToAllUsers (882.57s)
--- PASS: TestAccAWSEMRCluster_CoreInstanceGroup_InstanceType (894.86s)
--- PASS: TestAccAWSEMRCluster_CoreInstanceGroup_BidPrice (1019.45s)
--- PASS: TestAccAWSEMRCluster_CoreInstanceGroup_InstanceCount (1186.78s)
FAIL
FAIL    github.com/terraform-providers/terraform-provider-aws/aws       1759.028s
make: *** [testacc] Error 1


make testacc TESTARGS="-run=TestAccAWSEMRCluster_basic"
==> Checking that code complies with gofmt requirements...
TF_ACC=1 go test ./... -v -parallel 20 -run=TestAccAWSEMRCluster_basic -timeout 120m
?       github.com/terraform-providers/terraform-provider-aws   [no test files]
=== RUN   TestAccAWSEMRCluster_basic
=== PAUSE TestAccAWSEMRCluster_basic
=== CONT  TestAccAWSEMRCluster_basic
--- PASS: TestAccAWSEMRCluster_basic (417.90s)
PASS
ok      github.com/terraform-providers/terraform-provider-aws/aws       418.713s


make testacc TESTARGS="-run=TestAccAWSEMRCluster_terminationProtected"
==> Checking that code complies with gofmt requirements...
TF_ACC=1 go test ./... -v -parallel 20 -run=TestAccAWSEMRCluster_terminationProtected -timeout 120m
?       github.com/terraform-providers/terraform-provider-aws   [no test files]
=== RUN   TestAccAWSEMRCluster_terminationProtected
=== PAUSE TestAccAWSEMRCluster_terminationProtected
=== CONT  TestAccAWSEMRCluster_terminationProtected
--- PASS: TestAccAWSEMRCluster_terminationProtected (540.38s)
PASS
ok      github.com/terraform-providers/terraform-provider-aws/aws       541.182s


make testacc TESTARGS="-run=TestAccAWSEMRCluster_bootstrap_ordering"
==> Checking that code complies with gofmt requirements...
TF_ACC=1 go test ./... -v -parallel 20 -run=TestAccAWSEMRCluster_bootstrap_ordering -timeout 120m
?       github.com/terraform-providers/terraform-provider-aws   [no test files]
=== RUN   TestAccAWSEMRCluster_bootstrap_ordering
=== PAUSE TestAccAWSEMRCluster_bootstrap_ordering
=== CONT  TestAccAWSEMRCluster_bootstrap_ordering
--- PASS: TestAccAWSEMRCluster_bootstrap_ordering (457.91s)
PASS
ok      github.com/terraform-providers/terraform-provider-aws/aws       458.758

make testacc TESTARGS="-run=TestAccAWSEMRCluster_additionalInfo"
==> Checking that code complies with gofmt requirements...
TF_ACC=1 go test ./... -v -parallel 20 -run=TestAccAWSEMRCluster_additionalInfo -timeout 120m
?       github.com/terraform-providers/terraform-provider-aws   [no test files]
=== RUN   TestAccAWSEMRCluster_additionalInfo
=== PAUSE TestAccAWSEMRCluster_additionalInfo
=== CONT  TestAccAWSEMRCluster_additionalInfo
--- PASS: TestAccAWSEMRCluster_additionalInfo (405.64s)
PASS
ok      github.com/terraform-providers/terraform-provider-aws/aws       406.531s

@ghost ghost added service/emr Issues and PRs that pertain to the emr service. size/M Managed by automation to categorize the size of a PR. labels Jun 19, 2019
@ryndaniels ryndaniels requested a review from bflad June 19, 2019 16:00
@ryndaniels ryndaniels added the bug Addresses a defect in current functionality. label Jun 19, 2019
@bflad bflad self-assigned this Jun 20, 2019
@bflad bflad added this to the v2.16.0 milestone Jun 20, 2019
bflad
bflad approved these changes Jun 20, 2019
View reviewed changes
Copy link
Contributor

@bflad bflad left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One 🤔 but otherwise LGTM 🚀

--- PASS: TestAccAWSEMRCluster_MasterInstanceGroup_Migration_MasterInstanceType (394.96s)
--- PASS: TestAccAWSEMRCluster_additionalInfo (400.08s)
--- PASS: TestAccAWSEMRCluster_basic (422.02s)
--- PASS: TestAccAWSEMRCluster_CoreInstanceGroup_Migration_InstanceGroup (431.49s)
--- PASS: TestAccAWSEMRCluster_instance_group_names (461.86s)
--- PASS: TestAccAWSEMRCluster_updateAutoScalingPolicy (466.88s)
--- PASS: TestAccAWSEMRCluster_CoreInstanceGroup_Migration_CoreInstanceType (547.00s)
--- PASS: TestAccAWSEMRCluster_Kerberos_ClusterDedicatedKdc (554.49s)
--- PASS: TestAccAWSEMRCluster_CoreInstanceGroup_InstanceCount (557.40s)
--- PASS: TestAccAWSEMRCluster_CoreInstanceGroup_AutoscalingPolicy (565.34s)
--- PASS: TestAccAWSEMRCluster_instance_group_EBSVolumeType_st1 (565.31s)
--- PASS: TestAccAWSEMRCluster_instance_group (574.28s)
--- PASS: TestAccAWSEMRCluster_instance_group_update (585.54s)
--- PASS: TestAccAWSEMRCluster_MasterInstanceGroup_Migration_InstanceGroup (600.70s)
--- PASS: TestAccAWSEMRCluster_security_config (395.95s)
--- PASS: TestAccAWSEMRCluster_MasterInstanceGroup_Name (734.13s)
--- PASS: TestAccAWSEMRCluster_MasterInstanceGroup_InstanceType (817.77s)
--- PASS: TestAccAWSEMRCluster_CoreInstanceGroup_Name (849.00s)
--- PASS: TestAccAWSEMRCluster_bootstrap_ordering (414.41s)
--- PASS: TestAccAWSEMRCluster_terminationProtected (436.65s)
--- PASS: TestAccAWSEMRCluster_CoreInstanceGroup_BidPrice (983.02s)
--- PASS: TestAccAWSEMRCluster_keepJob (447.34s)
--- PASS: TestAccAWSEMRCluster_CoreInstanceGroup_InstanceType (994.51s)
--- PASS: TestAccAWSEMRCluster_MasterInstanceGroup_BidPrice (1010.89s)
--- PASS: TestAccAWSEMRCluster_Step_Basic (612.90s)
--- PASS: TestAccAWSEMRCluster_Step_Multiple (620.28s)
--- PASS: TestAccAWSEMRCluster_custom_ami_id (489.54s)
--- PASS: TestAccAWSEMRCluster_s3Logging (523.45s)
--- PASS: TestAccAWSEMRCluster_visibleToAllUsers (763.51s)
--- PASS: TestAccAWSEMRCluster_root_volume_size (765.58s)
--- PASS: TestAccAWSEMRCluster_Step_ConfigMode (932.29s)
--- PASS: TestAccAWSEMRCluster_tags (800.36s)

aws/resource_aws_emr_cluster.go Outdated Show resolved Hide resolved
@bflad bflad assigned ryndaniels and unassigned bflad Jun 20, 2019
@ryndaniels @bflad
Update aws/resource_aws_emr_cluster.go
e640b5e 
Co-Authored-By: Brian Flad <bflad417@gmail.com>
@ryndaniels ryndaniels merged commit 5889819 into master Jun 20, 2019
@ryndaniels ryndaniels deleted the rfd-retry-emr branch June 20, 2019 08:30
@bflad
Copy link
Contributor

bflad commented Jun 20, 2019

This has been released in version 2.16.0 of the Terraform AWS provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

@ghost
Copy link

ghost commented Nov 3, 2019

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thanks!

@ghost ghost locked and limited conversation to collaborators Nov 3, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Reviewers

@bflad bflad bflad approved these changes

Assignees

@ryndaniels ryndaniels

Labels
bug Addresses a defect in current functionality. service/emr Issues and PRs that pertain to the emr service. size/M Managed by automation to categorize the size of a PR.
Projects
None yet
Milestone
v2.16.0
Development

Successfully merging this pull request may close these issues.
2 participants
@ryndaniels @bflad