-
Notifications
You must be signed in to change notification settings - Fork 9.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
resource/aws_api_gateway_rest_api: Remove extraneous timeout for resource deletion #7554
Conversation
…urce deletion AWS API Gateway currently has a 1 `DeleteRestApi`/minute rate limit and when exceeding that returns `429 TooManyRequests` errors which are automatically retried by the AWS Go SDK. The resource was artificially timeboxing the `DeleteRestApi` API call to 10 minutes previously with no `RetryableError` condition. When running with parallelism over 10 (e.g. the acceptance testing runs at 20), there was a very high probability that one or more resource deletions would fail with the following error since the AWS Go SDK does not return the 429 error back to the calling code while retrying: ``` Error: errors during apply: timeout while waiting for state to become 'success' (timeout: 10m0s) ``` Removing the `Retry()` wrapper means the resource will retry the `DeleteRestApi` request without time limitations up to the configured provider `max_retries` (AWS Go SDK `MaxRetries`), which defaults to 25 and its exponential backoff generally lasts up to an hour. A previous full `TestAccAWSAPIGateway` (76 tests currently) acceptance test run resulted in 16 failures due to the timeout. Rerunning with this change yields no timeout related failures where anything over ~600 seconds would have failed previously. The `TestAccAWSAPIGatewayUsagePlan` failures are unrelated and will require separate addressing. ``` --- PASS: TestAccAWSAPIGatewayAccount_importBasic (9.38s) --- PASS: TestAccAWSAPIGatewayClientCertificate_basic (15.14s) --- PASS: TestAccAWSAPIGatewayDocumentationPart_basic (21.81s) --- PASS: TestAccAWSAPIGatewayDomainName_CertificateName (57.40s) --- PASS: TestAccAWSAPIGatewayDocumentationVersion_importBasic (64.03s) --- PASS: TestAccAWSAPIGatewayIntegrationResponse_basic (93.82s) --- PASS: TestAccAWSAPIGatewayAuthorizer_cognito (31.27s) --- PASS: TestAccAWSAPIGatewayDocumentationPart_importBasic (115.89s) --- PASS: TestAccAWSAPIGatewayGatewayResponse_basic (187.30s) --- PASS: TestAccAWSAPIGatewayBasePathMapping_basic (30.50s) --- PASS: TestAccAWSAPIGatewayClientCertificate_importBasic (9.61s) --- PASS: TestAccAWSAPIGatewayIntegration_basic (245.51s) --- PASS: TestAccAWSAPIGatewayDeployment_StageDescription (31.08s) --- PASS: TestAccAWSAPIGatewayDeployment_createBeforeDestoryUpdate (32.20s) --- PASS: TestAccAWSAPIGatewayMethod_customauthorizer (31.91s) --- PASS: TestAccAWSAPIGatewayIntegration_contentHandling (370.99s) --- PASS: TestAccAWSAPIGatewayAuthorizer_switchAuthType (58.16s) --- PASS: TestAccAWSAPIGatewayApiKey_basic (67.22s) --- PASS: TestAccAWSAPIGatewayDeployment_Description (25.98s) --- PASS: TestAccAWSAPIGatewayDocumentationVersion_importAllFields (517.75s) --- PASS: TestAccAWSAPIGatewayAuthorizer_basic (45.30s) --- PASS: TestAccAWSAPIGatewayResource_update (33.81s) --- PASS: TestAccAWSAPIGatewayDocumentationVersion_basic (600.43s) --- PASS: TestAccAWSAPIGatewayMethodSettings_Settings_RequireAuthorizationForCacheControl (43.25s) --- PASS: TestAccAWSAPIGatewayRequestValidator_basic (33.05s) --- PASS: TestAccAWSAPIGatewayMethod_basic (24.61s) --- PASS: TestAccAWSAPIGatewayApiKey_importBasic (142.18s) --- PASS: TestAccAWSAPIGatewayMethodSettings_Settings_UnauthorizedCacheControlHeaderStrategy (31.66s) --- PASS: TestAccAWSAPIGatewayMethodSettings_basic (773.05s) --- PASS: TestAccAWSAPIGatewayMethodSettings_Settings_CachingEnabled (817.51s) --- PASS: TestAccAWSAPIGatewayDeployment_basic (729.53s) --- PASS: TestAccAWSAPIGatewayMethodSettings_Settings_ThrottlingBurstLimit (30.70s) --- PASS: TestAccAWSAPIGatewayRestApi_EndpointConfiguration_Private (905.15s) --- PASS: TestAccAWSAPIGatewayDocumentationVersion_allFields (949.55s) --- PASS: TestAccAWSAPIGatewayMethodSettings_Settings_ThrottlingRateLimit (132.41s) --- FAIL: TestAccAWSAPIGatewayUsagePlan_importBasic (63.25s) testing.go:538: Step 0 error: Error applying: 1 error occurred: * aws_api_gateway_deployment.foo: 1 error occurred: * aws_api_gateway_deployment.foo: Error creating API Gateway Deployment: ConflictException: Another Deployment is in progress for rest api with id b4e9g4hos5 . Please try again later. status code: 409, request id: 63e82cc4-305e-11e9-925e-d98b99325ebe --- FAIL: TestAccAWSAPIGatewayUsagePlanKey_basic (39.18s) testing.go:538: Step 0 error: Error applying: 1 error occurred: * aws_api_gateway_deployment.test: 1 error occurred: * aws_api_gateway_deployment.test: Error creating API Gateway Deployment: ConflictException: Another Deployment is in progress for rest api with id 2hi2taru0i . Please try again later. status code: 409, request id: 7dd99050-305e-11e9-9b22-11a3b27a8062 --- FAIL: TestAccAWSAPIGatewayUsagePlan_description (54.72s) testing.go:538: Step 0 error: Error applying: 1 error occurred: * aws_api_gateway_deployment.test: 1 error occurred: * aws_api_gateway_deployment.test: Error creating API Gateway Deployment: ConflictException: Another Deployment is in progress for rest api with id 5fkt8pa7g7 . Please try again later. status code: 409, request id: 93e8d7f7-305e-11e9-82a4-e7660a766498 --- PASS: TestAccAWSAPIGatewayMethodSettings_Settings_LoggingLevel (34.52s) --- PASS: TestAccAWSAPIGatewayRestApi_openapi (19.54s) --- PASS: TestAccAWSAPIGatewayStage_accessLogSettings (250.56s) --- PASS: TestAccAWSAPIGatewayRestApi_policy (35.38s) --- PASS: TestAccAWSAPIGatewayMethodSettings_Settings_Multiple (230.89s) --- PASS: TestAccAWSAPIGatewayMethod_cognitoauthorizer (452.46s) --- PASS: TestAccAWSAPIGatewayMethodSettings_Settings_CacheDataEncrypted (1269.68s) --- PASS: TestAccAWSAPIGatewayAccount_basic (73.03s) --- PASS: TestAccAWSAPIGatewayMethodSettings_Settings_DataTraceEnabled (115.51s) --- PASS: TestAccAWSAPIGatewayUsagePlan_quota (47.29s) --- PASS: TestAccAWSAPIGatewayRestApi_api_key_source (125.12s) --- PASS: TestAccAWSAPIGatewayMethodResponse_basic (1374.18s) --- PASS: TestAccAWSAPIGatewayMethodSettings_Settings_MetricsEnabled (455.52s) --- FAIL: TestAccAWSAPIGatewayUsagePlan_productCode (62.26s) testing.go:538: Step 0 error: Error applying: 1 error occurred: * aws_api_gateway_deployment.test: 1 error occurred: * aws_api_gateway_deployment.test: Error creating API Gateway Deployment: ConflictException: Another Deployment is in progress for rest api with id ehysmxxa8d . Please try again later. status code: 409, request id: 7a57ee18-305f-11e9-b5ad-518807ce17fb --- PASS: TestAccAWSAPIGatewayUsagePlan_apiStages (63.55s) --- PASS: TestAccAWSAPIGatewayBasePathMapping_BasePath_Empty (1372.28s) --- PASS: TestAccAWSAPIGatewayUsagePlan_basic (61.01s) --- PASS: TestAccAWSAPIGatewayRestApi_EndpointConfiguration (1191.02s) --- PASS: TestAccAWSAPIGatewayDeployment_Variables (1566.24s) --- PASS: TestAccAWSAPIGatewayIntegration_integrationType (1637.29s) --- PASS: TestAccAWSAPIGatewayStage_basic (575.05s) --- PASS: TestAccAWSAPIGatewayRestApi_basic (1161.13s) --- PASS: TestAccAWSAPIGatewayMethod_customrequestvalidator (996.26s) --- PASS: TestAccAWSAPIGatewayUsagePlan_throttlingInitialRateLimit (390.33s) --- PASS: TestAccAWSAPIGatewayUsagePlan_throttling (480.69s) --- PASS: TestAccAWSAPIGatewayModel_basic (1131.61s) --- PASS: TestAccAWSAPIGatewayDocumentationPart_method (1892.44s) --- PASS: TestAccAWSAPIGatewayDeployment_StageName_Empty (1877.46s) --- PASS: TestAccAWSAPIGatewayVpcLink_importBasic (657.93s) --- PASS: TestAccAWSAPIGatewayAuthorizer_authTypeValidation (1752.95s) --- PASS: TestAccAWSAPIGatewayVpcLink_basic (684.31s) --- PASS: TestAccAWSAPIGatewayIntegration_cache_key_parameters (2028.42s) --- PASS: TestAccAWSAPIGatewayMethodSettings_Settings_CacheTtlInSeconds (2110.39s) --- PASS: TestAccAWSAPIGatewayResource_basic (881.84s) --- PASS: TestAccAWSAPIGatewayDocumentationPart_responseHeader (2282.80s) ```
This has been released in version 1.59.0 of the AWS provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading. |
I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues. If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thanks! |
AWS API Gateway currently has a 1
DeleteRestApi
/minute rate limit and when exceeding that returns429 TooManyRequests
errors which are automatically retried by the AWS Go SDK. The resource was artificially timeboxing theDeleteRestApi
API call to 10 minutes previously with noRetryableError
condition.When running with parallelism over 10 (e.g. the acceptance testing runs at 20), there was a very high probability that one or more resource deletions would fail with the following error since the AWS Go SDK does not return the 429 error back to the calling code while retrying:
Removing the
Retry()
wrapper means the resource will retry theDeleteRestApi
request without time limitations up to the configured providermax_retries
(AWS Go SDKMaxRetries
), which defaults to 25 and its exponential backoff generally lasts up to an hour.A previous full
TestAccAWSAPIGateway
(76 tests currently) acceptance test run resulted in 16 failures due to the timeout. Rerunning with this change yields no timeout related failures where anything over ~600 seconds would have failed previously. TheTestAccAWSAPIGatewayUsagePlan
failures are unrelated and will require separate addressing.