Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add retry for MWAA CreateEnvironment on ValidationException #27031

Merged
merged 5 commits into from
Oct 4, 2022
Merged

Add retry for MWAA CreateEnvironment on ValidationException #27031

merged 5 commits into from
Oct 4, 2022

Conversation

dhegberg
Copy link
Contributor

@dhegberg dhegberg commented Sep 30, 2022

Handles scenario where IAM permission used in validation are slow to propagate.

Description

Introduce a retry on ValidationException & InternalServiceException

InternalServiceException is a general service 500 that can be retried.

ValidationException will include the case where an recently created IAM role is experiencing permission propagation delay. This retry is currently internal to the CreateEnvironment API, but will be removed in an upcoming release.

Relations

Relates #26429

References

Output from Acceptance Testing

Acceptance tests fail, but the same tests on the previous commit also fail.

Verified MWAA Environment creation does succeed. Tested triggering ValidationError and verified multiple retries occurred.

==> Checking that code complies with gofmt requirements...
TF_ACC=1 go test ./internal/service/mwaa/... -v -count 1 -parallel 20 -run='TestAccMWAAEnvironment_basic'  -timeout 180m
=== RUN   TestAccMWAAEnvironment_basic
=== PAUSE TestAccMWAAEnvironment_basic
=== CONT  TestAccMWAAEnvironment_basic
    environment_test.go:24: Failed state verification, resource with ID us-west-2 not found
--- FAIL: TestAccMWAAEnvironment_basic (2359.15s)
FAIL
FAIL	github.com/hashicorp/terraform-provider-aws/internal/service/mwaa	2359.221s
FAIL
make: *** [testacc] Error 1

@github-actions
Copy link

Community Note

Voting for Prioritization

  • Please vote on this pull request by adding a 👍 reaction to the original post to help the community and maintainers prioritize this pull request.
  • Please see our prioritization guide for information on how we prioritize.
  • Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request.

For Submitters

  • Review the contribution guide relating to the type of change you are making to ensure all of the necessary steps have been taken.
  • For new resources and data sources, use skaff to generate scaffolding with comments detailing common expectations.
  • Whether or not the branch has been rebased will not impact prioritization, but doing so is always a welcome surprise.

@github-actions github-actions bot added needs-triage Waiting for first response or review from a maintainer. service/mwaa Issues and PRs that pertain to the mwaa service. size/M Managed by automation to categorize the size of a PR. labels Sep 30, 2022
@dhegberg dhegberg marked this pull request as ready for review September 30, 2022 05:00
@dhegberg
Copy link
Contributor Author

@breathingdust

Can you advise on the right use of isResourceTimeoutError as required by the Semgrep check?

If I'm interpreting correctly, it's used to trigger cleanup when the timeout occurs.

MWAA environments have a blocking status. If the Environment is in CREATING then DeleteEnvironment cannot be called.

Should the isResourceTimeoutError just return null (with a comment explaining that behavior)?

@ewbankkit ewbankkit removed the needs-triage Waiting for first response or review from a maintainer. label Oct 3, 2022
@ewbankkit ewbankkit changed the title Add retry for MWAA CreateEnvironment on ValidationException. Handles… Add retry for MWAA CreateEnvironment on ValidationException Oct 3, 2022
@ewbankkit ewbankkit added the eventual-consistency Pertains to eventual consistency issues. label Oct 3, 2022
@github-actions github-actions bot added size/XL Managed by automation to categorize the size of a PR. and removed size/M Managed by automation to categorize the size of a PR. labels Oct 4, 2022
Copy link
Contributor

@ewbankkit ewbankkit left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 🚀.

% ACCTEST_TIMEOUT=360m make testacc TESTARGS='-run=TestAccMWAAEnvironment_' PKG=mwaa ACCTEST_PARALLELISM=2
==> Checking that code complies with gofmt requirements...
TF_ACC=1 go test ./internal/service/mwaa/... -v -count 1 -parallel 2  -run=TestAccMWAAEnvironment_ -timeout 360m
=== RUN   TestAccMWAAEnvironment_basic
=== PAUSE TestAccMWAAEnvironment_basic
=== RUN   TestAccMWAAEnvironment_disappears
=== PAUSE TestAccMWAAEnvironment_disappears
=== RUN   TestAccMWAAEnvironment_airflowOptions
=== PAUSE TestAccMWAAEnvironment_airflowOptions
=== RUN   TestAccMWAAEnvironment_log
=== PAUSE TestAccMWAAEnvironment_log
=== RUN   TestAccMWAAEnvironment_full
=== PAUSE TestAccMWAAEnvironment_full
=== RUN   TestAccMWAAEnvironment_pluginsS3ObjectVersion
=== PAUSE TestAccMWAAEnvironment_pluginsS3ObjectVersion
=== CONT  TestAccMWAAEnvironment_basic
=== CONT  TestAccMWAAEnvironment_log
--- PASS: TestAccMWAAEnvironment_basic (2314.28s)
=== CONT  TestAccMWAAEnvironment_airflowOptions
--- PASS: TestAccMWAAEnvironment_log (2990.47s)
=== CONT  TestAccMWAAEnvironment_disappears
--- PASS: TestAccMWAAEnvironment_disappears (2505.37s)
=== CONT  TestAccMWAAEnvironment_pluginsS3ObjectVersion
--- PASS: TestAccMWAAEnvironment_airflowOptions (3730.01s)
=== CONT  TestAccMWAAEnvironment_full
--- PASS: TestAccMWAAEnvironment_full (2338.81s)
--- PASS: TestAccMWAAEnvironment_pluginsS3ObjectVersion (3079.22s)
PASS
ok  	github.com/hashicorp/terraform-provider-aws/internal/service/mwaa	8580.178s

@github-actions github-actions bot added documentation Introduces or discusses updates to documentation. tests PRs: expanded test coverage. Issues: expanded coverage, enhancements to test infrastructure. labels Oct 4, 2022
@ewbankkit
Copy link
Contributor

@dhegberg Thanks for the contribution 🎉 👏.

@ewbankkit ewbankkit merged commit 308c570 into hashicorp:main Oct 4, 2022
@github-actions github-actions bot added this to the v4.34.0 milestone Oct 4, 2022
@dhegberg dhegberg deleted the b-add_mwaa_create_environment_retry branch October 6, 2022 18:39
@github-actions
Copy link

github-actions bot commented Oct 6, 2022

This functionality has been released in v4.34.0 of the Terraform AWS Provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

For further feature requests or bug reports with this functionality, please create a new GitHub issue following the template. Thank you!

@github-actions
Copy link

github-actions bot commented Nov 7, 2022

I'm going to lock this pull request because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems related to this change, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Nov 7, 2022
@justinretzolk justinretzolk added the partner Contribution from a partner. label May 16, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
documentation Introduces or discusses updates to documentation. eventual-consistency Pertains to eventual consistency issues. partner Contribution from a partner. service/mwaa Issues and PRs that pertain to the mwaa service. size/XL Managed by automation to categorize the size of a PR. tests PRs: expanded test coverage. Issues: expanded coverage, enhancements to test infrastructure.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants