Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add "io2" as a valid EBS volume type to EMR validation routine #37740

Merged
merged 13 commits into from
Aug 27, 2024

Conversation

atsalolikhin-spokeo
Copy link

@atsalolikhin-spokeo atsalolikhin-spokeo commented May 28, 2024

Description

Adds support for io2 EBS volume storage type to aws_emr_cluster.

EC2 r5b instances support io2 since July 2021: https://aws.amazon.com/blogs/aws/amazon-ebs-io2-block-express-volumes-with-amazon-ec2-r5b-instances-are-now-generally-available/

https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-supported-instance-types.html says that r5b instances are supported in e.g., us-east-1 (Virginia), us-west-2 (oregon), etc.

AWS EMR FAQ says:

Q: What kind of EBS volumes can I attach to an instance?

Amazon EMR allows you to use different EBS Volume Types: General Purpose SSD (GP2), Magnetic and Provisioned IOPS (SSD).

EMR Management Guide says:

When you configure instance types in Amazon EMR, you can specify additional EBS volumes to add capacity beyond the instance store (if present) and the default EBS volume. Amazon EBS provides the following volume types: General Purpose (SSD), Provisioned IOPS (SSD), Throughput Optimized (HDD), Cold (HDD), and Magnetic.

Adding support for io2 allows us to add Provisioned IOPS (io2) EBS volumes.

The desired use case is to be able to provision an EMR cluster with io2 volumes to increase disk I/O performance for a large data processing task.

Relations

Relates #20248 (which was auto-closed but is still a pain point for us).
Closes #38837.
Closes #23029.

References

ArtsiomAntropau and others added 2 commits February 8, 2022 18:39
Fix aws_emr_managed_scaling_policy for compatibility with the behavior of emr_cluster that was fixed as a part of the next problem:

hashicorp#16924
hashicorp#7783
Copy link

Community Note

Voting for Prioritization

  • Please vote on this pull request by adding a 👍 reaction to the original post to help the community and maintainers prioritize this pull request.
  • Please see our prioritization guide for information on how we prioritize.
  • Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request.

For Submitters

  • Review the contribution guide relating to the type of change you are making to ensure all of the necessary steps have been taken.
  • For new resources and data sources, use skaff to generate scaffolding with comments detailing common expectations.
  • Whether or not the branch has been rebased will not impact prioritization, but doing so is always a welcome surprise.

@github-actions github-actions bot added size/XS Managed by automation to categorize the size of a PR. service/emr Issues and PRs that pertain to the emr service. labels May 28, 2024
@terraform-aws-provider terraform-aws-provider bot added the needs-triage Waiting for first response or review from a maintainer. label May 28, 2024
Copy link

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Welcome @atsalolikhin-spokeo 👋

It looks like this is your first Pull Request submission to the Terraform AWS Provider! If you haven’t already done so please make sure you have checked out our CONTRIBUTOR guide and FAQ to make sure your contribution is adhering to best practice and has all the necessary elements in place for a successful approval.

Also take a look at our FAQ which details how we prioritize Pull Requests for inclusion.

Thanks again, and welcome to the community! 😃

@justinretzolk justinretzolk added enhancement Requests to existing resources that expand the functionality or scope. and removed needs-triage Waiting for first response or review from a maintainer. labels May 28, 2024
@github-actions github-actions bot added the documentation Introduces or discusses updates to documentation. label May 28, 2024
@atsalolikhin-spokeo
Copy link
Author

info: @ewbankkit

@atsalolikhin-spokeo atsalolikhin-spokeo changed the title Add "io2" as a valid EBS volume type to emr validation routine Add "io2" as a valid EBS volume type to EMR validation routine May 31, 2024
@ewbankkit ewbankkit self-assigned this Aug 27, 2024
@github-actions github-actions bot added the prioritized Part of the maintainer teams immediate focus. To be addressed within the current quarter. label Aug 27, 2024
@ewbankkit ewbankkit requested a review from a team as a code owner August 27, 2024 20:25
@github-actions github-actions bot added the tests PRs: expanded test coverage. Issues: expanded coverage, enhancements to test infrastructure. label Aug 27, 2024
Copy link
Contributor

@ewbankkit ewbankkit left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 🚀.

% make testacc TESTARGS='-run=TestAccEMRInstanceGroup_instanceCount\|TestAccEMRInstanceGroup_basic' PKG=emr
make: Verifying source code with gofmt...
==> Checking that code complies with gofmt requirements...
TF_ACC=1 go1.22.6 test ./internal/service/emr/... -v -count 1 -parallel 20  -run=TestAccEMRInstanceGroup_instanceCount\|TestAccEMRInstanceGroup_basic -timeout 360m
=== RUN   TestAccEMRInstanceGroup_basic
=== PAUSE TestAccEMRInstanceGroup_basic
=== RUN   TestAccEMRInstanceGroup_instanceCountDecrease
=== PAUSE TestAccEMRInstanceGroup_instanceCountDecrease
=== RUN   TestAccEMRInstanceGroup_instanceCountCreateZero
=== PAUSE TestAccEMRInstanceGroup_instanceCountCreateZero
=== CONT  TestAccEMRInstanceGroup_basic
=== CONT  TestAccEMRInstanceGroup_instanceCountCreateZero
=== CONT  TestAccEMRInstanceGroup_instanceCountDecrease
--- PASS: TestAccEMRInstanceGroup_instanceCountCreateZero (459.14s)
--- PASS: TestAccEMRInstanceGroup_basic (674.17s)
--- PASS: TestAccEMRInstanceGroup_instanceCountDecrease (786.38s)
PASS
ok  	github.com/hashicorp/terraform-provider-aws/internal/service/emr	791.483s

@ewbankkit ewbankkit merged commit 99ca8dd into hashicorp:main Aug 27, 2024
45 checks passed
@ewbankkit
Copy link
Contributor

@atsalolikhin-spokeo Thanks for the contribution 🎉 👏.

@github-actions github-actions bot added this to the v5.65.0 milestone Aug 27, 2024
@github-actions github-actions bot removed the prioritized Part of the maintainer teams immediate focus. To be addressed within the current quarter. label Aug 29, 2024
Copy link

This functionality has been released in v5.65.0 of the Terraform AWS Provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

For further feature requests or bug reports with this functionality, please create a new GitHub issue following the template. Thank you!

Copy link

I'm going to lock this pull request because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems related to this change, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Sep 29, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
documentation Introduces or discusses updates to documentation. enhancement Requests to existing resources that expand the functionality or scope. service/emr Issues and PRs that pertain to the emr service. size/XS Managed by automation to categorize the size of a PR. tests PRs: expanded test coverage. Issues: expanded coverage, enhancements to test infrastructure.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Bug]: aws_emr_instance_group configured with instance_count = 0 spins up instance at apply then destroys
5 participants