Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix issue-10446, add test case for ebs_config.volumes_per_instance check #14858

Merged
merged 3 commits into from
Aug 28, 2020

Conversation

c4po
Copy link
Contributor

@c4po c4po commented Aug 26, 2020

Community Note

  • Please vote on this pull request by adding a 👍 reaction to the original pull request comment to help the community and maintainers prioritize this request
  • Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for pull request followers and do not help prioritize the request

Closes #10446

Release note for CHANGELOG:

BUG FIXES

* resource/aws_emr_cluster: Prevent EMR cluster recreate when `ebs_config.volumes_per_instance` is greater than 1

Output from acceptance testing:

$ make testacc TESTARGS='-run=TestAccAWSEMRCluster_ebs_config'
==> Checking that code complies with gofmt requirements...
TF_ACC=1 go test ./aws -v -count 1 -parallel 20 -run=TestAccAWSEMRCluster_ebs_config -timeout 120m
=== RUN   TestAccAWSEMRCluster_ebs_config
=== PAUSE TestAccAWSEMRCluster_ebs_config
=== CONT  TestAccAWSEMRCluster_ebs_config
2020/08/26 17:29:07 [INFO] Test: Using us-east-1 as test region
2020/08/26 17:29:07 [INFO] AWS Auth provider used: "EnvProvider"
--- PASS: TestAccAWSEMRCluster_ebs_config (576.89s)
PASS
ok  	github.com/terraform-providers/terraform-provider-aws/aws	576.948s

@c4po c4po requested a review from a team August 26, 2020 21:51
@ghost ghost added size/M Managed by automation to categorize the size of a PR. service/emr Issues and PRs that pertain to the emr service. tests PRs: expanded test coverage. Issues: expanded coverage, enhancements to test infrastructure. needs-triage Waiting for first response or review from a maintainer. labels Aug 26, 2020
@@ -1424,8 +1424,7 @@ func flattenEBSConfig(ebsBlockDevices []*emr.EbsBlockDevice) *schema.Set {
if ebs.VolumeSpecification.VolumeType != nil {
ebsAttrs["type"] = *ebs.VolumeSpecification.VolumeType
}
ebsAttrs["volumes_per_instance"] = 1

ebsAttrs["volumes_per_instance"] = len(ebsBlockDevices)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it possible to specify multiple ebs_config (multiple configurations with 1 or more volumes per instance)? e.g.

  master_instance_group {
    instance_type = "m4.large"

    ebs_config {
      size                 = 32
      type                 = "gp2"
      volumes_per_instance = 2
    }

    ebs_config {
      size                 = 50
      type                 = "gp2"
      volumes_per_instance = 2
    }
  }

If so, the volumes_per_instance needs to be calculated per configurations type.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think this is an issue here. The volumes_per_instance is an attribute of ebs_config.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, I get what you're saying.
I updated the PR and test case
but we can not dealing with 2 same ebs_config config like this

  master_instance_group {
    instance_type = "m4.large"

    ebs_config {
      size                 = 32
      type                 = "gp2"
      volumes_per_instance = 2
    }

    ebs_config {
      size                 = 32
      type                 = "gp2"
      volumes_per_instance = 1
    }
  }

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think not being able to handle that last configuration there is okay -- seems reasonable to say that should have one ebs_config with volumes_per_instance = 3 👍 Also thank you for adding the additional configuration to the test case!

@bflad bflad added bug Addresses a defect in current functionality. and removed needs-triage Waiting for first response or review from a maintainer. labels Aug 28, 2020
@bflad bflad added this to the v3.5.0 milestone Aug 28, 2020
@bflad bflad self-assigned this Aug 28, 2020
Copy link
Contributor

@bflad bflad left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great, thank you for this fix, @c4po 🚀

Output from acceptance testing (failure unrelated):

--- PASS: TestAccAWSEMRCluster_basic (418.45s)
--- PASS: TestAccAWSEMRCluster_configurationsJson (422.65s)
--- PASS: TestAccAWSEMRCluster_Step_Multiple (494.02s)
--- PASS: TestAccAWSEMRCluster_security_config (500.29s)
--- PASS: TestAccAWSEMRCluster_CoreInstanceGroup_AutoscalingPolicy (573.87s)
--- PASS: TestAccAWSEMRCluster_Kerberos_ClusterDedicatedKdc (629.50s)
--- PASS: TestAccAWSEMRCluster_Step_Basic (635.56s)
--- PASS: TestAccAWSEMRCluster_additionalInfo (704.09s)
--- FAIL: TestAccAWSEMRCluster_Ec2Attributes_DefaultManagedSecurityGroups (793.09s)
--- PASS: TestAccAWSEMRCluster_keepJob (380.07s)
--- PASS: TestAccAWSEMRCluster_disappears (806.34s)
--- PASS: TestAccAWSEMRCluster_terminationProtected (442.90s)
--- PASS: TestAccAWSEMRCluster_MasterInstanceGroup_InstanceType (881.22s)
--- PASS: TestAccAWSEMRCluster_MasterInstanceGroup_Name (938.63s)
--- PASS: TestAccAWSEMRCluster_Step_ConfigMode (958.46s)
--- PASS: TestAccAWSEMRCluster_CoreInstanceGroup_BidPrice (971.37s)
--- PASS: TestAccAWSEMRCluster_CoreInstanceGroup_InstanceType (1026.93s)
--- PASS: TestAccAWSEMRCluster_CoreInstanceGroup_Name (1041.45s)
--- PASS: TestAccAWSEMRCluster_step_concurrency_level (408.01s)
--- PASS: TestAccAWSEMRCluster_MasterInstanceGroup_BidPrice (1050.70s)
--- PASS: TestAccAWSEMRCluster_s3Logging (577.92s)
--- PASS: TestAccAWSEMRCluster_ebs_config (430.41s)
--- PASS: TestAccAWSEMRCluster_CoreInstanceGroup_InstanceCount (1167.50s)
--- PASS: TestAccAWSEMRCluster_MasterInstanceGroup_InstanceCount (1227.66s)
--- PASS: TestAccAWSEMRCluster_custom_ami_id (481.41s)
--- PASS: TestAccAWSEMRCluster_visibleToAllUsers (830.64s)
--- PASS: TestAccAWSEMRCluster_bootstrap_ordering (1382.29s)
--- PASS: TestAccAWSEMRCluster_tags (817.81s)
--- PASS: TestAccAWSEMRCluster_root_volume_size (816.65s)

@bflad bflad merged commit 2f8221a into hashicorp:master Aug 28, 2020
bflad added a commit that referenced this pull request Aug 28, 2020
@ghost
Copy link

ghost commented Sep 3, 2020

This has been released in version 3.5.0 of the Terraform AWS provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

For further feature requests or bug reports with this functionality, please create a new GitHub issue following the template for triage. Thanks!

@ghost
Copy link

ghost commented Sep 28, 2020

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thanks!

@ghost ghost locked and limited conversation to collaborators Sep 28, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
bug Addresses a defect in current functionality. service/emr Issues and PRs that pertain to the emr service. size/M Managed by automation to categorize the size of a PR. tests PRs: expanded test coverage. Issues: expanded coverage, enhancements to test infrastructure.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

EMR recreates resources when ebs_config.volumes_per_instance is greater than 1
2 participants