Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

resource/aws_batch_job_definition: Prevent extraneous differences with container properties missing environment, mount point, ulimits, and volumes configuration #12000

Merged
merged 2 commits into from
Feb 12, 2020

Conversation

bflad
Copy link
Contributor

@bflad bflad commented Feb 11, 2020

Community Note

  • Please vote on this pull request by adding a 👍 reaction to the original pull request comment to help the community and maintainers prioritize this request
  • Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for pull request followers and do not help prioritize the request

Closes #11998
Reference: #11488

Release note for CHANGELOG:

* resource/aws_batch_job_definition: Prevent extraneous differences with container properties missing `environment`, `mountPoints`, `ulimits`, and `volumes` configuration

Now that this resource is properly refreshing the container_properties attribute into the Terraform state, additional cases have been reported where the API canonicalizes the response with empty arrays. Here we account for environment, mountPoints, ulimits, and volumes.

Previous output from unit testing (before code update):

2020/02/11 12:37:58 [DEBUG] Canonical Batch Container Properties JSON are not equal.
First: {"command":["start.py","Ref::S3bucket","Ref::S3key"],"image":"example:image","jobRoleArn":"arn:aws:iam::123456789012:role/example","memory":2048,"vcpus":8}
Second: {"command":["start.py","Ref::S3bucket","Ref::S3key"],"environment":[],"image":"example:image","jobRoleArn":"arn:aws:iam::123456789012:role/example","memory":2048,"mountPoints":[],"ulimits":[],"vcpus":8,"volumes":[]}
--- FAIL: TestEquivalentBatchContainerPropertiesJSON (0.00s)
    --- FAIL: TestEquivalentBatchContainerPropertiesJSON/empty_environment,_mountPoints,_ulimits,_and_volumes (0.00s)
        container_properties_test.go:226: got false, expected true

Previous output from acceptance testing (before code update):

--- FAIL: TestAccAWSBatchJobDefinition_basic (13.16s)
    testing.go:640: Step 0 error: After applying this step, the plan was not empty:

        DIFF:

        DESTROY/CREATE: aws_batch_job_definition.test
          arn:                  "arn:aws:batch:us-west-2:--OMITTED--:job-definition/tf-acc-test-118932874341187373:1" => "<computed>"
          container_properties: "{\"command\":[\"echo\",\"test\"],\"environment\":[],\"image\":\"busybox\",\"memory\":128,\"mountPoints\":[],\"resourceRequirements\":[],\"ulimits\":[],\"vcpus\":1,\"volumes\":[]}" => "{\"command\":[\"echo\",\"test\"],\"image\":\"busybox\",\"memory\":128,\"vcpus\":1}" (forces new resource)
          id:                   "arn:aws:batch:us-west-2:--OMITTED--:job-definition/tf-acc-test-118932874341187373:1" => "<computed>"
          name:                 "tf-acc-test-118932874341187373" => "tf-acc-test-118932874341187373"
          retry_strategy.#:     "0" => "0"
          revision:             "1" => "<computed>"
          timeout.#:            "0" => "0"
          type:                 "container" => "container"

Output from acceptance testing:

--- PASS: TestAccAWSBatchJobDefinition_basic (16.80s)
--- PASS: TestAccAWSBatchJobDefinition_ContainerProperties_Advanced (16.84s)
--- PASS: TestAccAWSBatchJobDefinition_updateForcesNewResource (27.13s)

…h container properties missing environment, mount point, ulimits, and volumes configuration

Reference: #11998
Reference: #11488

Now that this resource is properly refreshing the `container_properties` attribute into the Terraform state, additional cases have been reported where the API canonicalizes the response with empty arrays. Here we account for `environment`, `mountPoints`, `ulimits`, and `volumes`.

Previous output from unit testing (before code update):

```
2020/02/11 12:37:58 [DEBUG] Canonical Batch Container Properties JSON are not equal.
First: {"command":["start.py","Ref::S3bucket","Ref::S3key"],"image":"example:image","jobRoleArn":"arn:aws:iam::123456789012:role/example","memory":2048,"vcpus":8}
Second: {"command":["start.py","Ref::S3bucket","Ref::S3key"],"environment":[],"image":"example:image","jobRoleArn":"arn:aws:iam::123456789012:role/example","memory":2048,"mountPoints":[],"ulimits":[],"vcpus":8,"volumes":[]}
--- FAIL: TestEquivalentBatchContainerPropertiesJSON (0.00s)
    --- FAIL: TestEquivalentBatchContainerPropertiesJSON/empty_environment,_mountPoints,_ulimits,_and_volumes (0.00s)
        container_properties_test.go:226: got false, expected true
```

Previous output from acceptance testing (before code update):

```
--- FAIL: TestAccAWSBatchJobDefinition_basic (13.16s)
    testing.go:640: Step 0 error: After applying this step, the plan was not empty:

        DIFF:

        DESTROY/CREATE: aws_batch_job_definition.test
          arn:                  "arn:aws:batch:us-west-2:--OMITTED--:job-definition/tf-acc-test-118932874341187373:1" => "<computed>"
          container_properties: "{\"command\":[\"echo\",\"test\"],\"environment\":[],\"image\":\"busybox\",\"memory\":128,\"mountPoints\":[],\"resourceRequirements\":[],\"ulimits\":[],\"vcpus\":1,\"volumes\":[]}" => "{\"command\":[\"echo\",\"test\"],\"image\":\"busybox\",\"memory\":128,\"vcpus\":1}" (forces new resource)
          id:                   "arn:aws:batch:us-west-2:--OMITTED--:job-definition/tf-acc-test-118932874341187373:1" => "<computed>"
          name:                 "tf-acc-test-118932874341187373" => "tf-acc-test-118932874341187373"
          retry_strategy.#:     "0" => "0"
          revision:             "1" => "<computed>"
          timeout.#:            "0" => "0"
          type:                 "container" => "container"
```

Output from acceptance testing:

```
--- PASS: TestAccAWSBatchJobDefinition_basic (16.80s)
--- PASS: TestAccAWSBatchJobDefinition_ContainerProperties_Advanced (16.84s)
--- PASS: TestAccAWSBatchJobDefinition_updateForcesNewResource (27.13s)
```
@bflad bflad added bug Addresses a defect in current functionality. regression Pertains to a degraded workflow resulting from an upstream patch or internal enhancement. service/batch Issues and PRs that pertain to the batch service. labels Feb 11, 2020
@bflad bflad requested a review from a team February 11, 2020 18:04
@ghost ghost added needs-triage Waiting for first response or review from a maintainer. size/M Managed by automation to categorize the size of a PR. tests PRs: expanded test coverage. Issues: expanded coverage, enhancements to test infrastructure. labels Feb 11, 2020
@bflad bflad removed the needs-triage Waiting for first response or review from a maintainer. label Feb 11, 2020
@theophilechevalier
Copy link
Contributor

Following #11998, I compiled the provider against your branch and I can confirm that all our batch stacks are back to normal.
Thanks @bflad 👍

@@ -19,10 +19,26 @@ func (cp *containerProperties) Reduce() error {
return aws.StringValue(cp.Environment[i].Name) < aws.StringValue(cp.Environment[j].Name)
})

if len(cp.Environment) == 0 {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is minor, but can you comment why you have to nil this out? I'm assuming due to marshaling?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll put comments on each of these, thanks

Copy link
Contributor

@paultyng paultyng left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

left one minor note, but otherwise LGTM

Copy link
Contributor

@gdavison gdavison left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 🚀

--- PASS: TestAccAWSBatchJobDefinition_ContainerProperties_Advanced (8.68s)
--- PASS: TestAccAWSBatchJobDefinition_basic (8.70s)
--- PASS: TestAccAWSBatchJobDefinition_updateForcesNewResource (13.46s)

@bflad bflad added this to the v2.49.0 milestone Feb 12, 2020
@bflad bflad merged commit 94c291c into master Feb 12, 2020
@bflad bflad deleted the b-aws_batch_job_definition-additional-equivalency branch February 12, 2020 21:34
bflad added a commit that referenced this pull request Feb 12, 2020
@ghost
Copy link

ghost commented Feb 14, 2020

This has been released in version 2.49.0 of the Terraform AWS provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

For further feature requests or bug reports with this functionality, please create a new GitHub issue following the template for triage. Thanks!

@ghost
Copy link

ghost commented Mar 27, 2020

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thanks!

@ghost ghost locked and limited conversation to collaborators Mar 27, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
bug Addresses a defect in current functionality. regression Pertains to a degraded workflow resulting from an upstream patch or internal enhancement. service/batch Issues and PRs that pertain to the batch service. size/M Managed by automation to categorize the size of a PR. tests PRs: expanded test coverage. Issues: expanded coverage, enhancements to test infrastructure.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

r/aws_batch_job_definition must always be replaced in 2.48
4 participants