Error creating compute environment for aws batch

[must012]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Terraform CLI and Terraform AWS Provider Version

Terraform v1.2.4
on linux_amd64
+ provider registry.terraform.io/hashicorp/aws v4.22.0

Affected Resource(s)

• aws_batch_compute_environment

Terraform Configuration Files

Please include all Terraform configurations required to reproduce the bug. Bug reports without a functional reproduction may be closed without investigation.

resource "aws_batch_compute_environment" "foo_environment" {
  compute_environment_name = "foo_compute_environment"

  compute_resources {
    allocation_strategy = "BEST_FIT"
    instance_role       = aws_iam_instance_profile.foo_profile.arn

    instance_type = [
      "optimal",
    ]
    
    launch_template {
      launch_template_id = aws_launch_template.foo_launch_template.id
      version            = aws_launch_template.foo_launch_template.latest_version
    }
    
    max_vcpus = 3
    min_vcpus = 0
    
    security_group_ids = [
      aws_security_group.foo_sg.id,
    ]

    subnets = [
      aws_subnet.tmp_subnet_1.id,
      aws_subnet.tmp_subnet_2.id,
    ]
    
    type = "EC2"
  }

  service_role = aws_iam_role.bar_role.arn
  state        = "ENABLED"
  type         = "MANAGED"
}

resource "aws_launch_template" "foo_launch_template" {
  name = "foo_launch_template"

  network_interfaces {
    associate_public_ip_address = true
    device_index = 0
    
    security_groups = [
    aws_security_group.foo_sg.id,
    ]
  }
}

Debug Output

https://gist.github.com/must012/a33cae57f0c4c89261a54995dc61f852

Expected Behavior

Creating a Resource(aws_batch_compute_environment)

As far as we tried, Terraform-aws requires Security Group to deploy resources of AWS Batch absolutely.
And Terraform-aws rules that we can't specify Security Group and Launch template. They are exclusively.
We'd like to deploy resources of AWS Batch only with Launch template (without Security group).
can create this option by setting the same options in the AWS console.

Actual Behavior

Error: error creating Batch Compute Environment (foo_compute_environment): : Error executing request, Exception : Either compute environment Security Groups or Network Interfaces in Launch template are exclusively allowed, RequestId: 20e65275-5945-4489-ad95-3a87ed61d2b9
│       status code: 400, request id: 20e65275-5945-4489-ad95-3a87ed61d2b9
│ 
│   with aws_batch_compute_environment.foo_environment,
│   on batch.tf line 1, in resource "aws_batch_compute_environment" "foo_environment":
│    1: resource "aws_batch_compute_environment" "foo_environment" {

However, resource creation failed with the above error.

Steps to Reproduce

1. terraform apply

References

• terraform document(compute environment)
  -- https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/batch_compute_environment#security_group_ids
• cloudformation documentation(compute environment)
  -- https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-batch-computeenvironment-computeresources.html#cfn-batch-computeenvironment-computeresources-securitygroupids

We think that this situation is not appropriate because we can deploy the resources only with Launch template using CloudFormation template. Is this correct?
Also we'd like to know if there is any way to avoid this situation?
In addition, is there any reason why security group is required?

[justinretzolk]

Hey @must012 👋 Thank you for taking the time to raise this. I took a quick look and this looks like a bug, where compute_resources.security_group_ids should indeed be optional, so long as a security group is specified in the launch template. I'm going to mark this as a bug so that one of our engineers or a community member can take a look at this when time allows.

For whoever picks this up; the information I found:

• The aws_batch_compute_environment resource uses batch.CreateComputeEnvironment to create these resources.

• The batch.CreateComputeEnvironmentInput type has a ComputeResources field, of type batch.ComputeResource that corresponds to the compute_resources argument for the resource.

• The batch.ComputeResource type has a field SecurityGroupIds (which corresponds to the security_group_ids argument), which has the following description:

  // The Amazon EC2 security groups associated with instances launched in the
  // compute environment. One or more security groups must be specified, either
  // in securityGroupIds or using a launch template referenced in launchTemplate.
  // This parameter is required for jobs that are running on Fargate resources
  // and must contain at least one security group. Fargate doesn't support launch
  // templates. If security groups are specified using both securityGroupIds and
  // launchTemplate, the values in securityGroupIds are used.

• Based on this, it seems that compute_resources.security_group_ids should be optional, with documentation stating that it is required when compute_resources.type is FARGATE.

[bigbitbus]

I also ran into this bug, waiting on the fix to be merged and released.

[Jacobsolawetz]

Also looking forward to merge!

[github-actions]

This functionality has been released in v4.38.0 of the Terraform AWS Provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

For further feature requests or bug reports with this functionality, please create a new GitHub issue following the template. Thank you!

[github-actions]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.