ECS Fargate volumesFrom container definition not working

[chris-edwards-pub]

Description

I have an ECS Fargate container definition that specifies volumesFrom. This value is not passed to AWS.

Snippet of terraform code:

        coredns-zone-sync = {
          cpu       = 256
          memory    = 1024
          essential = true
          image     = var.coredns_zone_sync_image

          volumesFrom = var.container_name
....

Terraform apply:

+ image                  = "xxxxx/coredns-zone-sync:1.2.0-SNAPSHOT.1712600310047"
                  + interactive            = false
                  + linuxParameters        = {
                      + initProcessEnabled = true
                    }
                  + logConfiguration       = {
                      + logDriver     = "awsfirelens"
                      + options       = {
                          + Name           = "datadog"
                          + TLS            = "on"
                          + dd_message_key = "log"
                          + dd_service     = "coredns-zone-sync"
                          + dd_source      = "coredns"
                          + dd_tags        = "xxx"
                          + provider       = "ecs"
                        }
                      + secretOptions = [
                          + {
                              + name      = "apiKey"
                              + valueFrom = "xxxi"
                            },
                        ]
                    }
                  + memory                 = 1024
                  + memoryReservation      = 100
                  + mountPoints            = []
                  + name                   = "coredns-zone-sync"
                  + portMappings           = []
                  + privileged             = false
                  + pseudoTerminal         = false
                  + readonlyRootFilesystem = false
                  + startTimeout           = 30
                  + stopTimeout            = 120
                  + systemControls         = []
                  + user                   = "0"
                  + volumesFrom            = []
                },

AWS json

        {
            "name": "coredns-zone-sync",
            "image": "xxxxx/coredns-zone-sync:1.2.0-SNAPSHOT.1712600310047",
            "cpu": 256,
            "memory": 1024,
            "memoryReservation": 100,
            "portMappings": [],
            "essential": true,
            "environment": [
                {
                    "name": "coredns_rds_mysql_password",
                    "value": "tmCFKu-vCv2nC"
                },
                {
                    "name": "coredns_rds_mysql_username",
                    "value": "root"
                },
                {
                    "name": "coredns_rds_mysql_server",
                    "value": "coredns-us-east-1.cnw50ppegiqf.us-east-1.rds.amazonaws.com"
                }
            ],
            "mountPoints": [],
            "volumesFrom": [],

• ✋ I have searched the open/closed issues and my issue is not listed.

⚠️ Note

Before you submit an issue, please perform the following first:

1. Remove the local .terraform directory (! ONLY if state is stored remotely, which hopefully you are following that best practice!): rm -rf .terraform/
2. Re-initialize the project root to pull down modules: terraform init
3. Re-attempt your terraform plan or apply and check if the issue still persists

Versions

• Module version 5.11.0:

• Terraform version:
  Terraform v1.7.5
  on linux_amd64

• Provider version(s):
  Terraform v1.7.5
  on linux_amd64

Reproduction Code [Required]

See snippet above

Expected behavior

I expect the volumesFrom to appear on the task definition in AWS

Actual behavior

The volumesFrom value doesnt get set in the container definition on aws.

Terminal Output Screenshot(s)

Additional context

[bryantbiggs]

we will need a reproduction in order to help troubleshoot or fix

[chris-edwards-pub]

Sure.. You'll have to supply your own subnet....

terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = ">= 4.66.1"
    }
  }
}

module "ecs_cluster" {
  source  = "terraform-aws-modules/ecs/aws"
  version = "5.11.0"

  cluster_name = "ecs-fargate-cluster-test"

  # Capacity provider
  fargate_capacity_providers = {
    FARGATE = {
      default_capacity_provider_strategy = {
        weight = 100
        base   = 1
      }
    }
  }

  services = {
    nginx-service = {
      cpu    = 512
      memory = 1024

      desired_count = 1

      enable_execute_command = true

      # Container definition(s)
      container_definitions = {

        nginx-1 = {
          cpu       = 256
          memory    = 512
          essential = true
          image     = "nginx:latest"

          readonly_root_filesystem = false

          memory_reservation = 100
        }

        nginx-2 = {
          cpu       = 256
          memory    = 512
          essential = true
          image     = "nginx:latest"

          volumesFrom = ["nginx-1"]

          readonly_root_filesystem = false

          memory_reservation = 100
        }

      }
      subnet_ids = ["subnet-XXXXXX"]
    }
  }
}

[bryantbiggs]

don't you need more details than what you are providing in your config per the API spec https://docs.aws.amazon.com/AmazonECS/latest/APIReference/API_VolumeFrom.html ?

[chris-edwards-pub]

If I go in manually to the task and add the volume like this… [image](https://github.com/terraform-aws-modules/terraform-aws-ecs/assets/6855954/1125d926-f52a-4a3e-89a5-e57a86025b3c) The JSON now shows the proper code… [image](https://github.com/terraform-aws-modules/terraform-aws-ecs/assets/6855954/e7d6ba1b-924c-4f3c-b43e-ba46a2e7791e) Anything I send in “volumesFrom” in the task definition is not sent. Chris

[bryantbiggs]

why are you specifiying this

volumesFrom = ["nginx-1"]

When your screen show shows it should look like this

volumesFrom = [{
  sourceContainer = "nginx-1"
  readOnly = false
}]

[chris-edwards-pub]

I tried both and neither worked, i was just guessing at this point.

[chris-edwards-pub]

I added the code exactly like you stated above and this was the terraform apply output and volumesFrom = []

+ resource "aws_ecs_task_definition" "this" {
      + arn                      = (known after apply)
      + arn_without_revision     = (known after apply)
      + container_definitions    = jsonencode(
            [
              + {
                  + cpu                    = 256
                  + environment            = []
                  + essential              = true
                  + image                  = "nginx:latest"
                  + interactive            = false
                  + linuxParameters        = {
                      + initProcessEnabled = true
                    }
                  + logConfiguration       = {
                      + logDriver = "awslogs"
                      + options   = {
                          + awslogs-group         = "/aws/ecs/nginx-service/nginx-1"
                          + awslogs-region        = "us-east-1"
                          + awslogs-stream-prefix = "ecs"
                        }
                    }
                  + memory                 = 512
                  + memoryReservation      = 100
                  + mountPoints            = []
                  + name                   = "nginx-1"
                  + portMappings           = []
                  + privileged             = false
                  + pseudoTerminal         = false
                  + readonlyRootFilesystem = false
                  + startTimeout           = 30
                  + stopTimeout            = 120
                  + systemControls         = []
                  + user                   = "0"
                  + volumesFrom            = []
                },
              + {
                  + cpu                    = 256
                  + environment            = []
                  + essential              = true
                  + image                  = "nginx:latest"
                  + interactive            = false
                  + linuxParameters        = {
                      + initProcessEnabled = true
                    }
                  + logConfiguration       = {
                      + logDriver = "awslogs"
                      + options   = {
                          + awslogs-group         = "/aws/ecs/nginx-service/nginx-2"
                          + awslogs-region        = "us-east-1"
                          + awslogs-stream-prefix = "ecs"
                        }
                    }
                  + memory                 = 512
                  + memoryReservation      = 100
                  + mountPoints            = []
                  + name                   = "nginx-2"
                  + portMappings           = []
                  + privileged             = false
                  + pseudoTerminal         = false
                  + readonlyRootFilesystem = false
                  + startTimeout           = 30
                  + stopTimeout            = 120
                  + systemControls         = []
                  + user                   = "0"
                  + volumesFrom            = []
                },
            ]
        )

[bryantbiggs]

this is the correct syntax for your example:

volumes_from = [{
  sourceContainer = "nginx-1"
  readOnly = false
}]

[chris-edwards-pub]

I swear I read the documentation here. I've spent two days on this. Your my hero today. Im sorry to have wasted your time.

Thanks,

Chris

[bryantbiggs]

you're not wasting anyones time 😅 - which is why I added it to the examples. Its not great, but its something. This is an unfortunate side effect of not having the container definition fully in Terraform hashicorp/terraform-provider-aws#17988

The container definition is just a json blob sent to ECS, and thats why you see the camel casing. But Terraform uses the snake case form, and without the container definition fully defined in Terraform natively, we have that weird edge case where the two casings meet and its a confusing pain point for a lot of folks unfortunately

[chris-edwards-pub]

Keep up the excellent work! :)

I'll remember the snake case form from now on.

[antonbabenko]

This issue has been resolved in version 5.11.1 🎉

[github-actions]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues. If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.