[ECS, Fargate] Support awslogs-endpoint task configuration option

[copumpkin]

Which service(s) is this request for?

Fargate, ECS

Tell us about the problem you're trying to solve. What are you trying to do, and why is it hard?

After @clareliguori helpfully pointed out that my issue #48 was mistaken and the issue was with the awslogs driver, I looked into the CloudWatch Logs VPCE support. Currently the only way to make it work is to update the VPC's DNS to point at the PrivateLink endpoint, which while fine for some situations, can be a bit coarse-grained for others (especially if there are nontrivial policies on the endpoint). As of a recent Docker (this commit adds it), the awslogs driver now supports the awslogs-endpoint configuration option, which would allow me to point my task directly at the logs VPCE that's appropriate for it. If I try to pass that into my task definition today, it tells me immediately that the option is invalid.

Are you currently working around this issue?

Just using DNS for the VPCE.

Edit: a broader way to phrase this might have been "Support Docker 18.09", as long as the front-end validation for the task definition schema notices that the new option is present 😄

[mailjunze]

By default "Private DNS" is not enabled for Cloudwatch endpoint, In order to use Cloudwatch with fargate You would need to "Enable Private DNS Name" for com.amazonaws.eu-west-1.logs. Go to the Cloudwatch Endpoint in VPC console > Actions > Modify Private DNS name. Also, make sure you're using the latest Fargate platform Version.

[pjelar]

Any update on this? I have a vpc endpoint with private DNS enable but I'm still unable to get logs.

[clareliguori]

The latest ECS AMIs now include Docker 18.09.9-ce:
https://docs.aws.amazon.com/AmazonECS/latest/developerguide/ecs-ami-versions.html

[clareliguori]

Re-opening, my mistake: the task definition log driver options are validated in ECS, and do not yet allow awslogs-endpoint