Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Docker push failing from ECS Fargate with proper Task roles #1233

Closed
yadavnikhil opened this issue Jun 28, 2019 · 3 comments · Fixed by #1235
Closed

Docker push failing from ECS Fargate with proper Task roles #1233

yadavnikhil opened this issue Jun 28, 2019 · 3 comments · Fixed by #1235

Comments

@yadavnikhil
Copy link
Contributor

Description

AuthConfig: no credentials found when running ECS container with Fargate type deployment.

Issue:
[DEBUG] DOCKER> No user and password set for ECR, checking EC2 instance role
[WARNING] DOCKER> Error while retrieving EC2 instance credentials: Connect to 169.254.169.254:80 [/169.254.169.254] failed: Invalid argument (connect failed)
[DEBUG] DOCKER> AuthConfig: no credentials found

Issue is when running ECS in Fargate, instance meta-data endpoint (169.254.169.254) is not accessible from within the container.
AWS has different metadata endpoint for containers running in ECS: https://docs.aws.amazon.com/AmazonECS/latest/userguide/task-metadata-endpoint-fargate.html

Endpoint to get the credentials for TaskRoleARN assigned to ECS task will be:
Ex. http://169.254.170.2$AWS_CONTAINER_CREDENTIALS_RELATIVE_URI

Existing plugin only checks instance endpoint & fails to get credentials when run in an ECS container for getting proper credentials.
Can this support be added so it can check additional URL when getting credentials when it is not able to connect to 169.254.169.254?

Info

  • d-m-p version : 0.30.0
  • Maven version (mvn -v) : 3.6.0

  • Docker version : ECS Fargate

  • If it's a bug, how to reproduce : run docker:push to ECR registry from ECS container running with Fargate and has Task IAM Role assigned with ECR access.

  • If it's a feature request, what is your use case : Able to authenticate to ECR from ECS container running in Fargate from Task IAM Role.

  • Sample project : [GitHub Clone URL]

@sebastiankirsch
Copy link
Contributor

sebastiankirsch commented Nov 6, 2019

@yadavnikhil is this working for you?
We're using ECS - not Fargate - but the mechanics are the same: https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task-metadata-endpoint-v2.html
The correct URI to fetch would be http://169.254.170.2$AWS_CONTAINER_CREDENTIALS_RELATIVE_URI - as you have documented above. Unfortunately, #1235 fetches http://169.254.170.2/$AWS_CONTAINER_CREDENTIALS_RELATIVE_URI (note the '/' after the IP) - which results in a 404 in our case.

Also, I'm not sure if the URI escaping in https://github.com/fabric8io/docker-maven-plugin/blob/master/src/main/java/io/fabric8/maven/docker/util/AuthConfigFactory.java#L347 is necessary?

Anyway, since this is a "relatively recent" addition I wonder if this is working for you?
And if it would be possible to remove the '/' in the code or if this would break your scenario...

@yadavnikhil
Copy link
Contributor Author

@sebastiankirsch Yes, this changes are working fine for us.
The idea is if you use fargate then only the above IP is valid. But when with ECS - no fargate, then you need to provide credentials is one of old ways, like docker creds etc, OR simply if you want credentials at runtime, you can assign the instance profile with ECR permissions to your ECS nodes & plugin will fetch the credentials for you. We using instance profile on ECS nodes for non-fargate type deployments.

@sebastiankirsch
Copy link
Contributor

@yadavnikhil
Like I referenced in the previous comment, ECS also supports the mentioned endpoint; it "just" struggles with the two back slashes after the host name.
But if it works with Fargate like that, either the $AWS_CONTAINER_CREDENTIALS_RELATIVE_URI variable doesn't contain a leading / or the metaservice in Fargate is more forgiving than the one of ECS.

I'll create a PR that handles both cases then.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
2 participants