Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

"aws: command not found" in local-exec after installing AWS CLI in Atlantis image #1605

Closed
camados opened this issue May 27, 2021 · 13 comments
Closed

Comments

@camados
Copy link

camados commented May 27, 2021

Hi, I'm using a Bridgecrew module that, on apply, executes aws sns commands in a local-exec. The aws cli was not installed into the base Atlantis image, so I went ahead and rebuilt my Atlantis image to include the AWS CLI, as well as updated the PATH and symlinked the aws executable with /bin/aws which local-exec uses. I'm still getting the "local-exec: /bin/aws not found" error. Can you help?

@camados camados changed the title Unable to use AWS CLI commands in local-exec Unable to use AWS CLI commands in local-exec after installing AWS CLI in Atlantis image May 27, 2021
@camados camados changed the title Unable to use AWS CLI commands in local-exec after installing AWS CLI in Atlantis image "aws: command not found" in local-exec after installing AWS CLI in Atlantis image May 27, 2021
@micborens
Copy link

micborens commented Jun 14, 2021

I am facing a pretty much similar problem as you but using the Helm and Kubernetes provider.

Error from atlantis plan

Error: Get "https://xxxx.gr7.eu-west-1.eks.amazonaws.com/apis/rbac.authorization.k8s.io/v1/namespaces/grafana/rolebindings/delete-pod-to-platform": getting credentials: exec: executable aws not found

It looks like you are trying to use a client-go credential plugin that is not installed.

To learn more about this feature, consult the documentation available at:
      https://kubernetes.io/docs/reference/access-authn-authz/authentication/#client-go-credential-plugins

my provider

  kubernetes {
    host                   = module.finder.this.endpoint
    cluster_ca_certificate = base64decode(module.finder.this.certificate_authority[0].data)
    exec {
      api_version = "client.authentication.k8s.io/v1alpha1"
      args        = ["eks", "get-token", "--cluster-name", module.finder.this.id, "--role-arn", module.accounts.tf_role]
      command     = "aws"
    }
  }
}

terraform registry doc:

https://registry.terraform.io/providers/hashicorp/kubernetes/latest/docs#exec-plugins

I work fine on my local because I have awscli installed.

@cilindrox
Copy link
Contributor

The following Dockerfile works and has the aws-cli available on $PATH for any commands/local-exec:

FROM runatlantis/atlantis:v0.17.1@sha256:74a1bfd6c863ef194a3bf15a4a1991a4cdc50375110a294599d7e02200a9dbb0

RUN apk --no-cache add aws-cli

@micborens
Copy link

Hello aws-cli is not the latest version.
from an alpine with awscli installed from apk cmd:

bash-5.1# aws --version
aws-cli/1.18.177 Python/3.8.10 Linux/5.10.25-linuxkit botocore/1.19.47

from an alpine with a manual installation:

bash-5.1# aws --version
aws-cli/2.2.27 Python/3.8.8 Linux/5.10.25-linuxkit exe/x86_64.alpine.3 prompt/off
bash-5.1#

I suggest you to install awscli using the second way (munual)

FROM runatlantis/atlantis:latest

# https://github.com/aws/aws-cli/issues/4685
ENV GLIBC_VER=2.33-r0

RUN curl -sL https://alpine-pkgs.sgerrand.com/sgerrand.rsa.pub -o /etc/apk/keys/sgerrand.rsa.pub \
    && curl -sLO https://github.com/sgerrand/alpine-pkg-glibc/releases/download/${GLIBC_VER}/glibc-${GLIBC_VER}.apk \
    && curl -sLO https://github.com/sgerrand/alpine-pkg-glibc/releases/download/${GLIBC_VER}/glibc-bin-${GLIBC_VER}.apk \
    && curl -sL https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip -o awscliv2.zip \
    && apk add --update --no-cache util-linux openssl python3 glibc-${GLIBC_VER}.apk glibc-bin-${GLIBC_VER}.apk \
    && unzip awscliv2.zip && aws/install \
    && rm -rf awscliv2.zip aws glibc-${GLIBC_VER}.apk glibc-bin-${GLIBC_VER}.apk \
        /usr/local/aws-cli/v2/*/dist/aws_completer \
        /usr/local/aws-cli/v2/*/dist/awscli/data/ac.index \
        /usr/local/aws-cli/v2/*/dist/awscli/examples \
        /var/cache/apk/*

@jseiser
Copy link

jseiser commented Jun 29, 2022

Is it not currently possible to run Atlantis in a dockerfile, while being able to utilize the aws cli for things like kubernetes provider?

Anything I try from the link above, fail to install with

ERROR: glibc-2.35-r0: trying to overwrite lib/ld-linux-x86-64.so.2 owned by gcompat-1.0.0-r4.

@ganeshk1928
Copy link

ganeshk1928 commented Jul 18, 2022

I’m also getting similar output

---> Running in 2eef0326330d
fetch https://dl-cdn.alpinelinux.org/alpine/v3.15/main/x86_64/APKINDEX.tar.gz
fetch https://dl-cdn.alpinelinux.org/alpine/v3.15/community/x86_64/APKINDEX.tar.gz
(1/2) Installing glibc (2.34-r0)
ERROR: glibc-2.34-r0: trying to overwrite lib/ld-linux-x86-64.so.2 owned by gcompat-1.0.0-r4.
ERROR: glibc-2.34-r0: trying to overwrite lib64/ld-linux-x86-64.so.2 owned by gcompat-1.0.0-r4.
(2/2) Installing glibc-bin (2.34-r0)
Executing glibc-bin-2.34-r0.trigger
1 error; 49 MiB in 48 packages
The command '/bin/sh -c apk add --no-cache glibc-${GLIBC_VER}.apk glibc-bin-${GLIBC_VER}.apk' returned a non-zero code: 1```

@ganeshk1928
Copy link

@jseiser any workaround ?

@cilindrox
Copy link
Contributor

cilindrox commented Jul 18, 2022

@ganeshk1928 see this SO answer.

ARG ALPINE_VERSION=3.15.4

FROM alpine:${ALPINE_VERSION}

ARG GLIBC_VERSION=2.34-r0
ARG AWSCLI_VERSION=2.6.1

# install glibc compatibility for alpine
RUN apk --no-cache add \
        binutils \
        curl \
    && curl -sL https://alpine-pkgs.sgerrand.com/sgerrand.rsa.pub -o /etc/apk/keys/sgerrand.rsa.pub \
    && curl -sLO https://github.com/sgerrand/alpine-pkg-glibc/releases/download/${GLIBC_VERSION}/glibc-${GLIBC_VERSION}.apk \
    && curl -sLO https://github.com/sgerrand/alpine-pkg-glibc/releases/download/${GLIBC_VERSION}/glibc-bin-${GLIBC_VERSION}.apk \
    && curl -sLO https://github.com/sgerrand/alpine-pkg-glibc/releases/download/${GLIBC_VERSION}/glibc-i18n-${GLIBC_VERSION}.apk \
    && apk add --no-cache \
        glibc-${GLIBC_VERSION}.apk \
        glibc-bin-${GLIBC_VERSION}.apk \
        glibc-i18n-${GLIBC_VERSION}.apk \
    && /usr/glibc-compat/bin/localedef -i en_US -f UTF-8 en_US.UTF-8 \
    && curl -sL https://awscli.amazonaws.com/awscli-exe-linux-x86_64-${AWSCLI_VERSION}.zip -o awscliv2.zip \
    && unzip awscliv2.zip \
    && aws/install \
    && rm -rf \
        awscliv2.zip \
        aws \
        /usr/local/aws-cli/v2/current/dist/aws_completer \
        /usr/local/aws-cli/v2/current/dist/awscli/data/ac.index \
        /usr/local/aws-cli/v2/current/dist/awscli/examples \
        glibc-*.apk \
    && find /usr/local/aws-cli/v2/current/dist/awscli/botocore/data -name examples-1.json -delete \
    && apk --no-cache del \
        binutils \
        curl \
    && rm -rf /var/cache/apk/*

looks like newer versions of the aws-cli break this glibc shim, but this old version seems enough for configuring the kubectl and helm provider using the new client.authentication.k8s.io/v1beta1 api.

You'll just have to extend FROM the current atlantis image and add the RUN commands from above, or use an init container for Atlantis, etc, and that'll get you a 2.x version of the aws-cli binary.

@ganeshk1928
Copy link

ganeshk1928 commented Jul 19, 2022

@cilindrox Can you share the full dockerfile with extended FROM if possible 😊 I'm unable to do that step 😔

Or

You can edit my dockerfile

# BASE IMAGE
FROM ghcr.io/runatlantis/atlantis:v0.19.6

# COPY ID_RSA KEY TO IMAGE
COPY id_rsa /home/atlantis/.ssh/id_rsa
RUN chmod 400 /home/atlantis/.ssh/id_rsa
RUN ssh-keyscan github.com > /home/atlantis/.ssh/known_hosts 
RUN chown -R atlantis:atlantis /home/atlantis/.ssh/

#INSTALLING JQ, CURL AND AWS CLI
ENV GLIBC_VER=2.34-r0
RUN apk --no-cache add binutils curl jq sudo
RUN curl -sL https://alpine-pkgs.sgerrand.com/sgerrand.rsa.pub -o /etc/apk/keys/sgerrand.rsa.pub
RUN curl -sLO https://github.com/sgerrand/alpine-pkg-glibc/releases/download/${GLIBC_VER}/glibc-${GLIBC_VER}.apk
RUN curl -sLO https://github.com/sgerrand/alpine-pkg-glibc/releases/download/${GLIBC_VER}/glibc-bin-${GLIBC_VER}.apk
RUN apk add --no-cache glibc-${GLIBC_VER}.apk glibc-bin-${GLIBC_VER}.apk
RUN curl -sL https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip -o awscliv2.zip && unzip awscliv2.zip && aws/install && rm -rf awscliv2.zip aws /usr/local/aws-cli/v2/*/dist/aws_completer /usr/local/aws-cli/v2/*/dist/awscli/data/ac.index /usr/local/aws-cli/v2/*/dist/awscli/examples
RUN rm glibc-${GLIBC_VER}.apk && rm glibc-bin-${GLIBC_VER}.apk
RUN rm -rf /var/cache/apk/*

#ADD ATLANTIS USER TO SUDO GROUP AND PROVIDE SUDO PREVILIGES
RUN echo '%wheel ALL=(ALL) ALL' > /etc/sudoers.d/wheel
RUN adduser atlantis wheel
RUN echo "atlantis ALL=(ALL) NOPASSWD: ALL" >> /etc/sudoers

USER atlantis

@cilindrox
Copy link
Contributor

cilindrox commented Jul 19, 2022

Sure, here's an example extending from the atlantis base:

FROM ghcr.io/runatlantis/atlantis:v0.19.6@sha256:2d1efea6ae65b075a29df211809f27d8239e77e75a6fa6222778cf04f4119aef

ARG GLIBC_VERSION=2.34-r0
ARG AWSCLI_VERSION=2.6.1

# gcompat (required and installed by atlantis) triggers a conflict when
# installing the glibc shim.
RUN apk --no-cache add \
		binutils \
	&& curl -sL https://alpine-pkgs.sgerrand.com/sgerrand.rsa.pub -o /etc/apk/keys/sgerrand.rsa.pub \
	&& curl -sLO https://github.com/sgerrand/alpine-pkg-glibc/releases/download/${GLIBC_VERSION}/glibc-${GLIBC_VERSION}.apk \
	&& curl -sLO https://github.com/sgerrand/alpine-pkg-glibc/releases/download/${GLIBC_VERSION}/glibc-bin-${GLIBC_VERSION}.apk \
	&& curl -sLO https://github.com/sgerrand/alpine-pkg-glibc/releases/download/${GLIBC_VERSION}/glibc-i18n-${GLIBC_VERSION}.apk \
	&& apk del gcompat \
	&& apk add --no-cache \
			glibc-${GLIBC_VERSION}.apk \
			glibc-bin-${GLIBC_VERSION}.apk \
			glibc-i18n-${GLIBC_VERSION}.apk \
	&& /usr/glibc-compat/bin/localedef -i en_US -f UTF-8 en_US.UTF-8 \
	&& curl -sL https://awscli.amazonaws.com/awscli-exe-linux-x86_64-${AWSCLI_VERSION}.zip -o awscliv2.zip \
	&& unzip awscliv2.zip \
	&& aws/install \
	&& rm -rf \
			awscliv2.zip \
			aws \
			/usr/local/aws-cli/v2/current/dist/aws_completer \
			/usr/local/aws-cli/v2/current/dist/awscli/data/ac.index \
			/usr/local/aws-cli/v2/current/dist/awscli/examples \
			glibc-*.apk \
	&& find /usr/local/aws-cli/v2/current/dist/awscli/botocore/data -name examples-1.json -delete \
	&& apk --no-cache del \
			binutils \
	&& rm -rf /var/cache/apk/*

Keep in mind I'm kinda playing with fire here, but it looks like gcompat was added for convenience on #2175 and it's not mission-critical. However, you might run into issues with some providers or custom workflows that depend on other binaries that aren't compiled against musl.

@ganeshk1928
Copy link

Thank you very much @cilindrox

@nitrocode
Copy link
Member

If you're using arm osx m1, then you need to run the docker build with --platform linux/arm64 or it will build but then aws --version will not work

This works

$ docker build --platform linux/amd64 -t atlantis-test .
$ docker run -it atlantis-test aws --version
WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested
aws-cli/2.7.19 Python/3.9.11 Linux/5.10.104-linuxkit exe/x86_64.alpine.3 prompt/off

This fails

$ docker build -t atlantis-test-arm .
$ docker run -it atlantis-test-arm aws --version
aws: error while loading shared libraries: libz.so.1: cannot open shared object file: No such file or directory

@gaspo53
Copy link

gaspo53 commented Mar 30, 2023

For those wanting aws-cli: https://hub.docker.com/r/chatwork/atlantis-aws

@nitrocode
Copy link
Member

nitrocode commented Mar 31, 2023

I wouldn't use a docker image you find on dockerhub unless it's official. It's highly encouraged to create a private Atlantis image for your organization so you can customize the necessary binaries that you workflows need.

It might be as simple as this

FROM ghcr.io/runatlantis/atlantis:v0.23.4

RUN apk add aws-cli

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

8 participants