Multiple Deploy Keys in docker build fails

[iamnoah]

Not exactly a bug, but I found it challenging to find a solution that works with docker build when using multiple deploy keys. The reason seems to be that the build container doesn't have the ssh and git config necessary to map the right key to the right repo.

For example:

      - name: Setup SSH
        uses: webfactory/ssh-agent@v0.5.0
        with:
          ssh-private-key: |
            ${{ secrets.DEPLOY_KEY_A }}
            ${{ secrets.DEPLOY_KEY_B }}
      - name: Works Well!
        run: |
          git clone github.com/me/private-repo-a
          git clone github.com/me/private-repo-b
      - name: Doesn't work :(
        run: |
          cat > Dockerfile <<EOF
          FROM debian
          RUN --mount=type=ssh git clone github.com/me/private-repo-a
          RUN --mount=type=ssh git clone github.com/me/private-repo-b
          EOF
          docker build --ssh default .

The docker build has access to the keys, but it doesn't use the right one for each repo, so one of the checkouts will fail.

My solution was to copy the config into the container:

run: |
  mkdir root-config
  cp -r ~/.gitconfig  ~/.ssh root-config/
  docker build ... .

And in my Dockerfile:

COPY root-config /root/
RUN sed 's|/home/runner|/root|g' -i.bak /root/.ssh/config

That works, but it feels pretty hacky. I was just wondering if anyone can come up with a better way/wanted to document a way to make it work.

[mpdude]

I don’t see how this is related to the action…?

Could you explain please?

[iamnoah]

@mpdude sorry, I've edited to add some context. The action is incredibly useful.

[mpdude]

Ok, I think I now understand what this is about.

Not sure I can really help, but we can keep this open for some time to get some visibility.

Thoughts:

• Is it possible to mount the relevant directories/config files at docker build time instead of having to copy them?

• You could try if the SSH config file supports ~ for paths. Maybe that could make the sed command unnecessary (if we can adapt in this action).

Does that help?

[shyim]

You should try the RUN like so RUN --mount=type=ssh

[mpdude]

I guess the problem is not having the SSH auth socket itself available, but the necessary mapping in .gitconfig and .ssh/config to map keys to repos.

[iamnoah]

I think ~ in the SSH config should work, but I have not found anything else that simplifies the setup unfortunately. There is a 2 year old buildkit issue for mounting directories as secrets during a build, which could make it safer to do (the COPY I do in my build is only safe because it is multi-stage and the home directory doesn't get copied over.)

[mpdude]

Buildkit support for such things would of course be best.

I also would not want to have ~/.ssh in intermediate layers of my images, but at least there is no sensitive data in this file.

I am not sure about .gitconfig, though – it might be that actions/checkout leaves a secret token in that?

[aingham]

@mpdude I'm interested in following this, as it's exactly the same issue that I came across - you've summarised it really well.

Copying the git and ssh configs and keys into the container, as you suggested, did work for me, but the problem with that is that the private keys are then sitting in the Docker container. It would be better if it could make use of the keys that have been made available to it via ssh agent forwarding - I guess you haven't found a way of doing that?

[mpdude]

@aingham See the initial comment how the SSH Agent socket can be mounted for a RUN command at build time. You don’t have to (and almost never should) copy private SSH keys into a Docker image unless you absolutely understand the implications.

@iamnoah any idea how we could proceed here?

[iamnoah]

So this answer suggests that we could copy just the public keys into the docker container, and use them as the IdentityFile in .ssh/config to pick the correct key from the agent. So no private keys would end up in the container. I'll test this out when I can.

.gitconfig should not have anything sensitive in it.

[iamnoah]

I didn't realize the key-* files in ~/.ssh already were just the public keys. So I think that while copying the contents of ~/.ssh into a container is less than ideal, it is at least not disclosing anything meant to be secret and is the best we can do with Docker at the moment.

Thanks for a very helpful action.

[hardbyte]

Very interested if someone has a non-hacky solution to this. It seems the logic that ssh-agent goes through to configure the git aliases needs to occur inside a docker container. For what its worth I ended up copying as well:

# Copy SSH config and public key details into image
COPY .gitconfig* /etc/gitconfig
COPY ssh/config* /etc/ssh/ssh_config
COPY ssh/ /home/runner/.ssh
RUN mkdir -p -m 0600 ~/.ssh && ssh-keyscan github.com >> ~/.ssh/known_hosts

# Test access:
RUN --mount=type=ssh pip install git+ssh://git@github.com/me/my-private-repo-a.git
RUN --mount=type=ssh pip install git+ssh://git@github.com/me/my-private-repo-b.git

[mrajancsr]

this is constantly breaking for me. What am i doing wrong here?

syntax = docker/dockerfile:1.0-experimental

FROM python:3.9

create a folder and cd into it

run mkdir temp_repo
run cd temp_repo

set folder as current working directory

workdir /temp_repo

move hello_world.py script from our local system to current workdir in docker

add hello_world.py .

copy the requirements file from current system to docker directory

copy requirements.txt /temp_repo
run pip install -r requirements.txt

copy .gitconfig* /etc/gitconfig
copy ssh/config* /etc/ssh/ssh_config
copy ssh /home/runner/.ssh
RUN mkdir -p -m 0600 ~/.ssh && ssh-keyscan github.com >> ~/.ssh/known_hosts

RUN --mount=type=ssh pip install git+ssh://git@github.com/my_company/repo1.git
RUN --mount=type=ssh git clone git+ssh://git@github.com/my_company/repo1.git
RUN --mount=type=ssh pip install git+ssh://git@github.com/my_company/repo2.git

copy timescale.pem /root

run apt-get update -y

cmd ["python", "-u", "hello_world.py"]

Its constantly failing on the last step with repository not found error. can someone please help?

[nicolo-kira]

While this is a bit of an older issue, I seemed to have solved this with our build process by doing something like this

Posting here for any other who come across this

    # Setup SSH_AUTH_SOCK to pull from other git repos during the build, using separate sockets via ssh-auth-sock to be able to define them in the docker build section
    - name: Setup SSH_AUTH_SOCK Repo A
      uses: webfactory/ssh-agent@v0.5.4
      with:
        ssh-private-key: ${{ secrets.REPO_A_DEPLOY_KEY }}
        ssh-auth-sock: /tmp/repo-a-${{ github.sha }}.sock

    - name: Setup SSH_AUTH_SOCK Repo B
      uses: webfactory/ssh-agent@v0.5.4
      with:
        ssh-private-key: ${{ secrets.REPO_B_DEPLOY_KEY }}
        ssh-auth-sock: /tmp/repo-b-${{ github.sha }}.sock

    # Sets up buildx to enable docker buildkit features
    - name: Set up Docker Buildx
      id: buildx
      uses: docker/setup-buildx-action@v2
      with:
        install: true

    - name: Build and push
      uses: docker/build-push-action@v3
      with:
        push: true
        tags: myregistry/myrepo:latest
        file: Dockerfile
        ssh: |
          default=${{ env.SSH_AUTH_SOCK }}
          repoa=/tmp/repo-a-${{ github.sha }}.sock
          repob=/tmp/repo-b-${{ github.sha }}.sock

This sets up separate ssh mount IDs you can reference in your build file:

# Note: the id here matches what is defined in the docker build and push step above

# Repo A action
RUN --mount=type=ssh,id=repoa git clone git@github.com:org/repo-a.git .

# Repo B Action
RUN --mount=type=ssh,id=repob yarn install --frozen-lockfile

[j-riebe]

@mpdude Is it possible to add the solution to this problem (initial issue) to the docs into a new section docker-build-push-Action + Deploy Keys?
I'd offer to create a PR, if thats ok with you.

[asparagusbeef]

@nicolo-kira the solution you suggested is the neated IMO. Including the one in the documentation. Note you can also remove the default=${{ env.SSH_AUTH_SOCK }} line, which is neat because some linters false positively flag this line.

@mpdude Is it possible to add this solution to the documentation? Its safer and easier to implement. I am unsure if there are any downsides to this approach.