Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Docker SSH Forward not working #141

Closed
doctenahasib opened this issue Oct 21, 2022 · 21 comments
Closed

Docker SSH Forward not working #141

doctenahasib opened this issue Oct 21, 2022 · 21 comments

Comments

@doctenahasib
Copy link

doctenahasib commented Oct 21, 2022

runs-on: ubuntu-22.04
env:
  ACTIONS_ALLOW_UNSECURE_COMMANDS: true
steps:
  - uses: actions/checkout@v2
  - uses: webfactory/ssh-agent@v0.7.0
    with:
      ssh-private-key: ${{ secrets.private-key }}
  - name: Build docker image
     run: docker compose build --ssh default

Hello I try to forward the SSH Key (multiple deploy keys ${{ secrets.private-key }}) to my docker compose build command where inside it does a PHP composer install and fetches private github repositories but without success. It says:

Cloning into bare repository '/root/.cache/composer/vcs/git-github.com-.....
ERROR: Repository not found.
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.

Does anyone face the same issue ? I tried from version 0.5.4 and up.
Btw it works locally on my mac.

@FSMaxB
Copy link

FSMaxB commented Oct 21, 2022

I had to enable buildkit with the DOCKER_BUILDKIT=1 environment variable. That was for pure docker without docker-compose though, so not sure if it applies.

@doctenahasib
Copy link
Author

doctenahasib commented Oct 21, 2022

@FSMaxB Does not work either with: DOCKER_BUILDKIT=1 docker build --ssh default . I get the same error.

@mpdude
Copy link
Member

mpdude commented Oct 24, 2022

Are you trying to use GitHub Deploy Keys during the Docker build process?

If so, does #78 or the recently merged #133 help?

If yes, any suggestions what should be added to the README?

@doctenahasib
Copy link
Author

@mpdude Yes I use Github Deploy Keys but the version 0.7.0 did not fix the problem on my side.

@mpdude
Copy link
Member

mpdude commented Oct 24, 2022

It all depends on how you mount the SSH socket into the Docker Build process and/or copy necessary config files into the images. I am afraid that this is nothing this action could do for you.

But the linked issues maybe contain configuration examples that show how you can achieve this?

@d3li0n
Copy link

d3li0n commented Oct 27, 2022

Not sure if it might help, but does your Dockerfile has these lines?

RUN mkdir ~/.ssh
RUN ssh-keyscan -t rsa github.com >> ~/.ssh/known_hosts
RUN --mount=type=ssh <command to install dependencies>

@doctenahasib
Copy link
Author

doctenahasib commented Oct 27, 2022

@mpdude This is how I mount the ssh into the docker build process and this is working fine on my local machine using mac os 12.6 but not in the CI using multiple deploy keys. When running composer install outside the docker container so basically in the CI it also works but that's not what I want.

@d3li0n Yes I have it in my Dockerfile which btw is being used as the base image.

ONBUILD RUN mkdir -p -m 0600 ~/.ssh && ssh-keyscan github.com >> ~/.ssh/known_hosts
ONBUILD RUN --mount=type=ssh composer install

@mpdude
Copy link
Member

mpdude commented Oct 27, 2022

you will need to copy SSH and Git config files into the build stage to make deployment key mappings work. Please see the README.

However I just notice that the section mentions a specific Docker build action, which you don't use. The problem/solution probably is the same.

@mpdude
Copy link
Member

mpdude commented Oct 27, 2022

@j-riebe Since you wrote #133 and have a good understanding of the problem:

Do you think we could write the README section in a more general way (i.e. for Docker, not a specific action supporting it)?

@mpdude
Copy link
Member

mpdude commented Oct 27, 2022

As a side note, personally I'd avoid using ssh-keyscan regularly, especially in automated builds. It blindly accepts whatever keys it is being presented at that moment, defeating the whole purpose of key-based host authentication.

Grab and verify the GH host keys once from their website/blog, and directly put it into your Dockerfile. The keys hardly ever change.

@doctenahasib
Copy link
Author

@mpdude @j-riebe What about dev environment. Since I am using the same Dockerfile for dev, ci and production servers, this forces me to create an empty root-config folder in the repository right ? Because otherwise when running docker in my local machine it would simply break as the Dockerfile has the COPY instruction. This works maybe fine for the CI but not for the other environments. Is there a way to avoid this root-config folder ?

@j-riebe
Copy link
Contributor

j-riebe commented Oct 27, 2022

Hey @doctenahasib,

I'd recommend using different Dockerfiles for your dev environment and the production build/ci process (à la "do one thing and do it well").

Your dev setup would most likely contain additional dev dependencies anyway that need to be handled separately. So you most likely end up building conditional statements into your Dockerfile or just use separate ones.

@j-riebe
Copy link
Contributor

j-riebe commented Oct 27, 2022

As your error description is not pretty verbose, could you try to turn off host key validation during the actual install?

RUN --mount=type=ssh GIT_SSH_COMMAND="ssh -v -o StrictHostKeyChecking=no" <INSTALL COMMAND>

Just to make sure, that at least the keys and settings are configured correctly.
I remember, that our build tool had some issues with the tweaked hostnames.
Although copying .ssh from the runner should theoretically make the modified .known_hosts available, that is created by the ssh-agent action (or not @mpdude ?).

Be aware that using this setting in production also has good chances of "defeating the whole purpose of key-based host authentication".

@doctenahasib
Copy link
Author

@j-riebe I added GIT_SSH_COMMAND and this is the output. Still not working. (Btw I masked some stuff with ***)

#9 1.714 Cloning into bare repository '/root/.cache/composer/vcs/git-github.com-***.git'...
#9 1.714 OpenSSH_8.9p1 Ubuntu-3, OpenSSL 3.0.2 15 Mar 2022
#9 1.714 debug1: Reading configuration data /etc/ssh/ssh_config
#9 1.714 debug1: /etc/ssh/ssh_config line 19: include /etc/ssh/ssh_config.d/*.conf matched no files
#9 1.714 debug1: /etc/ssh/ssh_config line 21: Applying options for *
#9 1.714 debug1: Connecting to github.com [140.82.112.3] port 22.
#9 1.714 debug1: Connection established.
#9 1.714 debug1: identity file /root/.ssh/id_rsa type -1
#9 1.714 debug1: identity file /root/.ssh/id_rsa-cert type -1
#9 1.714 debug1: identity file /root/.ssh/id_ecdsa type -1
#9 1.714 debug1: identity file /root/.ssh/id_ecdsa-cert type -1
#9 1.714 debug1: identity file /root/.ssh/id_ecdsa_sk type -1
#9 1.714 debug1: identity file /root/.ssh/id_ecdsa_sk-cert type -1
#9 1.714 debug1: identity file /root/.ssh/id_ed25519 type -1
#9 1.714 debug1: identity file /root/.ssh/id_ed25519-cert type -1
#9 1.714 debug1: identity file /root/.ssh/id_ed25519_sk type -1
#9 1.714 debug1: identity file /root/.ssh/id_ed25519_sk-cert type -1
#9 1.714 debug1: identity file /root/.ssh/id_xmss type -1
#9 1.714 debug1: identity file /root/.ssh/id_xmss-cert type -1
#9 1.714 debug1: identity file /root/.ssh/id_dsa type -1
#9 1.714 debug1: identity file /root/.ssh/id_dsa-cert type -1
#9 1.714 debug1: Local version string SSH-2.0-OpenSSH_8.9p1 Ubuntu-3
#9 1.714 debug1: Remote protocol version 2.0, remote software version babeld-ea310e90
#9 1.714 debug1: compat_banner: no match: babeld-ea310e90
#9 1.714 debug1: Authenticating to github.com:22 as 'git'
#9 1.714 debug1: load_hostkeys: fopen /root/.ssh/known_hosts2: No such file or directory
#9 1.714 debug1: load_hostkeys: fopen /etc/ssh/ssh_known_hosts: No such file or directory
#9 1.714 debug1: load_hostkeys: fopen /etc/ssh/ssh_known_hosts2: No such file or directory
#9 1.714 debug1: SSH2_MSG_KEXINIT sent
#9 1.714 debug1: SSH2_MSG_KEXINIT received
#9 1.714 debug1: kex: algorithm: curve25519-sha256
#9 1.714 debug1: kex: host key algorithm: ssh-ed25519
#9 1.714 debug1: kex: server->client cipher: chacha20-poly1305@openssh.com MAC: <implicit> compression: none
#9 1.714 debug1: kex: client->server cipher: chacha20-poly1305@openssh.com MAC: <implicit> compression: none
#9 1.714 debug1: expecting SSH2_MSG_KEX_ECDH_REPLY
#9 1.714 debug1: SSH2_MSG_KEX_ECDH_REPLY received
#9 1.714 debug1: Server host key: ssh-ed25519 SHA256:***
#9 1.714 debug1: load_hostkeys: fopen /root/.ssh/known_hosts2: No such file or directory
#9 1.714 debug1: load_hostkeys: fopen /etc/ssh/ssh_known_hosts: No such file or directory
#9 1.714 debug1: load_hostkeys: fopen /etc/ssh/ssh_known_hosts2: No such file or directory
#9 1.714 debug1: Host 'github.com' is known and matches the ED25519 host key.
#9 1.714 debug1: Found key in /root/.ssh/known_hosts:3
#9 1.714 debug1: rekey out after 1[342](https://github.com/***/actions/runs/3337933753/jobs/5524924702#step:6:343)17728 blocks
#9 1.714 debug1: SSH2_MSG_NEWKEYS sent
#9 1.714 debug1: expecting SSH2_MSG_NEWKEYS
#9 1.714 debug1: SSH2_MSG_NEWKEYS received
#9 1.714 debug1: rekey in after 134217728 blocks
#9 1.714 debug1: get_agent_identities: agent returned 3 keys
#9 1.714 debug1: Will attempt key: https://github.com/*** ED25519 SHA256:*** agent
#9 1.714 debug1: Will attempt key: https://github.com/*** ED25519 SHA256:*** agent
#9 1.714 debug1: Will attempt key: https://github.com/*** ED25519 SHA256:*** agent
#9 1.714 debug1: Will attempt key: /root/.ssh/id_rsa 
#9 1.714 debug1: Will attempt key: /root/.ssh/id_ecdsa 
#9 1.714 debug1: Will attempt key: /root/.ssh/id_ecdsa_sk 
#9 1.714 debug1: Will attempt key: /root/.ssh/id_ed25519 
#9 1.714 debug1: Will attempt key: /root/.ssh/id_ed25519_sk 
#9 1.714 debug1: Will attempt key: /root/.ssh/id_xmss 
#9 1.714 debug1: Will attempt key: /root/.ssh/id_dsa 
#9 1.714 debug1: SSH2_MSG_EXT_INFO received
#9 1.714 debug1: kex_input_ext_info: server-sig-algs=<ssh-ed25519-cert-v01@openssh.com,ecdsa-sha2-nistp521-cert-v01@openssh.com,ecdsa-sha2-nistp384-cert-v01@openssh.com,ecdsa-sha2-nistp256-cert-v01@openssh.com,sk-ssh-ed25519-cert-v01@openssh.com,sk-ecdsa-sha2-nistp256-cert-v01@openssh.com,rsa-sha2-512-cert-v01@openssh.com,rsa-sha2-256-cert-v01@openssh.com,ssh-rsa-cert-v01@openssh.com,sk-ssh-ed25519@openssh.com,sk-ecdsa-sha2-nistp256@openssh.com,ssh-ed25519,ecdsa-sha2-nistp521,ecdsa-sha2-nistp384,ecdsa-sha2-nistp256,rsa-sha2-512,rsa-sha2-256,ssh-rsa>
#9 1.714 debug1: SSH2_MSG_SERVICE_ACCEPT received
#9 1.714 debug1: Authentications that can continue: publickey
#9 1.714 debug1: Next authentication method: publickey
#9 1.714 debug1: Offering public key: https://github.com/*** ED25519 SHA256:*** agent
#9 1.714 debug1: Server accepts key: https://github.com/*** ED25519 SHA256:*** agent
#9 1.714 Authenticated to github.com ([140.82.112.3]:22) using "publickey".
#9 1.714 debug1: channel 0: new [client-session]
#9 1.714 debug1: Entering interactive session.
#9 1.714 debug1: pledge: filesystem
#9 1.714 debug1: client_input_global_request: rtype hostkeys-00@openssh.com want_reply 0
#9 1.714 debug1: client_input_hostkeys: searching /root/.ssh/known_hosts for github.com / (none)
#9 1.714 debug1: client_input_hostkeys: searching /root/.ssh/known_hosts2 for github.com / (none)
#9 1.714 debug1: client_input_hostkeys: hostkeys file /root/.ssh/known_hosts2 does not exist
#9 1.714 debug1: client_input_hostkeys: no new or deprecated keys from server
#9 1.714 debug1: Sending environment.
#9 1.714 debug1: channel 0: setting env GIT_PROTOCOL = "version=2"
#9 1.714 debug1: Sending command: git-upload-pack '***'
#9 1.714 ERROR: Repository not found.
#9 1.714 debug1: client_input_channel_req: channel 0 rtype exit-status reply 0
#9 1.714 debug1: channel 0: free: client-session, nchannels 1
#9 1.714 Transferred: sent 2204, received 2280 bytes, in 0.2 seconds
#9 1.714 Bytes per second: sent 13935.2, received 14415.8
#9 1.714 debug1: Exit status 1
#9 1.714 fatal: Could not read from remote repository.
#9 1.714 
#9 1.714 Please make sure you have the correct access rights
#9 1.714 and the repository exists.

@j-riebe
Copy link
Contributor

j-riebe commented Oct 27, 2022

@doctenahasib Those two lines

#9 1.714 debug1: Offering public key: https://github.com/*** ED25519 SHA256:*** agent
#9 1.714 debug1: Server accepts key: https://github.com/*** ED25519 SHA256:*** agent

seam to me, as if the wrong key was used.
I guess the *** are those parts that you masked, so you'd be able to do the following:

Check if the key that was "accepted" there is in fact the deploy key for the repo (masked too in line 1?).
If thats not the case (which is very likely) your workflow didn't copy the ssh-agent configuration from the runner (error source 1) into the correct place inside your container (error source 2).

In general this problem looks exactly like mentioned in the docs - the first known key might not be the right one.

BTW: Did you already try to use the docker/build-push-action?

@j-riebe
Copy link
Contributor

j-riebe commented Oct 27, 2022

And is the workflow at the top exactly what you are using or did you also "mask" some (very essential 😉) parts?

If thats the case, you forgot to copy the git and ssh config to your checked out repo.

- name: Prepare git and ssh config for build context 
  run: | 
    mkdir root-config 
    cp -r ~/.gitconfig ~/.ssh root-config/

If you don't do that, it doesn't matter what you do, Docker just won't know the files -> see docs regarding docker-build-push + Deploy keys for details.

@j-riebe
Copy link
Contributor

j-riebe commented Oct 27, 2022

#145

@doctenahasib
Copy link
Author

doctenahasib commented Oct 28, 2022

@j-riebe Thanks for your help. Finally I was able to make it work by doing simply this:

CI workflow file

- name: Prepare git and ssh config for build context
  run: |
    cp -r ~/.gitconfig ~/.ssh root-config/
    sed 's|/home/runner|/root|g' -i.bak root-config/.ssh/config
- name: Build docker image
  run: docker compose build --ssh default

Dockerfile

ONBUILD RUN mkdir -p -m 0600 ~/.ssh
ONBUILD COPY root-config /root/
ONBUILD RUN ssh-keyscan github.com >> ~/.ssh/known_hosts

I have created an empty root-config folder in my repo because in this way it works for both dev and ci environment. So basically it's copying an empty root-config folder on dev because I do not have deploy keys on my local machine but one ssh key which has access to all the private repositories. (The problem does not exist in dev with one ssh key). With this solution I keep having one Dockerfile.

@mpdude
Copy link
Member

mpdude commented Oct 29, 2022

@j-riebe thank you for the extensive support I was not able to provide!

@doctenahasib how could we improve the documentation, what would have helped you?

@j-riebe
Copy link
Contributor

j-riebe commented Oct 29, 2022

@mpdude See PR #145 for an update of the docs

@doctenahasib
Copy link
Author

@mpdude This applies only to multiple deploy keys as with one ssh key there is no issue. I would go with @j-riebe proposition. IMO only thing missing is specifying on how we should copy the root-config in the Dockerfile.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

5 participants