Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use SSH key source secret w/ git+ssh in requirements.txt? #280

Open
jeremy303 opened this issue Jun 19, 2018 · 12 comments
Open

Use SSH key source secret w/ git+ssh in requirements.txt? #280

jeremy303 opened this issue Jun 19, 2018 · 12 comments
Labels
help wanted openshift Issues that are relevant to openshift images as a whole, not specific to the s2i-python-container

Comments

@jeremy303
Copy link

Using the 3.6 S2I container, pip needs to pull a dependency from a private git repo as specified in my requirements.txt, e.g:

git+ssh://git@github.organization.org:github-user/githib-repo.git@some_tag

The same SSH key source secret that the OpenShift build is presently using to pull the application source code also permits access to this dependency, however the git+ssh clone is failing:

Collecting git+ssh://git@github.organization.org:github-user/githib-repo.git@some_tag (from -r requirements.txt (line 9))
Cloning ssh://git@github.organization.org:github-user/githib-repo.git@some_tag (to some_tag) to /tmp/pip-7tf7baqk-build
ssh: Could not resolve hostname github.organization.org: Name or service not known
fatal: Could not read from remote repository.

Is there anything I can do that would allow pip to make use of the same SSH key source secret used to clone my app code when installing requirements.txt dependencies?

Thanks.

@GrahamDumpleton
Copy link
Contributor

If understand what you want to do, you would need to use a build secret in the build configuration to have the appropriate SSH keys added into the build image. Are you using a build secret already?

@richardqa
Copy link

I have the same problem. I got failed when i deployed my app from Openshift template.
I think the secret resource only work for the GIT of the project, but not for some other GITs in the requirement file. How can I download other Repos GIT from a OpenShift template?

regards,

@jpramos123
Copy link

jpramos123 commented Feb 4, 2019

I'm facing a similar issue but using https.

I'm already using the source secret and build secret (both are the same secrets).
Inside the requirements.txt file the reference to the repo is the following:

git+https://gitlab.organization.com/group-user/repo.git

Collecting git+https://gitlab.organization.com/group-user/repo.git  (from -r
requirements.txt (line 1))
Cloning https://gitlab.organization.com/group-user/repo.git   to /tmp/pip-q4puhrby-build
fatal: could not read Username for 'https://gitlab.organization.com':  No such device or address
Command "git clone -q https://gitlab.organization.com/group-user/repo.git   /tmp/pip-q4puhrby-build" failed with error code 128 in None

Any idea on that?

@jpramos123
Copy link

Well, I tried what @GrahamDumpleton suggested and I still not able to clone the module from a private repository.

I created the ssh key with the following command: ssh-keygen -t rsa -C "jramos@organization.com" and updated the Private Key to OpenShift platform.

The build is able to clone my git project (which is in the same domain of the python module).

I created a ssh secret and assigned it to the Source Secret as well to the Build Secret.

My requirements.txt file is as follows:

git+ssh://git@gitlab.organization.com/group-user/repo.git
certifi==2018.1.18
chardet==3.0.4
click==6.7
 ...

I'm receiving the following error:

Collecting git+ssh://git@gitlab.organization.com/group-user/repo.git (from -r requirements.txt (line 1))
Cloning ssh://git@gitlab.organization.com/group-user/repo.git to /tmp/pip-rgq4xum2-build
Host key verification failed.
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.

Waiting for the feedback!

Thanks a lot!

@richardqa
Copy link

Well, I tried what @GrahamDumpleton suggested and I still not able to clone the module from a private repository.

I created the ssh key with the following command: ssh-keygen -t rsa -C "jramos@organization.com" and updated the Private Key to OpenShift platform.

The build is able to clone my git project (which is in the same domain of the python module).

I created a ssh secret and assigned it to the Source Secret as well to the Build Secret.

My requirements.txt file is as follows:

git+ssh://git@gitlab.organization.com/group-user/repo.git
certifi==2018.1.18
chardet==3.0.4
click==6.7
 ...

I'm receiving the following error:

Collecting git+ssh://git@gitlab.organization.com/group-user/repo.git (from -r requirements.txt (line 1))
Cloning ssh://git@gitlab.organization.com/group-user/repo.git to /tmp/pip-rgq4xum2-build
Host key verification failed.
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.

Waiting for the feedback!

Thanks a lot!

HI,
I didn't get it yet. The only solution that I got: ... in the moment to create your image with Dockerfile you attach your SSH public/private key, validate the keys with ssh-add. Thus, when you launch the App (oc new-app) the container already know these keys ssh and now It is able to install your requirements.txt.

On the other hand, I tried to get it using some resources of OpenShift but i hadn't sucessful

Regards,
Richard

@GrahamDumpleton
Copy link
Contributor

Maybe the issue here is my use of the term "build secret" and what is meant by that. Strictly speaking, it is a build input secret. So you need to add the private SSH key and mount it/copy it into the .ssh directory with the file having appropriate permissions so it isn't rejected. If have multiple keys, would need to have a .ssh/config file to map which to use. If have to create/copy stuff as assemble, remember to remove when done else they will stay in the image.

Only article that comes close to explaining it is:

What steps are you trying to use as is?

@GrahamDumpleton
Copy link
Contributor

Worth highlighting is that the more complicated scenario of needing to prepare .ssh directory is only needed for accessing private repositories from requirements.txt. That is, when needing to do checkout as part of the build steps themselves. If you are only trying to use private Git repository as input for source/docker built itself, follow:

@frenzymadness
Copy link
Member

Is this issue still relevant or did mentioned articles solved your problem?

@GrahamDumpleton
Copy link
Contributor

Leave it open for now as a prompt to create a proper cookbook recipe about it as the solution is non obvious. After documenting it, we might find a need to add some helper code in the assemble script to make it easier.

@torsava torsava added the openshift Issues that are relevant to openshift images as a whole, not specific to the s2i-python-container label Mar 19, 2019
@ihadgraft
Copy link

I got this working at one point when doing some initial testing and fortunately documented some things. Here's the solution I was able to work out:

MY_GIT_HOST=git-server.com

ssh-keyscan "$MY_GIT_HOST" > ~/git_known_hosts
echo 'ssh -i /path/to/private-key -o UserKnownHostsFile=/opt/app-root/src/git_known_hosts "$@"' > custom_ssh
chmod +x custom_ssh

export GIT_SSH=/opt/app-root/src/custom_ssh

After setting this up, git will use your custom command instead of ssh directly, i.e. any subsequent git calls will just work.

@frenzymadness
Copy link
Member

Hi @ihadgraft . Would you mind to transfer your last comment to a piece of documentation?

@tumregels
Copy link

Try this. Should work.

$ export GIT_SSH_COMMAND="ssh -i ~/.ssh/private_key_name"
$ pip install -r requirements.txt

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
help wanted openshift Issues that are relevant to openshift images as a whole, not specific to the s2i-python-container
Projects
Status: No status
Development

No branches or pull requests

8 participants