Unable to use custom EE image #9917
Comments
|
Can you try: |
|
the build process fails at the end: |
|
note: i have ansible-builder 0.6.0 here - 1.0.0a1 can't be installed, see ansible/ansible-builder#204 |
|
with ansible-builder 1.0.0 the build fails in the same way at the same step. |
|
Can you post your |
|
|
I just tried with your inputs and it worked fine for me. Can you please share which version of Docker you are using? |
|
lemmy@kumiko:~> docker --version |
|
Hmmm... Can you try updating and see what happens? |
|
|
Alright, I'm stumped. Now we bring out the hammer: |
|
I used an even bigger hammer: |
|
Can you paste the output of the rendered Dockerfile/Containerfile? Your output is showing up differently from mine. I would expect to see formatting something similar to: |
|
Wait a sec... I'm running with |
|
With |
|
I'd almost think it could be something in my ~/.ansible.cfg but then my local user on my k3s host does not have one. Right now I am having the very same problem with three different docker versions spread out over two different linux flavors, docker 18.x on debian buster, and 19x + 20.x on openSUSE... |
|
make that four different container runtimes on three different OS - podman 2.2.1 on RHEL 8.4 fails exactly the same way, at the same step. and I have created that user specifically for this test and haven't done anything with containers on that VM before... |
|
just out of curiosity: explains the "No such file or directory" error, at least... |
|
facepalm the base container "awx-ee" uses a non-root user. If I start that container "normally" i can't manually install collections - if I run the container with "-u root" i can: as normal user: as root: |
|
@lemmy04 The awx-ee doesn't flip the USER to 1000 until the very end of the Containerfile: https://github.com/ansible/awx-ee/blob/268fe3d79f99e2a44a11df7a216cb6cfdd43be9f/Containerfile#L29 Which you can see by manually patching the produced Containerfile by shoving Edit: works with Docker too: |
|
when I do the same thing (manually insert a "RUN whoami" I get "command not found" - and if I put in "RUN id -a" instead I see it already as UID 1000 just before the galaxy install stuff.... |
|
Oh, shit. Because using awx-ee as the base image. 🤦 My bad, should have realized that. I thought we added some code to ansible-builder to explicitly put In the meantime, you can hack your way through this by shoving |
|
that finishes now - but I think ansible-builder does two more steps after that "docker build"? Now, if i push the resulting image and re-run my failed job ir fails like this: cripple-windows | FAILED! => { as in, the missing collections were there, but the python modules listed in requirements.txt didn't get installed... |
|
I manually patched the ansible-builder main.py file to what is in ansible/ansible-builder#205... I think this whole shebang should move over into a bug on ansible-builder, just filed ansible/ansible-builder#206 |
|
@lemmy04 Were you able to get a build done successfully? I have been following this thread and followed the steps that you went through. I pulled the 0.1.1 version of the awx-ee. Ran that Pushed it up to dockerhub using podman. Then used ansible-builder to build an image and I have been getting this in my AWX output: and I managed to get a snippet of the pod contents before it terminated as being: Sorry @shanemcd for hijacking this thread....I thought i would add it here as it may provide a bit more context.... |
|
I'm also experiencing the exact behavior described by @weiyentan. I was previously (3-4 days ago) able to build EE imges just fine using this repo and simply modifying the The only thing I can think of that's changed is the ansible runner image. I cleared my local docker images at some point so I assume it pulled a newer version of this image considering the repo shows frequent updates to those tagged images. Does it make more sense to use different tag versions for newer image pushes to the EDIT: I realized I was using: build_arg_defaults:
ANSIBLE_RUNNER_IMAGE: 'quay.io/ansible/ansible-runner:stable-2.11-devel'I changed this to Thanks! |
|
So I've been able to get past the Run the ansible-builder build command as outlined in the documentation.
Then rerun the docker build command that the Ansible builder provided. Which will rebuild the container and reinstall the ansible-runner. After I do that I no longer get the |
|
You are a life saver @Bbett . I ower you a coffee/beer. That fixed my problem. |
|
@Bbett at this point I'm honestly confused about all steps to take or not take. |
|
Hi @DrackThor , Would love to help while waiting, which part is confusing you? I can add some clarity? |
|
@DrackThor here are my files. execution-environment.yml requirements.yml requirements.txt bindep.txt and finally context/run.sh Be sure to update the run.sh to be executable
Then build the container with the following command.
When I attempt to use the created container in AWX 19.0.0 I get the However if I go edit context/Dockerfile and run Then when I run the container in AWX 19.0.0 it works as expected. |
|
You can add reinstall Then build: |
|
@stasonspb that is correct, I didn't think to do that after I found that doing a reinstall of ansible-runner was correcting the issue. |
|
@Bbett I owe you several beers. That fix just solved a week of banging my head against a wall. |
Hi @weiyentan what's still unclear to me:
@Bbett thank you very much - also works for me now! I also struggled with this for almost three days now... |
|
You need bindep.txt. I wish some one can explain to me how I can add what I want from different repositories. But the requirement.yml you choose what you want. These ate galaxy things. Requirements.txt are what the old venv python libraries that you want. |
|
The runner image is relating to the engine that is driving ansible. 2.0 runner from pip3 seems to work with awx |
|
I've just tried to install awx 19.1.0 and use my previously built ee container, which brought me this error Traceback (most recent call last):
File "/var/lib/awx/venv/awx/lib64/python3.8/site-packages/awx/main/tasks.py", line 1377, in run
res = receptor_job.run()
File "/var/lib/awx/venv/awx/lib64/python3.8/site-packages/awx/main/tasks.py", line 2904, in run
return self._run_internal(receptor_ctl)
File "/var/lib/awx/venv/awx/lib64/python3.8/site-packages/awx/main/tasks.py", line 2962, in _run_internal
raise RuntimeError(detail)
RuntimeError: Sending stdin to podsteps to reproduce:
@shanemcd should I add additional information (eg. my config files etc) here, or should we open a new issue? |
|
@DrackThor Did you rebuild your EE on top of AWX EE 0.2.0? |
No, I've used this execution-environment.yaml: ---
version: 1
build_arg_defaults:
ANSIBLE_RUNNER_IMAGE: quay.io/ansible/ansible-runner:devel
PYTHON_BUILDER_IMAGE: quay.io/ansible/python-builder:latest
dependencies:
galaxy: requirements.yml
system: bindep.txt
python: requirements.txt
additional_build_steps:
append:
- RUN alternatives --set python /usr/bin/python3
- RUN pip3 uninstall --yes ansible-runner && pip3 install ansible-runner==2.0.0a1
- COPY --from=docker.avl.com/project-receptor/receptor:0.9.7 /usr/bin/receptor /usr/bin/receptor
- RUN mkdir -p /var/run/receptor
- ADD run.sh /run.sh
- CMD /run.sh
- RUN update-ca-trust force-enable
- ADD certificates/*.crt /etc/pki/ca-trust/source/anchors/
- RUN chmod 644 /etc/pki/ca-trust/source/anchors/*.crt && update-ca-trust extract
- COPY krb5.conf /etc/krb5.conf
- USER 1000
- RUN git lfs installI'll try using |
|
@DrackThor ahhh I think you need to bump to ansible-runner 2.0.0a2 |
|
There shouldn't be a need to have We should be pulling in the latest version by default. If not, then there is a problem some place |
I've got it running by building upon FROM quay.io/ansible/awx-ee:0.2.0
USER root
# install OS binaries
RUN yum -y install \
ca-certificates \
gcc \
git \
git-lfs \
krb5-devel \
krb5-libs \
krb5-workstation \
libcurl-devel \
libxml2-devel \
openssl-devel \
python3-jmespath \
python3-netaddr \
python3-passlib \
python3-pycurl \
python38-devel \
python38-pytz \
python38-pyyaml \
python38-requests \
qemu-img
# add Python dependencies and Ansible
# Galary dependencies
ADD requirements.yml /tmp/requirements.yml
ADD requirements.txt /tmp/requirements.txt
# upgrade pip
RUN /usr/bin/python3 -m pip install --upgrade pip
# install Ansible Galaxy collections
RUN ansible-galaxy collection install -r /tmp/requirements.yml --collections-path /usr/share/ansible/collections
# install Python dependencies
RUN pip install -r /tmp/requirements.txt
# add certificates
RUN update-ca-trust force-enable
ADD certificates/*.crt /etc/pki/ca-trust/source/anchors/
RUN chmod 644 /etc/pki/ca-trust/source/anchors/*.crt && update-ca-trust extract
# add Kerberos conf
COPY krb5.conf /etc/krb5.conf
USER 1000This way Kerberos support works as well 😃
|
Yah, it looks like you didn't use ansible-builder to create your dockerfile. Which is fine, but you now have development headers in your final image. https://github.com/ansible/network-ee is another EE, which has a more minimal dockerfile |
|
@pabelanger exactly, now I just extended the awx-ee:0.2.0 image. So imho there are two workarounds for this issue so far:
|
|
krb5 file can be mounted separately thru the crd resource.
…On Thu, 6 May 2021 at 01:22, DrackThor ***@***.***> wrote:
@DrackThor <https://github.com/DrackThor> ahhh I think you need to bump
to ansible-runner 2.0.0a2
I've got it running by building upon awx-ee:0.2.0 like mentioned in #10060
<#10060>
This image, in used in an Execution Environment on awx 19.1.0 works for me:
FROM quay.io/ansible/awx-ee:0.2.0
USER root
# install OS binaries
RUN yum -y install \
ca-certificates \
gcc \
git \
git-lfs \
krb5-devel \
krb5-libs \
krb5-workstation \
libcurl-devel \
libxml2-devel \
openssl-devel \
python3-jmespath \
python3-netaddr \
python3-passlib \
python3-pycurl \
python38-devel \
python38-pytz \
python38-pyyaml \
python38-requests \
qemu-img
# add Python dependencies and Ansible
# Galary dependencies
ADD requirements.yml /tmp/requirements.yml
ADD requirements.txt /tmp/requirements.txt
# upgrade pip
RUN /usr/bin/python3 -m pip install --upgrade pip
# install Ansible Galaxy collections
RUN ansible-galaxy collection install -r /tmp/requirements.yml --collections-path /usr/share/ansible/collections
# install Python dependencies
RUN pip install -r /tmp/requirements.txt
# add certificates
RUN update-ca-trust force-enable
ADD certificates/*.crt /etc/pki/ca-trust/source/anchors/
RUN chmod 644 /etc/pki/ca-trust/source/anchors/*.crt && update-ca-trust extract
# add Kerberos conf
COPY krb5.conf /etc/krb5.conf
USER 1000
This way Kerberos support works as well 😃
- I assume this is just a workaround and the use of ansible-builder
should be the way to go?
- thanks for your help so far! 😄
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#9917 (comment)>, or
unsubscribe
<https://github.com/notifications/unsubscribe-auth/ADJC2SFB34IGFPOV7WPOG3TTMFBBLANCNFSM427SFHJA>
.
|
|
Thanks for the discussion here folks. I think we learned a lot here. We're actively working on refining documentation and prioritizing UX-related issues as things around Execution Environments begin to stabilize. AWX 19.2.0 will be out later today, along with a newer version of the awx-ee. This is largely a bugfix release. See the changelog for more details. The protocols should be compatible with awx-ee 0.2.0, but in case you run into any issues, try building on top of the newest awx-ee to see if it helps. If you encounter any specific issues in the future please open a new issue. Seriously, thank you! |
ISSUE TYPE
SUMMARY
I have built a custom EE image (quay.io/mhomann/awx-ee-community), but when I try to use it for a job the job fails with a very uninformative "runtime error"
ENVIRONMENT
STEPS TO REPRODUCE
requirements.yml:
requirements.txt:
Observe the job to fail with the following error:
EXPECTED RESULTS
The job should be executed and give some meaningful result
ACTUAL RESULTS
ADDITIONAL INFORMATION
there is nothing in the system journal as to why the job failed except for something that looks like some failed tcp connection to something.
If i manually start a container with a shell from the image i've build i can manually execute ansible m win_ping and it works (within the limits of not having inventory etc etc).
The text was updated successfully, but these errors were encountered: