Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[3.0.1] podman build started failing with "runc: executable file not found in $PATH" although runc is in a recognized location #9459

Closed
riyad opened this issue Feb 21, 2021 · 7 comments · Fixed by #9906
Closed

[3.0.1] podman build started failing with "runc: executable file not found in $PATH" although runc is in a recognized location #9459

riyad opened this issue Feb 21, 2021 · 7 comments · Fixed by #9906
Labels
kind/bug Categorizes issue or PR as related to a bug. locked - please file new issue/PR Assist humans wanting to comment on an old issue or PR with locked comments. stale-issue

Comments

@riyad
Copy link
Contributor

riyad commented Feb 21, 2021 •
edited
Loading

Is this a BUG REPORT or FEATURE REQUEST? (leave only one on its own line)

/kind bug

Description

podman build ... with a Containerfile started failing with exec: "runc": executable file not found in $PATH.
Indeed runc is not in $PATH but in one of the locations listed in the engine.runtimes.runc config option. podman info picks it up correctly (see below).

  • Running buildah bud ... (with runc configured) works correctly (buildah 1.19.6)
  • Changing the runtime to crun (which also happens to be in $PATH) makes it work
  • Installing Podman 3.0.0 (i.e. downgrading from 3.0.1) makes it work again

Steps to reproduce the issue:

  1. Make sure podman uses runc

cat > Containerfile <<EOF
FROM alpine

RUN echo "does this work?"
EOF
$ podman build -f Containerfile 
STEP 1: FROM alpine
STEP 2: RUN echo "does this work?"
error running container: error creating container for [/bin/sh -c echo "does this work?"]: : exec: "runc": executable file not found in $PATH
Error: error building at STEP "RUN echo "does this work?"": error while running runtime: exit status 1

Describe the results you received:

podman build ... fails with an unexpected error:

$ podman build -f Containerfile 
STEP 1: FROM alpine
STEP 2: RUN echo "does this work?"
error running container: error creating container for [/bin/sh -c echo "does this work?"]: : exec: "runc": executable file not found in $PATH
Error: error building at STEP "RUN echo "does this work?"": error while running runtime: exit status 1

Describe the results you expected:

Should build a container as it did before and as it does with buildah

$ podman build --no-cache -f Containerfile         
STEP 1: FROM alpine
STEP 2: RUN echo "does this work?"
does this work?
STEP 3: COMMIT
--> 80dd1ca0a2c
80dd1ca0a2cab2b15edb1464ab12362dff495048f5cf57402abd8dbeea326948
$ buildah bud --no-cache -f Containerfile
STEP 1: FROM alpine
STEP 2: RUN echo "does this work?"
does this work?
STEP 3: COMMIT
Getting image source signatures
Copying blob cb381a32b229 skipped: already exists  
Copying blob 2c456cc9967d done  
Copying config ed22a73939 done  
Writing manifest to image destination
Storing signatures
--> ed22a739390
ed22a7393909e8972c44dce0a5dcf160f81790eedbb4c079edc23f4863b22430

Additional information you deem important (e.g. issue happens only occasionally):

This worked in Podman 3.0.0.

Output of podman version:

Version:      3.0.1
API Version:  3.0.0
Go Version:   go1.14.7
Built:        Thu Jan  1 01:00:00 1970
OS/Arch:      linux/amd64

Output of podman info --debug:

host:
  arch: amd64
  buildahVersion: 1.19.4
  cgroupManager: systemd
  cgroupVersion: v2
  conmon:
    package: 'conmon: /usr/libexec/podman/conmon'
    path: /usr/libexec/podman/conmon
    version: 'conmon version 2.0.26, commit: '
  cpus: 4
  distribution:
    distribution: ubuntu
    version: "20.10"
  eventLogger: journald
  hostname: acnologia
  idMappings:
    gidmap:
    - container_id: 0
      host_id: 1000
      size: 1
    - container_id: 1
      host_id: 100000
      size: 65536
    uidmap:
    - container_id: 0
      host_id: 1000
      size: 1
    - container_id: 1
      host_id: 100000
      size: 65536
  kernel: 5.8.0-43-generic
  linkmode: dynamic
  memFree: 1060929536
  memTotal: 16540594176
  ociRuntime:
    name: runc
    package: 'cri-o-runc: /usr/lib/cri-o-runc/sbin/runc'
    path: /usr/lib/cri-o-runc/sbin/runc
    version: 'runc version spec: 1.0.2-dev'
  os: linux
  remoteSocket:
    path: /run/user/1000/podman/podman.sock
  security:
    apparmorEnabled: false
    capabilities: CAP_CHOWN,CAP_DAC_OVERRIDE,CAP_FOWNER,CAP_FSETID,CAP_KILL,CAP_NET_BIND_SERVICE,CAP_SETFCAP,CAP_SETGID,CAP_SETPCAP,CAP_SETUID,CAP_SYS_CHROOT
    rootless: true
    seccompEnabled: true
    selinuxEnabled: false
  slirp4netns:
    executable: /usr/bin/slirp4netns
    package: 'slirp4netns: /usr/bin/slirp4netns'
    version: |-
      slirp4netns version 1.1.8
      commit: unknown
      libslirp: 4.3.1-git
      SLIRP_CONFIG_VERSION_MAX: 3
      libseccomp: 2.4.3
  swapFree: 1695019008
  swapTotal: 2147479552
  uptime: 21h 8m 7.85s (Approximately 0.88 days)
registries:
  search:
  - docker.io
  - quay.io
store:
  configFile: /home/riyad/.config/containers/storage.conf
  containerStore:
    number: 16
    paused: 0
    running: 1
    stopped: 15
  graphDriverName: overlay
  graphOptions:
    overlay.mount_program:
      Executable: /usr/bin/fuse-overlayfs
      Package: 'fuse-overlayfs: /usr/bin/fuse-overlayfs'
      Version: |-
        fusermount3 version: 3.9.3
        fuse-overlayfs: version 1.4
        FUSE library version 3.9.3
        using FUSE kernel interface version 7.31
    overlay.mountopt: nodev,metacopy=on
  graphRoot: /home/riyad/podman/storage
  graphStatus:
    Backing Filesystem: zfs
    Native Overlay Diff: "false"
    Supports d_type: "true"
    Using metacopy: "false"
  imageStore:
    number: 28
  runRoot: /run/user/1000/containers
  volumePath: /home/riyad/podman/storage/volumes
version:
  APIVersion: 3.0.0
  Built: 0
  BuiltTime: Thu Jan  1 01:00:00 1970
  GitCommit: ""
  GoVersion: go1.14.7
  OsArch: linux/amd64
  Version: 3.0.1

Package info (e.g. output of rpm -q podman or apt list podman):

Listing...
podman/unknown,now 100:3.0.1-1 amd64 [installed]
podman/unknown 100:3.0.1-1 arm64
podman/unknown 100:3.0.1-1 armhf
podman/unknown 100:3.0.1-1 s390x
@openshift-ci-robot openshift-ci-robot added the kind/bug Categorizes issue or PR as related to a bug. label Feb 21, 2021
@vrothberg
Copy link
Member

@rhatdan, we need to consolidate the runtime-selection code into c/common.

@rhatdan
Copy link
Member

rhatdan commented Feb 22, 2021

We need to pass the container runtime from podman to buildah

podman build --runtime crun ...

We fixed an issue in podman 3.0.1 for this, and looks like we are triggering a new bug.

@rhatdan
Copy link
Member

rhatdan commented Feb 22, 2021

#9368

@mheon
Copy link
Member

mheon commented Feb 22, 2021

@rhatdan I'm fairly sure this is more than that - I think Buildah is resolving the runtime differently than Podman, just using $PATH and not using the dedicated runtimes list in containers.conf.

@vrothberg
Copy link
Member

vrothberg commented Feb 23, 2021 •
edited
Loading

@rhatdan I'm fairly sure this is more than that - I think Buildah is resolving the runtime differently than Podman, just using $PATH and not using the dedicated runtimes list in containers.conf.

I share that thought, that's why I want us to consolidate the code in c/common if possible.

@buck2202
Copy link

buck2202 commented Feb 27, 2021 •
edited
Loading

Just confirming this is what I mentioned in #9365 (comment). Apologies that I didn't have time to open a new issue for it.

In my case, I installed cri-o-runc because the runc package on Ubuntu bionic was giving seccomp errors. podman info knows where it is

$ podman info
<snip>
  ociRuntime:
    name: runc
    package: 'cri-o-runc: /usr/lib/cri-o-runc/sbin/runc'
    path: /usr/lib/cri-o-runc/sbin/runc
    version: 'runc version spec: 1.0.2-dev'

but it is not in $PATH

$ which runc
$ dpkg-query -L cri-o-runc | grep /runc
/usr/lib/cri-o-runc/sbin/runc

$ cat test_dockerfile 
FROM ubuntu
RUN set -x && apt-get -q update

Default behavior:

$ podman build -t test -f test_dockerfile .
STEP 1: FROM ubuntu
✔ docker.io/library/ubuntu:latest
Getting image source signatures
Copying blob 83ee3a23efb7 [======================================] 27.2MiB / 27.2MiB
Copying blob f611acd52c6c done  
Copying blob db98fc6f11f0 done  
Copying config f63181f19b done  
Writing manifest to image destination
Storing signatures
STEP 2: RUN set -x && apt-get -q update
error running container: error creating container for [/bin/sh -c set -x && apt-get -q update]: : exec: "runc": executable file not found in $PATH
Error: error building at STEP "RUN set -x && apt-get -q update": error while running runtime: exit status 1

Manually passing path to runtime:

$ podman --runtime=/usr/lib/cri-o-runc/sbin/runc build -t test -f test_dockerfile .
STEP 1: FROM ubuntu
STEP 2: RUN set -x && apt-get -q update
+ apt-get -q update
Get:1 http://security.ubuntu.com/ubuntu focal-security InRelease [109 kB]
Get:2 http://archive.ubuntu.com/ubuntu focal InRelease [265 kB]
Get:3 http://security.ubuntu.com/ubuntu focal-security/main amd64 Packages [651 kB]
Get:4 http://security.ubuntu.com/ubuntu focal-security/universe amd64 Packages [671 kB]
Get:5 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 Packages [177 kB]
Get:6 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 Packages [21.6 kB]
Get:7 http://archive.ubuntu.com/ubuntu focal-updates InRelease [114 kB]
Get:8 http://archive.ubuntu.com/ubuntu focal-backports InRelease [101 kB]
Get:9 http://archive.ubuntu.com/ubuntu focal/universe amd64 Packages [11.3 MB]
Get:10 http://archive.ubuntu.com/ubuntu focal/multiverse amd64 Packages [177 kB]
Get:11 http://archive.ubuntu.com/ubuntu focal/main amd64 Packages [1275 kB]
Get:12 http://archive.ubuntu.com/ubuntu focal/restricted amd64 Packages [33.4 kB]
Get:13 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages [1060 kB]
Get:14 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 Packages [209 kB]
Get:15 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 Packages [29.6 kB]
Get:16 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 Packages [934 kB]
Get:17 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 Packages [4301 B]
Fetched 17.2 MB in 3s (5019 kB/s)
Reading package lists...
STEP 3: COMMIT test
--> 27375ea2553
27375ea2553c2785330194fa5d6f6b24fd8924dc6c8d1a72a1c459e51dfba3a4

Ensuring runtime is in $PATH:

$ sudo ln -s /usr/lib/cri-o-runc/sbin/runc /usr/local/bin
$ which runc
/usr/local/bin/runc
$
$ podman image rm test
Untagged: localhost/test:latest
Deleted: 27375ea2553c2785330194fa5d6f6b24fd8924dc6c8d1a72a1c459e51dfba3a4
$ podman build -t test -f test_dockerfile .
STEP 1: FROM ubuntu
STEP 2: RUN set -x && apt-get -q update
+ apt-get -q update
Get:1 http://security.ubuntu.com/ubuntu focal-security InRelease [109 kB]
Get:2 http://archive.ubuntu.com/ubuntu focal InRelease [265 kB]
Get:3 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 Packages [21.6 kB]
Get:4 http://archive.ubuntu.com/ubuntu focal-updates InRelease [114 kB]
Get:5 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 Packages [177 kB]
Get:6 http://archive.ubuntu.com/ubuntu focal-backports InRelease [101 kB]
Get:7 http://security.ubuntu.com/ubuntu focal-security/main amd64 Packages [651 kB]
Get:8 http://archive.ubuntu.com/ubuntu focal/multiverse amd64 Packages [177 kB]
Get:9 http://archive.ubuntu.com/ubuntu focal/universe amd64 Packages [11.3 MB]
Get:10 http://security.ubuntu.com/ubuntu focal-security/universe amd64 Packages [671 kB]
Get:11 http://archive.ubuntu.com/ubuntu focal/main amd64 Packages [1275 kB]
Get:12 http://archive.ubuntu.com/ubuntu focal/restricted amd64 Packages [33.4 kB]
Get:13 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 Packages [209 kB]
Get:14 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 Packages [934 kB]
Get:15 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 Packages [29.6 kB]
Get:16 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages [1060 kB]
Get:17 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 Packages [4301 B]
Fetched 17.2 MB in 3s (5098 kB/s)
Reading package lists...
STEP 3: COMMIT test
--> 5e5e4ca80c5
5e5e4ca80c5f3306d2624cf34d3e5c887073f68762fbdce95f9d873a53d8b22f

edit to add:

$ apt-cache madison containers-common 
containers-common |   100:1-14 | http://download.opensuse.org/repositories/devel:/kubic:/libcontainers:/stable/xUbuntu_18.04  Packages
$ apt-cache madison podman
    podman | 100:3.0.1-2 | http://download.opensuse.org/repositories/devel:/kubic:/libcontainers:/stable/xUbuntu_18.04  Packages
$ podman version
Version:      3.0.1
API Version:  3.0.0
Go Version:   go1.15.2
Built:        Thu Jan  1 00:00:00 1970
OS/Arch:      linux/amd64

@github-actions
Copy link

A friendly reminder that this issue had no activity for 30 days.

@rhatdan rhatdan mentioned this issue Mar 31, 2021
Merged
rhatdan added a commit to rhatdan/podman that referenced this issue Mar 31, 2021
@rhatdan
Should send the OCI runtime path not just the name to buildah
37d1bc4 
[NO TESTS NEEDED] Mainly because I have no idea how we would test
this.

Fixes: containers#9459

Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
mheon pushed a commit to mheon/libpod that referenced this issue Apr 16, 2021
@rhatdan @mheon
Should send the OCI runtime path not just the name to buildah
0d049e9 
[NO TESTS NEEDED] Mainly because I have no idea how we would test
this.

Fixes: containers#9459

Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
jmguzik pushed a commit to jmguzik/podman that referenced this issue Apr 26, 2021
@rhatdan @jmguzik
Should send the OCI runtime path not just the name to buildah
0dc1ff9 
[NO TESTS NEEDED] Mainly because I have no idea how we would test
this.

Fixes: containers#9459

Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
@github-actions github-actions bot added the locked - please file new issue/PR Assist humans wanting to comment on an old issue or PR with locked comments. label Sep 22, 2023
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Sep 22, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
kind/bug Categorizes issue or PR as related to a bug. locked - please file new issue/PR Assist humans wanting to comment on an old issue or PR with locked comments. stale-issue
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Should send the OCI runtime path not just the name to buildah rhatdan/podman
6 participants
@riyad @rhatdan @buck2202 @mheon @vrothberg @openshift-ci-robot