Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

rootless podman not using conmon_path from /etc/containers/libpod.conf #2174

Closed
sysrich opened this issue Jan 17, 2019 · 16 comments
Closed

rootless podman not using conmon_path from /etc/containers/libpod.conf #2174

sysrich opened this issue Jan 17, 2019 · 16 comments
Assignees
Labels
kind/bug Categorizes issue or PR as related to a bug. locked - please file new issue/PR Assist humans wanting to comment on an old issue or PR with locked comments. rootless

Comments

@sysrich
Copy link
Contributor

sysrich commented Jan 17, 2019

Is this a BUG REPORT or FEATURE REQUEST? (leave only one on its own line)

/kind bug

Description

running podman run hello-world as a non-root user produces the following error message

error creating libpod runtime: could not find a working conmon binary (configured options: [/usr/libexec/podman/conmon /usr/libexec/crio/conmon /usr/local/lib/podman/conmon /usr/local/libexec/crio/conmon /usr/bin/conmon /usr/sbin/conmon /usr/lib/crio/bin/conmon]): invalid argument

The container obviously does not start

/etc/containers/libpod.conf has conmon_path set to /usr/lib/podman/bin/conmon, which is the correct location for conmon on all *SUSE distributions.

The above error message clearly implies that podman has not read this parameter and instead is only looking in the default locations.

My expectation was that rootless podman should inherit its parameters from /etc/containers/libpod.conf and only be overriden by a ~.config/containers/libpod.conf if it is manually created by the user.

Steps to reproduce the issue:

  1. Install podman on openSUSE Tumbleweed version 20190115 or later. Alternatively install Kubic with the microos system role (the default) - podman is the default container runtime there.

  2. run podman run hello-world as a non-root user

  3. There is no step 3

Output of podman version:

Version:       1.0.0
Go Version:    go1.11.4
OS/Arch:       linux/amd64

Output of podman info:

host:
  BuildahVersion: 1.6-dev
  Conmon:
    package: podman-1.0.0-1.1.x86_64
    path: /usr/lib/podman/bin/conmon
    version: 'conmon version 1.12.0-dev, commit: '
  Distribution:
    distribution: '"opensuse-tumbleweed-kubic"'
    version: "20190115"
  MemFree: 11926519808
  MemTotal: 16487182336
  OCIRuntime:
    package: runc-1.0.0~rc6-2.1.x86_64
    path: /usr/bin/runc
    version: |-
      runc version 1.0.0-rc6
      spec: 1.0.1-dev
  SwapFree: 0
  SwapTotal: 0
  arch: amd64
  cpus: 4
  hostname: r2d2
  kernel: 4.20.0-1-default
  os: linux
  rootless: false
  uptime: 7h 35m 10.36s (Approximately 0.29 days)
insecure registries:
  registries: []
registries:
  registries:
  - docker.io
store:
  ConfigFile: /etc/containers/storage.conf
  ContainerStore:
    number: 3
  GraphDriverName: btrfs
  GraphOptions: null
  GraphRoot: /var/lib/containers/storage
  GraphStatus:
    Build Version: 'Btrfs v4.19.1 '
    Library Version: "102"
  ImageStore:
    number: 3
  RunRoot: /var/run/containers/storage
@openshift-ci-robot openshift-ci-robot added the kind/bug Categorizes issue or PR as related to a bug. label Jan 17, 2019
@giuseppe
Copy link
Member

yes, at the moment we don't honor /etc/containers/libpod.conf for rootless containers, but use directly the version in the user config directory.

Is conmon installed at some path that the default libpod configuration doesn't know about?

@sysrich
Copy link
Contributor Author

sysrich commented Jan 17, 2019

/usr/lib/podman/bin/conmon is in the default libpod configuration documented here: https://github.com/containers/libpod/blob/master/libpod.conf

But the error message implies that podman 1.0.0 is not honoring that default libpod configuration either and looking at a different list of paths somehow.

@vrothberg
Copy link
Member

/usr/lib/podman/bin/conmon is in the default libpod configuration documented here: https://github.com/containers/libpod/blob/master/libpod.conf

But the error message implies that podman 1.0.0 is not honoring that default libpod configuration either and looking at a different list of paths somehow.

I agree. It seems that when generating ~/.config/containers/libpod.conf the hard-coded defaults are used instead of copying specific values from /etc/containers/libpod.conf.

@vrothberg
Copy link
Member

Podman should definitely do that for config options. openSUSE is shipping a libpod.conf tailored specifically for openSUSE-specific paths to avoid costly lookups for binaries.

@sysrich
Copy link
Contributor Author

sysrich commented Jan 17, 2019

I agree. It seems that when generating ~/.config/containers/libpod.conf the hard-coded defaults are used instead of copying specific values from /etc/containers/libpod.conf.

I agree, and it seems to me the hard coded defaults are different from the defaults written in the default libpod.conf from https://github.com/containers/libpod/blob/master/libpod.conf

@giuseppe
Copy link
Member

Podman should definitely do that for config options.

I am not sure about it. We should be careful and cherry-pick only the options that can be inherited safely from the global configuration. Some of them would override the defaults tailored for the rootless mode.

@giuseppe
Copy link
Member

maybe we should maintain a white list for those that must be inherited from /etc/containers/libpod.conf the first time the rootless configuration is created

@vrothberg
Copy link
Member

ACK. We shouldn't blindly copy but need to pick the ones that make sense.

@afbjorklund
Copy link
Contributor

afbjorklund commented Jan 22, 2019

I had the same issue, and had to patch libpod because of it (lack of rootless configuration)

sed -e 's|"/etc/cni|"/usr/local/etc/cni|' -i libpod.conf
sed -e 's|"/etc/cni|"/usr/local/etc/cni|' -i libpod/runtime.go
sed -e 's|/usr/libexec/|/usr/local/lib/|' -i libpod.conf
sed -e 's|/usr/libexec/|/usr/local/lib/|' -i libpod/runtime.go
sed -e 's|/usr/share/|/usr/local/share/|' -i libpod/runtime.go

Otherwise it is not able to find CNI, which means that rootless podman is not able to start up.

Unfortunately it also means it is "dirty", so configuration would be preferred.

Git Commit: "4e0c0ecbc383531cd1b38db9027583974a72070d-dirty"

@rhatdan
Copy link
Member

rhatdan commented Jan 22, 2019

We should definitely get the executable paths from libpod.conf

@giuseppe
Copy link
Member

opened a PR here: #2222

@sysrich would it be possible for you to try if it solves the issue you had?

@afbjorklund
Copy link
Contributor

Would it be possible to add no_pivot_root and cni_config_dir too ?

I think that cgroup_manager is determined automatically (?), so it is OK...

# CGroup Manager - valid values are "systemd" and "cgroupfs"
cgroup_manager = "cgroupfs"

# Whether to use chroot instead of pivot_root in the runtime
no_pivot_root = true

# Directory containing CNI plugin configuration files
cni_config_dir = "/usr/local/etc/cni/net.d/"

@mheon
Copy link
Member

mheon commented Jan 28, 2019

The systemd cgroup manager doesn't work at all with rootless, so we shouldn't bother sourcing that - it needs to stay unconditionally set to "cgroupfs"

The other two would be nice to have, though

@giuseppe
Copy link
Member

I can add cni_plugin_dir and no_pivot_root, although rootless containers do not support CNI plugins, so it won't make any difference for now.

cni_config_dir doesn't probably make sense. Even if there will be rootless CNI plugins, the configuration for rootless will be different than from the host

@afbjorklund
Copy link
Contributor

The cni_plugin_dir was OK, since it looked in multiple paths (including /usr/local/lib/cni)

I seem to recall some error from CNI, but maybe that was just normal podman and not rootless ?

@afbjorklund
Copy link
Contributor

Got some other error now: ERRO[0001] slirp4netns failed
Guess I will look into that particular podman error separately.

The other error was (as I suspected) when running as root:
ERRO[0000] error configuring network namespace for container 7dcf3329d7bd6f6372967fcfffaf78bac904246910005bcafc4c4d0260397bca: Missing CNI default network

So no_pivot_root would be enough.

giuseppe added a commit to giuseppe/libpod that referenced this issue Feb 5, 2019
if some paths are overriden in the global configuration file, be sure
that rootless podman honors them.

Closes: containers#2174

Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
@github-actions github-actions bot added the locked - please file new issue/PR Assist humans wanting to comment on an old issue or PR with locked comments. label Sep 24, 2023
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Sep 24, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
kind/bug Categorizes issue or PR as related to a bug. locked - please file new issue/PR Assist humans wanting to comment on an old issue or PR with locked comments. rootless
Projects
None yet
Development

No branches or pull requests

7 participants