NixOS setuid wrapper prevent running sudo in user namespace

[nh2]

On NixOS 18.03:

$ sudo -u nobody echo hi
hi

$ /run/current-system/sw/bin/unshare -Upf --map-root-user -- sudo -u nobody echo hello
sudo: /nix/store/3xsjm8rfpy0ysfjs1pcybj33fsigszgp-wrapper.c:203: main: Assertion `!(st.st_mode & S_ISUID) || (st.st_uid == geteuid())' failed.
Segmentation fault

The assertion in question is this:

nixpkgs/nixos/modules/security/wrappers/wrapper.c

Line 203 in f85a82a

assert(!(st.st_mode & S_ISUID) || (st.st_uid == geteuid()));

If I'm able to to run sudo -u nobody outside the user namespace, why should I lose this capability inside a user namespace where I am root?

Could the assertion be safely relaxed so that I can still drop privileges to nobody inside my faked root environment (using sudo, after executing some commands as that fake root user)?

CC @ixmatus

[Ekleog]

Note: this breaks with all security wrappers, not just sudo (just duped with su in #49100)

[Chiiruno]

I get the same problem with steam-run.
(also did this with sudo, same thing)

steam-run su
su: /nix/store/v6l2sacryfr88yqq0pq7sia8wfgm9q31-wrapper.c:203: main: Assertion `!(st.st_mode & S_ISUID) || (st.st_uid == geteuid())' failed.

Is there any way around this?

[ixmatus]

Sorry for the very late reply, the original Github issue email message must have got lost in my inbox :-/

It has been a while since I've looked at the wrapper code so I will need to familiarize myself first before I can provide an answer to the question asking if it is safe to relax the assertion. I'll see if I can spend some time doing that later today.

[nyarly]

Pinging this: I'm trying to use exim from a Rails app run under systemd, and I trigger this same assertion.

[scintill]

Running the activation script in a chroot in the namespace creates wrappers with the right ownership (though it also reported other errors.) Unfortunately, in the case of trying to drop to regular user with su or chroot, I then get setgid: Invalid argument or chroot: failed to set supplemental groups: Operation not permitted. I believe it has to do with the kernel checking gid_map, but don't understand more than that yet.

Edit: if you're already root in the namespace, you can also just bypass the wrapper (execute the path in e.g. su.real), but I ran into issues that are probably not strictly NixOS-specific, such as the above, or sudo bailing because it's .so files are not owned by root as viewed from within the namespace.

[stale]

Thank you for your contributions.

This has been automatically marked as stale because it has had no activity for 180 days.

If this is still important to you, we ask that you leave a comment below. Your comment can be as simple as "still important to me". This lets people see that at least one person still cares about this. Someone will have to do this at most twice a year if there is no other activity.

Here are suggestions that might help resolve this more quickly:

1. Search for maintainers and people that previously touched the related code and @ mention them in a comment.
2. Ask on the NixOS Discourse.
3. Ask on the #nixos channel on irc.freenode.net.

[vs49688]

Still a problem, just hit this on 20.03 when trying to use Gitea with nullmailer.

[tobiasBora]

I also get the same error, but when running a normal appImage (appimage runs fuse):

$ nix-shell -p "(steam.override { extraPkgs = pkgs: [pkgs.fuse]; nativeOnly = true;}).run"
$ steam-run bash
bash-4.4$ ./LittleWeeb-0.4.0.303.AppImage 
fusermount: /nix/store/v6l2sacryfr88yqq0pq7sia8wfgm9q31-wrapper.c:203: main: Assertion `!(st.st_mode & S_ISUID) || (st.st_uid == geteuid())' failed.

[nixos-discourse]

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/help-with-a-derivation-package/8686/6

[legendofmiracles]

The same happens to nixos-enter when run without root.

[jbalme]

Bubblewrap breaks setuid inside containers by design which is what causes the check to fail. Unfortunately, this means that even if the check were removed, the functionality that requires setuid in the first place would just fail later down the line.

The only way around this would be passing a message to a setuid binary on the host, as Flatpak does with flatpak-spawn.

[tobiasBora]

I've the feeling that bubblewrap creates more troubles than it solves, because bubblewrap's goal is to jail apps while (as far as I understand) NixOs only uses bubblewrap to get user namespaces… Wouldn't it be easier for NixOs to simply write a short C code that only provides user namespaces?

[jbalme]

Nix runs on more than just NixOS and many distros do not have user namespaces enabled for security reasons.

[Kreyren]

I seem to have this issue in #126744 (set as duplicate of this issue)

[Atemu]

Duplicate with some more discussion and repro steps: #69338

[jbalme]

I did some looking into making a unprivileged userns container with unshare, and setuid is non-trivial to work there too. It will be possible to get setuid to a "fake" root, but getting "real" root inside the container doesn't seem possible, as "real" root no longer has uid 0 inside the container.

This is probably by design - if a user can create an arbitrary container with an arbitrary host setuid binary, they can surely design the container such that they can escalate privileges (eg. by swapping out libc out from under the setuid binary.)

I suppose we could look into a privileged mountns script that drops back to the capabilities of the calling user.

[rnhmjoj]

Just found this in a different situation:

unshare -mr /run/current-system/sw/bin/mount --bind a b   # WORKS
unshare -mr /run/wrappers/bin/mount --bind a b   # FAILS

Can't we at least change the wrapper to check for an unshared user namespace and call directly the real binary? It's pretty easy to detect by looking at /proc/self/ns/user.

[jbalme]

Can't we at least change the wrapper to check for an unshared user namespace and call directly the real binary? It's pretty easy to detect by looking at /proc/self/ns/user.

That will only work if we're root in the container, and only for things that a fake root can do (mounts inside the container, but not bind to port 80 or capture packets/trace from host processes, etc...)

Correct me if I'm wrong, but the primary use case for userns on Nix is to emulate a FHS distro, but while preserving the user's environment and Nix functionality, is it not? I do have to wonder if we're simply using the wrong tool for that.

[nixos-discourse]

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/build-a-yocto-rootfs-inside-nix/2643/28

[jraygauthier]

@matthewbauer: Might be related:
Was bitten by this when attempting to deploy a script that used sudo alongside with its dependency using nix-bundle. Found no way to make sudo available to the bundled script. Only workaround I found was to adapt my script so that sudo is not required (root access already available) and requiring my client to invoke the bundle using sudo (external to the bundle).

[nixos-discourse]

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/build-a-yocto-rootfs-inside-nix/2643/29

[futile]

Stumbled upon this as well. Thanks @rnhmjoj, your comment lead me to this general workaround:

$ PATH="/run/current-system/sw/bin:$PATH" ./command-that-runs-in-userns.sh

This might be overly generic, or might need to be adapted in combination with other environments etc., but seems like a simple workaround.

Edit:

This version might be a bit cleaner:

$ PATH=${PATH//:\/run\/wrappers\/bin/} ./command-that-runs-in-userns.sh

It simply removes all entries /run/wrappers/bin from PATH (and it assumes that there are entries before and after it in PATH) and thus doesn't otherwise reorder entries.

Edit: Of course this doesn't work for fusermount/fusermount3, because the path is hard-compiled into the executable there...:

nixpkgs/pkgs/os-specific/linux/fuse/common.nix

Line 55 in 320c5dc

export NIX_CFLAGS_COMPILE="-DFUSERMOUNT_DIR=\"/run/wrappers/bin\""

[Atemu]

I've had a look into how Flatpak solves this issue and it seems like most apps use Polkit to authenticate. SteamVR (the main reason I'm here) uses it too.

AFAIUI, it's an IPC-based privilege escalation mechanism which enables it to work in containers. It needs a privileged daemon on the host of course though.

Perhaps we could build a sudo substitute on top of polkit?

[CMCDragonkai]

This is quite annoying, it's not explained anywhere that unshare basically doesn't work on NixOS due to these wrappers. At least this should be documented somewhere and the fix is: #42117 (comment)

[Jayman2000]

I created an FHS environment so that I could run some #!/bin/bash scripts, and I ran into this bug. The workaround that I came up with is to have the user run

sudo nix-shell

, and then, once the shell starts, run

sudo -u <their-username> bash

I also created an example implementation of this workaround.

[symphorien]

see #231673

[nixos-discourse]

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/cannot-start-rootless-podman-compose-from-dynamic-user-systemd-service/34081/1

[nixos-discourse]

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/sudo-not-working-inside-a-buildfhsuserenv-no-new-privileges-flag/38971/4