How to share files between rstudio/rocker and external folders with podman

[agila5]

Dear all, I have one question regarding the use of rocker/rstudio with podman in rootless mode.

I noticed that, by default, the external files are not writable from Rstudio (and vice versa) since only the owner of those files has write access and, by default, the owner is not the rstudio user used within the container.

By default, podman maps the external user group to root user group UID/GID 0 while the rstudio user has UID/GID 1000.

The only workaround we found is:

1. Set UMASK 002 while running the container;
2. Set UMASK 002 in my external user environment;
3. Add the mapped group rstudio to my external user groups (e.g. in my case the rstudio user maps to 494215 GID and we added that group to my groups);
4. Add the SETGID attribute to all shared folders to enable writability to all users in the same group.

Do you know if there is an easier solution?

[eitsupi]

https://github.com/containers/podman/blob/main/troubleshooting.md#2-cant-use-volume-mount-get-permission-denied

As you read this explanation, you may find it worthwhile to try changing the UID and GID of rstudio users by settings the following two options.
However, I am not sure if RStudioServer will work well in that case.

rocker-versioned2/scripts/init_userconf.sh

Lines 7 to 8 in eef72ff

	USERID=${USERID:=1000}
	GROUPID=${GROUPID:=1000}

https://www.rocker-project.org/use/managing_users/

Check the user id on the host (id) and pass this value to the docker container as an environmental variable, -e USERID=$UID, where $UID is the local user id.

[cboettig]

@agila5 yeah, this is tricky in podman as you say. In standard docker runtime this is relatively simple, since the container has root permissions we can remap the rstudio user inside the container to the desired UID/GUIDs provided as you see in the script linked above.

Actually I'm impressed you found a work-around in podman at all, and thanks for sharing it here! It does look a little cumbersome but not too bad, and may be a great help to any of our other podman users. If anyone does have a better solution in podman I'd love to see it too.

[hute37]

rstudio on podman: workaround 2

description

podman, in root-less mode, implements a different user-mapping than docker.
In podman, the (outer) user running the container is mapped as user root inside the container.
Everything is stored under the user home (~/.local) e no interaction with a privileged daemon is required.

The "mapping" problem with rstudio is caused by the introduction of an "unprivileged" container user that owns the rsession behind rstudio-server (user rstudio, uid=1000, gid=1000)
This user get remapped following /etc/subuid /etc/subgid definition, causing sharing problems with outer volume directories.

A possible solution is to run the container directly as user "root".

This setting looks like a security hole, but in practice is much more secure than the docker (dangerous) practice of unrestricted daemon access (group docker membership), more unsecure than "sudo NOPASSWD".
This setting is more secure than uncontainerized setup (unless granted, ~/.ssh keys cannot be accessed)

• https://opensource.com/article/18/10/podman-more-secure-way-run-containers
• https://opensource.com/article/18/12/podman-and-user-namespaces

workaround

To run as root, a small patch is required in the image, in rserver.conf to enable uid=0 execution:

##
# enable UID 0 user session

sed -i '/auth-minimum-user-id/d'      /etc/rstudio/rserver.conf
echo    'auth-minimum-user-id = 0' >> /etc/rstudio/rserver.conf
sed -i '/auth-minimum-user-id/d'      /etc/rstudio/disable_auth_rserver.conf
echo    'auth-minimum-user-id = 0' >> /etc/rstudio/disable_auth_rserver.conf

##
# avoid /etc/bash.bashrc message

touch /root/.sudo_as_admin_successful

to run the image a possible command could be:

podman run --rm --ulimit=host -p 8787:8787 \
    -e PASSWORD=Sec3et  -e USER=root -e USERID=0 -e GROUPID=0 -e ROOT=true  \
    -v ~/work:/root/work:Z  "image-name"

NOTE

porman rootless requires --ulimit=host option

---

example

file access

in container (rstudio terminal)

root@0d4780b2023a:~/work/vs/dve-sample-r/inst/extdata/int/temp# umask
0022
root@0d4780b2023a:~/work/vs/dve-sample-r/inst/extdata/int/temp# ls -l
total 4
-rw-r--r-- 1 root root 8 Feb 10 23:53 test-inner.R
-rw-rw-r-- 1 root root 0 Feb 12 23:51 test-outer.R
root@0d4780b2023a:~/work/vs/dve-sample-r/inst/extdata/int/temp# 

in host, as outer normal user

user@host:~/work/vs/dve-sample-r/inst/extdata/int/temp$ umask
0002
user@host:~/work/vs/dve-sample-r/inst/extdata/int/temp$ ls -l
total 4
-rw-r--r-- 1 user user 8 Feb 11 00:53 test-inner.R
-rw-rw-r-- 1 user user 0 Feb 13 00:51 test-outer.R
user@host:~/work/vs/dve-sample-r/inst/extdata/int/temp$ 

processes

in container

root@0d4780b2023a:# ps -ef
UID         PID   PPID  C STIME TTY          TIME CMD
root          1      0  0 00:12 ?        00:00:00 s6-svscan -t0 /var/run/s6/services
root         35      1  0 00:12 ?        00:00:00 s6-supervise s6-fdholderd
root        285      1  0 00:12 ?        00:00:00 s6-supervise rstudio
rstudio+    287    285  0 00:12 ?        00:00:00 /usr/lib/rstudio-server/bin/rserver --server-daemonize 0
root        336    287  0 00:12 ?        00:00:05 /usr/lib/rstudio-server/bin/rsession -u root --session-use-secure-cookies 0
root        374    336  0 00:12 pts/0    00:00:00 bash -l
root        401    374  0 00:47 pts/0    00:00:00 ps -ef

in host

user@host:~$ ps -fu $USER

UID         PID   PPID  C STIME TTY          TIME CMD
user   73175  44428  0 01:11 pts/4    00:00:03 podman run --rm --ulimit=host -p 8787:8787 -e PASSWORD=Sec3et -v /home/gp21012/work:/root/work:Z -e USER=root -e USERID=0 -e GROUPID=0 -e ROOT=true ubdems/dve-base
user   73222  73175  0 01:12 pts/4    00:00:00 /usr/bin/slirp4netns --api-socket /run/user/21012/libpod/tmp/0d4780b.net --disable-host-loopback --mtu 65520 --enable-sandbox -c -e 3 -r 4 --netns-type=path /run/user/21012/netns/cni-b37 tap0
user   73225      1  0 01:12 ?        00:00:00 /usr/bin/conmon --api-version 1 -c 0d4780b2 -u 0d4780 -r /usr/bin/runc -b /home/user/.local/share/containers/storage/vfs-containers/0d4780b/userdata -p /run/user/1000/vfs-containers/0d4780b/userdata/pidfile -l k8s-file:/home/user/.local/share/containers/storage/vfs-containers/0d4780b2/userdata/ctr.log --exit-dir /run/user/1000/libpod/tmp/exits --socket-dir-path /run/user/1000/libpod/tmp/socket --log-level error --runtime-arg --log-format=json --runtime-arg --log --runtime-arg=/run/user/1000/vfs-containers/0d4780b2023a2d00/userdata/oci-log --conmon-pidfile /run/user/1000/vfs-containers/0d4780b20/userdata/conmon.pid --exit-command /usr/bin/podman --exit-command-arg --root --exit-command-arg /home/user/.local/share/containers/storage --exit-command-arg --runroot --exit-command-arg /run/user/1000 --exit-command-arg --log-level --exit-command-arg error --exit-command-arg --cgroup-manager --exit-command-arg cgroupfs --exit-command-arg --tmpdir --exit-command-arg /run/user/1000/libpod/tmp --exit-command-arg --runtime --exit-command-arg runc --exit-command-arg --storage-driver --exit-command-arg vfs --exit-command-arg --events-backend --exit-command-arg journald --exit-command-arg container --exit-command-arg cleanup --exit-command-arg --rm --exit-command-arg 


user   73237  73225  0 01:12 ?        00:00:00 s6-svscan -t0 /var/run/s6/services
user   73277  73237  0 01:12 ?        00:00:00 s6-supervise s6-fdholderd
user   73537  73237  0 01:12 ?        00:00:00 s6-supervise rstudio
user   73764  38460  0 01:12 pts/2    00:00:00 /usr/lib/firefox/firefox -contentproc -childID 18 -isForBrowser -prefsLen 10331 -prefMapSize 253059 -jsInitLen 279340 -parentBuildID 20220106144528 -appDir /usr/lib/firefox/browser 38460 true tab
user   73788  73539  0 01:12 ?        00:00:05 /usr/lib/rstudio-server/bin/rsession -u root --session-use-secure-cookies 0 --session-root-path / --session-same-site 0 --launcher-token 473C89DE --r-restore-workspace 2 --r-run-rprofile 2

[eitsupi]

rocker-versioned2/scripts/init_userconf.sh

Lines 32 to 41 in 89d36d2

	if [ "$USERID" -lt 1000 ]; then # Probably a macOS user, https://github.com/rocker-org/rocker/issues/205
	echo "$USERID is less than 1000"
	check_user_id=$(grep -F "auth-minimum-user-id" /etc/rstudio/rserver.conf)
	if [[ -n $check_user_id ]]; then
	echo "minumum authorised user already exists in /etc/rstudio/rserver.conf: $check_user_id"
	else
	echo "setting minumum authorised user to 499"
	echo auth-minimum-user-id=499 >>/etc/rstudio/rserver.conf
	fi
	fi

I am wondering if there is no need to set a lower limit to 499 here.
I don't understand why this don't allow 0.

Related to #293, #488

[hute37]

I'm working on a project template that supports rstudio (with git support) running in a rootless podman image.

• https://gitlab.com/ub-dems-public/ds-labs/dve-sample-r

# to build images:

./build.sh setup

Here, during image build I run this script to enable internal root user in (rootless) podman container.

• init_ubs-userconf.sh

My only issue is that, when run this on an Azure VM, the useradd -m root -u 0 cause a security warning:

Detected suspicious use of the useradd command

---

# to run rstudio (in container)

./runtime.sh rstudio

• docker/r-images/Makefile (line 429)

Notes

• The rstudio session password is a random string stored in a private user file
• the internal home is mapped to a project sub directory, to enable rstudio persistence (settings, etc)
• to enable git in the internal image, copy ~/.ssh keys under mapped home
• the external workspace and data directories are mounted internally under the mapped home
• all filesystem operation are managed by current user (external uid,gid)

[eitsupi]

@hute37 Thank you for sharing this!
It would be great if we could implement the solution you shared with us here.

[seal20]

I have found a way to run rocker image rootless with podman and sharing folder (without having to run rstudio as root in the container). All praise goes to Erik Sjölund https://lists.podman.io/archives/list/podman@lists.podman.io/thread/PZZQU2YDGVBHKONNXPMDVXHEBFGWGL3W/

I am not sure I fully understand the intermediate mapping of uid, but it seems to work flawlessly.

From the mailing post slightly modifier for rocker tidyverse latest image.

#!/bin/bash
uid=1000
gid=1000
subuidSize=$(( $(podman info --format "{{ range
.Host.IDMappings.UIDMap }}+{{.Size }}{{end }}" ) - 1 ))
subgidSize=$(( $(podman info --format "{{ range
.Host.IDMappings.GIDMap }}+{{.Size }}{{end }}" ) - 1 ))
podman run -d
  --uidmap $uid:0:1 \
  --uidmap 0:1:$uid \
  --uidmap $(($uid+1)):$(($uid+1)):$(($subuidSize-$uid)) \
  --gidmap $gid:0:1 \
  --gidmap 0:1:$gid \
  --gidmap $(($gid+1)):$(($gid+1)):$(($subgidSize-$gid)) \
  docker.io/rocker/tidyverse:latest

Here is an example bash script using tidyverse:latest I use it to spawn pet container "on demand" using tidyverse:latest. I can open as many as I want on different ports, and work on different projects at the same time (I use firefox temp container to open each rstudio-rserver at the same time).

BE CAREFUL: it disables authentication of the rstudio server!

#!/bin/bash

############################################################
# Help                                                     #
############################################################
help()
{
   # Display Help
   echo "This script will create a podman container for rstudio server"
   echo "The container is based on rocker tidyverse latest"
   echo "A working directory can be mapped to the container."
   echo "Access the container on a random port or the port defined on localhost "
   echo ""
   echo "Syntax: create_rstudio_pod [-w|n|p|u]"
   echo "options:"
   echo "w     Project folder path that will be mounted in /home/rstudio/analysis folder (default: /home/$USER/tmp)"
   echo "n     Name of the container (default: random)"
   echo "u     Update the image (default: FALSE"
   echo "h     Print this Help."
   echo
}

############################################################
# Main program                                             #
############################################################

##define default
PORT=$(comm -23 <(seq 8787 9999 | sort) <(ss -Htan | awk '{print $4}' | cut -d':' -f2 | sort -u) | shuf | head -n 1)
WORKING_FOLDER="$HOME"/tmp
CONT_NAME=$(date +%Y%m%d%H%M )_rstudio

## Read argument
while getopts ":hw:n:u" option; do
    case $option in
        h) # display Help
            help
            exit;;
        w) # store working folder
            WORKING_FOLDER="$OPTARG";;
        n) # name for the container
            CONT_NAME="$OPTARG";;
        u)  UPDATE=TRUE;;
        \?) # Invalid option
         echo "Error: Invalid option"
         exit;;
   esac
done

## function to create the containers
# fix permission
# see: https://lists.podman.io/archives/list/podman@lists.podman.io/thread/PZZQU2YDGVBHKONNXPMDVXHEBFGWGL3W/
WORKING_FOLDER_PATH=$(readlink -f "$WORKING_FOLDER")

uid=1000
gid=1000
subuidSize=$(( $(podman info --format "{{ range
.Host.IDMappings.UIDMap }}+{{.Size }}{{end }}" ) - 1 ))
subgidSize=$(( $(podman info --format "{{ range
.Host.IDMappings.GIDMap }}+{{.Size }}{{end }}" ) - 1 ))

function create_con () {
        if  podman run  -dq \
            --ulimit=host \
            --name="$CONT_NAME" \
            -p ${PORT}:8787 \
            -v "$WORKING_FOLDER_PATH":/home/rstudio/analysis:Z \
            --env DISABLE_AUTH=true   \
            --uidmap $uid:0:1 \
            --uidmap 0:1:$uid \
            --uidmap $(($uid+1)):$(($uid+1)):$(($subuidSize-$uid)) \
            --gidmap $gid:0:1 \
            --gidmap 0:1:$gid \
            --gidmap $(($gid+1)):$(($gid+1)):$(($subgidSize-$gid)) \
            docker.io/rocker/tidyverse:latest >/dev/null; then

            echo "Your pet rstudio server $CONT_NAME was created."
            echo "You can access it from http://localhost:$PORT"
            echo "Your working folder is $WORKING_FOLDER"; else

            echo "Command failed"
            fi
            }

if [ "$UPDATE" = 'TRUE' ] ; then
        podman pull docker.io/rocker/tidyverse:latest
        create_con
            else
        create_con
fi

[hute37]

In the "workaround 2" above, there is a problem related to the user login (as root) made by the rstudio-server.
This change of user context "discards" additional group membership of the external "host" user.
This happens in podman by-design. It could be a security issue to enable default full group inheritance.

Running podman with --group-add keep-groups works only for initial context, but get lost in container "root" user context after rstudio-server login.

In a practical scenario, I had to share a (cifs) mounted drive among several users, all belonging the the same "data" group.
While i was able to share data in console/script R session (as root, by default mapping), disk sharing failed in rstudio because of lost group membership.

Running the server in --env DISABLE_AUTH=true could be helpful.

I'll try the script for the "data sharing problem" ...

---

warning:

a password-less terminal enabled web server must be bound to localhost on a "single-user"¹ machine, remotely accessible via ssh port forwarding.
Anything else is a HUGE security hole!

In any case (root or not), you have to consider the possible uploading your ssh GitHub pipeline enabled private keys to a malicious server, hidden inside some dependent package you install in rstudio ...

Footnotes

1. local-domain sockets could be used on a multi-user machine ? ↩