Create jovyan in Dockerfile, reset UID at startup

[parente]

• Create user jovyan with UID=1000 in the default users group in the Dockerfile
• Set group ownership of user home and conda to root to avoid 'users' group from host access when mounted
• Set stick bit on both paths so root owns subdirs too
• Change jovyan UID if NB_UID is specified and is not the default 1000

@costerwi Can you give this a shot and see if it retains what you're looking for via the NB_UID runtime env var plus fixes the problems you were correcting in your latest PRs? Meanwhile, I'm going to see if it fixes both the bash and julia kernel install problems people have reported with the current setup.

[parente]

Argh, docker fun:

drwx--S--- 4 jovyan root 4096 Sep 11 02:58 share
jovyan@a7de91967404:~/.local$ cd share/
bash: cd: share/: Permission denied

So after installing ijulia, the /home/jovyan/.local folder gets created. Later, when the script switches the jovyan uid to 1005 from 1000 (for testing), the jovyan user can no access the .local folder even though that user clearly owns the folder. All other folders (e.g., .jupyter) it has no problems with. Poking around a bit in the docker issues, it looks like this could be a storage driver specific thing (e.g., AUFS).

I tried various workarounds to no avail. Might be at a dead end.

[parente]

Found slight workarounds for the docker permission weirdness for both ijulia and bash kernel. Will kick off builds of all the other subimages tonight and see if anything else breaks with this change.

[parente]

I've locally rebuilt every stack using the new minimal-notebook image defined by the Dockerfile changes in this PR. I've confirmed the changes fix #30 and make #25 possible without any serious hoop-jumping. I've run each image in the following two ways (i.e., with and without a UID change for jovyan):

make dev STACK=scipy-notebook'
make dev STACK=scipy-notebook DARGS='-e NB_UID=1005'

In every case, I've confirmed the containers to be working properly, meaning:

• all expected kernels appear and function
• jovyan has full access to its home directory
• jovyan can conda install / upgrade anything in any conda environment

Remaining problem: I don't know what I don't know. For instance, does setting the default group for jovyan to users to avoid having to reset groups everywhere when the UID changes cause a problem for certain host mounted use cases?

I'm just looking to be a bit more careful up front this time to avoid having to come back to updating the base again and again with slight tweaks for different user / mount scenarios.

@rgbkrk Can I get a quick sanity check?

[rgbkrk]

Looking really good. Only thing I'm wondering is what the reasoning is for ending as root user in the final user bound images.

[parente]

Turns out that if you don't switch back to root, the CMD from the base image launches as the last USER. 👀

[costerwi]

I agree; this has great promise. Simpler is better!

[costerwi]

Why set the group ownership to root?

[parente]

Exactly what I wanted to ask you. :) The -N on useradd puts the jovyan user in the default users group (GID=100). Anything that user touches in a host mount home or work dir will be accessible by GID=100 on the host unless the root of the mount has proper permissions. To head off any "Oops, I didn't think to protect against a container / host group overlap as well as UID overlap", I figured root was a safe bet.

But I agree it's quirky. I'm open to suggestions here.

[costerwi]

I didn't notice the -N switch but I like the effect. Coincidentally, I am using the users group on my host CentOS system where the GID is also 100 so I'd prefer that over root but I respect that it may not satisfy all applications.
This must be a common Docker problem... Some research may be required for a sensible suggestion.

[parente]

Long running issue with lots of solutions but no clear best practice:
moby/moby#7198

The real solution probably lies in docker + kernel support for user namespace mapping: moby/moby#12648

My gut tells me we should treat whatever we do as a patch for the time being and so simpler is better. We can state the limitations and exactly what's going to happen in host mounts in the READMEs. My vote is for calling out user=jovyan (fixed), primary group = users (fixed), UID=configurable via env var NB_UID (default: 1000) and GID=100 (fixed).

[rgbkrk]

Turns out that if you don't switch back to root, the CMD from the base image launches as the last USER.

I guess I assumed that was the opinionated behavior we wanted to provide people, so they don't run as root by default.

[parente]

supervisord runs as root and changes to the jovyan user. If / when we switch away from that (issue #24), it'll need to change, yes.

[rgbkrk]

Oh durr. I totally forgot about that. So sorry.

[rgbkrk]

👍

[parente]

Last set of pushes now has:

• users group set for /home/jovyan and /opt/conda (KISS)
• READMEs stating defaults for jovyan (uid, group, gid, and what it owns)
• Fix for r-notebook

Going to put this into master and trigger the DH builds. datascience-notebook will fail to rebuild because IJulia has a regression at the moment and I can't find a way to point at a specific commit in Julia.