docker image for kubernetes jupyterhub

[KrisThielemans]

We're trying to get an image ready to launch on jupyterhub with kubernetes. Current attempt is on https://github.com/paskino/SIRF-SuperBuild/tree/add_jovyan_user/docker

This does not use docker-compose, so our .yml files are not used.

Difficulties

• sirf_env.sh is not sourced
• the jupyter notebook started in /home/jovyan
• gadgetron is not started (but we can tell the user to do that, as they need to do it on the VM anyway)
• user permissions for /opt (read-only) - see related discussion in docker images and permissions problems for /devel #464, docker: slow startup of container the very first time #555 and docker: no longer chown /opt #558 although they concentrate on using docker-dompose - causing problems (solvable by doing sudo chown -R jovyan:users /opt/SIRF-Exercises/)
  • would not be able to modify notebooks
  • cannot download data

    $ /opt/SIRF-Exercises/scripts/download_data.sh -p ~/data ~/data_download
    ....
    creating data_path.py in /opt/SIRF-Exercises/lib/sirf_exercises/data_path.py
    ./download_data.sh: line 204: /opt/SIRF-Exercises/lib/sirf_exercises/data_path.py: Permission denied
    

• currently starting JupyterLab, where'd prefer to use the classic interface (can be done by adding tree at the URL) (This is a jupyterlab (system-wide!) setting, not under our control
• cannot plot anything (This disappears when launching classic jupyter)

plt.figure()
plt.plot(1,2)
Javascript Error: IPython is not defined
[<matplotlib.lines.Line2D at 0x7f46d0e47580>]

[paskino]

I built the image with docker-compose. The problem is how the image is run, which we don't know. I believe that with a lengthy docker terminal command one can achieve the same result as with docker-compose.

I can make such that the directory where we clone the notebooks are rw for the user jovyan.

[KrisThielemans]

some information on the set-up.

• it uses KubeSpawner
• Information on configuration of JupyterHub:
  https://zero-to-jupyterhub.readthedocs.io/en/latest/jupyterhub/customizing/user-environment.html

[KrisThielemans]

Suggestion was to use nbgitpuller, as suggested by the JupyterHub doc to get and sync the notebooks.

NBgitpuller syncs from github using the following strategy: https://jupyterhub.github.io/nbgitpuller/topic/automatic-merging.html.

Given our problems with notebooks and conflicts, even with nbstripout, I'm not sure if this will work. @paskino, we will have to be careful with this.

An alternative is to just copy /opt/SIRF-Exercises, which is possible via a poststart hook, see https://zero-to-jupyterhub.readthedocs.io/en/latest/jupyterhub/customizing/user-environment.html#about-user-storage-and-adding-files-to-it

[paskino]

I tried plotting and it worked

[paskino]

A possible solution to the LD_LIBRARY_PATH is adding it in the system /etc/environment

Another option is to install in /opt/pyvenv (in the conda environment). However, conda does not set LD_LIBRARY_PATH but uses RPATH, I believe.

[KrisThielemans]

you can set env variables for conda environments, see here. However, I'm not sure if kubespawner actually activates the env. (I guess so, as you can run sirf)

However, conda does not set LD_LIBRARY_PATH but uses RPATH, I believe.

It uses RPATH when you build executables/libraries. This has nothing to do with us at the moment.

[paskino]

I believe SIRF compiles python packages with swig and installs them in the python module directory so they are found. CIL instead doesn't do that and our (little) C++ library needs to be found.

[KrisThielemans]

I believe SIRF compiles python packages with swig and installs them in the python module directory so they are found. CIL instead doesn't do that and our (little) C++ library needs to be found.

This confuses me a bit.

• swig doesn't compile anything at all. It just creates a C++ file. We tell CMake to then compile, see e.g. here.
• LD_LIBRARY_PATH applies for run-time stuff, not compilation stuff. I see that CIL has a number of shared libraries, so their location has indeed to be in the LD_LIBRARY_PATH (just like the ITK, ISMRMRD and Gadgetron library)

A possible solution to the LD_LIBRARY_PATH is adding it in the system /etc/environment

I think far preferable would be to add a file /etc/ld.conf.d/sirif-cil.conf with the install location and run ldconfig (as root). See e.g.
https://unix.stackexchange.com/questions/425251/using-ldconfig-and-ld-so-conf-versus-ld-library-path

[KrisThielemans]

what about the radical solution to install in /usr/local? Stuff like path and LD_LIBRARY_PATH is no longer needed...

[paskino]

OK, it seems like this https://github.com/paskino/SIRF-SuperBuild/releases/tag/jupyterhub_v0.0.0 is the first jupyterhub config that works. We'll build on that.

[KrisThielemans]

This is currently still in the above fork. We will merge this for the next version.