Rebase on jupyter/minimal-notebook:4.0

[parente]

@rgbkrk First crack at updating docker-demo-images to 4.0. Need many more eyes and testing, particularly against how it'll be run at tmpnb.org.

Key points

• Now based on the jupyter/minimal-notebook:4.0 image from docker-stacks to DRY
• It builds to completion with all kernels showing when the notebook server starts. Fixes the various defects currently open against this repo like docker build . broken on java install #63, bash kernel not loading. Need to update to two step process #64, docker build -t <tag> . broken still. Line 36... kernelspec install-self --user #68.
• No version pinning throughout (yet).
• Inverts the original Dockerfile install order so that site specific customizations come after the install of all the demo-wares. This change means site-specific install steps can later be split off as a subimage, with a Dockerfile inheritance chain like jupyter/minimal-notebook -> jupyter/demo -> jupyter/tmpnb-demo. Then other folks can simply start FROM jupyter/demo and add their own site customizations without having to rebuild the massive middle image.
  • But for the time being, I haven't done that split so there's just one Dockerfile. Trying to KISS until everything works. Splitting is an easy later refactor.

I added comments galore throughout the Dockerfile.

Known problems

• It won't work if the static_url_prefix in the jupyter config is in play because the CDN site does not have the piece parts yet. So for testing, I simply removed that part of the config for now. (And since it's near the end of the Dockerfile now, I didn't have to wait 3 hours to test again! Yay!)
• The IHaskell widget demos don't work. There's kernel comm errors in the JS log. I did have to remove the version pinning on ihaskell-widgets to get the install to work but it looks that version installed anyway (and might be the only released version anyway). Briefly perusing the IHaskell commits, maybe this is still a WIP for 4.0 compatibility? Or simply unreleased? /cc @gibiansky @sumitsahrawat
• The Julia logo doesn't appear. JS console reports http://mydomain:8889/kernelspecs/julia-0.3/logo-64x64.png Failed to load resource: the server responded with 404 Not sure why considering the logo is right where it needs to be. For comparison, the python2 logo is in its proper spot and loads just fine:

jovyan@266f76e05efd:~/.local/share/jupyter/kernels/julia-0.3$ ls -l                     
total 12                                                                                
-rw-r--r-- 1 jovyan users  195 Sep 13 05:00 kernel.json                                 
-rw-r--r-- 1 jovyan users 1310 Sep 13 05:00 logo-32x32.png                              
-rw-r--r-- 1 jovyan users 2843 Sep 13 05:00 logo-64x64.png                              
jovyan@266f76e05efd:~/.local/share/jupyter/kernels/julia-0.3$ cd ../python2                              
jovyan@266f76e05efd:~/.local/share/jupyter/kernels/python2$ ls -l                       
total 12                                                                                
-rw-r--r-- 1 jovyan users  168 Sep 13 04:58 kernel.json                                 
-rw-rw-r-- 1 jovyan users 1084 May  8 20:15 logo-32x32.png                              
-rw-rw-r-- 1 jovyan users 2180 May  8 20:15 logo-64x64.png                              

Unknowns

• Does it work with tmpnb? I know jupyter/minimal-notebook does because I've tried it recently.
• Anything else broken? I didn't test every kernel with every sample notebook.

DevOps impact

• The jupyter static dir is now /opt/conda/lib/python3.4/site-packages/notebook/static/custom/ which appears to be nginx mounted according to the original Dockerfile comments. That'll need to change on deploy.

[rgbkrk]

Oh my gosh, thanks.

Our cdn.jupyter.org setup is nginx + static files, fronted by fastly. We used to serve directly out of nginx, especially when we were hosting the latest master of ipython/ipython for the longest time. With a pinned version, this changes.

With this PR, we'll have to update the try directory. Example: http://cdn.jupyter.org/notebook/try/custom/custom.js

We should probably rely on versioning there too.

[rgbkrk]

This is the only part I'm questioning, but I'm assuming the originating files already have it and we're relying on it to stay consistent?

[parente]

The jupyter/minimal-notebook stack has all this and more: https://github.com/jupyter/docker-stacks/blob/master/minimal-notebook/jupyter_notebook_config.py

[rgbkrk]

The Julia logo is a known issue. I patched that in IJulia not so long ago, seems like we were waiting on a release.

[rgbkrk]

One deprecation note: should we migrate the tmpnb example to using jupyter/minimal-notebook and delete the jupyter/minimal image from docker hub? The alternative is making jupyter/minimal just "FROM jupyter/minimal-notebook" or set up hooks to create minimal from the same spot.

[ghost]

Thanks for the ping @parente.

IHaskell and the ihaskell-widgets package both support 4.0 now in the github master branch. The new widgets added to IPywidgets, i.e Valid, Proxy and ProxyPlace, have yet to be implemented. I will work on them ASAP, and then we can get a new release out. /cc @gibiansky

[rgbkrk]

Would it be ok if we rely on this and for Haskell to not be supported until it's ready?

[parente]

@rgbkrk As I'm learning more about tmpnb, dawns on me that the containers are running the notebook server as root not jovyan because of the way the command is passed.

su jovyan -c '--command="su jovyan -c '/opt/conda/bin/ipython notebook \
                --NotebookApp.base_url={base_path} \
                --ip=0.0.0.0 \
                --port={port}"'

should do the trick on launch. Or we can wait for jupyter/docker-stacks#24 to get resolved.

[rgbkrk]

Would specifying the user as part of the call to docker-py work well? (same as -u from docker land).

[parente]

Yes it will, but is that an exposed option to orchestrate?

[rgbkrk]

No, but it could always be added. It's too bad we can't just run straight from the notebook server. Open to tini as well.

[ghost]

New releases: ihaskell-0.8.0.0 and ihaskell-widgets-0.2.0.0.

[parente]

Sweet! Will update later today and try.

[parente]

Confirmed ihaskell widgets working. Rebased on master to fix conflict. Looking into tini over in docker-stacks for simplifying container startup.

[parente]

One deprecation note: should we migrate the tmpnb example to using jupyter/minimal-notebook and delete the jupyter/minimal image from docker hub? The alternative is making jupyter/minimal just "FROM jupyter/minimal-notebook" or set up hooks to create minimal from the same spot.

How about marking jupyter/minimal DEPRECATED on docker hub in its description, referring people to jupyter/minimal-notebook, and leaving the deprecated image there for a time before deleting it?

[rgbkrk]

How about marking jupyter/minimal DEPRECATED on docker hub in its description, referring people to jupyter/minimal-notebook, and leaving the deprecated image there for a time before deleting it?

That's fine. As long as we also change the tmpnb README's to point to jupyter/minimal-notebook.

[parente]

Round-up:

• Swap tini in for supervisord docker-stacks#38 needs merging to master, then backport to 4.0 branch
• jupyter/minimal-notebook needs rebuild on dockerhub (along with all other images)
• cdn.jupyter.org needs update of resources, maybe versioned
• /opt/conda/lib/python3.4/site-packages/notebook/static/custom/ is now the dir with the custom files within container (what needs to change? tmpnb orchestrate option to copy out files?)
• After Swap tini in for supervisord docker-stacks#38 merges, tmpnb start command for orchestrate should be something like:

--command="start-notebook.sh \
                --NotebookApp.base_url={base_path} \
                --port={port}"

Upon launch, the container entrypoint should be tini (as specified in the base image) while the initial command will be start-notebook.sh. That script automatically switches to the jovyan user and passes all other arguments through as args to jupyter notebook.

• Deprecate jupyter/minimal in favor of jupyter/minimal-notebook both in the jupyter/minimal Docker Hub description and in the tmpnb Makefile and README.

[parente]

New ihaskell failure today during build (whereas last week all was well).

cabal: Error: some packages failed to install:
ihaskell-widgets-0.2.0.0 depends on singletons-2.0.0.1 which failed to
install.
singletons-2.0.0.1 failed during the building phase. The exception was:
ExitFailure 1
The command '/bin/sh -c cabal update &&     cabal install cpphs &&     cabal install gtk2hs-buildtools &&     cabal install ihaskell-0.8.0.0 --reorder-goals &&     cabal install ihaskell-widgets HTTP Chart Chart-cairo &&     ihaskell install &&     rm -fr $(echo ~/.cabal/bin/* | grep -iv ihaskell) ~/.cabal/packages ~/.cabal/share/doc ~/.cabal/setup-exe-cache ~/.cabal/logs' returned a non-zero code: 1
make: *** [build] Error 1

Update Doh. https://hackage.haskell.org/package/singletons-2.0.0.1/reports/1

[rgbkrk]

Today I'll carve out some of these as well as give you access to the CDN @parente. We do have versioned assets up but we haven't publicly declared what's available.

[rgbkrk]

Ah but I see we'd want to version the try content from the CDN.

[parente]

Not clear on what that versioning scheme would look like or how it relates to the options on the launching of the tmpnb containers.

If you've got cycles in the late afternoon today, we can pair on it if you'd like.

[rgbkrk]

2-4pm central I'm packed, but any time on either side of that wide open.

[parente]

Might be able to do 1 PM CST. Let's take off issue to gitter and see what time works out. I'll ping there.

[parente]

https://jupyter.hackpad.com/Ops-Log-9242015-DObACmQcdks

@rgbkrk I think this is ready to go in to match the "latest" image now on Docker Hub. (Unless you don't want to merge til after the try site is updated.)

[rgbkrk]