Make a docker-compose reference implementation of ScriptedForms with JupyterHub

[SimonBiggs]

Create a reference implementation with Docker and docker-compose which includes JupyterHub and likely a postgres installation, (or maybe something like elasticsearch, I'm not sure).

Set up a neat default way for each result entered into the scriptedforms to be stored within the database and have the data be labelled with the time filled out and the JupyterHub user who fill out the form.

Then, included with it a second JupyterLab extension based on holoviews http://holoviews.org/gallery/index.html which helps display the results from previously filled out ScriptedForms from all users. Data points should be able to be hovered and it should be possible to see which JupyterHub user filled in that data item.

[SimonBiggs]

@alysivji I was reading some of your blog posts and I noticed quite a speciality in docker-compose. Might you be keen on working on this with me?

[alysivji]

@SimonBiggs Sure, would love to help out! I'm going to an Open Source jam session tmrw so this works out pretty well timing-wise.

I heard about ScriptedForms on an old episode of Python Bytes. Haven't gotten around to trying it out just yet, will work thru the Quickstart tmrw.

Could I get a bit more context on what is required for this issue to be resolved?

[SimonBiggs]

The target use case is to have a tool where someone can create a ScriptedForms server within their work intranet. Having multiple users, each with a JupyterHub sign in users can fill out the forms from anywhere in the centre and have their results saved. The results should be saved in such a way that a history of the results is then searchable within the JupyterLab interface for trending the historical records.

The forms themselves should be accessible via a persistent volume shared via samba so that windows machines can edit the ScriptedForms that are on the server.

The aim is to have a JupyterLab integrated, flexible, modular, version of something like the following:

http://qatrackplus.com/index.html#features

[alysivji]

Went thru the Quickstart yesterday. Really cool project; wish I had something like it 2-3 years ago!

So I'm envisioning a cookiecuter template that can be used to bootstrap an intranet installation.

Spin up the following containers:

• jupyterhub (starting to read the documentation on how to do this)
  • will require a way to set up accounts that users can modify as needed
  • samba for windows mounts
• scriptedforms server
  • haven't look into how this connects with the notebook server (don't think i'll need to worry about it too much)
• postgres db
  • is this for scriptedforms applications?
    • set schema?
  • storing and visualizing client data?
    • have instructions on how to load their data

All the work will be stored in a docker directory in the main repo. Will have a docker-compose.yml in the main folder. Could have a Makefile provide a simple api abstraction over docker-compose.yml as most use cases will only require 2-3 commands.

[SimonBiggs]

Yes that sounds perfect. By the sounds of it I think I'll learn a few things from this. That will be nice :).

Postgres db is for people to use to store data for their own ScriptedForms applications. So for now, just let it sit there. Down the track, separate from this issue, I will make a standard way for ScriptedForms to save to postgres, and then a standard way to display data. But for now if postgres is just sitting there that will be sufficient.

The ScriptedForms server is actually just a notebook server. One and the same. In fact, once ScriptedForms is installed you can run a notebook server and then change the url to /scriptedforms/use/formname.md. Also visa versa, you can run ScriptedForms and change the url to /tree and you're back to the notebook.

As such, JupyterHub and the scriptedforms servers might need to be more tightly intertwined. I briefly read something about a docker spawner for JupyterHub allowing each user to use their own notebook server docker image, and then also read about a docker spawner using "docker in docker" so that it can work when JupyterHub is itself a docker container. I can't remember where I read that though.

Using a coockicutter, and creating a makefile API around docker-compose sounds perfect.

[alysivji]

Awesome. I'll do a first pass without JupyterHub and we can go from there.

Looking forward to dealing with docker inside docker! My brain is already hurting (in a good way :D)