Instructions/Image to build “locally” on a 2i2c hub

[jbusecke]

I am hearing from researchers at LEAP that they have trouble building images locally.

I realize this can be hard on a variety of machines (windows seems to cause trouble), so here is a suggestion:

Can we provide a docker image with docker, repo2docker, etc that we could run on a 2i2c hub, so that user can create and test their repo before pushing to github (and thus creating a potentially big image in the registry via the CI)?

Happy to help with testing this

[yuvipanda]

@jbusecke I think the path forward here is to deploy https://2i2c.org/blog/2024/jupyterhub-binderhub-gesis/ to LEAP.

I see you should have access to https://imagebuilding-demo.2i2c.cloud/. Log in and try it out, particularly the 'build your own image' functionality. Currently it requires you to specify github repos in the form of /, but that's being worked on. This will build and run the same way repo2docker-action runs, and is the long term path forward.

A possible goal (working alongside @batpad) is to deploy this on the NASA VEDA hub by end of next quarter. We can target LEAP as well to make this possible.

This is a better path forward than trying to get docker working within docker containers on the hub.

What do you think?

[jbusecke]

This is great to see @yuvipanda!

Quick comments/questions:

• Will the user have to specify the repo each time, and the image will be rebuilt on the fly? Or will built images be cached somehow?
• Depending on the answer to above, I think about two things: Longterm reproducibility (e.g. a docker image build from a non-locked conda env file might be different depending on the time it was created) and image storage utilization (longer term question, but how can we ensure to only retain tested tags of a given image).
• I am actually not able to get the demo working (I tried with https://github.com/ocean-transport/scale-aware-air-sea/ (which has a Dockerfile). Could you given an example how this would work?

Thank you so much for working on this!

[yuvipanda]

@jbusecke if i type ocean-transport/scale-aware-air-sea into the 'github repository' box (instead of the whole url) it works. There's UX work to be done here but the core functionality works. Try that again?

There's optimistic caching, same way as mybinder.org. The git ref (which is currently just HEAD so main or master, but will be user inputtable soon) is used to tag the built image. However, they may get cleaned up over time, and a rebuild will be triggered. Also each hub has its own cache so that would also cause rebuilds as the repository is used across different hubs.

So the answer to long term reproducibility would be to use lockfiles! I think that's kinda the same regardless of using repo2docker-action or using repo2docker in this dynamic fashion.

I hope being able to deploy this throughout will help us expand the pool of people who can make images