A Case for Jupyter

[jcmkk3]

I've been working on and off over the last few weeks to get Janet working with Jupyter. Much of this time has been spent just gaining familiarity with C and C++ (the kernel implementation that I was attempting to use). I have mostly been spinning my wheels and realize that I'm in way over my head. Even if I do get it working, it will be too much of a hack job. While I'm abandoning implementing this myself, I did want to outline the value proposition in case someone else more capable would be interested in picking it up.

For those not too familiar, Jupyter is an ecosystem of projects built around common APIs for the purpose of interactive computing. In practice, this manifests itself most commonly as developing in notebooks (example in python). I think that there are probably many similarities to lispy staples like network repls and repl driven development, but I don't come from that world and can't speak too deeply about it. Notebooks are not the only way to work in Jupyter, however (see below). The real essence of Jupyter is a focus on interactive computing with rich outputs (it works just fine with textual output as well) and a surrounding ecosystem.

To use Janet in Jupyter, a kernel would need to be implemented. This kernel allows the rest of the Jupyter ecosystem to interface with Janet. Luckily, Janet is built with C so it can be implemented using the xeus reference kernel. The documentation is very detailed and there is a minimalistic example that can be used as a starting point. I believe that for someone with C and C++ knowledge, it would be relatively straight forward.

Below are some additional details, if anyone's interested in the greater Jupyter ecosystem. Many of these would just work by implementing a kernel, however, some would need some additional support for Janet.

If anyone is interested, I'm happy to provide any support that I can (most likely my knowledge of the ecosystem itself).

More About Jupyter

Editor Tools

• Jupyter Lab
• Jupyter Notebooks / Jupyter Lab Classic
• VSCode
• Emacs
• nteract
• QTConsole
• hydrogen

Use Case Ideas

• Teaching/experimenting
• Interactive development
• Rich run artifacts from some sort of scheduled job (see papermill)
• Documentation (interactive or exported to HTML)
• Data exploration (charting)
• Dashboards
• Other simple web apps (could be a local tool)

Other Interesting Projects

• jupytext
• jupyter widgets (example in python)
• voila
• papermill
• scrapbook

[sogaiu]

@jcmkk3 Thanks for your detailed post.

I used Python notebooks some years back when going through some fast.ai courses. My impression at the time was that it was nice for presenting results (and replicating the results of others) but for things that got larger I felt quite a bit of frustration (e.g. searching within a single browser page looking for the appropriate thing to evaluate).

Perhaps things have improved in recent times :)

Do you know of any good resources that summarize / give rationale for why the notebook / Jupyter approach is a good one?

[jcmkk3]

Hi sogaiu,

Thanks for taking a look. I think that the notebook format is often what people think when they hear Jupyter, but there are also many non-notebook uses of Jupyter. I like to think of it as a collection of specifications and tooling around an interactive-first way of programming.

If you were following along with fast.ai, then you were probably using the classic jupyter notebook environment. The standard at this point is Jupyter Lab, which is much more IDE like. VSCode is also investing heavily into notebooks and bring that whole ecosystem into the mix (see a preview). I also use neovim and send chunks of code to QTConsole.

I often start with exploring ideas in a notebook, then moving chunks of code into a file and importing them back into the notebook to perform interactive testing, documentation, and just getting a feel for the api that I'm creating. With python, I can use the autoreload functionality to keep my imported code in sync with my files. This is not part of Jupyter, but it does work well for this workflow.

I am in analytics by trade so that definitely has an effect on how I code. Data exploration is very conducive of this sort of workflow, but I've found that it also works well when I'm writing convenience scripts for little cli tools, exploring a web api, or trying to get a feel for a new package.

Some do push the notebook format pretty far like Jeremy Howard from fast.ai. He created nbdev (also a video) so that he could do everything in a notebook.

[sogaiu]

@jcmkk3 Thanks a lot for the detailed response.

It's going to take me a while to get through the content :)

Yes, I think Jupyter Lab was not what we were using at the time I was going through fast.ai material.

It was interesting to see that the youtube page for the Jeremy Howard video that you mentioned had a link to a video by Joel Grus: https://www.youtube.com/watch?v=7jiPeIFXb6U

This one seems to be at the opposite end of the spectrum so perhaps it will be worth comparing when thinking about notebooks. (Though it appears to not cover Jupyter Lab).

I hope to come to understand both the notebook and non-notebook aspects of Jupyter before too long.

Thanks again!

[andrewchambers]

Thanks for these links, this definitely looks like a fun project to investigate.