.NET Interactive Notebooks should have a programmatic way of executing them

[SQLvariant]

Is your feature request related to a problem? Please describe.
.NET Interactive Notebooks should have a programmatic way of executing them.

Describe the solution you'd like
When executing .NET Interactive Notebooks in this manner, the Notebook should first be cloned, and results stored in the new copy of the Notebook.

This opens up new opportunities for how notebooks can be used. For Example:

You can already execute SQL Notebooks using the Invoke-SqlNotebook command (available in the SqlServer PowerShell module). When you execute a SQL Notebook this way the Invoke-SqlNotebook command will automatically make a copy of the Notebook you give it, execute all the code cells inside it, and store the results with the code cells. This allows you to share open the output Notebook and have all the documentation & results in a single file.

This makes the output Notebook extremely portable. If someone is running into an issue, they can share the commands they’re running & the precise results they received with me. When using Notebooks for troubleshooting, this means someone can also share the exact error message.

For .NET Interactive Notebooks, hopefully there could be a single programmatic way of executing them, which would work no matter which of the .NET Interactive kernels the user was planning to use.

Once something like this this is in place, it makes it easier for .NET Interactive users to head in the direction of what the Papermill folks have done. Here’s an excerpt from their GitHub page, talking about parameters.

This opens up new opportunities for how notebooks can be used. For example:

Perhaps you have a financial report that you wish to run with different values on the first or last day of a month or at the beginning or end of the year, using parameters makes this task easier.

Do you want to run a notebook and depending on its results, choose a particular notebook to run next? You can now programmatically execute a workflow without having to copy and paste from notebook to notebook manually.

Describe alternatives you've considered
Doug Finke has already created a very useful Invoke-ExecuteNotebook command, which will execute a Notebook using the PowerShell kernel & accepts parameter input. ( repo | short video demo ). The Invoke-ExecuteNotebook command also lets you save the resulting output Notebook file either locally, in a Gist, or in an Azure Blob store.

[Szeraax]

Wow, this is so interesting to read and is VERY similar to my comment yesterday about the limitations of the .dib format not saving output in any way.

One key difference between what I was thinking of and what you propose here is that I suggested storing n-runs within the dib file itself and letting documentation + code + historical output all live in 1 place (thankfully cells are collapsable!)

How do you like the current Sql notebook implementation of copying the file on each run? How would you feel about instead the results getting saved within the file? Same either way to you or prefer one over the other?

In my mind, I worry about getting too many files if it copies every time. The other question I have is what .net compiled notebooks (standalone executables) should do when run on the command line. Should they create a copy of the markdown and the outputs to the new file?

[Szeraax]

@SQLvariant btw

[teauxfu]

In case anyone ends up here looking for a way to do this, here's a possible workaround that involves using a .ipynb file and Papermill as OP mentioned: https://stackoverflow.com/a/75941908/13090245. Probably good enough if you're not mixing multiple languages.

I agree that this would be an excellent addition to the core tool though. Something like dotnet-interactive run notebook.dib output.dib