Enable automated execution of notebooks

[jonsequitur]

Automation of notebooks (i.e., direct execution from the command line) is a common use case and is available to .NET Interactive Jupyter users through Papermill. Directly supporting automation has the potential to simplify authoring and deployment for .NET-only users.

Goals include:

1. Make code as portable as possible between the interactive authoring and automation modes.

2. Provide a way to accept input such that the same code works on both the command line and during interactive use, and is capable of displaying command line help, informative errors, etc.

   Example:

   #!csharp
   #!args
   
   ParseResult ParseCommandLine(string args)
   {
       var root = new RootCommand
       {
           new Option<int>("--number-of-things"),
           new Argument<FileInfo>()
       };
   
       return root.Parse(args);
   
   }
   
   #!csharp
   
   var something = DoSomethingWith(Args<int>("--number-of-things")); 

   In this example:

   • Command line automation would trigger argument parsing and store the parsed values for use by all subkernels.
   • Interactive use would trigger a user prompt for the complete command line input.
   • Identical strings, whether provided on the command line or via a user prompt, should result in identical parsed values when accessed in subsequent submissions.

3. Output formats should be specifiable so that the automation run can control, for example, output formats (e.g. plaintext, HTML, Markdown, .ipynb)

[jonsequitur]

Related: #890.

[Szeraax]

this is very interesting! I was only coming at this from an powershell approach where I only ever want to get plaintext output saved, but if I have an automated run of my notebook that generates a graphic, I would like to be able to get that out and into the file system too.

Having said that, I see less use for taking output cells and converting them to mark down. Are people likely to use cells to generate markdown format data? Or other rich text? Same question for HTML, and even though VSCode probably does ultimately display all output cells as HTML and could emit that type out, it would mean that running compiled notebooks now need a LOT of rendering capabilities in place if you want that feature to have parity across vs/dotnet/compiled.

I would suggest scapping md and html output. Stick with plain text (and possibly graphics) output in a simple manner so that compiled notebooks can stay lean, but still provide all the same capabilities as a .dib run via dotnet or VSCode.

As for outputting output, seems like it could be nice to just take all output cells, concatenate them, and then spit it out to stout. Graphics being output would be a bit more complicated and I'm not sure what would likely be good there.

[Regenhardt]

Sending the whole output to stdout might be a good start as that already lets people handle it themselves, like saving it to a file (dotnet run generateGraph > graph.png) or use it as a test (dotnet run e2e-tests.dib in a CI/CD pipeline).

[jonsequitur]

Relevant to this issue, dotnet-repl provides a number of notebook automation features.

https://github.com/jonsequitur/dotnet-repl