AsyncModalDagExecutor raises when run from a jupyter notebook

[TomNicholas]

returns

╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮
│ /tmp/ipykernel_495009/1422487590.py:1 in <module>                                                │
│                                                                                                  │
│                                                                                                  │
│ /home/tom/Documents/Work/Code/cubed/cubed/core/array.py:112 in compute                           │
│                                                                                                  │
│   111 │   │   """Compute this array, and any arrays that it depends on."""                       │
│ ❱ 112 │   │   result = compute(                                                                  │
│   113 │   │   │   self,                                                                          │
│                                                                                                  │
│ /home/tom/Documents/Work/Code/cubed/cubed/core/array.py:300 in compute                           │
│                                                                                                  │
│   299 │   _return_in_memory_array = kwargs.pop("_return_in_memory_array", True)                  │
│ ❱ 300 │   plan.execute(                                                                          │
│   301 │   │   executor=executor,                                                                 │
│                                                                                                  │
│ /home/tom/Documents/Work/Code/cubed/cubed/core/plan.py:154 in execute                            │
│                                                                                                  │
│   153 │   │   │   │   [callback.on_compute_start(dag) for callback in callbacks]                 │
│ ❱ 154 │   │   │   executor.execute_dag(                                                          │
│   155 │   │   │   │   dag, callbacks=callbacks, array_names=array_names, **kwargs                │
│                                                                                                  │
│ /home/tom/Documents/Work/Code/cubed/cubed/runtime/executors/modal_async.py:178 in execute_dag    │
│                                                                                                  │
│   177 │   def execute_dag(self, dag, callbacks=None, array_names=None, **kwargs):                │
│ ❱ 178 │   │   asyncio.run(                                                                       │
│   179 │   │   │   async_execute_dag(                                                             │
│                                                                                                  │
│ /home/tom/miniconda3/envs/cubed/lib/python3.9/asyncio/runners.py:33 in run                       │
│                                                                                                  │
│   32 │   if events._get_running_loop() is not None:                                              │
│ ❱ 33 │   │   raise RuntimeError(                                                                 │
│   34 │   │   │   "asyncio.run() cannot be called from a running event loop")                     │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
RuntimeError: asyncio.run() cannot be called from a running event loop

This SO answer seems to suggest that this could be fixed by changing this line in the executor

https://github.com/tomwhite/cubed/blob/74145fb826d21496ca86b4e64b0fcb1c55d8a931/cubed/runtime/executors/modal_async.py#L178

[tomwhite]

I tried some of the suggestions from the SO page, but then I get RuntimeWarning: coroutine 'async_execute_dag' was never awaited, and can't see an obvious way to avoid it.

It looks like the easiest fix may be to use nest_asyncio by putting the following at the top of the notebook:

import nest_asyncio
nest_asyncio.apply()

What do you think?

BTW AsyncPythonDagExecutor has the same problem and is useful for reproducing without using Modal.