Enhancement idea - unthrowables

[joemarshall]

I've done some work with exception unthrowing which allows doing roughly the same stuff with keyboard processing but without having to use shared memory - it instead throws an exception to jump right out to javascript and allows messages to come in before reconstructing the python stack to go back in where it was originally.

There are some disadvantages to my approach, most obviously that you can't interrupt in a with statement or in a try block that uses a finally, but on the plus side it doesn't require anything complicated in relation to server or browser side permissions, and maintains the javascript execution model nicely. I needed it because I have students who have iOS devices or use Safari on Mac, so they don't have shared memory support.

I think it would be nice to have these two complementary approaches integrated into pyodide_interrupts somehow. i.e. pyodide interrupts allowing:
a) Regular interrupt without throwing out
b) Regular interrupt that jumps right out to javascript where current python state can be resumed or cancelled.
c) On demand jump out to javascript with a parameter (I use this for things like doing time.sleep, and it could be used for doing synchronous IO in python when the underlying javascript is async), again current python stack can be resumed if wanted.

The code is all here
https://github.com/joemarshall/unthrow
and there's a demo on here:
https://joemarshall.github.io/websensors/pyodide/console_async.html
which is just a normal console, but you can do things like

import time
time.sleep(100)

while True:
  print("WOO")

while True:
  pass

and all should work and be interruptable with ctrl+c without killing the whole python state.

[hoodmane]

I think this package in it's current version is a bit obsolete because better functionality has been directly integrated into pyodide. I think this might be better as an issue on pyodide. Currently in pyodide I have patched in an interrupt into the main loop of ceval:

https://github.com/iodide-project/pyodide/blob/a5e21ba75a6debc235abe027176c2cf5b894ce68/cpython/patches/0001-Add-pyodide-callback.patch#L138

I added it for every "slow dispatch" function, but like the rest of the CPython main loop maintenance stuff it is skipped by FAST_DISPATCH. However, most of the rest of the thread state stuff is behind the eval_breaker flag which is normally set by the OS / a different thread when stuff needs to be done and never gets set. I also added the function PyPyodide_SetPyodideCallback which sets a C function to execute. Perhaps it'd be good to set up a system so that you could register a Python (or javascript?) function there.

Another issue is integration with PyErr_CheckSignals so that the interrupt system works correctly even when native C extensions are doing work (at least if they work with the interrupt system natively). With this we could make a pyodide.checkInterrupt javascript function that allows "native" javascript extensions to also cooperate by periodically calling this. I have an implementation of this here:
pyodide/pyodide#1294

Regular interrupt without throwing out

Sounds like a useful thing to have. Again, you can set any one function like this with PyPyodide_SetPyodideCallback but it has to be a C function and it can't interact correctly with pyodide.setInterruptBuffer since that also sets the one PyodideCallback. I'm not sure what the best fix for this is: maybe set one C function callback that maintains a list of stuff to do? Some of the stuff can be spaced out on clocks if it's expensive.

resumable interrupt

My main concern is how you deal with situations where the native browser stack has stuff on it that can't be rewound. Do you have any way to inspect the stack for this and bail out if it's not going to work? It seems to me that it could really trash everything if the native stack and Python stack get out of sync (though I haven't thought very carefully about what sorts of problems you could see). I'd be much happier if we could raise an error "Stack could not be rewound" than if it fatally crashes pyodide.

[hoodmane]

By the way, this unthrowables thing is really really cool! (But also scary dark magic.)
It's a shame Safari can't implement Atomics and then we could use threads to do this stuff like normal people.

who have iOS devices or use Safari on Mac

People really use repls / program on their phones? I guess they must, but it doesn't seem very pleasant. Of course the Safari on Mac issue is much simpler, it's probably possible to ask them to use Chrome or Firefox instead.