Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Running a python model instead #100

Draft
wants to merge 5 commits into
base: main
Choose a base branch
from
Draft

Running a python model instead #100

wants to merge 5 commits into from

Conversation

Peter9192
Copy link
Member

@Peter9192 Peter9192 commented Jan 14, 2025

I've been playing around to try and run python code from the web application. As such, we could have an analogous implementation in Python and use the nice web application to manage and analyse our experiments.

There are multiply ways to achieve this:

  • Pyodide: running a python interpreter in the browser.
    Advantage: easy (?!), actual python code is being executed
    Disadvantage: long load time as it needs to initialize the full pyodide environment (though only inside the web worker)
  • Transcrypt: transpiling the python code to js/ts: https://www.transcrypt.org/
    Advantage: easy and fast, code can be optimized during build
    Disadvantage: for the purpose of "proof of concept", this is not much different than just writing js/ts directly
  • Compiling python code directly to WASM, e.g. something like py2wasm
    Advantage: skips the overhead of loading pyodide and interpreting the python code; could scale to larger models and then be more efficient than js/ts
    Disadvantage: more challenging, would make more sense with more performance oriented libraries like Rust or C++

I decided to try the first option. Some notes

  • Pyodide docs on running in webworker https://pyodide.org/en/stable/usage/webworker.html#
  • However, we are using comlink; comlink doesn't like pyodide objects
  • It would be much easier if we simply exposed the run method only, instead of the full BMI with all controls
  • To still use BMI and comlink, we need a wrapper that takes care of properly forwarding function calls to python object and results back to js
  • Some nice ideas in this blogpost: https://adamemery.dev/articles/pyodide-react

I tried several approaches that go me almost there. Finally I managed to forward model output through comlink, but only through serialization on the python side and deserialization on the ts side. Then, comlink will do something similar again. How (un)desireable is that?

This PR still needs some work to reconcile the best elements of the different approaches I've tried and end up with a fully working version.

@Peter9192
Copy link
Member Author

Alternatively could keep BMI in ts and only CLASS in Python. Pro: comlink wraps ts object instead of pyodide. Con: more communication across pyodide boundary; Python BMI not self-sufficient (but is that necessary?)

@Peter9192 Peter9192 mentioned this pull request Jan 15, 2025
5 tasks
@Peter9192
Copy link
Member Author

Perhaps run should be implemented by the client rather than on the BMI instance, that would simplify life. Also see #101

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant