pyodide working, numpy getting close

[SimonBiggs]

I have an instance set up at http://jyve.simonbiggs.net:

When using the pyodide kernel it can recognise to pull numpy.js just at the moment however it appears to not be accurately picking up the "locateFile" function. Instead it is trying to find numpy.data at http://jyve.simonbiggs.net/lab/numpy.data.

I shall do a bit more digging another day.

Cheers.
Simon

[bollwyvl]

Very nice progress: is this with the newest pyodide?

…

[SimonBiggs]

Yup

[SimonBiggs]

ScriptedForms branch of the following:

https://github.com/SimonBiggs/jyve

[SimonBiggs]

Also, what are your thoughts about using the Google drive extensions? I haven't looked into it but I imaging Google would have API limitation options...

[SimonBiggs]

So I thought about it further. Showers are good for that. I think I don't want to expose anyone's data from other services to this creature. So instead of using Google drive or the GitLab API I'll use firebase nosql storage. That way the only data that can be leaked or lost is the data on my website.

I'll only save text, so hopefully 1 GB will go a long way.

[bollwyvl]

On Fri, Oct 5, 2018, 19:30 Simon Biggs ***@***.***> wrote: So I thought about it further. Showers are good for that. I think I don't want to expose anyone's data from other services to this creature. So instead of using Google drive or the GitLab API I'll use firebase nosql storage. That way the only data that can be leaked or lost is the data on my website. I'll only save text, so hopefully 1 GB will go a long way.

Google drive will change out from underneath you, and is not worth your development time. Firebase, too, has changed many times. Google forms, on the other hand, are fantastic... If you can deal with an append-only workflow. They are actually my go-to for small projects, as either the Google sheets UI or downloading them as csv to pandas can do all the stuff. You just need a per-sheet, write-only url fragment, and if you are willing to require a Google account and a pop-up, you can get Good Data from them. Meant to extend this demo on jyve, but you get the idea: https://github.com/deathbeds/deathbeds.github.io/blob/master/deathbeds/2018-07-04-Are-Google-Forms-a-Good-Idea-for-Notebooks.ipynb If you need some data to flow back out, You can push the envelope a bit and rig up all kinds of crazy workflows, where a public sheet reads from a named range of a private sheet. —

…

You are receiving this because you commented. Reply to this email directly, view it on GitHub <#33 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AACxRASUbHOrZURuuv9OSOmJp_hpPi1Jks5uh-uYgaJpZM4XKdSm> .

[SimonBiggs]

That is a brilliant idea. I shall look at that. It might just be perfect.

[SimonBiggs]

Can forms be created via the API? As in could users log in with their Google account, create a ScriptedForm and have the ScriptedForm mimic itself as a Google Form. Then as they fill out their ScriptedForm it would then use the API to fill out the created Google Form...

[bollwyvl]

https://developers.google.com/apps-script/reference/forms/ Same deal, but maybe it's a separate ui./token

…

On Fri, Oct 5, 2018, 21:53 Simon Biggs ***@***.***> wrote: Can forms be created via the API? As in could users log in with their Google account, create a ScriptedForm and have the ScriptedForm mimic itself as a Google Form. Then as they fill out their ScriptedForm it would then use the API to fill out the created Google Form... — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#33 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AACxRKwgKoolrbKUuDvkBiqRjOTbiODgks5uiA0JgaJpZM4XKdSm> .

[SimonBiggs]

@bollwyvl I think that might only work inside the "App Scripts" that you write within the Google Office suite... https://developers.google.com/apps-script/

[SimonBiggs]

Maybe it is possible:

https://developers.google.com/apps-script/api/how-tos/execute

[bollwyvl]

Grrr. The scopes there are: https://www.googleapis.com/auth/forms https://www.googleapis.com/auth/forms.currentonly So the pop-up would ask some thing like "allow BigCorp scripted forms to modify all your Google forms". This is better, but not great. You could have a "master" form that stored json serialized forms: - fields - date time - owner - form id then use the scripts API to watch for new rows to update the forms by form. Even if you made a malicious form, the worst it could do is create malicious rows in YOUR form, tagged with your name. Then end user submissions would land in each form's spreadsheet, no security problems.

…

On Sat, Oct 6, 2018, 01:53 Simon Biggs ***@***.***> wrote: Maybe it is possible: https://developers.google.com/apps-script/api/how-tos/execute — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#33 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AACxRNriJeins6CpYi3j9585KIxnNGvkks5uiEU-gaJpZM4XKdSm> .

[PythonLinks]

I just used numpy in pyodide yesterday. It is part of the tutorial I am giving next week.

[SimonBiggs]

Within jyve?

…

On Thu., 16 May 2019, 7:45 am PythonLinks, ***@***.***> wrote: I just used numpy in pyodide yesterday. It is part of the tutorial I am giving next week. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#33?email_source=notifications&email_token=ABSBK642K6JRV3FIKZPF4ITPVR76HA5CNFSM4FZJ2STKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODVQBESY#issuecomment-492835403>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABSBK64OHVVVPJOHRDLXEZLPVR76HANCNFSM4FZJ2STA> .

[PythonLinks]

No.
Good point. It was from iodide.
Do you know specifically what the problem is?

[SimonBiggs]

Haven't looked at the issue since late last year.

[bollwyvl]

Like I said on that other issue, my plan was to wait until lab 1.0 to do another serious attempt... But by all means, give it a go! Pyodide has changed a lot since then!

…

On Thu, May 16, 2019, 01:56 Simon Biggs ***@***.***> wrote: Haven't looked at the issue since late last year. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#33?email_source=notifications&email_token=AAALCRD675TYONNLR75FGZTPVTZSFA5CNFSM4FZJ2STKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODVQXXZY#issuecomment-492927975>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAALCREWS72DYCD4RX25VLDPVTZSFANCNFSM4FZJ2STA> .

[SimonBiggs]

Like I said on that other issue, my plan was to wait until lab 1.0 to do another serious attempt... But by all means, give it a go! Pyodide has changed a lot since then!

Heya @bollwyvl, I'm thinking of giving this another deep dive. Before I take the plunge do you have any initial pointers?

My first key issue is that running the pyodide kernel hangs the browser rendering. An example of this can be seen by going to http://jyve.simonbiggs.net/ creating a pyodide kernel notebook and then running:

import time

while True:
    print('boo')
    time.sleep(10)

This will freeze up the interface and the kernel can't be reset.

I believe I can fix this by running the pyodide kernel within a webworker instead. There's a bug with this in builds of pyodide I was previously using, but it appears there may be a solution to that pyodide/pyodide#441 (comment).

[bollwyvl]

Very much still interested, but haven't had a compelling use case to do the grunt work. I investigated some ways to run it a bit more cleanly, perhaps with a second ServiceManager. I got pretty far with lab 1.0, but the kernel/session ground kept changing under me... At this point I'm holding out to pick it back up for lab 2.0, which should be released presently. Unfortunately I don't have a lot of guidance, as I'll have to relearn the code base anyway... But wasm is clearly the future of jyve, as a number of other runtimes have become available (elixir, etc).

…

On Tue, Feb 4, 2020, 05:06 Simon Biggs ***@***.***> wrote: Like I said on that other issue, my plan was to wait until lab 1.0 to do another serious attempt... But by all means, give it a go! Pyodide has changed a lot since then! Heya @bollwyvl <https://github.com/bollwyvl>, I'm thinking of giving this another deep dive. Before I take the plunge do you have any initial pointers? My first key issue is that running the pyodide kernel hangs the browser rendering. An example of this can be seen by going to http://jyve.simonbiggs.net/ creating a pyodide kernel notebook and then running: import time while True: print('boo') time.sleep(10) This will freeze up the interface and the kernel can't be reset. [image: Screenshot from 2020-02-04 20-55-14] <https://user-images.githubusercontent.com/6559099/73734143-4748ba00-4791-11ea-9ecd-6adb96b4e133.png> I believe I can fix this by running the pyodide kernel within a webworker instead. There's a bug with this in builds of pyodide I was previously using, but it appears there may be a solution to that pyodide/pyodide#441 (comment) <pyodide/pyodide#441 (comment)> . — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#33?email_source=notifications&email_token=AAALCRHT5L464MIV2WCKRSTRBE43ZA5CNFSM4FZJ2STKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEKXBIVI#issuecomment-581833813>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAALCRBYC7HTL24EHNBPJCTRBE43ZANCNFSM4FZJ2STA> .

[SimonBiggs]

Kk, I'll hold off until JLab 2.0 also

[SimonBiggs]

@bollwyvl pyodide is unfortunately falling into an unmaintained state, from where I stand, pyodide has the possibility of making Jyve be truly something amazing. Might you be in a position to comment on the following proposal:

https://bugs.python.org/issue40280

If you were able it would also be awesome to have a hand in creating a patch, if it comes to that. I suspect a similar process may also be needed for emscripten, but I think it best to address the hardest issues first, in case it just may not be viable.

In the end I was hoping to get the pyodide patches merged upstream and then be able to version bump so that threading and webworkers run without issue. First step though, get the work merged upstream instead of having it go stagnant.

[bollwyvl]

Whew, that's probably a bit above my paygrade. Language runtime aside, there are so many thorny issues raised there. I have a lot of opinions about what one might be able to do once we get to "better", but I am woefully uninformed about the cPython implementation, and how that might be pushed forward. @jtpio do you have any thoughts on the matter?

[jtpio]

Hey sorry for the late answer. Unfortunately I haven't been able to look into pyodide yet, but I agree this would be the ultimate use case for Jyve and similar frontends.