Augment Pyodide with pre-fetched packages details

[WebReflection]

This MR is a first step toward better Pyodide packages integrations and feedback.

The current logic fetches only Pyodide versions that are unknown from the official documentation, then makes those details available to pyodide only when packages are part of the config.

The loading is also not bound with the main core to avoid adding 2.x Kb to the final Polyscript size, which is completely useless with any other interpreter to date.

This also predates the ability to bring a similar logic to MicroPython and others though!

[ntoll]

OK. So my understanding is this will give us the relevant metadata needed for more "human" checks and messaging around packages. Right?

[WebReflection]

@ntoll yes, and this should be a WIP (I'll make one soon) as the rest of the logic hasn't been finalized yet.

Basically this provides all pyodide official packages available, and their version, per each release.

I've explicitly ignored releases lower than 27 because I think we shouldn't support too much way outdated pyodide, also this is a new feature so previous projects will run just as fine as before.

The thing I am a little bit skeptical is about keeping the versions, as the reference numbers might not tell the whole story because in Pyodide 0.28.3, as example, old bokeh with its old JS dependencies might work too.

If I remove that information we're left with an opportunity, per each field, to provide a list of JS/CSS modules instead that should land out of the box on the page, making inclusion of more complex packages as easy as cake but that logic is also missing, because I haven't found yet a way to automatically fetch required dependencies out of Pyodide packages, when it comes to the Web, but I have some idea around it.

Once this last bit is done we can:

• inform the user a package for the currently running Pyodide version is not available
• automatically add dependent scripts on the main thread, if a package requires such dependent scripts on the main to work
• pave the way for a JSON "schema" that provides packages by unique name and meta data to land properly on PyScript

... profit?

Happy to discuss any of these topics but the initial file is something I wanted to push already because it avoids reaching out Pyodide documentation site every time we need to generate this metadata so that we don't DDoS them and we fetch only latest, whenever that lands.

[WebReflection]

@ntoll this MR now does the following:

• it updates Pyodide to use its latest
• it has already the most updated Pyodide packages graph
• there is initial logic to throw errors when a package name (not url, just name with or without version) is not available but ...
• the logic runs only if index_url is not specified because by no mean we can orchestrate validation around foreign / custom packages URLs in here

As both coincident and reflected-ffi have been updated to remove the ugly hack needed for Pyodide 0.28.3 and as Pyodide 0.29 also brings in back py-game-ce I think we should move forward with this current/initial packages validation state so that we can release with latest MicroPython and Pyodide ... thoughts?

[ntoll]

I agree it would be good to move forward with a release containing the latest runtimes. 👍