[WIP] proof of concept: HTMLCanvas bitmap context in pyodide #115

Vipitis · 2025-09-17T00:14:20Z

two evenings of tinkering but I can feel a bit of progress. Rendercanvas looks very web inspired, so I am reading a lot of things between the docs, pyodide docs, pyodide source that sound close but are ever so slightly different. Plus I don't have any webdev experience, it's more like a learning opportunity.

for those that want to give it a try - you can essentially just load the .html as a static page with the python script inserted. But you will need to build wheels and load them locally. Installing from pypi via micropip doesn't include these changes.

some todos:

examples/html_canvas.py

Vipitis · 2025-09-17T22:38:47Z

feel like some stuff breaks due to the python name mangling when using double underscores in combination with the subclass... which happens quite a bit - so I am sorta surprised it still works this far.

But I can't find any such references for Pyodide which would be odd if that is a known limitation.

Vipitis · 2025-09-20T01:05:18Z

feel like some stuff breaks due to the python name mangling

that wasn't the case. I actually fell through the line here

rendercanvas/rendercanvas/base.py

Lines 450 to 451 in fa7defc

    
           context = self._canvas_context 
        
           if context:

~~because I hadn't implemented context as a class. I took a lot of inspiration form the bitmap scripts and got it working~~
I also registered the auto backend successfully - meaning if you build the wheel and then load it statically. The examples noise.py and snake.py work out of the box (although not events yet). But the weekend has a few more days :)

auto_demo.mp4

E: turns out that wasn't true either and I am using the existing bitmaprenderingcontext due to how get_context is implemented.

Vipitis · 2025-09-20T20:11:46Z

singular keydown event works... so more events and other types shouldn't be impossible. However I will get to that another day.

snake_events.mp4

Vipitis · 2025-09-21T18:28:17Z

tried all day to make it work for the docs ... but either the .whl don't get included as static files or pyodide has trouble importing the wheel. I also wanted to automate the iframe inclusion with sphinx-gallery but seems like you need to either modify the gen_rst function or write an image scaper to append some custom rst to the examples... so I just added the comment to the relevant examples instead 🤷

classic "works on my machine", so have a video of what could have been instead:

doc_embed.mp4

Korijn · 2025-09-21T20:06:05Z

Even so it's awesome how far you managed to take this!

rendercanvas/html.py

Korijn · 2025-09-23T11:09:15Z

docs/backends.rst

+            pythonCode = `
+                # Use python script as normally
+                from rendercanvas.auto import RenderCanvas, loop
+                canvas = RenderCanvas(title="Example")


I would argue that providing a constructor like this would be more conventional for the web:

Suggested change

canvas = RenderCanvas(title="Example")

canvas_el = document.getElementById("canvas")

canvas = RenderCanvas(canvas_el, title="Example")

Since often there are multiple canvas elements on the page, users should be able to control which is used.

One of the goal was to keep the python code portable between auto backends. So passing a string to __init__() would work even on backends where this kwarg isn't used like glfw and the user doesn't need to use any pyodide specific code in python. (Once we have a wgpu-py version for browser, most examples should just work without changes to shadertoy, pygfx or fastplotlib etc).

I also losely followed the idea of https://pyodide.org/en/stable/usage/sdl.html#setting-canvas where they provide a specific API to accessing the canvas, although I not using it.

Maybe I can write a little multi canvas example to see if my approach works.

I have zero webdev experience, so my design decisions are directed to the python devs wanting to write their python code (like myself).

I have zero webdev experience, so my design decisions are directed to the python devs wanting to write their python code (like myself).

I hear you, but just because you can use python the language, doesn't mean you can "ignore" the environment it's running in! I don't mind what kind of API you choose (I value portability as well) as long as the user can control which <canvas> is used.

I imagine python devs turning to browsers will often do so because they want to use the browser's capabilities to build the UI they have in mind. It's easy to envision applications with multiple canvases embedded in a richer UI.

it shouldn't be impossible to support both.
canvas_el: [str|HTMLCanvasElement] = "canvas"

would work even on backends where this kwarg isn't used like glfw

apparently kwargs don't get ignored when a different auto backend is selected because the base class calls super.__init__(*args, *kwargs). We could use the title arg as I am not sure if that has a use in the browser, but that seems janky.

For resizing, have a look at window.ResizeObserve: I use this code in another project to get the physical size (it's PScript, so you need to convert to Python/JS depending on where it runs):

# Inside the ResizeObserve callback ... entry = entries.find(lambda entry: entry.target is self.node) if entry.devicePixelContentBoxSize: # Best if we have the physical pixels ... psize = [ entry.devicePixelContentBoxSize[0].inlineSize, entry.devicePixelContentBoxSize[0].blockSize, ] else: # ... but not all browsers support that (see issue #423) ... if entry.contentBoxSize: lsize = [ entry.contentBoxSize[0].inlineSize, entry.contentBoxSize[0].blockSize, ] else: # even more backward compat lsize = [entry.contentRect.width, entry.contentRect.height] ratio = get_pixel_ratio() psize = Math.floor(lsize[0] * ratio), Math.floor(lsize[1] * ratio)

In general the standard reference material for browser APIs is on MDN, in this case see this page for example usage: https://developer.mozilla.org/en-US/docs/Web/API/ResizeObserver

finally found a bit of time to implement this - also tried to make the demo page react and it seems to work. Altough I am not sure if this matches real webframeworks

I'm pretty sure we can hook things up in a clean way, but I will need to sit down and play a bit to get it right.

feel free to commit into this branch or similar if you have the capacity. I am very much out of my depth with the web stuff and also don't have too much time currently (settling into new job and new university) to sit down for some long evenings and figure it out.

almarklein · 2025-09-23T11:26:15Z

rendercanvas/html.py

+from pyodide.ffi import run_sync, create_proxy
+from js import document, ImageData, Uint8ClampedArray, window
+
+# TODO event loop for js? https://rendercanvas.readthedocs.io/stable/backendapi.html#rendercanvas.stub.StubLoop


I think asyncio can be used in Pyodide, right?

edit: should have read more before reacting. So is pyodide.webloop a sort of more native loop implementation?

yes, they overwrite a couple of functions.
But I should try to use the async loop class and see if this whole class can be avoided.

edit: you can't find much on it in the docs so I looked at the source and went from there. Perhaps other prominent pyodide packages provide some specific insight. I pretty much learned all my async and js knowledge from attempting this. Surprised myself it sorta works. But this obviously means I might be doing something wrong.

Can I ask why a loop is needed at all? Why not just implement call_later with setTimeout and leave everything else to the browser's native loop?

Loop is made to fit into the existing rendercanvas implementation, although it can likely be simpler with more changes. Very much out of my depth here.

The docstrings here says as much, although I am not sure if that's happening all the way down. https://github.com/pyodide/pyodide/blob/bcd0235bff5351d1dc383e4f3e34b9917fdf3281/src/py/pyodide/webloop.py#L280L294

Yeah it is: https://github.com/pyodide/pyodide/blob/bcd0235bff5351d1dc383e4f3e34b9917fdf3281/src/py/pyodide/webloop.py#L330

The point is that in the browser there already is an event loop driving the page, and you can not get a reference to it. So I am just wondering what the point of having an event loop abstraction is inside a browser, apart from compatibility.

okay, so I tried with the asyncio loop and it works just fine. Also not calling loop.run() at the end of an example still works.

so should I drop my loop class and just import the asyncio loop instead?

so should I drop my loop class and just import the asyncio loop instead?

Yes, let's just do that. It's the simplest to do, and makes sure all works together if the user uses asyncio for other stuff.

one more consideration that might be complex to implement: use the browsers requestAnimationFrame with a callback to run the loop? I spotted something like this here: https://github.com/pyglet/pyglet/blob/5d1fe49db83bd2c060d97226e1334f06a706b1fc/pyglet/app/async_app.py#L64

Using requestAnimationFrame is the standard approach for rendering, since it automatically pauses when the tab becomes hidden for example.

yeah, with some of the wgpu examples I am having issues where the animation stops randomly or when you change tab etc... it just suspends and doesn't recover. So we might need a webloop or whole sheduler... the rendercanvas bitmap examples seems to survive tab switching

rendercanvas/html.py

Korijn · 2025-10-07T21:03:21Z

rendercanvas/html.py

            return res

-
+        self._pointer_inside = False # keep track for the pointer_move event


You might want to look into:

Pointer capturing

Mouse leave and enter events

Maybe you already have and this comment is entirely moot

almarklein · 2025-10-08T13:50:06Z

rendercanvas/html.py

+        # char ... it's not this
+        # def _html_char(proxy_args):


Have a look at the implementation of char events in jupyter_rfb vispy/jupyter_rfb#119

I looked at that initially, I tried to avoid actually changing the webpage, but maybe there is no way around it. Which also opens new questions like should the python code spawn new canvases?
Also all the global events might mess up when there are multiple canvases...

Vipitis added 3 commits September 16, 2025 02:09

js bitmap context

c81ab50

might need a loop -.-

393b590

working loop

f0f533d

Korijn reviewed Sep 17, 2025

View reviewed changes

examples/html_canvas.py Outdated Show resolved Hide resolved

assing js_array directly

d62dc39

Vipitis added 3 commits September 20, 2025 02:02

add context class

93e3639

fix channels

bd9b3ab

register auto backend

9017288

Vipitis added 2 commits September 20, 2025 21:29

working events!

a10970d

fix pixel order

e62274a

Vipitis added 8 commits September 20, 2025 22:25

cleanup testing code

d79656c

remove unused context class

8d51188

add all events

1044d0f

add basic documentation

d393c52

typos pass

8b71ed7

embed examples into docs

f197d7d

maybe fix wheel location

529c3ec

maybe fix files

7025f87

Korijn reviewed Sep 21, 2025

View reviewed changes

rendercanvas/html.py Outdated Show resolved Hide resolved

Korijn reviewed Sep 23, 2025

View reviewed changes

almarklein reviewed Sep 23, 2025

View reviewed changes

Korijn reviewed Sep 23, 2025

View reviewed changes

rendercanvas/html.py Outdated Show resolved Hide resolved

Vipitis added 3 commits September 24, 2025 00:36

add canvas selector

58bc775

add multicanvas example

51459d3

use asyncio loop

6cf0ba5

Vipitis added 2 commits September 25, 2025 15:52

ruff format

d9f8fc0

add canvas element arg

fcd2d1c

almarklein reviewed Sep 26, 2025

View reviewed changes

rendercanvas/html.py Outdated Show resolved Hide resolved

Vipitis added 3 commits September 26, 2025 15:13

simplify selector argument

3e13446

icorrect type hints

75e3ddc

enbled wgpu context

879399e

This was referenced Sep 27, 2025

[WIP] js_webgpu backend prototype pygfx/wgpu-py#753

Draft

Supporting JS/webgpu as a backend for e.g. PyScript pygfx/wgpu-py#407

Open

Vipitis commented Sep 29, 2025

View reviewed changes

rendercanvas/html.py Outdated Show resolved Hide resolved

Vipitis added 4 commits October 4, 2025 17:17

fix button ids in pointer events

7544b77

add resize event

2dd4824

make the example resize

053f2db

fix pointer_move just inside or down

ef8d9d3

Korijn reviewed Oct 7, 2025

View reviewed changes

almarklein reviewed Oct 8, 2025

View reviewed changes

	canvas = RenderCanvas(title="Example")
	canvas_el = document.getElementById("canvas")
	canvas = RenderCanvas(canvas_el, title="Example")

		return res


		self._pointer_inside = False # keep track for the pointer_move event

[WIP] proof of concept: HTMLCanvas bitmap context in pyodide #115

Are you sure you want to change the base?

[WIP] proof of concept: HTMLCanvas bitmap context in pyodide #115

Conversation

Vipitis commented Sep 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Vipitis commented Sep 17, 2025

Uh oh!

Vipitis commented Sep 20, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Vipitis commented Sep 20, 2025

Uh oh!

Vipitis commented Sep 21, 2025

Uh oh!

Korijn commented Sep 21, 2025

Uh oh!

Uh oh!

Korijn Sep 23, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Korijn Sep 23, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

almarklein Sep 26, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

almarklein Sep 23, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Vipitis Sep 23, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Korijn Sep 23, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Vipitis Oct 8, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Vipitis commented Sep 17, 2025 •

edited

Loading

Vipitis commented Sep 20, 2025 •

edited

Loading

Korijn Sep 23, 2025 •

edited

Loading

Korijn Sep 23, 2025 •

edited

Loading

almarklein Sep 26, 2025 •

edited

Loading

almarklein Sep 23, 2025 •

edited

Loading

Vipitis Sep 23, 2025 •

edited

Loading

Korijn Sep 23, 2025 •

edited

Loading

Vipitis Oct 8, 2025 •

edited

Loading