Displaying ipycanvas in google colab?

[caranha]

Hello! I'm preparing a tutorial using ipycanvas, and while it works perfectly in my local jupyter installation, I can't get the same code to display in google colab (for remote students).

I know this is much more likely to be a google colab issue than a ipycanvas issue, but I was wondering if you know any workarounds from experience?

This is the code that I'm trying to run.

from ipycanvas import Canvas
testipycanvas = Canvas(width = 20, height = 20)
testipycanvas.fill_style = "green"
testipycanvas.fill_rect(0,0,20)
display(testipycanvas)

(I have already !pip install'ed the module with no errors. The code above gives me a blank output with no error messages)

[martinRenou]

Hello! Thanks for opening an issue.

Custom widgets libraries are not supported by Google colab yet, you can follow the discussion here: googlecolab/colabtools#60

[caranha]

Alright, thank you for the quick answer!

[martinRenou]

Note that it should work in Jupyter Notebook, JupyterLab, Voila, Pycharm and VSCode though, maybe on more platforms that I am not aware of :)

[caranha]

Thank you! Unfortunately we recommended Colab to the students who couldn't install stuff in their computers when we opened this lecture about 4 months ago, so now we're stuck with that decision until the end of the course :-P

Not doing that mistake again next year :-D

[caranha]

Also apparently the correct issue on googlecolab about custom widget libraries is now on googlecolab/colabtools#498

[timkpaine]

A helpful related discussion about not trying to build support for proprietary colab APIs one by one into jupyter extensions cytoscape/ipycytoscape#148

[GuillaumeFavelier]

There is some update about this on https://github.com/googlecolab/colabtools. For short, they are working on experimental support for custom widgets. Some working examples using ipyleaflet, ipympl or qgrid are already available! I think that would be awesome if ipycanvas could join this list 😁

The proposal for the API is in: https://github.com/googlecolab/colab-cdn-widget-manager
And potential issues can be discussed in: https://github.com/googlecolab/colab-cdn-widget-manager/issues

---

On mne-tools/mne-python, we rely on ipyvtklink (which is based on ipycanvas) for EEG and MEG data analysis and visualization:

output.mp4

Related to mne-tools/mne-python#8080 (comment)

[martinRenou]

Thanks for commenting @GuillaumeFavelier :) Did you give it a try in Colab? If ipympl and ipyleaflet are working, I am hoping that ipycanvas could work out of the box now!

EDIT:
I believe you "just" need to install ipycanvas and add the following to the beginning of your cell:

display(Javascript('''
  google.colab.widgets.installCustomManager('https://ssl.gstatic.com/colaboratory-static/widgets/colab-cdn-widget-manager/6a14374f468a145a/manager.min.js');
'''))

[GuillaumeFavelier]

Yes! I tried the clock.ipynb example:

from IPython.display import Javascript

display(Javascript('''
  google.colab.widgets.installCustomManager('http://127.0.0.1:9897/manager.dev.js');
'''))

from ipycanvas import Canvas, hold_canvas
import numpy as np

CLOCK_RADIUS = 100

canvas = Canvas(width=CLOCK_RADIUS*2.5, height=CLOCK_RADIUS*2.5)
canvas.translate(CLOCK_RADIUS * 1.2, CLOCK_RADIUS * 1.2)
display(canvas)

def clear_drawing():
    canvas.clear_rect(-CLOCK_RADIUS * 1.2, -CLOCK_RADIUS * 1.2, canvas.width, canvas.height)

def minutes_vec(minutes):
    a = minutes * np.pi/30
    return [np.sin(a), -np.cos(a)]

def draw_dial():
    ht = 10
    mt = 6
    ho = 20
    mo = 10

    canvas.text_align = 'center'
    canvas.text_baseline = 'middle'

    canvas.line_width = 2
    canvas.stroke_circle(0, 0, CLOCK_RADIUS)

    for m in range(60):
        a = m * np.pi/30
        x, y = np.sin(a), -np.cos(a)

        canvas.line_width = 1
        if m % 5 == 0:
            canvas.stroke_line(x * (CLOCK_RADIUS - ht), y * (CLOCK_RADIUS - ht), x * CLOCK_RADIUS, y * CLOCK_RADIUS)
            canvas.font = '12px serif'
            canvas.stroke_text(str(m), x * (CLOCK_RADIUS + mo), y * (CLOCK_RADIUS + mo))
            canvas.font = '16px serif'
            canvas.stroke_text(str(m//5 if m > 0 else 12), x * (CLOCK_RADIUS - ho), y * (CLOCK_RADIUS - ho))
        else:
            canvas.stroke_line(x * (CLOCK_RADIUS - mt), y * (CLOCK_RADIUS - mt), x * CLOCK_RADIUS, y * CLOCK_RADIUS)

def draw_hands(minutes):
    ms = 35
    hs = 50

    hrs = minutes // 60
    mins = minutes % 60

    mv = minutes_vec(mins)
    hv = minutes_vec(hrs * 5 + (mins / 12))

    canvas.line_width = 5
    canvas.stroke_line(0, 0, mv[0] * (CLOCK_RADIUS - ms), mv[1] * (CLOCK_RADIUS - ms))
    canvas.stroke_line(0, 0, hv[0] * (CLOCK_RADIUS - hs), hv[1] * (CLOCK_RADIUS - hs))

def draw_clock(hours, minutes):
    with hold_canvas(canvas):
        clear_drawing()
        draw_dial()
        draw_hands((hours % 12) * 60 + minutes)

import datetime
import ipywidgets as widgets

now = datetime.datetime.now()

hour_text = widgets.IntText(value = now.hour, continuous_update=True, layout={'width': '50px'})
minute_text = widgets.IntText(value = now.minute, continuous_update=True, layout={'width': '50px'})

def on_text_change(change):
    draw_clock(int(hour_text.value), int(minute_text.value))

hour_text.observe(on_text_change, names='value')
minute_text.observe(on_text_change, names='value')

display(widgets.HBox([hour_text, widgets.Label(value=':'), minute_text]))

on_text_change(0)

But unfortunately, I did not get any output except the displayed time (we expect the clock to appear).

I shared the results of my experiment in mne-tools/mne-python#8080 (comment)

[martinRenou]

Ok, thanks for trying. There might be a useful error showing up in the JavaScript console when trying that. If you can report what you see it would be great, otherwise I will give it a try myself 😃

[martinRenou]

I just gave it a try. With a simpler example, and it seems to display the Canvas:

The Canvas is there, but then I don't seem to be able to draw on it unfortunately.

[GuillaumeFavelier]

I don't know if it helps but I tried the clock example again but this time I opened the Javascript console and this is what I see:

[martinRenou]

Thanks. I am seing the same when trying to draw something on the canvas, I will open an issue on Colab and we can follow there.

[martinRenou]

See googlecolab/colab-cdn-widget-manager#8

[martinRenou]

ipycanvas seems to work very nicely with Colab now thanks to @blois 's amazing work!

!pip install -q ipycanvas

from google.colab import output
output.enable_custom_widget_manager()

from ipycanvas import Canvas
testipycanvas = Canvas(width = 20, height = 20)
testipycanvas.fill_style = "green"
testipycanvas.fill_rect(0,0,20)
display(testipycanvas)

[GuillaumeFavelier]

Awesome news, I can't wait to try it out 🚀
Thank you for all the people involved!

[ships]

hi friends, i am excited about this new feature but I believe am getting the issue mentioned above, where my output area is sized adaptively for the canvas dimensions but doesn't actually draw the canvas, with no error message. I created this notebook in january- is there a chance there is some versioning/state associated with my notebook that is preventing me from using the new feature?

thank you for this great product!

[blois]

I'm seeing this happen on the second execution of the cell- if I restart the runtime then it works for the first execution but then not the second. Will try to dig in but nothing comes to mind that would be causing this.

[dave-epstein]

Having the same issue. Only thing I see in console is DevTools failed to load source map: Could not load content for https://cdn.jsdelivr.net/npm/ipycanvas@0.12/dist/index.js.map: HTTP error: status code 404, net::ERR_HTTP_RESPONSE_CODE_FAILURE but not sure if that is related?

[ships]

I found that when I change my line to

!pip install -q ipycanvas==0.11

the canvas works, and does not vanish on subsequent runs of the cell.

Even in this case I also get the error DevTools failed to load source map: Could not load content for https://cdn.jsdelivr.net/npm/ipycanvas@0.11.0/dist/index.js.map: HTTP error: status code 404, net::ERR_HTTP_RESPONSE_CODE_FAILURE , so I don't think that is related.

[martinRenou]

@blois the new ipycanvas version needs a CanvasManager widget model instance around at all times, this model is only created once when you import ipycanvas. Could it be that it is not recreated JavaScript-side in the subsequent cells executions?

[blois]

Opened googlecolab/colab-cdn-widget-manager#24 this sounds similar to jslink which currently has a sort of one-off approach.

[martinRenou]

Thanks a lot @blois

[martinRenou]

For anyone having issues with ipycanvas in Colab, please downgrade to 0.11 as @ships suggested above.

!pip install -q ipycanvas==0.11

[dave-epstein]

Hey, @martinRenou, is there any fix that would allow me to take advantage of the performance improvements with put_image_data from #251? That function is super important to my use case in Colab... Thanks!

[WhatIThinkAbout]

Unfortunately, even downgrading ipycanvas to v11 still seems to have some problems in Google Colab.

In the example below, shown in VS Code, I draw a green square onto the canvas in the first cell, then can display the same canvas again in the next cell, and finally target both the canvases to draw a small blue square from the final cell:

However, when I try the same thing from Colab, the second cell doesn't display the green square, yet strangely the blue square does get drawn to both canvases. Here's the same code in Colab:

As you can see, the green square wasn't drawn when the second cell just tried to show the canvas. Is there any way to work around this, as Colab seems to be a bit hit-or-miss when it actually comes to showing the canvas?

[martinRenou]

Unfortunately, it's unlikely the behavior you are describing here will be fixed.

You can probably workaround this behavior by passing sync_image_data=True to the Canvas constructor:

canvas = Canvas(sync_image_data=True)

This parameter makes the actual pixels part of the widget model, syncing those pixels between canvas views with best effort.

[WhatIThinkAbout]

It seems that even with sync_image_data=True somethings still don't work (Again using v0.11)

from ipywidgets import Image
from ipycanvas import Canvas

sprite1 = Image.from_file(os.path.join(level.working_directory,f'images/baby_robot_0.png'))
sprite2 = Image.from_file(os.path.join(level.working_directory,f'images/baby_robot_1.png'))

canvas = Canvas(width=300, height=300, sync_image_data=True)

canvas.fill_style = "#a9cafc"
canvas.fill_rect(0, 0, 300, 300)

canvas.draw_image(sprite1, 50, 50)
canvas.draw_image(sprite2, 100, 100)

canvas2 = Canvas(width=600, height=300, sync_image_data=True)

canvas2.fill_style = "#caa9fc"
canvas2.fill_rect(0, 0, 600, 300)

# Here ``canvas`` is the canvas from the previous example
canvas2.draw_image(canvas, 0, 0)
canvas2.draw_image(canvas, 300, 0)

canvas2

In this example, taken from the documentation of copying from one canvas to another, the canvas copy doesn't work. When displaying 'canvas2' it only shows a coloured rectangle. Whereas 'canvas' shows the canvas with image, so it appears that it hasn't been able to copy from one canvas to the other.

canvas:

canvas2:

Would you know if there's anyway to get this to work?

[martinRenou]

You are describing another example use-case here. If you want you can open your own issue to not pollute this one. I'd be happy to help you there.