What *is* Canvas desynchronized attribute?

[juj]

Trying to understand what the Canvas WebGL/2D context creation desynchronized attribute is, and what are the pros and cons of using that attribute.

It is clear that the benefit is that "drawing will go faster" and/or "drawing will have less latency". Everyone always wants faster operation, less latency and less overhead, so the attribute sounds great. But there must be a drawback, or otherwise we would not need an attribute at all, but would just always enable it.

What is the drawback?

Reading the wordings of the spec from different sources, I am getting confused by the following ways that I find being able to interpret the spec:

[1] states

[If] desynchronized is true, then the user agent may optimize the rendering of the
canvas [...] by desynchronizing the canvas paint cycle from the event loop

This wording suggests that desynchronized is/can be an optimization where browser is allowed to detach canvas rendering to occur decoupled/faster than some internal in-order buffering of the browser event queue.

Drawback: Painting may appear out of order with respect to other observed web events(?)

[2] states

Insofar as this mode involves bypassing the usual paint mechanisms, rasterization,
or both, it might introduce visible tearing artifacts.

The mention of tearing artifacts suggests that desynchronized is/can be implemented as a vsync disabled flip mode.

Drawback: Scanline tearing when presentation occurs without waiting for vsync vertical refresh.

[3] states

A popular technique for reducing latency is called front buffer rendering,
also known as single buffer rendering, where rendering happens in parallel
and racily with the scanning out process.

This suggests that desynchronized is/can be implemented by browser as a front buffer only rendering mode, where the swap chain does not even contain two buffers to do double buffering. Note that WebGL 3D & 2D canvas (sprite) renderers do not work in a top-down scanning fashion, so in front buffer rendering mode, the rendered objects can appear one by one on the screen as they are drawn e.g. back to front.

Drawback: To ensure consistent view of full "final" images only, one will need to implement double buffering manually if rendering algorithm is complex (e.g. a clear(); for(sort(objects, backToFront)) draw(object);

Although actually doing front-buffer rendering seems pretty radical for web. Perhaps instead, wording in [3] was off-by-one, and intended to read that desynchronized is/can be implemented by UA as double-buffering instead of triple-buffering? (the original name lowLatency suggests that it would be an optimization that avoids an extra buffer copy) In that case,

Drawback: the usual that comes with double-buffering instead of triple-buffering: if one buffer has finished rendering, and another one being presented, the CPU must pause until next vsync to get a new buffer to draw to. In triple-buffering mode, the CPU can drive a third frame with a GPU while one is being shown, and another one is in the queue.

Reading the technical prose for Chrome at [4],

every JS loop activation (i.e. at 60Hz), causes a blit of the contents of
the Context to a GpuMemoryBuffer that is then sent to the Display
Compositor directly, skipping the latency introduced by the Renderer
Compositor queue.

suggests similar to [1], that in Chrome at least desynchronized is implemented as a way that the Canvas swap can occur independent of the rest of the page composition swap.

Pros: Skips an intermediate surface blit from canvas -> page -> screen, and goes directly canvas -> screen?

Drawback: a. Requires setting alpha=false on WebGL contexts. Both Chrome and Firefox developers have recommended before that WebGL contexts should never be created with alpha=false due to performance problems with 24-bit RGB memory layout (though I am not sure why alpha=false could not be implemented as RR8G8BB8X8 as opposed to R8G8B8A8, but that is well beyond this bug)

b. The internal Chrome doc states a very important constraint that should be part of the public spec if this constraint still holds?

every JS loop activation (i.e. at 60Hz), causes a blit of the contents of the Context to a GpuMemoryBuffer

WebGL has had a property that if one does not perform any WebGL API draw calls within a requestAnimationFrame() body, then that loop tick should not present anything (e.g. a black screen) but it should keep the old contents. The above sentence suggests that under desynchronized, that is no longer the case, but a WebGL page should render a new frame in every single rAF() tick?

---

sidenote: is [4] out of date? The code example there has

var context_3d = canvas_3d.getContext("webgl", 
                                      {lowLatency: true, desynchronized: true, alpha: false});

but this lowLatency: true did not make it? Should it be

var context_3d = canvas_3d.getContext("webgl", 
                                      {desynchronized: true, alpha: false});

---

In summary, putting all of the documentation [1]-[4] together, desynchronized can enable in an UA

• a front-buffered only,
• or double-instead-of-triple-buffered,
• and/or vsync-desynchronized
• and/or decoupled-from-DOM-and-event-loop rendered canvas?

What of this is correct? Did I misinterpret some of the spec wording?

Which of the above drawbacks can one have? Should I double-buffer my painter's algorithm page manually (e.g. in a WebGL FBO) if I want to use desynchronized and avoid the back-to-front rendering showing up? Should I expect to observe vsync tearing?

If I am not implementing a 2D digital artist pen paint application, nor a VR application, but a regular game-type WebGL application, should I look at using desynchronized?

• (pros) the fact that it promises that possibly an extra blit canvas->page->screen could be avoided, and [4] states reduces the latency and power consumption even further suggests that it would be a great way to save power especially on mobile.
• (pros) If desynchronized means [1] and app does not care about getting events queued in synchronization, then it would be a nice small latency win.

• (cons) If desynchronized means "vsync off", like [2] suggests, it would mean visual glitching on the output.
• (cons) If desynchronized means "front-buffering only", like [3] states, would it still be faster to use it, and flip FBO->canvas manually?
• (cons) If desynchronized means "must render at each rAF() tick", cannot use it if app is not configured for it (Emscripten OFFSCREEN_FRAMEBUFFER case)
• (cons) If desynchronized requires using alpha=false, would any performance benefit be worth it given that alpha=false has performance problems (at least in Chrome)?

[juj]

CC @jdashg , @kenrussell , @kainino0x

[annevk]

FWIW, cf529a3 should link to more context.

[kainino0x]

I believe everything you said is correct. desynchronized loosens quite a few guarantees, including allowing DOM-decoupling and tearing/front-buffer rendering.

It should be possible to get just the DOM-decoupled behavior by rendering through transferControlToOffscreen(), even on the main thread.

• (cons) If desynchronized requires using alpha=false, would any performance benefit be worth it given that alpha=false has performance problems (at least in Chrome)?

As it can go through a very different compositing path, on some systems I think the alpha=false problem wouldn't present an issue (e.g. with front-buffer rendering). But on systems where desynchronized=true doesn't have an effect (or is implemented differently), the alpha=false requirement could definitely still be an issue. AFAIK there's unfortunately no way to determine what the best configuration is going to be for a given system, right now.

[kenrussell]

@juj good questions and observations - I also think your four points about what desynchronized can enable in an UA are correct. Some of your links ([1], [2], [3] above) all seem to point to the same place in the spec. If you think the documentation should be revised then it would be best to either post a pull request or file an issue pointing to specific lines of the spec source files.

desynchronized was specifically added for pen drawing applications. To avoid large amounts of flickering on single-buffered (front buffer rendering) platforms - currently ChromeOS - it must be used in conjunction with preserveDrawingBuffer:true. It's not intended as a general purpose flag for WebGL applications.

As @kainino0x points out, the best approach to decouple the presentation of WebGL's rendering from other HTML composition is to use transferControlToOffscreen() on the HTMLCanvasElement. This won't tear or flicker, but does allow a latency reduction, and also allows rendering to be done from a web worker. This is still a fairly new code path so if you find any problems with it, like non-smooth animation, please file bugs about them and notify us about their filing via email. Thanks.