Canvas2D Filters

[fserb]

A javascript interface for using SVG filters within canvas.

The SVG filter exposes deep, flexible drawing modifiers for 2d graphics. Integrating these into canvas 2d should be technically feasible once an interface is defined.

Working proposal: https://github.com/fserb/canvas2D/blob/master/spec/filters.md

(cc @whatwg/canvas )

[litherum]

Conceptually, adding filters to Canvas2D seems reasonable.

Adding full SVG-filter support seems like a large amount of new API for limited benefit. In practice, most content uses only a fairly small set of filter types. If we're going to add filter support, we should start with a small set of popular filters, and only expand the set if compelling needs arise. Also, we should start with the "linked-list" model of filters that the css filter() function accepts, rather than arbitrary graphs that are allowed by SVG.

[mysteryDate]

I agree with @litherum. I felt a little odd writing up all the SVG filters into the doc. I have a hard time imagining the practical usage for the lighting filters, for example. Do we have any idea what a "small set of popular filters" would be? GaussianBlur, ConvolveMatrix, ColorMatrix and Blend seem like good initial candidates to me.

[Kaiido]

I'm not sure what would be the real benefit of this...
We can already apply all these filters through url(#svg-filter).

This should theoretically not even require access to the DOM (thinking of OffscreenCanvas), since we can generate such a filter from data:// URI or a blob:// URI _{(though Chrome has a bug where they don't allow blob URIs yet, IIRC per specs they should and FF does support it).}

So the example in the explainer can already be achieved: https://jsfiddle.net/hogyqxw1/ and it seems everything in this proposal becomes neat API wrapping that could be done by a library.

---

What would be great though is that we finally have a load event for these filters, and that implementers support it in Worker thread too (they don't currently).

[mysteryDate]

I think the idea is precisely to provide a more user-friendly API for a feature that's already implemented but relatively hidden and difficult to use.

[kdashg]

There is somewhat perverse value in hard-to-optimize paths being hard to use, so as to keep devs pointed at the APIs we want them to use, and can optimize well.

[mysteryDate]

@jdashg Why are filters hard-to-optimize? Aren't they just shaders?
What APIs would you want devs to use to accomplish this?

[kdashg]

Use an API with shaders, like WebGL. Not all canvas2d implementations are implemented with shaders, since not all canvas2d implementations are hardware accelerated.
If you want it to be fast, use WebGL or libraries that do.

[fserb]

OffscreenCanvas can't support URL filters properly. Even if we could somehow address this, it's not clear at this point that any web engine can build a SVG on a WebWorker (and not sure if we ever want to do that), which would still leaves us with a weird situation of "we have this capability but no way for users to describe what they want".

I'd also argue that the URL filter to SVG is mostly a hack to allow filters to be described, as opposed to a properly well designed spec that describes Canvas filter capabilities. Having a clear and friendlier interface would be an overall win.

I'm not sure that "because some filters can be slow performing right now, let's make it hard for people to use it" is a good argument. If there was a fundamental performance problem with a feature, sure, make more expensive things harder. But if filters are useful and people start using it more, then eventually it will be worth it for us to optimize for it. Making it convoluted to use just to prevent people from using it at all sounds bad.

[kdashg]

We basically can't afford to optimize all fast-paths for all use-cases, so design-wise, we try to shepherd devs towards the always-fast APIs and have them write the fast-paths for their multitude of fast-paths that they care about.

If we want to keep shipping things that have the most impact for the most users, we don't want to design a system that encourages us to clean up after devs that string together friendly interfaces that are hard to optimize.

Put another way, I want to focus on providing the web with features that aren't possible today, rather than signing us up for perpetual maintenance for keeping these other APIs fast through sheer force of engineering.

This is not about making it convoluted, but cutting our losses and not making it easier to fall into the performance pits that we already have coming from svg.

[Kaiido]

it's not clear at this point that any web engine can build a SVG on a WebWorker

It seems indeed none can do it yet, but OffscreenCanvas support is limited to Blink based browsers currently.

(and not sure if we ever want to do that)

There are already at least two places where the specs currently expect this feature:

• OffscreenCanvasRenderingContext2D and its filter property.
• createImageBitmap() with a Blob representing an SVG image.

Also one advantage of the current CanvasFilter is its direct parallel with CSS. If you really want to go the Primitives interface way, I guess it might be good to consider how this could fit there too.