Canvas Floating Point Color Types

[ccameron-chromium]

Request for position on an emerging web specification

• WebKittens who can provide input: weinig@apple.com

Information about the spec

• Spec Title: Canvas Floating Point Color Types
• Spec URL: https://github.com/w3c/ColorWeb-CG/blob/master/canvas_float.md
• GitHub repository: https://github.com/w3c/ColorWeb-CG/

Design reviews and vendor positions

• TAG Design Review:
• Mozilla standards-positions issue: Canvas Floating Point Color Types mozilla/standards-positions#708

Anything else we need to know

This allows float16 (MTLPixelFormatRGBA16Float or kCVPixelFormatType_64RGBAHalf) as the backing format of an HTML canvas, in addition to the current 8-bit default. It also adds float32 as an option for ImageData (for the corresponding getImageData and putImageData calls).

[annevk]

Not using Float16Array because it's not in JS feels like "shipping the org-chart". If Float16Array is the right thing to do we should add it to JavaScript.

Also, this really ought to be proposal against the HTML Standard.

[ccameron-chromium]

Not using Float16Array because it's not in JS feels like "shipping the org-chart". If Float16Array is the right thing to do we should add it to JavaScript.

I went back just now and did some digging on this. The main reference is an es-discuss thread that has been going on-and-off from 2014 through 2018. This link picks up the various threads:

I had been under the impression that a decision had been taken against Float16Array. That is not the case. The thinking has been more along the lines of "it's a lot of work and there aren't that many applications beyond OpenGL interop".

The arguments for doing Float16Array have only grown since 2014.

• At this point almost all architectures have a hardware instruction for doing conversion to and from float16 (this is F16C aka CVT16 on amd64 CPUs, and from what I can tell ARMv8 has fairly extensive support).
• Graphics is still using float16, and there just isn't motion towards float32 (perhaps except for some very niche applications).
• Machine learning has become a huge consumer of float16.
• Memory is still important, if not more important (we have more pixels than ever), and float16 is half of the memory.
• Power is still important, if not more important (from mobile to datacenter), and memory is bandwidth is power.

I agree with your assessment. We should add Float16Array now, with/before this proposal (or definitively say we will never do it, if some reason comes up that we aren't seeing).

Also, this really ought to be proposal against the HTML Standard.

Followed up on email about making sure I get the right venue for the discussion.

(Of course, it's all quite sadmaking that this means that what looked like a very simple proposal now has big dependencies and will take longer to get shipping).

[othermaciej]

Failure to use Float16Array sounds like concerns: integration.

[ccameron-chromium]

I have slightly changed my position on the above. I do believe that we should add Float16Array. I believe we should not block Canvas Floating Point Color Types on Float16Array. See the ColorWeb-CG position here.

[annevk]

I outlined my concerns with that proposal in the HTML issue. Thanks for working on opening that!

[ccameron-chromium]

There have been some updates to these plans in whatwg/html#8708 (comment)