Canvas 2D context color serialization

[annevk]

Presumably the goal of #6562 was to enable new kind of color values. And these can be set. But they cannot be serialized as both fillStyle and strokeStyle use https://html.spec.whatwg.org/#serialisation-of-a-color which has not been updated. That serialization probably needs to switch on the color space?

cc @ccameron-chromium @whatwg/canvas

[ccameron-chromium]

The fillStyle and strokeStyle parameters are CSS colors. The color space in which those parameters are specified is independent of the color space of their CanvasRenderingContext2D. Serialization of non-sRGB colors is covered in CSS Color Level 4.
https://www.w3.org/TR/css-color-4/#serializing-color-values

The scope of #6562 is 2D canvas and ImageData.

The WebGL API with analogous scope is drawingBufferColorSpace/unpackColorSpace.

[annevk]

It is indeed covered there, but that's not what the fillStyle and strokeStyle getters use, right? (I'm also not sure if we can directly use that or if there are some differences in serialization for the sRGB space.)

[ccameron-chromium]

To the original question:

That serialization probably needs to switch on the color space?

No. The values of fillStyle and strokeStyle are independent of the color space of CanvasRenderingContext2D.

Is there disagreement on this?

[annevk]

I meant the color space of the color.

[ccameron-chromium]

Then this is in the context of CSS Color Level 4's new color syntax, and not #6562.

It appears that canvas prefers serializing to hex notation over rgb syntax for opaque colors (but not for non-opaque colors, which use rgba). In CSS Color Level 4's serialization, both rgb and rgba are preferred over hex notation. We should probably have the canvas color serialization carve out "here is the area where we are different", and then explicitly fall back to regular CSS serialization.

[annevk]

I guess in my mind that issue made using non-sRGB color values more meaningful and therefore we should have considered this.

[mysteryDate]

CSS Color 4 has a concept of legacy color syntax. Essentially, pre-css color 4 colors (rgb, rgba, hsl, hsla) get serialized as they always have. Non-legacy colors are serialized as described in csswg.sesse.net/css-color-4.

The simplest solution would just be to do exactly what CSS is doing, while preserving "opaque colors are hex" quirk from https://html.spec.whatwg.org/#serialisation-of-a-color.

ctx.fillStyle = "rgb(255, 0, 255)";
ctx.fillStyle;   // '#ff00ff'
ctx.fillStyle = "rgb(255, 0, 255, 0.5)"
ctx.fillStyle;   // 'rgb(255, 0, 255, 0.5)'
ctx.fillStyle = "color(dIsPlAy-P3  0.964  0.763  0.787)"
ctx.fillStyle;   // 'color(display-p3 0.96 0.76 0.79)'

Could we even link to https://csswg.sesse.net/css-color-4/#serializing-color-function-values?
We obviously need some tests in wpt to cover this.

[annevk]

Yeah, it's fine to link CSS's algorithms and the more we can reuse the better I think. As long as sRGB continues to do the same thing I think we're good.

[mysteryDate]

Yeah, it's fine to link CSS's algorithms and the more we can reuse the better I think. As long as sRGB continues to do the same thing I think we're good.

Great! Should we add this to some sort of agenda or should I just make a PR with the spec changes to this repo?

[annevk]

Just a PR is fine. And you can count WebKit as supportive of fixing this.

[Loirooriol]

Browsers aren't interoperable:

	Gecko	Blink/WebKit
rgb(255, 0, 255)	#ff00ff	#ff00ff
rgba(255, 0, 255, 1)	#ff00ff	#ff00ff
rgba(255, 0, 255, 0)	rgba(255, 0, 255, 0)	rgba(255, 0, 255, 0)
color(srgb 1 0 1)	#ff00ff	color(srgb 1 0 1)
color(srgb 1 0 1 / 1)	#ff00ff	color(srgb 1 0 1)
color(srgb 1 0 1 / 0)	rgba(255, 0, 255, 0)	color(srgb 1 0 1 / 0)
color(srgb none -1 2 / 3)	#0000ff	color(srgb none -1 2)
color(display-p3 0.96 0.76 0.79)	#febfc9	color(display-p3 0.96 0.76 0.79)

[annevk]

I think ideally we do whatever requires keeping around the least amount of state while retaining full precision.

From that perspective it seems nice that if color(srgb ...) is compatible with #... you'd serialize to the latter (so you can parse into the same model). However, maybe implementations already keep around some of the input state anyway for some CSS-related reason and therefore what Blink/WebKit do is overall simpler and preferable.

What Gecko does for inputs that are not compatible with #... does not seem acceptable as they lose precision.

@tabatkins @svgeesus what are the chances of the CSS WG taking over the definition of canvas CSS color serialization? It doesn't seem good for us to continue maintaining this. Looking at https://csswg.sesse.net/css-color-4/#serializing-color-values we'd have to special case quite a bit to get our desired results.

[svgeesus]

Yeah, it's fine to link CSS's algorithms and the more we can reuse the better I think. As long as sRGB continues to do the same thing I think we're good.

I agree. And if Canvas needs a special "for Canvas, serialize legacy sRGB as hex" we can certainly add that sort of language.

When serializing colors, in addition to the syntactic form, the required minimum precision (bit depth) needs to be considered and also, it is better to avoid color space conversions. Serializing color(display-p3 0.96 0.76 0.79) as | #febfc9 or indeed rgb(99.61% 74.9% 78.82%) is problematic because 1/3 of all display-p3 colors can only be represented in sRGB by using unbounded coordinates (greater than 100% or less than 0) and because the minimum precision for rgb() is only 8 bits.

@tabatkins @svgeesus what are the chances of the CSS WG taking over the definition of canvas CSS color serialization? It doesn't seem good for us to continue maintaining this. Looking at https://csswg.sesse.net/css-color-4/#serializing-color-values we'd have to special case quite a bit to get our desired results.

Sure, that makes total sense. Just tell us what you need.

[svgeesus]

@Loirooriol wrote:

Browsers aren't interoperable:

I read those results as "Gecko strictly follows the current Canvas https://html.spec.whatwg.org/#serialisation-of-a-color" which, as @annevk says, has not yet been updated. I assume that when it has been updated and points to CSS Color serialization, Gecko will then produce the same results as Blink and WebKit currently do, right?

[annevk]

@svgeesus here is what I think we want:

1. For non-sRGB just do as CSS already does.
2. For sRGB I'd be inclined to always serialize using HTML existing serialization algorithm, which is roughly what Gecko appears to be doing. (Except that I'd also want the CSS WG to take over the maintenance of that algorithm.)
3. For sRGB that goes outside one byte per color channel I'm less sure. 1 probably makes the most sense? Not even sure if that's implemented or if implementations just round and therefore it ends up in 2.

(I noticed that HTML also has https://html.spec.whatwg.org/#colours for <input type=color>. That will also need some revisiting. I'll bring that up separately, but something to keep in mind.)

[svgeesus]

@annevk I agree with your 1 and 2, for the existing 8-bit Canvas 2D which has just sRGB and display-p3 as options. For the future, something to bear in mind with float canvas is that serializing as 8-bit will be lossy. Your 3 seems to depend on that, so we are good for now, and when float gets implemented then for sRGB seralising as color(srgb ...) makes the most sense. CSS is already doing that for some cases where the extra precision or the handling of wide gamut values encoded in extended sRGB is needed.

[annevk]

I guess one thing that isn't specified right now is what precision we parse and store the colors in. It's clear at the 2D context level we only operate on one byte per channel, but it's not clear when setting fillStyle. Per the current definition of that setter it seems to support arbitrary precision per channel (we just store a CSS color).

And I think that's probably what we want as well, even though it becomes lossy once you start painting. Otherwise the API starts to expose too many underlying mode switches going forward.

[Loirooriol]

Not even sure if that's implemented or if implementations just round and therefore it ends up in 2

Servo was just rounding to a RGBA with three 8-bit ints and one 32-bit float. But Firefox changed the shared style API so now Servo stores the full color. Then the logic that I tentatively adopted is: if the color would serialize (as a computed color with currentcolor resolved) using the legacy rgb() or rgba() functions, then use the HTML algorithm. Otherwise, serialize like in CSS. I think that matches the proposed above.

[svgeesus]

if the color would serialize (as a computed color with currentcolor resolved) using the legacy rgb() or rgba() functions, then use the HTML algorithm.

However, the current HTML algorithm uses either #rrggbb or rgba() depending on whether there is unity alpha.

2. For sRGB I'd be inclined to always serialize using HTML existing serialization algorithm, which is roughly what Gecko appears to be doing. (Except that I'd also want the CSS WG to take over the maintenance of that algorithm.)

Which is why I am restarting this discussion, to agree on text which would go in CSS Color 4 as a replacement to the HTML one.

[svgeesus]

I guess one thing that isn't specified right now is what precision we parse and store the colors in. It's clear at the 2D context level we only operate on one byte per channel

Until the float canvas proposal is adopted but even then, we can have different serialization for integer and float 2D contexts.

CSS Color 4 says:

The precision with which sRGB component values are retained, and thus the number of significant figures in the serialized value, is not defined in this specification, but must at least be sufficient to round-trip eight bit values. Values must be rounded towards +∞, not truncated.

[svgeesus]

@annevk you said

It would be great to get some help from @svgeesus @tabatkins and @LeaVerou on resolving #8917 so HTML can clearly link a way to serialize CSS color values (for the Hexadecimal state we can continue to maintain that ourselves). I think we have a plan there, but let me know if there's anything still unclear (prolly best in that issue).

I hadn't realized that maintaining the hex serialization part in HTML was a goal. I thought it was all being moved over to CSS, which makes total sense because of the necessary branching logic (if it is sRGB and if the values are in [0..255] and are 1 byte per component and the alpha is exactly 1 then output #RRGGBB). It seems cleaner to keep it all together.

[annevk]

I suppose it would be better if <input type=color> and <canvas> would both use https://html.spec.whatwg.org/multipage/canvas.html#serialisation-of-a-color. I had forgotten that HTML contained two color serialization algorithms currently.

Using that algorithm for 24-bit sRGB colors + alpha sounds good.

This has some implications for the design of <input type=color colorspace> which I'm going to spell out here just to keep it all in one place:

• Hexadecimal as a state no longer makes sense. Perhaps Limited sRGB with "limited-srgb" as keyword is reasonable? This gives you either #... or rgba(...) (when there's alpha) and always limited to 8-bit per color component.
• This includes having to round (in addition to conversion to sRGB) when the end user selects a color outside that range as we want to decouple what the end user picks from the representation the web developer wants to some extent. (Although of course user agents are encouraged to expose this mismatch somehow, but we don't want the submission format to dictate the picker and conversely the picker shouldn't dictate the submission format.)
• Does CSS offer rounding for sRGB?

[svgeesus]

I like limited-srgb (which defines both a color space and the range of valid values) a lot more than hexadecimal (which gives a syntax, but what it means is left as an exercise).

• Does CSS offer rounding for sRGB?

Yes, although note here is no route to serialize as hex. rgb() and rgba() accept decimal (not integer) as input, and all the sRGB colors (except color(srgb)) are required to maintain at least 8 bits per component to serialize out. as rgb() or rgba() depending on unity alpha or not.

We could easily add a "serialize as hex" option if it would be helpful for canvas.

The precision with which sRGB component values are retained, and thus the number of significant figures in the serialized value, is not defined in this specification, but must at least be sufficient to round-trip eight bit values. Values must be rounded towards +∞, not truncated.

color(srgb) accepts unbounded floats on input and serializes out with minimum 10 bits per component

[annevk]

From HTML's perspective we have these requirements:

• 2D canvas: stores a CSS color and when that color happens to use 8-bit per component it needs to be serialized as #... (when opaque) or rgba(...) (when not opaque). I think it would make sense if we could set a boolean named parameter called "HTMLCompatible" or some such when serializing to enable that.
• <input type=color>: stores a CSS color, but when serializing that color a) needs to be converted to a color space according to the colorspace attribute b) when that is Limited sRGB that color b1) needs to be rounded to 8-bits per component and b2) use "HTMLCompatible" just like 2D canvas

I think for <input type=color>:

• HTML should probably do the color space conversion upon the CSS color.
• HTML should probably do the rounding as well, although I could also see this being an additional named parameter, but I don't see much utility in that unless there's multiple callers needing it.

So here is what this would mean for the CSS Color specification for maximum clarity:

• It needs to define a "serialize a CSS color" operation that takes a CSS color and outputs a string (doh).
• It needs to define a "HTMLCompatible" named parameter for that operation that influences the serialization when the passed CSS color is a) in the 'srgb' color space and b) uses 8-bits per component. In particular for colors meeting those requirements it will use https://html.spec.whatwg.org/multipage/canvas.html#serialisation-of-a-color (which it defines itself so that definition can be removed from HTML).

Does that make sense? Once that's in place I can update the <input type=color> PR and perhaps also create a separate PR to ensure 2D canvas is properly defined.

[svgeesus]

Canvas is currently limited to 8 bits/component but I would like to write this bit of CSS Color such that it needs small extension rather than a total rewrite once canvas gets the float option.

It isn't clear to me what is expected if the color picker returns an sRGB result which isn't Limited sRGB, i.e. it has more than 8 bits/component and/or has extended range values. In other words a) is true and b) is false. Or would that just be the "not HTMLCompatible" path and thus return color(srgb float float float / float) which would be my preference for that case?

@ccameron-chromium

[annevk]

@svgeesus yes, that's how we should handle those. Note that "Limited sRGB" and associated rounding (which HTML will handle) is for <input type=color> only. 2D canvas won't have such rounding. There the expectation is more that what you put in you get out (modulo parsing/serialization), just that if your input happens to match the range of HTMLCompatible it comes out as HTMLCompatible.

@domenic also pointed out that for <input type=color alpha> we also don't have to pass HTMLCompatible is true (as that's not a legacy code path) so we won't, although we would still round to 8 bits to match the expectations around "Limited sRGB".

[ccameron-chromium]

Is there something preventing us from doing the suggestion in this comment:
#8917 (comment)

It seems like a simple solution.

I think ideally we do whatever requires keeping around the least amount of state while retaining full precision.
From that perspective it seems nice that if color(srgb ...) is compatible with #... you'd serialize to the latter (so you can parse into the same model).
However, maybe implementations already keep around some of the input state anyway for some CSS-related reason and therefore what Blink/WebKit do is overall simpler and preferable.

I disagree with that goal. And there's a ton of complexity in answering if a color "is a color compatible with #...". All of the implementations are required to keep the "was this color specified using a legacy syntax" state around anyway.

Canvas is currently limited to 8 bits/component but I would like to write this bit of CSS Color such that it needs small extension rather than a total rewrite once canvas gets the float option.

I agree very strongly with this goal. Let's not hyper-optimize things around the rapidly vanishing 8-bit sRGB land.

[annevk]

@ccameron-chromium that solution doesn't preserve backwards compatibility as CSS doesn't serialize RGB inputs in the same way that canvas does today. See #10481 and w3c/csswg-drafts#10550 for the current proposal here. There might well be some tweaks possible, but I'm pretty certain that this needs to be the model by-and-large.

[svgeesus]

@annevk I'm looking over this discussion again and working on a spec edit to capture it.

Is this solely for use with Canvas or also other places?

So the requirements are:

1. color space is sRGB (P3 canvases won't use this)
2. alpha is exactly 1
3. source is 8 bits per component (that is easier to express if this is for Canvas only) and bounded by 0..255 (no extended range).

otherwise you get the CSS serialization. Right?

[annevk]

Why is it easier to express if this is for canvas only? We also want this for <input type=color>.

I also don't think it depends on the color space of the canvas. It only depends on the color space of the color. If you use a P3 color on a RGB canvas it shouldn't really get converted at this level.

[svgeesus]

Why is it easier to express if this is for canvas only? We also want this for <input type=color>.

Its okay, I found suitable language in the serializing alpha section about "internal 8 bit representation" so I re-used that.

I also don't think it depends on the color space of the canvas. It only depends on the color space of the color. If you use a P3 color on a RGB canvas it shouldn't really get converted at this level.

Agreed, so to make that clearer I will add an example with a P3 canvas and

imageData = context.getImageData(0, 0, 1, 1, { colorSpace: "srgb" });

[annevk]

@svgeesus this will not impact getImageData(). We only use serialization of CSS colors in fillStyle and friends. See web-platform-tests/wpt#47148 for tests I wrote around this. Note that there's a comment there from implementers that color(srgb ...) should roundtrip and not become #.... Which seems reasonable.

[svgeesus]

Okay. Have a look at

w3c/csswg-drafts@073f754

except I pushed that before seeing your comment just now. Will look at the WPT for better examples.

Comments welcome.

[svgeesus]

And now directly viewable here

[annevk]

Thanks, I left feedback in the CSS WG issue thread here: w3c/csswg-drafts#10550 (comment).