Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Radial gradients and negative radii with variations or non-normal color stops #367

Open
drott opened this issue Aug 25, 2022 · 74 comments
Open

Comments

@drott
Copy link
Collaborator

drott commented Aug 25, 2022

When implementing COLRv1 variations we're running into problems when varying color stops of COLRv1 PaintVarRadialGradient into the negative, or varying the radii of PaintVarRadialGradient into negative values.

Two scenarios can lead to that:

  1. Simply attaching a variation axis to r0 and r1 and specifying negative values as part of the axis range.
  2. Having a smaller circle A and a larger circle B, and then shifting color stops near the center of A to smaller values so that they would be outside the vector from c0 to c1. In most implementations the color stops need to normalized to the 0 to 1 range and the approach used is usually to interpolate new circle centers and radii, so that they are equivalent to the original gradient definitions, but color stops are in the range of [0, 1]. This interpolation can result in one of the two interpolated circles having a negative radius.

It seems to me like in principle this is currently well-defined in the spec - and nothing should be drawn for circles with a radius < 0.

But with this definition we run into implementation problems when the color line does not end exactly at the 0-radius circle. Since the gradient implementations expect normalized color stops, we would need to manually interpolate a color for color stop 1 or 0 at the zero-radius circle. But specifying manually a start/end interpolated color and truncating the color line then means that the shader is not aware of the full color-line and repeat modes stop working correctly after the first iteration of the color line.

Changing the shader/gradient programs seems highly difficult as they are currently optimized for CSS gradients - which by their definition of circles etc. do not run into this problem.

@jfkthame and I discussed possible solutions for this situation, and I am inviting Jonathan to suggest what a possible solution to this might look like.

@jfkthame
Copy link

For scenario (1), using a variation axis to alter r0 or r1, one reasonable argument would be that because radii are unsigned values (stored as UFWORD in the font table), it is impossible for them to become negative, and so the resolved value when variations are applied should simply be clamped at zero. This would avoid the question of what it means to have a circle of negative radius. And when applied to an explicit variation of the radius, authors should have no trouble understanding that the radius cannot become negative, and so variations cease to have any further effect when that value is reached.

Another argument would be that when the radius is negative, nothing at all should be rendered for the gradient -- which is what is implied by the HTML canvas spec: "If either of r0 or r1 are negative, then an "IndexSizeError" DOMException must be thrown". So when a radius varies to a negative value, the gradient becomes undefined and disappears.

If this issue were only about applying variations to the radii, I think either of these answers would be entirely reasonable, and readily implementable.

However, neither of these answers give a satisfactory result in scenario (2); and I think that as far as possible, we should aim for consistency between the two scenarios, rather than addressing them separately.

In case (2) the gradient is defined by two circles with non-negative radii, but normalization of the color line results in a "projected" starting or ending circle whose projected radius becomes negative. Note that the specification of the gradient uses perfectly valid circles; the fact that we end up with a negative projected radius is an implementation detail that should not cause a complete failure to render.

As Dominik notes, "slicing" the definition of the normalized gradient at the zero-radius position isn't a workable approach because repeat/reflect modes will fail to work correctly. And in this case, clamping the projected radius at zero, while implementable, would give a result that is difficult for authors to relate to their specification of the gradient, because of the inconsistency it would introduce between how the center position and the radius are treated.

There is, I think, another reasonable interpretation of what it means for one of the circles defining a gradient to have a negative radius: the gradient is simply rendered using the absolute values of the radii. This avoids any implementation problems of the other interpretations. And intuitively, it makes some sense. If we consider the radius of the circle as a vector of length r from the center (in any arbitrary direction), we can picture what happens as it varies towards zero: the circle shrinks. When it reaches zero, the circle collapses to a point. Continuing the variation, the radius vector begins to extend in the opposite direction from the center: and so the circle that it defines begins to grow again.

This matches the behavior I have observed in the FontGoggles testing tool, and is, I think, the most reasonable and implementable solution given the available painting APIs.

I suggest, therefore, that the COLRv1 spec should explicitly say that:

(a) When variations would cause one or both of the radii in PaintVarRadialGradient to become negative, it is resolved to the absolute value.

(b) When the color line used in Paint[Var]RadialGradient has a non-[0.0 - 1.0] range, the stop offsets are normalized to the range [0.0 - 1.0], and the center coordinates and radii are interpolated/extrapolated accordingly; if the resulting radii become negative, their absolute values are used.

@PeterConstable
Copy link
Collaborator

PeterConstable commented Aug 29, 2022

Re Jonathan's suggestion of using absolute value, is there a gotcha in this part of the algorithm spec:

For all values of ω where r(ω) > 0, starting with the value of ω nearest to positive infinity and ending with the value of ω nearest to negative infinity, draw the circular line with radius r(ω) centered at position (x(ω), y(ω)), with the color at ω, but only painting on the parts of the bitmap that have not yet been painted on in this step of the algorithm for earlier values of ω.

The parameter ω starting at +ve infinity could represent either a r<0 end or the opposite end, depending on how c0 and c1 are specified. But either way, once pixels are painted, they're not re-painted. That would work if painting starts at the other end away from r<0, but not if it starts at the r<0 end.

@rsheeter
Copy link
Contributor

The intepretation of negative radius as just going the opposite direction but still producing the same circle proposed in #367 (comment) makes a lot of sense to me, at least on initial reading.

@rsheeter
Copy link
Contributor

The "hourglass" interpretation also seems defensible but iiuc it's less implementable so to me, given COLRv1 deliberately aspires to be implementation friendly, we should take the |r| interpretation.

@drott
Copy link
Collaborator Author

drott commented Aug 31, 2022

I think the change of the radius depending on the color line would need to be explicitly explained or specified, as otherwise that is suprising and not clear from the rest of the text. This is due to the extrapolation towards a zero-radius circle that can still display the smallest color stop, or in other words due to the internal implementation detail of attempting to normalize the color stops to 0,1.

image

@PeterConstable
Copy link
Collaborator

PeterConstable commented Sep 4, 2022

There's an odd behaviour that arises from taking the absolute value of the circle radii in combination with normalizing the colour line, interpolating circles to the normalized colour line, and then using the absolute values of the interpolated circle radii. I've crafted an HTML file that illustrates this: TestRadialGradientDefs.html.zip

If the beginning colour stop offset is < 0, we project an alternate of circ0, circ0Alt. Since circ0 is smaller than circ1, circ0Alt is even smaller. Decreasing the stop offset will, of course, eventually make the circ0Alt radius hit 0.

image

Now suppose the radius of circ0 is variable, and we start reducing its radius. Of course, that will also cause the radius of the projected circle, circ0Alt, to hit 0 and go negative even sooner.

image

If we continue to decrease the radius of circ0 but don't take the absolute value of the radius for circ0Alt, the latter will remain (<=) 0 as the circ0 radius is further decreased...

image

... until the radius circ0 has reached 0 and then the absolute value of circ0 radius gets large enough again.

But if you use the absolute value of the circ0Alt radius, then the behaviour gets weirder as the radius of circ0 continues to decrease. The radius of circ0Alt reaches 0, then immediately starts to get bigger while the radius of circ0 continues to get smaller but hasn't yet reached 0.

image

But that happens only for a short bit. When the radius of circ0 gets to 0, the radius of circ0Alt reaches a local maximum. Then it decreases again while circ0 grows because the absolute value of its radius is used. After the radius of circ0Alt reaches 0 again, it finally starts increasing monotonically as the delta for R0 gets more negative.

Repro steps using the HTML in the attached .zip:

  1. Click the checkbox to use the absolute value for the radii of interpolated circles.
  2. Set the begin stop offset to -0.4.
  3. Using the slider for R0 deltas, make the delta increasingly negative.
  4. Observe:
  • At R0.delta = -22, the radius of circ0Alt reaches 0.
  • At R0.delta = -40, the radius of circ0 reaches 0, and the radius of circ0Alt hits a local maximum.
  • At R0.delta = -58, the radius of circ0Alt has returned to 0.
  • For R0.delta decreasing below -58, the radius of circ0Alt continues to grow.

Given that odd behaviour, it might make more sense to use the absolute value of radii for circ0 and circ1, but not for the projected circles.

@drott
Copy link
Collaborator Author

drott commented Sep 5, 2022

Given that odd behaviour, it might make more sense to use the absolute value of radii for circ0 and circ1, but not for the projected circles.

I agree with you that the absolute value should be only taken once, but not of the original circle, but only for the final projected circles. In that sense, I don't think this step of the diagram reflects the intended behaviour.

image

Here, the projected circle on the left should be visible and the dotted lines indicating the filled are should be tangent to that.

@PeterConstable
Copy link
Collaborator

If the dotted lines aligned to the projected circle rather than the original circle, that would be diverging from the WHATWG createRadialGradient() algorithm spec.

@jfkthame
Copy link

jfkthame commented Sep 5, 2022

Assuming you're referring to https://html.spec.whatwg.org/multipage/canvas.html#dom-context-2d-createradialgradient, I'm not really understanding how it's relevant:

context.createRadialGradient() is spec'd to throw an exception if either radius is negative; so it can't create a gradient with negative radii.

And gradient.addColorStop() is spec'd to throw an exception if a stop offset is outside the interval [0.0 .. 1.0]. So it cannot set a color stop outside of the space between the two circles, such that our "normalization" of the color line would result in a projected circle that could become negative-radius.

So (AFAICT) that spec simply doesn't address the situations we're dealing with here.

@jfkthame
Copy link

jfkthame commented Sep 6, 2022

I've filed https://bugzilla.mozilla.org/show_bug.cgi?id=1789380 with a patch to adjust Firefox's behavior here such that it matches what Dominik has implemented in Chrome Canary.

@PeterConstable
Copy link
Collaborator

I'm trying to update my HTML page, but encountering odd behaviour given that both ends of the colour line might require projection beyond [0,1]. Is there a build of either browser that can be downloaded with the revised behaviour?

Something that puzzles me about this is that, intuitively, it seems that shifting stop offsets should not affect the shape of the cone/cylinder (that should be determined solely by the defined circles), yet this seems to entail that stop offsets < 0 or > 1 can alter the shape.

@drott
Copy link
Collaborator Author

drott commented Sep 7, 2022

Up to date FF nightly and Chrome Canary now have the same behavior, I checked this morning.

@PeterConstable
Copy link
Collaborator

I've revised my HTML page to do what we discussed—extrapolate circles c0' and c1' from c0 and c1 for a colour line with begin stop offset < 0 or end stop offset > 1, and take the absolute value of the radii for those extrapolated circles (but not for r0 or r1 if deltas make them < 0). After revising calculations, I compared with the change Jonathan made in Gecko, and the logic is the same.

TestRadialGradientDefs_Revised.zip

In this revised HTML page, I also added an extra feature: Previously, I drew lines to show the envelope cylinder/cone defined by c0 and c1. Now, in addition, I draw separate lines to show the envelope cylinder/cone defined by c0' and c1'. This highlights a concern with the proposed revision of the algorithm.

As spec'd in OT1.9, the shape of the envelope cylinder/cone is determined solely by the specification of the circles, c0 and c1. In particular, the shape is not affected by the location of stop offsets. With the proposed revision of the algorithm, that continues to hold true but only so long as the radius of the extrapolated circles remains >= 0.

Here's an example with a begin offset at -0.6 and end offset at 1.2. The circles with dotted outlines are c0 and c1; the circles with solid outlines are c0' and c1'.

image

Note that the shape of the cone obtained from c0' and c1' is the same as that obtained from c0 and c1.

Now suppose the begin stop offset were variable and animated to move to -1.3. Here's what happens:

image

Note that the cone obtained from c0' and c1' is a different shape than the cone obtained from c0 and c1.

This behaviour is visible using the test font and html page that Dominik has shared (separately in email) using the latest Firefox Nightly build (106.0a1 (2022-09-07)). In the following video clip, the COL1 variation axis of the font is manipulated to change the begin stop offset. In particular, notice what happens with the three radial gradients in the bottom line (same except for different extend modes).

Firefox.Nightly.2022-09-07.17-23-58_Trim.mp4

First, COL1 is moved from 0.02 to 0.36, moving the begin stop away from the tip of the cone—everything is fine. But then COL1 is manipulated to move the begin stop toward the tip of the cone. The stop reaches the tip at around -0.46, and up to that point the shape of the cone has not been affected. But beyond that the shape of the cone changes; this is when the projected radius for c0' goes negative, and its absolute value is taken.

I think this is a problem in a couple of ways. First, it hinders an effect that designers may want to use in variable fonts, having a gradient flow off the end of the cone. Secondly, unless we spec this as required behaviour, it will lead to non-interoperability of fonts between different implementations. For example, the following video clip shows basically the same scenario with the same test font, but this time in a test app using DWriteCore (Experimental release):

Font.Viewer.2022-09-07.17-38-48_Trim.mp4

(Note: Nick has a bug in his implementation vis-à-vis the OT1.9 COLR spec, overlooking the clause, "For all values of ω where r(ω) > 0," leading to the cone being continued beyond the tip. While it is a bug, it is helpful in showing the colour line continuing beyond the tip without the stop offset affecting the shape.)

@PeterConstable
Copy link
Collaborator

PeterConstable commented Sep 8, 2022

Somewhat ironically, we began discussing a year and a half ago (#148) whether there needed to be colour stops within the [0,1] range, and I asked whether we'd encounter any constraint in existing graphics libraries. The response I got was, "If any graphics library has such limitations, implementation of COLRv1 on top of that library needs to add extrapolated points as necessary." But what wasn't considered then is whether it would be ok to have the color line affect the shape of the gradient rather than just determining the colours in the gradient.

@drott
Copy link
Collaborator Author

drott commented Sep 9, 2022

If the implied suggestion in the previous comment is to restrict to [0,1] for color stops, or restrict to >= 0 for radii, I don't think that will work without somewhat counterintuitive special casing and redefinitions of how color lines are to be interpreted. Please note that constraining to a [0,1] interval in drawing was one of the reasons for the problems we've had with PaintVarSweepGradient.

We're faced with the following observations:

  • Constraining color stops to [0,1] is not a sufficient solution to this problem, because the radius on either end can turn negative from variations applying to the radius itself.
  • Constraining the radii to >= 0 is not sufficient either, as color stops can lie outside the [0,1] range and shaders can't handle negative radii and expect [0, 1] normalised stops (Docs: Skia, Cairo) - The practical approach is interpolation: Two implementations choose to solve this this by normalising color stops and interpolating new circles accordingly.
  • Combining multiple shaders and applying clipping in between them is not deemed practical and may lead to pixel artefacts and/or performance issues.

To define some kind of clamping then, we would end up with a somewhat complicated definition of what to constrain to, i.e. clamp to the extrapolated position where the combination of color stop offset and radius slope (between the two circles) hits 0. But then we're back to adapting the result to shader API: We would need to give the shader a manually interpolated color stop and change the color line, which leads to repeat mode problems:

  • If the shader does not receive full/"unclamped" information about all color stops, it can't repeat/reflect correctly, as not all colors and stops are specified and thus not visible to the shader.
  • If we specify color stops outside the expected range for the shader, we get undefined behaviour. As an example, the Skia shader does then not repeat correctly, see this Skia Fiddle.

I think this is a problem in a couple of ways. First, it hinders an effect that designers may want to use in variable fonts, having a gradient flow off the end of the cone. Secondly, unless we spec this as required behaviour, it will lead to non-interoperability of fonts between different implementations.

Of course the goal is to achieve tight interop, that's why we're discussing this issue.

I agree that the "geometrical" effect is undesirable, however:

  • I (and I believe @jfkthame) consider it the least intrusive adjustment to the spec: it only occurs in nice scenarios: when the gradient is a "cone" and when the projected circle reaches < 0 - where the circles are not contained within each other. Regarding the cone effect: I don't think a gradient cone should be use for glyph geometry - a PaintGlyph clipping mask should be use for that instead.
  • this proposed solution is feasible to implement with a number of existing, often hardware-accelerated shaders.
  • this proposed solution does not introduce discontinuities or unexpected reinterpretations of the color line
  • it keeps repeat modes intact

@PeterConstable
Copy link
Collaborator

I wouldn't suggest any change affecting radial or (as we have in OT1.9.1 alpha) sweep gradients, neither of which have any such issues. But radial / two-point conical gradients are different because they implicitly define a shape. I've asked some other graphics people within MS and heard the same intuition: the colour line definition should not affect the shape.

PDF, HTML Canvas 2K, Skia, Cairo all support this type of gradient, and in all of them the colour line does not affect the shape. But they also impose constraints that---so far---we have not: the colour stop offsets for this type of gradient must be within [0.0, 1.0], and the circle radii must be > 0.

If we were to clamp the stop offsets for radial gradients to [0, 1] and clamp the circle radii to >=0, then I think it would provide the following:

  • it would be easy to describe in the spec and would be a less intrusive revision,
  • it would be easily to implement in Skia and Cairo, and is closer to those and to Canvas 2D and PDF,
  • it does not introduce discontinuities or unexpected reinterpretation of the colour line,
  • it keeps repeat modes intact, and
  • it would provide a more intuitive behaviour in which the colour line does not affect the shape.

To define some kind of clamping then, we would end up with a somewhat complicated definition of what to constrain to,

Not complicated at all: as suggest above, there is no extrapolation of circles, no interpolation of intermediate colours, and only one extra check beyond what is already spec'd: if stop offset (for a given variation instance) is < 0, set to 0; if > 1, set to 1.

Regarding the cone effect: I don't think a gradient cone should be use for glyph geometry - a PaintGlyph clipping mask should be use for that instead.

We, the format requires PaintRadialGradient to be used in a PaintGlyph fill, so PaintGlyph as a clipping mask is necessarily going to be involved. Yet the geometry is implicit in this type of gradient, just as it would be for mesh gradients. We should expect fonts will take advantage of that, e.g. having a clipping mask that doesn't mask all non-painted portions of the gradient.

But also—and more importantly—because the colour line is affecting the shape, it also affects the iso-colour contours of the gradient. Even with a PaintGlyph clipping mask, this will be noticeable, and could be frustrating for designers.

I can be convinced that the concern I raise isn't a blocker. But I want to be cautious with this since we need to get it right, and soon (we know there are more implementations on the way), and would not want to end up with something that designers forever find counterintuitive. Hence, I'd really like to see a wider range of input than just you, Jonathan and me.

@jfkthame
Copy link

I think we have essentially two options to resolve the issue here in a well-defined and widely-implementable way:

(a) Let the radius-variation and color-line normalization & extrapolation math do its unconstrained thing; and then if we find ourselves with negative radii -- which many shader APIs cannot handle -- we take the absolute value, as currently implemented in Firefox NIghtly & Chrome Canary. (A minor alternative would be to clamp negative radii to zero instead of taking the abs value, but there's no particular benefit to that -- the result is still a distortion of the "ideal" geometry.)

(b) Avoid allowing negative values to arise, by clamping the effect of any variations on the radii and restricting the color stop range to [0, 1] so that extrapolation of the circles is not required to normalize the stops. The gradient is then directly implementable with common shader APIs.

As noted, approach (a) has the disadvantage that in some cases it leads to an unexpected change to the shape of the gradient. But a designer can avoid this by not venturing into those areas of the color-line and radius-variation space. Essentially, if the designer respects the constraints of (b) in their definition of the gradient, instead of relying on the implementation to clamp things, the result will be the same and the gradient shape will be unaffected.

The main reservation I have about approach (b) is that it means a given color line may behave quite differently for radial gradients than the same color line used by a linear gradient. This seems to me at least as unfortunate (and unexpected for designers) as the unexpected effect of the color stop range on the shape of the radial gradient.

Consider a simple color line with three stops: [red @ 0.0, green @ 0.5, blue @ 1.0], and two gradients that share this color line: a linear gradient from (0, 500) to (1000, 500), and a radial gradient from (0, 500; r=500) to (1000, 500; r=500). These will be similar-looking gradients progressing from left to right, with the difference that the radial gradient has curved color contours while the linear gradient has straight.

However, now suppose we vary the color stop offsets by adding 1.0 to all of them. For the linear case, this simply shifts the colors within the gradient to the right (entirely intuitive). But for the radial (curved) gradient, under proposal (b) the offsets all get clamped to 1.0, and the result is -- what? Presumably the gradient collapses to a simple red/blue fill with a curved edge between them? Whereas (a) would allow the colors of the curved gradient to shift rightwards just like its linear counterpart.

Perhaps, if we choose (b) here (to avoid distorting the geometry of the radial gradient in these edge cases) we should apply the same clamping of the color stop offsets to all gradients, so that color lines behave more consistently?

@PeterConstable
Copy link
Collaborator

PeterConstable commented Sep 10, 2022

Consider a simple color line with three stops...

It may or may not be obvious to designers to craft a font in such a way that a ColorLine table is actually shared, but the example is valid. Note, though, that your example has the two circles with identical radius. That's a special case in which the distortion issues don't arise. If, instead the example had (0, 500; r=500) to (1000, 500; r=600). then the curved contours will start changing, which the designer might not expect.

A further critique of option (b) that had not occurred to me earlier is that, if a color stop offset were to vary to < 0 and get clamped to 0, then that will affect not just colors at one end of the gradient but also how the pattern mirrors and reflects. So, for example, if two stops toward the tip of a cone were getting compressed because both offsets are varying but one is clamped, then that compression will also occur in each repeat/mirror of the pattern.

I wouldn't want to end up clamping for all gradients: conceptually, the colour line is infinitely long, and it shouldn't matter what sub-range is used to specify its pattern. I think the radial gradient behaviour as currently spec'd is great. It's really unfortunate that some graphics implementations didn't allow for more flexibility in the colour line description. The kinds of animated variation behaviours we can provide in COLRv1 could also be desirable in other contexts.

Side note: it turns out that Direct2D's behaviour for colour lines in which the begin offset is > 0 or the end offset is < 1 is that the begin colour is padded out to 0 / end colour is padded out to 1. And if offsets are outside [0, 1], then the offset is ignored. So, if a stop were animated out of the [0,1] range, then that colour will disappear from the (repeated / mirrored) pattern.

@jfkthame
Copy link

It may or may not be obvious to designers to craft a font in such a way that a ColorLine table is actually shared...

True, though whether it is actually shared or not is irrelevant; independently-defined gradients that happen to have similar color-stop placement (and are perhaps subjected to similar variations) would also be expected to behave similarly whether they're linear or radial, but under option (b), they may fail to do so.

A further critique of option (b) that had not occurred to me earlier is that, if a color stop offset were to vary to < 0 and get clamped to 0, then that will affect not just colors at one end of the gradient but also how the pattern mirrors and reflects. So, for example, if two stops toward the tip of a cone were getting compressed because both offsets are varying but one is clamped, then that compression will also occur in each repeat/mirror of the pattern.

Yes; that's essentially the same issue as Dominik mentioned earlier:

If the shader does not receive full/"unclamped" information about all color stops, it can't repeat/reflect correctly...

In view of these issues with (b), I'm finding myself increasingly convinced that (a) is the most reasonable solution available to us. I don't like the fact that it allows the color-stop offsets to perturb the gradient geometry, but this is something that can be spec'd and implemented interoperably; and designers can be warned by a note in the spec about the cases where rendering will deviate from the "ideal" shape.

@PeterConstable
Copy link
Collaborator

A different variant of (b) could be to treat stops outside [0,1] as D2D currently does: ignore them. It would still affect the pad/repeat/reflect patterns, so from your example comparing similar radial and linear gradients, it would create a difference — unless the same were also done for linear.

If the Cairo and Skia shaders can't be modified to be more flexible wrt the colour line, it does seem that we'd need to choose between compromising the colour pattern or compromising the geometry.

@nedley
Copy link

nedley commented Sep 15, 2022

I am uncomfortable with any solution where the geometry of the gradient is influenced by its color stops or vice versa. COLRv1 implementations should clip or ideally not draw any section with negative radius.

By way of comparison, https://www.w3.org/TR/css-images-3/#radial-color-stops is clear about the behavior of CSS color stops outside the normalized range: such stops will still mathematically influence colors in the normalized range but are not themselves drawn. Since this model is well-defined and already implemented, I suggest hewing as closely to it as possible by allowing stops outside [0, 1], selecting colors only from the normalized range.

My general concern here is that this conversation feels like we are trying to backsolve for some number of existing implementations, rather than figuring out the correct behavior and specifying that.

@jfkthame
Copy link

I don't think the CSS gradients spec really addresses the radial-gradient issue here, because it doesn't allow for defining separate starting and ending circles, such that the focus of the gradient falls outside the circles.

@nedley
Copy link

nedley commented Sep 15, 2022

I believe COLRv1 radial gradients with r1 > r0 can be expressed as a CSS radial gradient by extrapolating to the starting point, at least geometrically. But my point (of argumentation) is that, since I am not aware of any implementation that exactly maps to what COLRv1 allows, we should avoid a Procrustean resolution as much as we can.

@drott
Copy link
Collaborator Author

drott commented Sep 15, 2022

@nedley thanks for commenting and looking into this issue. I agree the geometry impact is suboptimal, however, we've from various angles hit the barrier of then having to change fundamentally and maintain modified hardware accelerated shader programs if we don't have a mapping to the existing ones. We find that these are usually optimised towards covering the usual CSS cases - but in this scenario we're hitting situations that are outside of the behaviour of the equivalent CSS gradients.

From the drawRadialGradient documentation I find that the CoreGraphics API also expects 0 and 1 normalized color stops. How tightly optimized are the CoreGraphics shaders, and how easy or difficult would you find it to change these and the CoreGraphics API to accept non-normalized color stop offsets or handling negative radii? (as clipping or color line truncation are not complete solutions) - would this be a solution you would prefer to finding a reasonable mapping to existing shaders?

In the design of COLRv1 we've usually tried to map to existing primitives to avoid overhead implementing new primitives in the graphics libraries - I tend to think it makes more sense to lean in the direction of following that principle here as well.

@nedley
Copy link

nedley commented Sep 15, 2022

From the drawRadialGradient documentation I find that the CoreGraphics API also expects 0 and 1 normalized color stops.

Core Graphics doesn't support repeated gradients at all, so I wouldn't consider it to be prior art in this case.

@litherum
Copy link

litherum commented Sep 15, 2022

Imagine someone is visualizing a satellite communicating with Earth, and they want to show the communication waves by animating the color stop locations on a radial gradient cone. This seems like a natural use case for radial gradients and animations, which much of this thread seems to be about.

For most of the animation, they will achieve their effect. But then when the values and points align just right, the communication cone starts widening? This wouldn't be what the author wanted.

I understand that extrapolating + running an absolute value is computationally simple, but I don't think it serves font creators / users well.

@drott
Copy link
Collaborator Author

drott commented Sep 15, 2022

Imagine someone is visualizing a satellite communicating with Earth [...] For most of the animation, they will achieve their effect. But then when the values and points align just right, the communication cone starts widening? This wouldn't be what the author wanted.

This visualization is entirely possible without any unwanted side effects at all if color stop offset 0 and radius 0 are chosen and the source and destination circle center points are moved. What I am trying to make clear: The geometric effects of the proposed solution here only affects IMO very rare edge case situations in which negative radii or negative color stops are chosen or occur due to variations.

@nedley
Copy link

nedley commented Sep 15, 2022

Let me put it differently: if it is a choice between compromising the geometry or the color line, we would rather compromise the color line.

@PeterConstable
Copy link
Collaborator

his visualization is entirely possible ...

If the visualization Myles has in mind is what I think it is, then your suggestion does not provide that visualization. Moving the destination circle makes the wave stretch, not propagate; moving both circles would appear like a pulse that is moving but not spreading; increasing the two radii would appear like a spreading wave, but only a short pulse. If you wanted to visualize a continuous emmission that spreads, the circles would need to remain the same and you'd need colour stops that animate from before the tip of the cone then along the cone.

@PeterConstable
Copy link
Collaborator

Let me put it differently: if it is a choice between compromising the geometry or the color line, we would rather compromise the color line.

Going back to the start of this thread:

But specifying manually a start/end interpolated color and truncating the color line then means that the shader is not aware of the full color-line and repeat modes stop working correctly after the first iteration of the color line.

There appear to be two conflicting perspectives:

  • The extended colour line should not be compromised, therefore we need to compromise the geometry.
  • The geometry should not be compromised.

There is also a third perspective, which is that neither colour nor geometry should need to be compromised, and I showed above Nick's current DWriteCore implementation which doesn't require any compromise to the colour or geometry. But that is in conflict with one other perspective: that this should be implementable using existing Cairo cairo_pattern_create_radial() and Skia MakeTwoPointConical() APIs.

@jfkthame
Copy link

there is a very dramatic discontinuity going from 0 to a small negative number

That's not my observation with the implementation I'm working on (which does not directly implement the algorithm from the spec, but maps the gradient specified by the font to existing shader APIs as we've been discussing here); the behavior as a radius passes through zero is nicely continuous.

@niklasb-ms
Copy link

That's not my observation with the implementation I'm working on (which does not directly implement the algorithm from the spec, but maps the gradient specified by the font to existing shader APIs as we've been discussing here); the behavior as a radius passes through zero is nicely continuous.

I get that, but if the algorithm described in the spec produces bad results for negative radii, then we either need to change description of the algorithm or disallow negative radii. I say "if" because it's possible there's just an error in my implementation of the algorithm. An extremely literal implementation would be impractical, since step 3 involves looping over all possible floating point values of w. Instead, I compute the color of a given point by solving for the largest value of w, such that the given point is on the circle centered at x(w), y(w) with radius r(w) and where r(w) is greater than 0.

It would take some analysis to determine whether the algorithm in the spec needs revising to work with radii less than zero. Or we could just require that radii be non-negative. If time is a concern, the latter seems easier, and I don't think it's a bad solution.

@PeterConstable
Copy link
Collaborator

PeterConstable commented Sep 30, 2022

r(ω) can perfectly well be greater than 0 for some values of ω even if one or both of r0 and r1 are negative.

If both are negative, how could r(ω) could ever be positive?

Something else that's not obvious for (3): Again, suppose circle1 is constant and the radius of circle0 is varying to and beyond 0 (i.e., goes negative). As mentioned earlier, as r0 decreases, the cone tip moves toward circle1.

But how far does it go? In this example, the radius of circle0 is varying, but the position of the center c0 is not. When r0 is zero, the tip is at c0. What's the math that indicates that the tip moves any further toward c1? Is the tip approaching c1 asymptotically? Or does the math allow it to progress beyond c1?

I'm starting to wonder if adopting (3) would require us to specify explicitly the math used to derive circle0ʹ (the moving tip) rather than implicitly assume it's clear what is expected.

I think I have conceptually in my mind how to describe what is happening geometrically (corollary: tip approaches c1 asymptotically, similarly to r0 > r1 and growing very large, with the tip approaching c1 from the opposite direction). But it's definitely non-trivial and requires modifying how it's described from when r0 >= 0 to when r0 < 0. This is swinging me away from option (3) toward option (1).

@PeterConstable
Copy link
Collaborator

PeterConstable commented Sep 30, 2022

Btw, I have prototyped a codepen implementing radial gradients in HTML Canvas 2D using stop offsets not limited to [0, 1]. This assumes that radii are clamped to non-negative. Color transitions smoothly when the begin or end stop offset moves past the cone tip, using an interpolated color at the tip for the normalized colour line.

In this prototype, I've started on some parts that would be needed to implement repeat or mirror extend modes (in particular, creating a clip for a section of the cone), but have only completed implementing pad extend mode.

I was surprised at how many distinct cases for pad extend mode needed separate logic:

  • cylinder with begin/end the same
  • cylinder with begin/end different
  • cone with begin/end the same and beyond the tip
  • cone with begin/end the same but not beyond the tip
  • cone with begin/end different and both beyond the tip
  • cone with begin/end different and neither beyond the tip
  • cone with begin/end different and end beyond the tip
  • cone with begin/end different and begin beyond the tip

I'm curious if I'm overlooking a simpler approach.

@jfkthame
Copy link

If both are negative, how could r(ω) could ever be positive?

The algorithm says:

For real values of ω:
...
Let r(ω) = (r₁-r₀)ω + r₀

When, for example, r₁ = -1 and r₀ = -2, the expression (r₁-r₀)ω + r₀ simplifies to r(ω) = ω - 2, so it'll clearly be positive whenever ω > 2. Or swapping the radii: when r₁ = -2 and r₀ = -1, we get r(ω) = -ω - 1, which is positive for ω < -1.

The only case, I think, where r(ω) will never become positive is when the radii are negative and equal, so that r₁-r₀ is zero and the entire term that varies with ω collapses. That's the "imaginary cylinder" that never converges to a tip and then projects into visible space.

@drott
Copy link
Collaborator Author

drott commented Sep 30, 2022

Yes, I agree with @jfkthame - negative and unequal will have a projection, even if it's potentially far. Negative and equal r won't render anything. So in the spec we need to describe what happens and how it's handled: a) Radii becoming negative through variations b) far color stops require projected circles for shaders (in the form of a note for implementors) and may end up with negative radii c) negative radii are handled as follows: projected to a positive repetition according to repeat mode (1x for repeat or 2x for reflect), or truncated in case of pad.

@PeterConstable
Copy link
Collaborator

The algorithm says:...

Of course. Instead of reading, I was relying on recollected intuition of what it's doing--which (at least, for Canvas) is interpolating and extrapolating positive radii.

Helping readers intuitively understand that, from two negative radii, a positive radius can be extrapolated would be one more challenge. (Forget any intuition; just pay careful attention to the math.)

@PeterConstable
Copy link
Collaborator

PeterConstable commented Sep 30, 2022

Repeat/mirror modes may be easier since there's a pattern repeating along the entire colour line, so we can project any two circles separated by wavelength to normalize the colour line.

With two negative (and different) radii, how to explain what to draw for pad mode? Normalize a color line using the tip as one of the projected circles, and what other circle, with the color line normalized how?

I assume offsets 0 and 1 still align to those "circles". The tip is where r(ω) = 0, hence (r₁-r₀)ω + r₀ = 0 or ω = r₀ / (r₀ - r₁).

Side rambling while I try to wrap my head around this:

For a given r₀ < 0, as r₁ approaches -∞, ω approaches 0 from the negative side; and as r₁ < 0 approaches 0, ω from the > 1 side. In between, as r₁ approaches r₀, ω approaches ±∞.

For a given r₁ < 0, as r₀ approaches -∞, ω approaches 1 from the <1 side; and as r₀ < 0 approaches 0, ω approaches ∞, but jumps to 0 for r₀ = 0. And in between, as r₀ approaches r₁, ω approaches ±∞. (Two values of r₀ yield +∞?)

??

For pad, any colours need to be determined by stops with offsets not "beyond the tip" (in negative radius territory) or, if all are beyond the tip, the stop closest to the tip. So to provide an algorithm, you need to explain how to determine the tip offset and determine which portion(s) of the (non-repeating!) colour line align to radii > 0. The offset for the tip isn't difficult to explain in terms of the math. But it's harder to explain in general which offsets will provide a radius > 0 given that (with (r₁-r₀)ω + r₀) the magnitude of r₀, relative magnitudes of r₁ and r₀, and the value of ω are all contributing factors. And it's still not clear to me whether parts of the colour line could ever cross discontinuities or multiple negative radii ranges (i.e., discontiguous +ve radius ranges — hyperbolic conic sections come to mind).

image


Addendum:

  • r(ω) = (r₁-r₀)ω + r₀

Hence, r(ω) > 0 if

  • (r₁-r₀)ω + r₀ > 0
  • (r₁-r₀)ω > -r₀
  • ω > r₀ / (r₀ - r₁) if r₁ > r₀
    or
  • ω < r₀ / (r₀ - r₁) if r₁ < r₀

For given instance values of r₀, r₁, the circular cone sections will have +ve radii for a continuous range of ω from ωtip to ∞ if r₁ > r₀, or from ωtip to -∞ if ₁ < r₀,.

So, maybe the explanation is just in terms of the math:

If variation deltas result in r₀ or r₁ becoming negative:

  • If r₀ = r₁, paint nothing. Else...
  • Determine the offset of the cone tip: ωtip = r₀ / (r₀ - r₁)
  • If r₁ > r₀, paint the region for ω from +∞ to ωtip, as described.
  • Else (i.e., r₁ < r₀), paint the region for ω from ωtip to -∞, as described.

For implementations that have constraints regarding stop offsets, there will be several more details for the implementer to figure out in terms of using the appropriate portion of the specified color line, with interpolated color values and inter-/extrapolated circles as necessary.

@jfkthame
Copy link

Here's a javascript implementation of rendering a radial gradient directly using the algorithm from the spec, i.e. iterating over "all" values of ω and painting circles using the computed values of x(ω), y(ω) and r(ω). Obviously, it's an approximation, in order to run in reasonable time, but it lets you experiment with varying the radii and exploring how the cone is automatically "projected" by the algorithm when the radii are negative.

@PeterConstable
Copy link
Collaborator

Thanks. That's definitely helpful — and easier to implement by direct application of the algorithm and drawing circles rather than using createRadialGradient() 😉.

I forked it and widened the radius controls and increased their range by 10x to confirm what I thought: as either radius gets largely negative, the tip approaches the center of the opposite circle asymptotically.

image

And it helped clarify for me that, if the radii are both negative but only slightly different, the tip goes to ±∞ in the direction of whichever circle has the larger r.

image

It's still not intuitive to think of in terms of conic sections, if for no other reason than the ±∞ discontinuity for r0 = r1. And, for pad mode, because the tip quickly goes outside c0 to c1, stops within the [0, 1] range except the first or last (depending on the direction) may have no visible effect unless there is a stop well outside [0, 1]:

image

@niklasb-ms
Copy link

Ok, my bad. I just debugged this and it turns out I had an explicit check for negative radii. If either r0 or r1 was negative, I would treat the gradient as ill-formed and draw nothing. When I remove that check it works fine.

Given that, I'm fine with allowing negative radii and feel confident that the existing algorithm described in the spec works fine for negative radii.

I thought I remembered there being language in the spec saying radii cannot be negative, hence the check, but I must have been mistaken because I can no longer find such language.

@PeterConstable
Copy link
Collaborator

as either radius gets largely negative, the tip approaches the center of the opposite circle asymptotically.

Similarly, as either radius gets largely positive, the tip approaches the center of the opposite circle asymptotically, but from the opposite direction.

@PeterCon
Copy link

PeterCon commented Oct 1, 2022

I've started to incorporate changes into OT1.9.1 alpha. In the COLR table chapter, in the prose section on color lines, I've added clarifications regarding color stop ordering and variations, and also added a note regarding platforms constraining offset ranges and the possible implementation workaround of normalizing color stop offsets. There's a bit more on that that will need to get added in the section on radial gradients, as well as discussion of variation deltas resulting in negative radii.

@drott
Copy link
Collaborator Author

drott commented Oct 3, 2022

I've started to incorporate changes into OT1.9.1 alpha.

Thanks! (would you mind preparing a patch for this repo as well?)

My feedback on some of the wording there, only looked at color line section so far.

As stated above, color stops can be defined in a font with offsets in the range [-2, 2).

I suggest to drop that sentence, and then say...

However, some application platforms could restrict color line specifications to a limited range, such as [0, 1].

To me, this reads like the application platform would somehow restrict the font. I would rephrase that more widely:

"With the above definition, color lines may exceed the [0,1] interval or be defined on a narrower interval than [0,1]. When mapping such color lines specified to application platform provided gradient implementations, one may find that the latter require color stops to be exactly aligned within a [0, 1] interval to work correctly.

Implementations can work around such constraints by normalizing the offsets of the defined color stops.

Yes.

For exammple, if constrained to the range [0, 1], map the lowest stop offset to 0, map the largest offset to 1, and linearly interpolate normalized offsets for other stops in between.

Typo, s/exammple/example/.

@jfkthame
Copy link

jfkthame commented Oct 3, 2022

As stated above, color stops can be defined in a font with offsets in the range [-2, 2).

If you do keep some form of this statement, it may be worth noting that although color stops can only be defined in this range (because of the 2.14 data type used), their offsets could subsequently be adjusted by variation deltas to positions outside this interval.

moz-v2v-gh pushed a commit to mozilla/gecko-dev that referenced this issue Oct 3, 2022
…i. r=gfx-reviewers,lsalzman

This implements the adjusted behavior agreed with Dominik in
googlefonts/colr-gradients-spec#367
and email discussion. Dominik is updating Blink to have the
same behavior.

Differential Revision: https://phabricator.services.mozilla.com/D158429
pull bot pushed a commit to TheRakeshPurohit/skia that referenced this issue Oct 3, 2022
Implementation of proposed resolution in
googlefonts/colr-gradients-spec#367.

When radii become zero, in pad case, truncate to a an interpolated
color. In other modes, find a suitable repetition of the color line
which can be specified using positive radii, but paints equivalently.

Tests added for testing a couple of edge cases for negative radii.

Thanks to Jonathan Kew for providing early test builds of Firefox to
compare implementations.

Bug: skia:13708
Cq-Include-Trybots: luci.skia.skia.primary:Test-Android-Clang-GalaxyS20-GPU-MaliG77-arm64-Release-All-Android_NativeFonts
Change-Id: Id924780291e563349048e5de6bdd512c5cc2494e
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/584376
Commit-Queue: Ben Wagner <bungeman@google.com>
Commit-Queue: Dominik Röttsches <drott@chromium.org>
Auto-Submit: Dominik Röttsches <drott@chromium.org>
Reviewed-by: Ben Wagner <bungeman@google.com>
@PeterConstable
Copy link
Collaborator

Updating per feedback from drott, jfkthame.

(would you mind preparing a patch for this repo as well?)

I'll work more on this tomorrow.

@PeterConstable
Copy link
Collaborator

I've completed drafting changes to the COLR spec for OT191α:

  • In the "Color lines" section, added a note on color line normalization (mentioned earlier, with feedback incorporated).
  • In the "Linear gradients" section, added a sub-section, "Note on implementation of linear gradients", providing guidance on normalization of the color line and the required adaptation of the geometric definition.
  • In the "Radial gradients" section:
    • Small revision in the introductory paragraphs.
    • Added a paragraph toward the end of the section describing negative radii in variable fonts and the mathematical interpretation, along with a note on specific calculation of offsets for the tip and for r(ω) > 0.
    • Add a sub-section, "Not on implementation of radial gradients" describing platform implementations requiring non-negative radii, that simple color line normalization can result in negative radii, and then several points to guide implementers on working around that combination of issues.

I didn't make any (further) changes in the "Sweep gradients" section since I think the revisions that Dominik and I worked out earlier in the summer already accommodated color line definitions for intervals other than [0, 1].

See the delta version for tracked changes.

Please review and provide feedback on the OT191α revisions; then I'll prepare a PR for this repo.

@drott
Copy link
Collaborator Author

drott commented Oct 5, 2022

Thanks, Peter, feedback on the updated wording:

For example, if constrained to the range [0, 1], map the lowest stop offset to 0, map the largest offset to 1, and linearly interpolate normalized offsets for other stops in between. For correct alignment to the geometry of a gradient definition, it can be necessary to interpolate or extrapolate alternate geometric values to align with the normalized 0 and 1 stop offsets.

The paragraph in the color line section as a whole works well now, thanks for the update. I would perhaps generalize this part of the note still, as with interpolated color stops one will always need updated geometry.

Proposed new wording:

For example, for a specified range narrower or wider than the [0,1] interval, map the lowest stop offset to 0, map the largest offset to 1, and linearly interpolate normalized offsets for other stops in between. Then, for correct alignment to the geometry of a gradient definition, interpolate or extrapolate alternate geometric values to align with the normalized 0 and 1 stop offsets.

This can be done as follows: if omin is the minimum stop offset in the defined color line, omax is the maximum offset, and vector v = p₁ - p₀, then let p₀′ = p₀ + ominv p₁′ = p₀ + omaxv p₂′ = p₂ + ominv

FWIW, this is not how we implement this. We compute a new p0-p3 gradient, compare implementation note in nanoemoji and Skia code, then we interpolate along that vector for the shifted color stops. So I can't vouch for correctness of the the projected p2' that you're describing. I would suggest to strip this part starting from:

This would require deriving alternate geometric points for the linear gradient, p₀′, p₁′ and p₂′. […]

And replace it with:

and interpolate geometric coordinates for the gradient parameters so that an equivalent gradient is drawn.

(which would be more similar to the radial section as well IMO.)

Implementation note for radial looks good to me, thanks for phrasing that.

...then I'll prepare a PR for this repo.

🤩 Thanks.

jamienicol pushed a commit to jamienicol/gecko that referenced this issue Oct 5, 2022
…i. r=gfx-reviewers,lsalzman

This implements the adjusted behavior agreed with Dominik in
googlefonts/colr-gradients-spec#367
and email discussion. Dominik is updating Blink to have the
same behavior.

Differential Revision: https://phabricator.services.mozilla.com/D158429
@PeterConstable
Copy link
Collaborator

For example, for a specified range narrower or wider than the [0,1] interval...

This overlooks the case of the specified range overlapping without being narrower or wider than [0, 1]. I think the current wording for the first sentence is fine. I will revise the second, though.

I would suggest to strip this part starting from...

I don't see a need to remove all of that, but I will revise the opening sentence along the lines you suggest so as not to assume p₀′, p₁′ and p₂′, then provide details for p₀′, p₁′ and p₂′ as one possibility.

@PeterConstable
Copy link
Collaborator

OK, I've revised taking your comments into account.

  • remarks in the Color lines section are generalized and worded such that geometric adjustments are always needed
  • in Note on implementation of linear gradients, initial wording about deriving alternate geometric points is generalized, and then I provide details for p₀′, p₁′ and p₂′ as before, but then added similar details if using p₀ and derived p₃.

@drott
Copy link
Collaborator Author

drott commented Oct 6, 2022

Thanks for the update - LGTM with remaining comments below:

the definition of a color line can span an interval greater than, narrow than,

Typo: *narrower

Then, for correct alignment to the geometry of a gradient definition, it would be necessary to interpolate or extrapolate alternate geometric values to align with the normalized 0 and 1 stop offsets.

Nit / Optional: Strip "it would be necessary to", sounds like a filler to me here.

p₃′ = p₀ + omaxv

I think this should be p₃′ = p₃ + omaxv - perhaps it may make sense for the second section, where the scaling is described for p0 and p3 to call the vector w or something other than v to emphasize it's not the same one, i.e. it's p3-p0 there and not p1-p0 as in the first case.

As a side note: Looking closer at the note using the p0, p1, p2 descriptions, it seems correct to me - as the only thing we need to maintain after the scaling is that difference vector between p0 and p2 stays the same, so that the gradient incline stays the same.

@PeterConstable
Copy link
Collaborator

Thanks for catching the typo. I'll shorten that other sentence, and use w as suggested.

I think this should be p₃′ = p₃ + omaxv

No, this is analogous to p₁′. Consider the case when omax = 1.

This brings to mind a special case for normalization: If omin == omax, then it might be necessary to add one or two stops to provide a non-zero-length interval.

@drott
Copy link
Collaborator Author

drott commented Oct 6, 2022

I think this should be p₃′ = p₃ + omaxv

No, this is analogous to p₁′. Consider the case when omax = 1.

Yes, you're right, sorry, I got confused. Corresponding code here.

This brings to mind a special case for normalization: If omin == omax, then it might be necessary to add one or two stops to provide a non-zero-length interval.

I am not sure which case you have in mind - intuitively I would say there's no extra wording required, as we already have:

"If either point p₁ or p₂ is the same as point p₀, the gradient is ill-formed and must not be rendered.

If line p₀p₂ is parallel to line p₀p₁ (or near-parallel for an implementation-determined definition), then the gradient is ill-formed and must not be rendered."

I think that's sufficient for saying what should happen if color stops / interpolated points fall onto the same exact location, isn't it?

@jfkthame
Copy link

jfkthame commented Oct 6, 2022

This brings to mind a special case for normalization: If omin == omax, then it might be necessary to add one or two stops to provide a non-zero-length interval.

That's exactly what I did in https://bugzilla.mozilla.org/show_bug.cgi?id=1793611 in order to make it possible to "project" the radial-gradient geometry appropriately when the specified color line is zero-length.

Example: with https://jfkthame.github.io/test/radial/index.html, paste

s1.value = 2; s2.value = 2; s3.value = 2; update() 

into the console; this creates a hard red/blue transition that occurs beyond the right-hand defining circle.

@PeterConstable
Copy link
Collaborator

If someone approves PR #368, then I think this can be closed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

8 participants