Separate blend mode from premultiplied alpha in CanvasItemMaterial

[badlogic]

Describe the project you are working on

Spine Runtime for Godot, which is a proprietary, source available piece of software. However, this proposal is useful for all 2D rendering (and possibly 3D rendering as well)

Describe the problem or limitation you are having in your project

CanvasItemMaterial only supports one blend mode for premultiplied textures, namley the equivalent to the post-multiplied blend mode BLEND_MODE_MIX.

Texture filtering of translucent pixels, especially in image border regions, generally results in rendering artifacts due to the nature of post-multiplicative alpha compositing. These artifacts usually manifest at the edges between opaque and translucent pixel borders.

See http://www.realtimerendering.com/blog/gpus-prefer-premultiplication/ for a great overview of the issue.

Note that the Fix alpha border import setting (aka bleeding) is a suboptimal solution, that fails in many real world scenarios, due to the undecidablity how to blend multiple neighboring pixels into a bleed pixel.

Premultiplied alpha is the only general solution to fix these texture filtering artifacts without complex and insufficient preprocessing like edge bleeding on the source images.

The only preprocessing step needed is to premultiply imported pixel colors with their respective alpha, something Godot already supports through the texture import property Premult Alpha: https://docs.godotengine.org/en/stable/tutorials/assets_pipeline/importing_images.html

CanvasItemMaterial offers 4 blend modes for post multiplied pixel color blending, but only one of these 4 blend modes (mix) for premultiplied alpha pixel color blending. https://docs.godotengine.org/en/stable/classes/class_canvasitemmaterial.html#enum-canvasitemmaterial-blendmode

This limits the usefulness of premultiplied alpha support in Godot severly.

Describe the feature / enhancement and how it helps to overcome the problem or limitation

The CanvasItemMaterial blend modes BLEND_MODE_ADD, BLEND_MODE_SUB, and BLEND_MODE_MUL should also work for premultiplied textures.

Describe how your proposal will work, with code, pseudo-code, mock-ups, and/or diagrams

Two ways to implement this:

1. Remove BLEND_MODE_PREMULT_ALPHA and instead add a boolean property premultiplied_alpha. The canvas rasterizer implementation(s) then set the blend equations properly. This would be a breaking change.
2. Retain the current blend mode enum, and extend it with 4 new enum values BLEND_MODE_PREMULT_ALPHA_MIX, BLEND_MODE_PREMULT_ALPHA_ADD, BLEND_MODE_PREMULT_ALPHA_SUB, and BLEND_MODE_PREMULT_ALPHA_MUL. BLEND_MODE_PREMULT_ALPHA would be an alias for BLEND_MODE_PREMULT_ALPHA_MIX for backward compatibility.

The rasterizer code in 3.x to be modified can be found here:
https://github.com/godotengine/godot/blob/3ba980379dfb72f5d238640835d8e77c76cf3c99/drivers/gles3/rasterizer_canvas_gles3.cpp#L256-L304
https://github.com/godotengine/godot/blob/3ba980379dfb72f5d238640835d8e77c76cf3c99/drivers/gles2/rasterizer_canvas_gles2.cpp#L1735

The rasterizer code in 4.x to be modified can be found here:
https://github.com/godotengine/godot/blob/f4b0c7a1ea8d86c1dfd96478ca12ad1360903d9d/servers/rendering/renderer_rd/renderer_canvas_render_rd.cpp#L2031-L2086

The rendering backend enums and corresponding enum exposed to users would also need to be changed.

Overall, the changes are extremely low risk for option 2, as code paths for the current functionality would stay the same. In case of option 1, current code paths would also be retained, but due to the removal of BLEND_MODE_PREMULT_ALPHA user code would break.

If this enhancement will not be used often, can it be worked around with a few lines of script?

No, it can not be worked around as neither shaders, nor materials, not RenderingServer allow direct manipulation of the blend equations. Everything must go through the CanvasItemMaterial enum BlendMode.

Is there a reason why this should be core and not an add-on in the asset library?

See previous answer, the core API does not expose setting the blend equation in any way other than the fixed blend modes.

[Calinou]

Related to #3431.

I recommend going with the compatibility-breaking approach for 4.0 (new boolean).

For 3.x, we can add new enum options at the end of the blend mode enum. I'm not sure if we should add a new enum option for "premultiplied alpha mix" – we could reuse the existing option and rename its property hint in the editor (which doesn't break compatibility).

[reduz]

I kind of agree that having a bool setting may work better. In general Godot does not use premultiplied alpha by default because cost vs benefit its for the most part an advanced feature that many users do not understand well and makes little difference to them, but it would probably make sense to make this a bool setting rather than a specific blend mode, to make the workflow easier when using it.

[reduz]

Another option I thought for a while, given we now do this in materials, is to automatically premultiply the alpha in the shader (not the texture) when using regular blend, which should also fix most use cases transparently.

[badlogic]

I have to brush up on blend equations again (which I'll do if this gets greenlight and I can implement it), but iirc, premultiplication in the shader would not be sufficient, as the destination alpha/color blend would have to be different as well. I'll investigate.

…

On Mon, Apr 25, 2022, 09:27 Juan Linietsky ***@***.***> wrote: Another option I thought for a while, given we now do this in materials, is to automatically premultiply the alpha in the shader (not the texture) when using regular blend, which should also fix most use cases transparently. — Reply to this email directly, view it on GitHub <#4433 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAD5QBAYMT5GSJKUGY3PAPLVGZCNZANCNFSM5UE7RYXQ> . You are receiving this because you authored the thread.Message ID: ***@***.***>