Subtract blend mode produces different results with different renderers

[allenwp]

Godot version

master commit 2ab0f17 and later

System information

Windows 11 NVIDIA 980 Ti

Issue description

When working on #76334 I noticed that the RenderingDevice abstraction uses a different blend mode for BLEND_MODE_SUB than the OpenGL driver. RenderingDevice uses BLEND_OP_SUBTRACT while OpenGL uses GL_FUNC_REVERSE_SUBTRACT.

In Godot 3, OpenGL uses GL_FUNC_REVERSE_SUBTRACT just like Godot 4's OpenGL driver.

RenderingDevice Abstraction:

godot/servers/rendering/renderer_rd/forward_mobile/scene_shader_forward_mobile.cpp

Lines 221 to 222 in b7032b5

	blend_attachment.alpha_blend_op = RD::BLEND_OP_SUBTRACT;
	blend_attachment.color_blend_op = RD::BLEND_OP_SUBTRACT;

godot/servers/rendering/renderer_rd/forward_clustered/scene_shader_forward_clustered.cpp

Lines 210 to 211 in b7032b5

	blend_attachment.alpha_blend_op = RD::BLEND_OP_SUBTRACT;
	blend_attachment.color_blend_op = RD::BLEND_OP_SUBTRACT;

godot/servers/rendering/renderer_rd/renderer_canvas_render_rd.cpp

Lines 2124 to 2125 in b7032b5

	attachment.alpha_blend_op = RD::BLEND_OP_SUBTRACT;
	attachment.color_blend_op = RD::BLEND_OP_SUBTRACT;

OpenGL:

godot/drivers/gles3/rasterizer_scene_gles3.cpp

Line 2177 in 4c677c8

glBlendEquation(GL_FUNC_REVERSE_SUBTRACT);

godot/drivers/gles3/rasterizer_canvas_gles3.cpp

Line 738 in 4c677c8

glBlendEquation(GL_FUNC_REVERSE_SUBTRACT);

This results in a different blend result when using the Subtract blend mode in different renderers:

Forward+:

Mobile:

Compatibility:

There appear to be some (relatively) minor differences between the Forward+ and Mobile renderers, but the difference to the OpenGL renderer is substantial because of the different blend mode function that is used.

Steps to reproduce

1. Create some objects, such as canvas items or 3D meshes.
2. Create a material for these objects and change its blend mode to Subtract.
3. Change the renderer and note that it appears different based on the renderer.

Important note:
If you are testing with a Godot 4 version earlier than master commit 2ab0f17, you must also set the Depth Draw Mode to Never as described in #76334 to correctly see the Subtract blend mode.

Minimal reproduction project

blend-subtract.zip

[BastiaanOlij]

This is likely a result of the OpenGL renderer being based on the Godot 3 renderer and may just be an oversight. @clayjohn whats your take on this?

[clayjohn]

This is likely a result of the OpenGL renderer being based on the Godot 3 renderer and may just be an oversight. @clayjohn whats your take on this?

I think the Vulkan renderer mistakenly uses BLEND_OP_SUBTRACT instead of BLEND_OP_REVERSE_SUBTRACT For reference, BLEND_OP_SUBTRACT takes the pixel's color subtracts the background color from it and then sets the background color to the result.

I.e. if you want to write a white pixel to a grey background with BLEND_OP_SUBTRACT you get (1, 1, 1) - (0.5, 0.5, 0.5) = (0.5, 0.5, 0.5) while with BLEND_OP_REVERSE_SUBTRACT you get (0.5, 0.5, 0.5) - (1, 1, 1,) = (0, 0, 0). My expectation is that a blend subtract mode would subtract a given color from the background rather than the other way around.

To illustrate, here is a simple example with a white background and a middle grey square on top. In Vulkan the white background is subtracted from the grey resulting in all black where they intersect

In OpenGL the grey is subtracted from the white so the final result is still grey

[allenwp]

Given that the Subtract blend mode in Godot has historically (pre-4.0) mapped to GL_FUNC_REVERSE_SUBTRACT, I would expect equivalent behaviour from the new 4.0 renderers.

I think it would be good to know if there was a deliberate choice and use case that prompted the change in subtract blend mode behaviour. If there wasn’t, then it would make most sense to change the new 4.0 renderers to BLEND_OP_REVERSE_SUBTRACT sooner rather than later to make things easier for users upgrading from 3 to 4.x.

[allenwp]

After reading @clayjohn's comments about what he would expect for the behaviour of a "subtract" blend mode, I became curious what the expected behaviour would be for a user who is not at all familiar with graphics API's SUBTRACT vs. REVERSE_SUBTRACT... As a way of determining what this might be for graphics artists, I tried out the Photoshop "Subtract" blend mode and found it to be the same as REVERSE_SUBTRACT in OpenGL/Vulkan:

Photoshop:

Godot 3 (or OpenGL driver in Godot 4):

I can immediately think of some use cases for the REVERSE_SUBTRACT style blend mode, such as a weird ghost object that is sapping away light from its (background) environment. It is harder for me to think of a use case where the background should be subtracted from the foreground object, as in the case with SUBTRACT.

[clayjohn]

Thanks for investigating a little deeper. I think the best course of action is to switch the RD renderers to use REVERSE_SUBTRACT and make the change in 4.1

[allenwp]

Cool, I'll put together a PR sometime in the next week-ish if nobody else has got to it before me.