Vulkan: Environment background_mode set to Canvas causes 2D viewport, tooltips, context menus, and popups to be blank (DisplayServer regression)

[axblount]

---

Bugsquad note: This issue has been confirmed several times already. No need to confirm it further.

---

Godot version

v4.0.dev.calinou (29bdd00)

System information

Windows 10, Vulkan, RX 580 (Driver Version 20.45.35.03-210217a-364232E-RadeonSoftwareAdrenalin2020)

Issue description

When the WorldEnvironment's background_type is set to canvas (BG_CANVAS) all tooltips, context menus, and pop up windows appear as blank grey squares. They still work if you know where to click.

I'm using the latest build from https://hugo.pro/projects/godot-builds/

Steps to reproduce

1. Create a 3D scene.
2. Add a WorldEnvironment child.
3. Add a new Environment.
4. Change the environment's background type to "Canvas"

Now hover over exported properties, right click nodes, open anything on the menu bar.

Minimal reproduction project

GodotTest.zip

Bugsquad edit (keywords for easier searching): background mode, background type, bg, canvas, dropdown

[nonunknown]

I can confirm this issue! in 2D its even worse, because everything disappear in the scene:

[jsmnbom]

Note that setting the Canvas Max Layer to a negative value makes everything show up again, but none of the environement properties seem to actually apply in that case.

This effectively means it's impossible to use the wordenvironment (and therefore glow and such) with 2D in current master as far as I can tell?

[jsmnbom]

Okay so I (a complete C++ novice) spent some time trying to debug this today, and I think I've found part of the cause. (note that i might also be completely wrong so take this with a grain of salt).

When WorldEnvironment's background_type is set to canvas (BG_CANVAS) then the method render_empty_scene gets called to (as far as i can tell) render a transparent 3d scene on top of the canvas. This method seems to be broken though, since it does not pass the environment that it finds on to scene_render->render_scene.
Proposed fix:

- scene_render->render_scene(p_render_buffers, &camera_data, PagedArray<RendererSceneRender::GeometryInstance *>(), PagedArray<RID>(), PagedArray<RID>(), PagedArray<RID>(), PagedArray<RID>(), PagedArray<RID>(), PagedArray<RID>(), RID(), RID(), p_shadow_atlas, RID(), scenario->reflection_atlas, RID(), 0, 0, nullptr, 0, nullptr, 0, nullptr);
+ scene_render->render_scene(p_render_buffers, &camera_data, PagedArray<RendererSceneRender::GeometryInstance *>(), PagedArray<RID>(), PagedArray<RID>(), PagedArray<RID>(), PagedArray<RID>(), PagedArray<RID>(), PagedArray<RID>(), environment, RID(), p_shadow_atlas, RID(), scenario->reflection_atlas, RID(), 0, 0, nullptr, 0, nullptr, 0, nullptr);

This turns out to not be enough to fix the issue completely, unfortunately. It seems that the tonemapper effect (desponsible for glow, tonemapping etc.) makes the entire viewport black (NOT the clear color, but completely black). I've spent a ton of time (compilation takes around 5min on my potato machine) and other than narrowing it down to the tonemapper, I cannot figure out what's going wrong :(

Probably useless but screenshot after the "fix":

[jsmnbom]

Okay so forgot what i said about it being the tonemapper. I think the problem just shows there because of what texture the tonemapper gets passed. Unfortunatly, I have no idea how to debug what texture gets passed to a shader (espcially since it's just an RID, and i have no idea how to use those properly), so I am once again stuck.

My guess (100% a guess, I am too noob to actually figure out how this works), is that the internal texture in p_viewport->render_buffers, somehow does not contain the previously drawn 2d canvas content from RSG::canvas->render_canvas(p_viewport->render_target, ...) calls. Again weather that's how it actually works I have no idea.

Might try again when I have more time, but I think this issue might run deeper than I am able to fix. (sorry for the spam on this issue, do say if these updates are not allowed, i just thought they might help someone in the future.)

[h0lley]

I've had a look at it as well and pretty much came to the same conclusion as @jsmnbom.

a couple of things to note:

I am almost certain that @jsmnbom is correct that this call needs to get environment passed:

godot/servers/rendering/renderer_scene_cull.cpp

Line 3271 in 9c746b3

scene_render->render_scene(p_render_buffers, &camera_data, PagedArray<RendererSceneRender::GeometryInstance *>(), PagedArray<RID>(), PagedArray<RID>(), PagedArray<RID>(), PagedArray<RID>(), PagedArray<RID>(), PagedArray<RID>(), RID(), RID(), p_shadow_atlas, RID(), scenario->reflection_atlas, RID(), 0, 0, nullptr, 0, nullptr, 0, nullptr);

In addition, it should probably get passed scenario->camera_effects (which now holds the settings for the DOF effect) as well (it's the two empty RIDs right before p_shadow_atlas). Only by passing those arguments will the tonemapper code for the various effects be invoked.

then, I've confirmed that the current state is pretty much how it's always been since the initial Vulkan implementation. so unfortunately it's hard to compare it with the state when it was last working.

next, I've toyed around with RenderDoc to attempt confirm @jsmnbom's suspicion that the issue is not down due to a bug in the tonemapper but rather that it gets passed a black texture. If I interpret it correctly, this really seems to be the case. So it would be interesting to inspect the render buffers - however, I too have no clue how to debug RIDs. It doesn't seem to be documented anywhere.

[julhe]

I did some deeper diging and the issues seems to be not only the environment not beeing passed, but also the canvas is rendering to a completly different texture: p_viewport->render_buffers->render_target. The 3D and post-processing pipeline however uses render_buffers->internal_texture and only copies in the very last moment into render_target .

The simple solution would be to copy the contents of render_target into internal_texture. That seems to be the easiet arround here:

godot/servers/rendering/renderer_rd/forward_clustered/render_forward_clustered.cpp

Lines 1559 to 1561 in 8d15814

	case RS::ENV_BG_CANVAS: {
	keep_color = true;
	} break;

I've tried things like copy_effects->copy_to_fb_rect() but that doesn't work for "Vulkan reasons"🙃, need to do more digging there.

[h0lley]

anyone considering addressing this please be also aware of this related proposal godotengine/godot-proposals#4564
if 2D and 3D post processing is to be separated, then the solution to this issue might look different

[sketchyfun]

This is still an issue in Godot 4.0 Beta 1

All my backgrounds use this method to draw 2D skies behind the 3D world, so it currently breaks the entire view in-game

[Notatea]

@sketchyfun indeed so. This issue makes any kind of WorldEnvironment post-processing impossible to use in 2D projects.

If used under a SubViewport Environment, everything in a related SubViewport will dissapear, but Godot UI will be unaffected.
Setting Canvas Max Player to -1 will "un-disappear" things in SubViewport, but none of the efects would be applied.

Tested both on Windows 11 (R5 2600, RX580 4G) and Linux (i5 8265U, intel UHD620) - tried on latest Ubuntu and Fedora releases.

I've looked into it, and I also think that the issue has something to do with canvas rendering texture target. But sadly, my knowledge isn't sufficient to fix this :(

I do agree with @h0lley that 2D and 3D post-processing separation should be considered, though.

[INdIEDeV016]

Still an issue in Beta 2...

[Novark]

Issue persists in Beta 3 [v4.0.beta3.official (Windows 10 x64)]

[SewerynKaminski]

Issue persists in Beta 3 [v4.0.beta3.official [01ae26d] (Linux Ubuntu 22.04 x64)]

[sketchyfun]

Issue persists in Beta 4 v4.0.beta4.official [e675154] (Windows 10)

[MaybeAnonymous]

Issue persists in Beta 4 v4.0.beta4.official [e675154] (Artix Linux)

[Calinou]

As noted in OP, please stop posting further confirmation comments as it causes unnecessary notifications to be created for everyone watching this repository. It will not help the issue be resolved faster 🙂

Future comments not bringing significant new information will be removed.

[coreybasden]

As far as I can tell a worldenvironment node set to canvas mode is THE way to make 2d things glow. Seems pretty game-breaking to be left untouched for over a year or am I missing something? Are people just not using glow effects in godot?

[h0lley]

As far as I can tell a worldenvironment node set to canvas mode is THE way to make 2d things glow. Seems pretty game-breaking to be left untouched for over a year or am I missing something? Are people just not using glow effects in godot?

people are using Godot 3. that's the price we pay for using alpha/beta software.
in terms of core contributors, the people who are part of the "rendering team" seem to be mostly focused on 3D (things like OpenGL3 and XR) at the moment. this is not something we can force.

on top of that, there's uncertainty if restoring the Godot 3 implementation is even the way to go. see here. hopefully they settle on a way to go forward with this soon. personally I'd love for this to be prioritized, too, but github issues are not the place for discussing these preferences. you can 👍 the report to indicate your interest in a fix.