Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improvements to blur look #20

Open
wants to merge 2 commits into
base: main
Choose a base branch
from
Open

Conversation

SolarLiner
Copy link

This PR tweaks a few things to produce more aethestic results. No changes to the shaders have been made, making these changes easy to review ;)

  1. The blur node has been moved after tonemapping. Before, blurring would create boxy regions around bright values, but moving the blurring after the tonemapping means no large values are present anymore, preventing those boxy regions from appearing.
  2. The addressing mode on the blur input sampler has been changed to "Mirrored Repeat", reducing the streaking at the edges, as well as the apparent brightening on the edges of the image. This is less visible but still makes a good change in scenes with a lot of dynamic range (e.g. space scenes).

Here's a before and after of this change:

Before:

Screenshot 2024-12-02 at 11 41 49

After:

Screenshot 2024-12-02 at 11 43 30

@SolarLiner SolarLiner mentioned this pull request Dec 2, 2024
@atbentley
Copy link
Owner

atbentley commented Feb 2, 2025

Thanks for the bringing up these issues @SolarLiner, and the contribution!

While your screenshot comparision looks like a big improvement I am hesitent to just merge this in because it breaks the debanding. Currently debanding for this blur shader piggybacks off the deband dithering that bevy applies during the tonemapping node. So by blurring after tonemapping we lose the debanding.

I wanted to understand this a bit better so I did some investigating.

So I tried to make a reproduction of this and it looks like 'bright values' are a result of using an emmisive standard material with values greater than 1.0, with hdr turned on. Does that sound right?

For example, I was able to reproduce something similar to your screenshot using:

StandardMaterial {
    base_color: Color::WHITE,
    emissive: LinearRgba::new(100.0, 100.0, 100.0, 1.0),
    ..default()
};

Since the blur shader is based off the gaussian blur in the bevy dof shader, I went in and checked what happens there what happens with a similar material. And indeed for bright values and a large circle of confufision the dof shader also gets blown out. Both the guassian variant as well as the bokeh variant. This is not stricly a problem in the dof shader though since the circle of confusion is typically smaller than what we want to use when trying to create blurry backgrounds.


The implementation on main has clear issues with emmisive materials in hdr mode. This branch has issues with banding. I think I'm not going to go ahead with the approach this PR just yet. There are some paths forward:

  1. Make changes to the shader to prevent values getting blown out. This could be as simple as some clamping but I'm also thinking of using something other than the guassian blur.
  2. Implement alternative debanding that does not rely on the tonemapping debanding.

I'm not suggesting this is something you have to go off and do, I might get around to either of these myself but currently this is not super high priority for me.


Here is a before/after of the deband_dither examples, zoom in on the blur regions to see the bands.

Before:
before

After:
after

I hope the banding is visible in the upload, if not the example can be run with cargo run --example deband_dither.

@SolarLiner
Copy link
Author

SolarLiner commented Feb 3, 2025

So I tried to make a reproduction of this and it looks like 'bright values' are a result of using an emmisive standard material with values greater than 1.0, with hdr turned on. Does that sound right?

Yes and no. Most PBR scenes will have values that fall outside the 0..1 range, because lights and emissive materials are configured to some realistic values (sometimes even taking real-life luminosity outputs). Even though there is an exposure stage that then dials the values back down to a manageable range, the dynamic range of a typical scene (especially outside in daylight) will be too high to fit in 0..1, which is why we need tonemapping.

Since the blur shader is based off the gaussian blur in the bevy dof shader, I went in and checked what happens there what happens with a similar material. And indeed for bright values and a large circle of confufision the dof shader also gets blown out. Both the guassian variant as well as the bokeh variant. This is not stricly a problem in the dof shader though since the circle of confusion is typically smaller than what we want to use when trying to create blurry backgrounds.

Bokeh simulation is emulating something that happens before the camera sensor, so it makes sense that the effect is applied before tonemapping (it's working with "raw light" values, and not "camera pixel" values, so to speak). The result is that bright lights will indeed overpower their surrounding, which is very much something that happens with physical cameras as well (look at out of focus street lamps in pictures, for example).

In the case of this plugin, the blurring is done "as-if" we had a translucent material placed on top of an already finalized image (like looking at a picture with a piece of rough acrylic on top), which means this blurring is not done as a physical emulation of light transport, but as an added effect on top of an already captured image (i.e. we are now working after the camera sensor, and therefore in said "camera pixel" values).

Tonemapping is an emulation of the camera film/sensor response to the physical light it's seeing. Any process happening before is a physical process of light transport (ie. depth of field, motion blur, chromatic aberration, etc.). Anything happening after is an added effect to the captured image, as if done using an image processing program (ex. color correction, contrast enhancement and sharpening, etc.).

So the fundamental issue is with the ordering of the screen-space shaders. We are separately showing the normal output image that Bevy generates, and a separate image that seems to have been taken before the camera sensor and shown directly raw.

This works okay for "transparent" or "mild" tonemapping, but would start showing major differences if the tonemapping used is more stylized or aggressive, or has non-linear behavior (i.e. bounded tonemappers will eventually clip high value pixels to white, but unbounded tonemappers (like Reinhard) will never clip, and so no matter the raw color values, the relative saturation will be kept post-tonemapping). Tonemapping also has the role of outputing an image with colors in the 0..1 range, and clamping is essentially a tonemapping step as well; it does not make sense to blur a separately-tonemapped image.

That putting the blur step after tonemapping breaks debanding is a separate issue that needs to be fixed either here or in Bevy (it could be an issue with their ordering of screen-space processes as well), but this PR fixes a conceptual problem as much as a technical one. It also aligns this plugin with how "blurred UI" is implemented in most games.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants