Add port of AMD's Robust Contrast Adaptive Sharpening #7422

Elabajaba · 2023-01-30T07:03:20Z

Objective

TAA, FXAA, and some other post processing effects can cause the image to become blurry. Sharpening helps to counteract that.

Solution

This is a port of AMD's Contrast Adaptive Sharpening (I ported it from the SweetFX version, which is still MIT licensed). CAS is a good sharpening algorithm that is better at avoiding the full screen oversharpening artifacts that simpler algorithms tend to create.

This is a port of AMD's Robust Contrast Adaptive Sharpening (RCAS) which they developed for FSR 1 (and continue to use in FSR 2). RCAS is a good sharpening algorithm that is better at avoiding the full screen oversharpening artifacts that simpler algorithms tend to create.

Future Work

Consider porting this to a compute shader for potentially better performance. (In my testing it is currently ridiculously cheap (0.01ms in Bistro at 1440p where I'm GPU bound), so this wasn't a priority, especially since it would increase complexity due to still needing the non-compute version for webgl2 support).

Changelog

Added Contrast Adaptive Sharpening.

JMS55

Awesome work, the added sharpness is a definite improvement :)

Would be nice to have this as a compute shader, with an IFDEF keeping the old version for WebGL. That can be a future PR though.

crates/bevy_core_pipeline/src/contrast_adaptive_sharpening/mod.rs

crates/bevy_core_pipeline/src/contrast_adaptive_sharpening/node.rs

crates/bevy_core_pipeline/src/contrast_adaptive_sharpening/contrast_adaptive_sharpening.wgsl

crates/bevy_core_pipeline/src/contrast_adaptive_sharpening/mod.rs

Elabajaba · 2023-02-03T01:28:47Z

What do people think about shortening the internal names from ContrastAdaptiveSharpening to CAS (eg. ViewContrastAdaptiveSharpeningPipeline -> ViewCASPipeline, ContrastAdaptiveSharpeningNode -> CASNode).

I think it's probably better to not change the user facing ContrastAdaptiveSharpeningSettings to CASSettings so that it keeps sharpening in the name.

edit: Though CAS is probably more recognizable than ContrastAdaptiveSharpening since that's what it's called in options menus, and is what AMD advertises it as.

JMS55 · 2023-02-03T01:37:49Z

I like keeping the public API with the full name, but it's totally fine to shorten the internal type names. I did the same for TAA and SSAO.

We may even want to rename it to just SharpeningSettings or something - the contrast adaptive part could just be an implementation detail.

JMS55 · 2023-02-06T19:04:41Z

@Elabajaba needs rebasing due to stageless.

Elabajaba · 2023-02-06T19:36:18Z

Rebased, ~~just not sure if I need to add #[reflect(Component)] to ContrastAdaptiveSharpeningSettings (like what is added in #7527).~~ Added #[reflect(Component)] so reflection will work.

Since this is also modifying the FXAA example (and TAA removes it), this either needs to be merged before TAA and then add the sharpening system into the new anti-aliasing example, or I can rebase it onto TAA and it can be merged after (or just change it to use the AA example once TAA is merged).

alice-i-cecile · 2023-02-13T18:48:07Z

@Elabajaba can you please resolve the comments you've addressed to help reviewers?

robtfm

looks good, and shows a decent improvement with fxaa. i guess it'll be an even better impact with taa. i didn't grok the shader math but it looks in line with the reference.

it's not a matter for this PR, but i think we need to put some thought into node ordering. more nodes is making construction of core pipeline increasingly fragile; omitting a node (like FXAA) will cause ordering issues with nodes that order relative to it.
this isn't a problem as long as all these things are always in the core pipeline but it is not scalable and not open to non-core modification... something we can improve as part of rendergraph yeeting maybe.

crates/bevy_core_pipeline/src/contrast_adaptive_sharpening/node.rs

JMS55 · 2023-03-12T22:51:34Z

it's not a matter for this PR, but i think we need to put some thought into node ordering. more nodes is making construction of core pipeline increasingly fragile; omitting a node (like FXAA) will cause ordering issues with nodes that order relative to it.
this isn't a problem as long as all these things are always in the core pipeline but it is not scalable and not open to non-core modification... something we can improve as part of rendergraph yeeting maybe.

Strongly agreed, that's part of what I opened the render node issues about. Imo, we should have labels be separate from nodes. So we always add labels and can order relative to them, but it doesn't matter if the node is in the graph or not. We should also pull out the node ordering into the core_3d file, instead of defining it per-plugin, which is more fragile.

Elabajaba · 2023-03-29T06:25:28Z

This has been updated to use Robust Contrast Adaptive Sharpening (RCAS) instead of CAS. RCAS is newer and was developed for FSR 1 (it's also used in FSR2), and uses fewer texture samples compared to CAS (5 instead of 9).

superdump

Minor changes, basically looks good. :) Thanks!

crates/bevy_core_pipeline/src/contrast_adaptive_sharpening/mod.rs

...bevy_core_pipeline/src/contrast_adaptive_sharpening/robust_contrast_adaptive_sharpening.wgsl

crates/bevy_core_pipeline/src/contrast_adaptive_sharpening/mod.rs

JMS55 · 2023-04-01T15:33:10Z

Maybe we should add CAS to the TAA/FXAA bundles. We should at least link to CAS from the TAA docs, where it mentions the softness of TAA.

Also, thoughts on calling the user-facing types just Sharpening?

superdump · 2023-04-01T16:51:52Z

Maybe we should add CAS to the TAA/FXAA bundles. We should at least link to CAS from the TAA docs, where it mentions the softness of TAA.

Sharpening by default makes sense if they are practically blurry. I didn't really feel that our TAA looked blurry with our implementation / configuration, but I haven't had chance to play with it much yet. And I think last time I checked, FXAA at higher quality settings didn't look blurry, but lower quality settings did. However... I have mostly been looking at it on a MacBook Pro with hidpi display, so things will naturally look sharper. I should test it without the winit scale factor 1.0 setting.

Also, thoughts on calling the user-facing types just Sharpening?

I could imagine there being other sharpening methods long-term. I think it's better to keep specific implementations specifically-named and then when it becomes clear what the situation is, use an enum to group the variants. I don't think we're there yet though. So what I mean is, build the individual pieces of functionality with full flexibility, and then add convenience API on top when it makes sense to do so.

UI tweaks

Elabajaba · 2023-04-01T20:55:43Z

Changed it to use a linear 0.0 to 1.0 value for sharpening strength, visually it looks pretty linear imo.

DGriffin91 · 2023-04-02T23:34:38Z

@superdump

And I think last time I checked, FXAA at higher quality settings didn't look blurry, but lower quality settings did.

This is actually backwards. FXAA is both sharper and faster at lower settings. (Part of why I didn't use the word quality in the plugin, but sensitivity instead)

I agree that FXAA shouldn't enable CAS by default.

alice-i-cecile added C-Feature A new feature, making something new possible A-Rendering Drawing game state to the screen labels Jan 31, 2023

Elabajaba force-pushed the cas branch from 88c3042 to ac731a1 Compare January 31, 2023 03:54

Elabajaba marked this pull request as ready for review January 31, 2023 06:16

Elabajaba changed the title ~~Add AMD's Contrast Adaptive Sharpening~~ Add port of AMD's Contrast Adaptive Sharpening Jan 31, 2023

IceSentry requested a review from JMS55 January 31, 2023 21:52

JMS55 suggested changes Feb 2, 2023

View reviewed changes

JMS55 reviewed Feb 3, 2023

View reviewed changes

crates/bevy_core_pipeline/src/contrast_adaptive_sharpening/mod.rs Outdated Show resolved Hide resolved

JMS55 approved these changes Feb 3, 2023

View reviewed changes

JMS55 added this to the 0.10 milestone Feb 6, 2023

Elabajaba force-pushed the cas branch from 5912d95 to f6f0964 Compare February 6, 2023 19:15

superdump self-requested a review February 7, 2023 12:41

Elabajaba force-pushed the cas branch 2 times, most recently from 92fa303 to 7a44693 Compare February 12, 2023 21:16

Elabajaba force-pushed the cas branch from 7a44693 to b86b23c Compare February 17, 2023 19:52

mockersf modified the milestones: 0.10, 0.11 Feb 28, 2023

robtfm suggested changes Mar 12, 2023

View reviewed changes

crates/bevy_core_pipeline/src/contrast_adaptive_sharpening/node.rs Outdated Show resolved Hide resolved

Elabajaba force-pushed the cas branch from c4222f6 to 067612c Compare March 13, 2023 03:08

Elabajaba force-pushed the cas branch from 8eb0436 to 6a0fd31 Compare March 22, 2023 20:01

Elabajaba added 4 commits March 28, 2023 02:18

AMD CAS port

69a770a

modify fxaa example for a CAS playground

b03a4a8

formatting

427eaf3

clippy

aa33369

Elabajaba force-pushed the cas branch from 6a0fd31 to afc3fe0 Compare March 29, 2023 06:20

Elabajaba changed the title ~~Add port of AMD's Contrast Adaptive Sharpening~~ Add port of AMD's Robust Contrast Adaptive Sharpening Mar 29, 2023

Elabajaba requested review from robtfm and JMS55 and removed request for superdump, robtfm and JMS55 March 30, 2023 01:17

superdump requested changes Apr 1, 2023

View reviewed changes

JMS55 reviewed Apr 1, 2023

View reviewed changes

crates/bevy_core_pipeline/src/contrast_adaptive_sharpening/mod.rs Outdated Show resolved Hide resolved

UI tweaks

dd2583c

Elabajaba added 2 commits April 1, 2023 16:40

Merge pull request #1 from JMS55/cas-ui-tweaks

ef72d11

UI tweaks

linear sharpening_strength from 0.0 to 1.0

dd41c12

Elabajaba added 2 commits April 1, 2023 17:04

don't extract when sharpening_strength is 0

2b6d738

fma, spacing and link to source

8185c62

superdump approved these changes Apr 1, 2023

View reviewed changes

JMS55 approved these changes Apr 2, 2023

View reviewed changes

alice-i-cecile added the S-Ready-For-Final-Review This PR has been approved by the community. It's ready for a maintainer to consider merging it label Apr 2, 2023

robtfm approved these changes Apr 2, 2023

View reviewed changes

alice-i-cecile added this pull request to the merge queue Apr 2, 2023

Merged via the queue into bevyengine:main with commit 09f1bd0 Apr 2, 2023

This was referenced Jun 28, 2023

News: Release 0.11 (wrong branch ... see #657) bevyengine/bevy-website#656

Closed

News: Release 0.11 bevyengine/bevy-website#657

Merged

rparrett mentioned this pull request Jul 15, 2024

Types related to Contrast Adaptive Sharpening should be Cas* instead of CAS* #14341

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add port of AMD's Robust Contrast Adaptive Sharpening #7422

Add port of AMD's Robust Contrast Adaptive Sharpening #7422

Elabajaba commented Jan 30, 2023 •

edited

Loading

JMS55 left a comment

Elabajaba commented Feb 3, 2023 •

edited

Loading

JMS55 commented Feb 3, 2023

JMS55 commented Feb 6, 2023

Elabajaba commented Feb 6, 2023 •

edited

Loading

alice-i-cecile commented Feb 13, 2023

robtfm left a comment

JMS55 commented Mar 12, 2023

Elabajaba commented Mar 29, 2023

superdump left a comment

JMS55 commented Apr 1, 2023 •

edited

Loading

superdump commented Apr 1, 2023

Elabajaba commented Apr 1, 2023

DGriffin91 commented Apr 2, 2023

Add port of AMD's Robust Contrast Adaptive Sharpening #7422

Add port of AMD's Robust Contrast Adaptive Sharpening #7422

Conversation

Elabajaba commented Jan 30, 2023 • edited Loading

Objective

Solution

Future Work

Changelog

JMS55 left a comment

Choose a reason for hiding this comment

Elabajaba commented Feb 3, 2023 • edited Loading

JMS55 commented Feb 3, 2023

JMS55 commented Feb 6, 2023

Elabajaba commented Feb 6, 2023 • edited Loading

alice-i-cecile commented Feb 13, 2023

robtfm left a comment

Choose a reason for hiding this comment

JMS55 commented Mar 12, 2023

Elabajaba commented Mar 29, 2023

superdump left a comment

Choose a reason for hiding this comment

JMS55 commented Apr 1, 2023 • edited Loading

superdump commented Apr 1, 2023

Elabajaba commented Apr 1, 2023

DGriffin91 commented Apr 2, 2023

Elabajaba commented Jan 30, 2023 •

edited

Loading

Elabajaba commented Feb 3, 2023 •

edited

Loading

Elabajaba commented Feb 6, 2023 •

edited

Loading

JMS55 commented Apr 1, 2023 •

edited

Loading