Initial DLSS implementation #19864

JMS55 · 2025-06-29T18:43:47Z

Objective

Closes NVIDIA DLSS (Deep Learning Super Sampling) #8420

Solution

Initial implementation of DLSS upscaling, via https://github.com/bevyengine/dlss_wgpu.
Future PRs will work more on transparency, exposure, working with custom viewports, fixing inevitable resolution override bugs, etc.
DLSS framegen is not planned, but ray-reconstruction is for solari.
FSR3/4, XeSS2, and MetalFX temporal upscaling should be easy to add now if a future contributor wants to. In the future we could have an auto-temporal AA component that handles DLSS/FSR/XeSS/MetalFX/TAA fallbacks automatically.

Testing

Did you test these changes? If so, how?
- Run the anti_aliasing example
Are there any parts that need more testing?
- Different types of scene content and rendering effects to make sure they work with DLSS/upscaling
- Reviewing dlss_wgpu code

Showcase

github-actions · 2025-06-29T18:44:26Z

It looks like your PR has been selected for a highlight in the next release blog post, but you didn't provide a release note.

Please review the instructions for writing release notes, then expand or revise the content in the release notes directory to showcase your changes.

JMS55 · 2025-06-29T18:56:19Z

@cart not sure what licensing concerns there are with this. See the license notes in https://github.com/JMS55/dlss_wgpu/blob/main/README.md.

github-actions · 2025-06-29T19:02:53Z

You added a new example but didn't add metadata for it. Please update the root Cargo.toml file.

JMS55 · 2025-06-29T19:05:58Z

You added a new example but didn't add metadata for it. Please update the root Cargo.toml file.

Lies D:

github-actions · 2025-06-29T19:32:27Z

You added a new example but didn't add metadata for it. Please update the root Cargo.toml file.

cart · 2025-07-01T20:32:03Z

Rad! My only concern here is that we're taking a dependency on an external crate that depends on a specific version of wgpu. This will make upstream bevy wgpu updates harder, as it will require collaboration with owner of the crate (in this case, you, which is much better than a stranger, but still a risk).

@cart not sure what licensing concerns there are with this. See the license notes in https://github.com/JMS55/dlss_wgpu/blob/main/README.md.

I think this is ok, given that the onus is on developers to wire up the SDK themselves.

JMS55 · 2025-07-01T22:03:50Z

Theoretically it's an optional feature, so it shouldn't block wgpu upgrades. Of course I'm happy to update it asap.

I'm also happy to put it in the bevy repo, up to maintainers. I don't have a preference either way.

github-actions · 2025-07-06T19:37:47Z

You added a new example but didn't add metadata for it. Please update the root Cargo.toml file.

alice-i-cecile · 2025-08-12T21:16:21Z

As part of your reviews, please take a look at (and leave feedback on https://github.com/bevyengine/dlss_wgpu. We need to be confident in (and ideally have multiple people comfortable working with) both this particular PR and the underlying crate.

cart

I'm working on some changes to remove the need to hard code DLSS into bevy_render.

In the interim, please try to answer my open questions :)

crates/bevy_render/src/lib.rs

crates/bevy_anti_aliasing/Cargo.toml

crates/bevy_render/src/renderer/mod.rs

cart · 2025-08-13T01:22:10Z

crates/bevy_render/src/renderer/mod.rs

+    #[cfg(any(not(feature = "dlss"), feature = "force_disable_dlss"))]
+    let instance = Instance::new(&instance_descriptor);
+
+    #[cfg(all(feature = "dlss", not(feature = "force_disable_dlss")))]


What if a different feature needed to enable DX12 specific instance features? Using Instance::from_hal, at a glance, feels like we're throwing away backend support (or at least, we're throwing away the ability to customize other backends).

Like my previous comment, I don't want to merge until we understand these bounds.

Same as above, you wouldn't ever use the dlss feature on platforms where you didn't intend to support Vulkan.

We could also add DX12 support to dlss_wgpu, but it's just a lot of extra work for no real gain.

cart · 2025-08-14T00:27:07Z

I've pushed my port to #20565 (which includes those changes). This won't build until bevyengine/dlss_wgpu#8 is merged. The diff will be cleaner when #20565 merges.

crates/bevy_anti_aliasing/src/dlss/mod.rs

crates/bevy_render/Cargo.toml

# Objective Currently registering additional required Vulkan features requires either hard-coding them into `bevy_render` (see the current [DLSS proposal](#19864)), or forcing the plugin to take full manual control over wgpu initialization. Neither is an acceptable or scalable option for a modular engine like Bevy. ## Solution * Add a new `raw_vulkan_init` Cargo feature, that when enabled switches to wgpu's raw Vulkan init path, which accepts callbacks that allow checking and requiring additional Vulkan features. * Add a new `WgpuRawVulkanInitSettings` resource, which provides wgpu Vulkan Instance and Device init callbacks, which can be used to detect and register vulkan features. * These callbacks can register arbitrary features in the new `AdditionalVulkanFeatures` resource, which is inserted into the RenderApp at the same time that the RenderDevice is. This enables plugins to register initialization callbacks, which must happen _before_ RenderPlugin. They can then feed off of `AdditionalVulkanFeatures`, after the renderer is initialized, to detect if a given feature is supported. Due to the current lifecycles, this is best accomplished with either: * A separate "init plugin" that is registered before RenderPlugin, and a "logic plugin" that does everything else. This should be used if the plugin logic needs `Plugin::build()` access to render device state, which needs to be registered _after_ RenderPlugin to have access. The proposed DLSS feature needs this pattern. * A single "init plugin" that is registered before RenderPlugin and does everything. Use this pattern if you can as it is simpler. With deferred plugin init, we could remove the need for this split. --------- Co-authored-by: JMS55 <47158642+JMS55@users.noreply.github.com>

# Objective For example, in: - #19864 (comment) The link to the directory containing the release notes is broken. ## Solution Fix the link. ## Testing Copy pasted the new link in URL bar.

cart

Just pushed some changes to adapt to the latest raw_vulkan_init API. I think this is in a good spot to land now. DLSS is a bit more cordoned-off. I think we've landed on the right "middle ground" pattern for proprietary APIs like this:

Separate repo for the bindings to those APIs, with instructions on how to comply with the SDK license requirements.
Bevy-specific logic in the bevy repo, behind an opt-in cargo flag
Do not bake proprietary features into foundational libs (ex: bevy_render, bevy_ecs, etc), even in an opt-in way

Initial DLSS implementation

c75b2b4

JMS55 added the A-Rendering Drawing game state to the screen label Jun 29, 2025

github-project-automation bot added this to Rendering Jun 29, 2025

JMS55 added C-Feature A new feature, making something new possible M-Needs-Release-Note Work that should be called out in the blog due to impact M-Deliberate-Rendering-Change An intentional change to how tests and examples are rendered labels Jun 29, 2025

JMS55 added this to the 0.17 milestone Jun 29, 2025

Update example description

af1b47c

Rename variable

eb8c77f

alice-i-cecile added S-Needs-Review Needs reviewer attention (from anyone!) to move forward X-Contentious There are nontrivial implications that should be thought through labels Jun 30, 2025

Merge commit '1579256709dd64dcc4910b0bad5e81622d9ede0c' into dlss3

951c3f9

JMS55 added 6 commits July 6, 2025 17:15

Fix merge

742ebfa

Switch to git dep

463c66d

Merge branch 'main' into dlss3

950f3e8

Add release notes

bf56980

Modify release notes

6b960b0

Add note about DlssSupported

cf1a1f7

janhohenheim mentioned this pull request Jul 8, 2025

Move renderer initialization from build time to RenderStartup #20021

Closed

JMS55 added 3 commits July 7, 2025 21:40

Merge commit '5e3927ba489f597dd189f63286dc7985840db1b5' into dlss3

1b71463

Fix resolution override to work even if the camera lacks a viewport

10c96e8

Update release notes

04f488b

JMS55 added 2 commits August 12, 2025 11:17

Merge commit 'c0eb89eedcf0f451a26ea71e456532b8b8311a29' into dlss3

2bf8837

Fix merge

7b3f14b

cart requested changes Aug 13, 2025

View reviewed changes

This was referenced Aug 13, 2025

Opt-in raw vulkan initialization with hooks #20565

Merged

Factor out extension logic bevyengine/dlss_wgpu#8

Merged

Port to "raw vulkan init"

3bfd551

JMS55 commented Aug 14, 2025

View reviewed changes

crates/bevy_render/Cargo.toml Outdated Show resolved Hide resolved

Address feedback

c349dcd

ariofrio mentioned this pull request Aug 15, 2025

Fix typo in release-notes directory link in GitHub workflow #20581

Merged

JMS55 added 5 commits August 15, 2025 09:34

Merge commit 'a76e0a20d9ea44f9f042e15a8624024f41e1f5dd' into dlss3

da202cc

Use published dlss_wgpu

bbf69b3

Fix feature

9374266

Credit cart

20b7d61

Remove unused import

4d6d6ad

JMS55 requested review from cart and tychedelia August 15, 2025 14:04

JMS55 added S-Needs-Review Needs reviewer attention (from anyone!) to move forward and removed S-Needs-Design This issue requires design work to think about how it would best be accomplished labels Aug 15, 2025

Fix some things

11ad770

cart approved these changes Aug 15, 2025

View reviewed changes

cart enabled auto-merge August 15, 2025 20:54

cart added this pull request to the merge queue Aug 15, 2025

Merged via the queue into bevyengine:main with commit 1bb17a9 Aug 15, 2025
36 checks passed

github-project-automation bot moved this to Done in Rendering Aug 15, 2025

Uh oh!

Initial DLSS implementation #19864

Initial DLSS implementation #19864

Uh oh!

Conversation

JMS55 commented Jun 29, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Objective

Solution

Testing

Showcase

Uh oh!

github-actions bot commented Jun 29, 2025

Uh oh!

JMS55 commented Jun 29, 2025

Uh oh!

github-actions bot commented Jun 29, 2025

Uh oh!

JMS55 commented Jun 29, 2025

Uh oh!

github-actions bot commented Jun 29, 2025

Uh oh!

cart commented Jul 1, 2025

Uh oh!

JMS55 commented Jul 1, 2025

Uh oh!

github-actions bot commented Jul 6, 2025

Uh oh!

alice-i-cecile commented Aug 12, 2025

Uh oh!

cart left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

cart Aug 13, 2025

Choose a reason for hiding this comment

Uh oh!

JMS55 Aug 13, 2025

Choose a reason for hiding this comment

Uh oh!

cart commented Aug 14, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

cart left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

JMS55 commented Jun 29, 2025 •

edited

Loading