Skip to content

Commit

Permalink
Deferred Renderer (#9258)
Browse files Browse the repository at this point in the history 
# Objective

- Add a [Deferred
Renderer](https://en.wikipedia.org/wiki/Deferred_shading) to Bevy.
- This allows subsequent passes to access per pixel material information
before/during shading.
- Accessing this per pixel material information is needed for some
features, like GI. It also makes other features (ex. Decals) simpler to
implement and/or improves their capability. There are multiple
approaches to accomplishing this. The deferred shading approach works
well given the limitations of WebGPU and WebGL2.

Motivation: [I'm working on a GI solution for
Bevy](https://youtu.be/eH1AkL-mwhI)

# Solution
- The deferred renderer is implemented with a prepass and a deferred
lighting pass.
- The prepass renders opaque objects into the Gbuffer attachment
(`Rgba32Uint`). The PBR shader generates a `PbrInput` in mostly the same
way as the forward implementation and then [packs it into the
Gbuffer](https://github.com/DGriffin91/bevy/blob/ec1465559f2c82001830e908fc02ff1d7c2efe51/crates/bevy_pbr/src/render/pbr.wgsl#L168).
- The deferred lighting pass unpacks the `PbrInput` and [feeds it into
the pbr()
function](https://github.com/DGriffin91/bevy/blob/ec1465559f2c82001830e908fc02ff1d7c2efe51/crates/bevy_pbr/src/deferred/deferred_lighting.wgsl#L65),
then outputs the shaded color data.

- There is now a resource
[DefaultOpaqueRendererMethod](https://github.com/DGriffin91/bevy/blob/ec1465559f2c82001830e908fc02ff1d7c2efe51/crates/bevy_pbr/src/material.rs#L599)
that can be used to set the default render method for opaque materials.
If materials return `None` from
[opaque_render_method()](https://github.com/DGriffin91/bevy/blob/ec1465559f2c82001830e908fc02ff1d7c2efe51/crates/bevy_pbr/src/material.rs#L131)
the `DefaultOpaqueRendererMethod` will be used. Otherwise, custom
materials can also explicitly choose to only support Deferred or Forward
by returning the respective
[OpaqueRendererMethod](https://github.com/DGriffin91/bevy/blob/ec1465559f2c82001830e908fc02ff1d7c2efe51/crates/bevy_pbr/src/material.rs#L603)

- Deferred materials can be used seamlessly along with both opaque and
transparent forward rendered materials in the same scene. The [deferred
rendering
example](https://github.com/DGriffin91/bevy/blob/deferred/examples/3d/deferred_rendering.rs)
does this.

- The deferred renderer does not support MSAA. If any deferred materials
are used, MSAA must be disabled. Both TAA and FXAA are supported.

- Deferred rendering supports WebGL2/WebGPU. 

## Custom deferred materials
- Custom materials can support both deferred and forward at the same
time. The
[StandardMaterial](https://github.com/DGriffin91/bevy/blob/ec1465559f2c82001830e908fc02ff1d7c2efe51/crates/bevy_pbr/src/render/pbr.wgsl#L166)
does this. So does [this
example](https://github.com/DGriffin91/bevy_glowy_orb_tutorial/blob/deferred/assets/shaders/glowy.wgsl#L56).
- Custom deferred materials that require PBR lighting can create a
`PbrInput`, write it to the deferred GBuffer and let it be rendered by
the `PBRDeferredLightingPlugin`.
- Custom deferred materials that require custom lighting have two
options:
1. Use the base_color channel of the `PbrInput` combined with the
`STANDARD_MATERIAL_FLAGS_UNLIT_BIT` flag.
[Example.](https://github.com/DGriffin91/bevy_glowy_orb_tutorial/blob/deferred/assets/shaders/glowy.wgsl#L56)
(If the unlit bit is set, the base_color is stored as RGB9E5 for extra
precision)
2. A Custom Deferred Lighting pass can be created, either overriding the
default, or running in addition. The a depth buffer is used to limit
rendering to only the required fragments for each deferred lighting
pass. Materials can set their respective depth id via the
[deferred_lighting_pass_id](https://github.com/DGriffin91/bevy/blob/b79182d2a32cac28c4213c2457a53ac2cc885332/crates/bevy_pbr/src/prepass/prepass_io.wgsl#L95)
attachment. The custom deferred lighting pass plugin can then set [its
corresponding
depth](https://github.com/DGriffin91/bevy/blob/ec1465559f2c82001830e908fc02ff1d7c2efe51/crates/bevy_pbr/src/deferred/deferred_lighting.wgsl#L37).
Then with the lighting pass using
[CompareFunction::Equal](https://github.com/DGriffin91/bevy/blob/ec1465559f2c82001830e908fc02ff1d7c2efe51/crates/bevy_pbr/src/deferred/mod.rs#L335),
only the fragments with a depth that equal the corresponding depth
written in the material will be rendered.

Custom deferred lighting plugins can also be created to render the
StandardMaterial. The default deferred lighting plugin can be bypassed
with `DefaultPlugins.set(PBRDeferredLightingPlugin { bypass: true })`

---------

Co-authored-by: nickrart <nickolas.g.russell@gmail.com>
  • Loading branch information
@DGriffin91 @nickrart
DGriffin91 and nickrart authored Oct 12, 2023
1 parent c8fd390 commit a15d152
Show file tree
Hide file tree
Showing 40 changed files with 2,959 additions and 485 deletions.
10 changes: 10 additions & 0 deletions Cargo.toml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -611,6 +611,16 @@ description = "Illustrates bloom configuration using HDR and emissive materials"
category = "3D Rendering"
wasm = true

[[example]]
name = "deferred_rendering"
path = "examples/3d/deferred_rendering.rs"

[package.metadata.example.deferred_rendering]
name = "Deferred Rendering"
description = "Renders meshes with both forward and deferred pipelines"
category = "3D Rendering"
wasm = true

[[example]]
name = "load_gltf"
path = "examples/3d/load_gltf.rs"
Expand Down
4 changes: 2 additions & 2 deletions assets/shaders/array_texture.wgsl
View file
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
#import bevy_pbr::mesh_vertex_output MeshVertexOutput
#import bevy_pbr::mesh_view_bindings view
#import bevy_pbr::pbr_types STANDARD_MATERIAL_FLAGS_DOUBLE_SIDED_BIT
#import bevy_pbr::pbr_types STANDARD_MATERIAL_FLAGS_DOUBLE_SIDED_BIT, PbrInput, pbr_input_new
#import bevy_core_pipeline::tonemapping tone_mapping
#import bevy_pbr::pbr_functions as fns

Expand All @@ -16,7 +16,7 @@ fn fragment(

// Prepare a 'processed' StandardMaterial by sampling all textures to resolve
// the material members
var pbr_input: fns::PbrInput = fns::pbr_input_new();
var pbr_input: PbrInput = pbr_input_new();

pbr_input.material.base_color = textureSample(my_array_texture, my_array_texture_sampler, mesh.uv, layer);
#ifdef VERTEX_COLORS
Expand Down
29 changes: 18 additions & 11 deletions crates/bevy_core_pipeline/src/core_3d/main_opaque_pass_3d_node.rs
View file
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
use crate::{
clear_color::{ClearColor, ClearColorConfig},
core_3d::{Camera3d, Opaque3d},
prepass::{DepthPrepass, MotionVectorPrepass, NormalPrepass},
prepass::{DeferredPrepass, DepthPrepass, MotionVectorPrepass, NormalPrepass},
skybox::{SkyboxBindGroup, SkyboxPipelineId},
};
use bevy_ecs::{prelude::*, query::QueryItem};
Expand Down Expand Up @@ -34,6 +34,7 @@ impl ViewNode for MainOpaquePass3dNode {
Option<&'static DepthPrepass>,
Option<&'static NormalPrepass>,
Option<&'static MotionVectorPrepass>,
Option<&'static DeferredPrepass>,
Option<&'static SkyboxPipelineId>,
Option<&'static SkyboxBindGroup>,
&'static ViewUniformOffset,
Expand All @@ -53,12 +54,24 @@ impl ViewNode for MainOpaquePass3dNode {
depth_prepass,
normal_prepass,
motion_vector_prepass,
deferred_prepass,
skybox_pipeline,
skybox_bind_group,
view_uniform_offset,
): QueryItem<Self::ViewQuery>,
world: &World,
) -> Result<(), NodeRunError> {
let load = if deferred_prepass.is_none() {
match camera_3d.clear_color {
ClearColorConfig::Default => LoadOp::Clear(world.resource::<ClearColor>().0.into()),
ClearColorConfig::Custom(color) => LoadOp::Clear(color.into()),
ClearColorConfig::None => LoadOp::Load,
}
} else {
// If the deferred lighting pass has run, don't clear again in this pass.
LoadOp::Load
};

// Run the opaque pass, sorted front-to-back
// NOTE: Scoped to drop the mutable borrow of render_context
#[cfg(feature = "trace")]
Expand All @@ -69,23 +82,17 @@ impl ViewNode for MainOpaquePass3dNode {
label: Some("main_opaque_pass_3d"),
// NOTE: The opaque pass loads the color
// buffer as well as writing to it.
color_attachments: &[Some(target.get_color_attachment(Operations {
load: match camera_3d.clear_color {
ClearColorConfig::Default => {
LoadOp::Clear(world.resource::<ClearColor>().0.into())
}
ClearColorConfig::Custom(color) => LoadOp::Clear(color.into()),
ClearColorConfig::None => LoadOp::Load,
},
store: true,
}))],
color_attachments: &[Some(
target.get_color_attachment(Operations { load, store: true }),
)],
depth_stencil_attachment: Some(RenderPassDepthStencilAttachment {
view: &depth.view,
// NOTE: The opaque main pass loads the depth buffer and possibly overwrites it
depth_ops: Some(Operations {
load: if depth_prepass.is_some()
|| normal_prepass.is_some()
|| motion_vector_prepass.is_some()
|| deferred_prepass.is_some()
{
// if any prepass runs, it will generate a depth buffer so we should use it,
// even if only the normal_prepass is used.
Expand Down
126 changes: 116 additions & 10 deletions crates/bevy_core_pipeline/src/core_3d/mod.rs
View file
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,9 @@ pub mod graph {
pub mod node {
pub const MSAA_WRITEBACK: &str = "msaa_writeback";
pub const PREPASS: &str = "prepass";
pub const DEFERRED_PREPASS: &str = "deferred_prepass";
pub const COPY_DEFERRED_LIGHTING_ID: &str = "copy_deferred_lighting_id";
pub const END_PREPASSES: &str = "end_prepasses";
pub const START_MAIN_PASS: &str = "start_main_pass";
pub const MAIN_OPAQUE_PASS: &str = "main_opaque_pass";
pub const MAIN_TRANSPARENT_PASS: &str = "main_transparent_pass";
Expand All @@ -24,13 +27,16 @@ pub mod graph {
}
pub const CORE_3D: &str = graph::NAME;

// PERF: vulkan docs recommend using 24 bit depth for better performance
pub const CORE_3D_DEPTH_FORMAT: TextureFormat = TextureFormat::Depth32Float;

use std::{cmp::Reverse, ops::Range};

pub use camera_3d::*;
pub use main_opaque_pass_3d_node::*;
pub use main_transparent_pass_3d_node::*;

use bevy_app::{App, Plugin};
use bevy_app::{App, Plugin, PostUpdate};
use bevy_ecs::prelude::*;
use bevy_render::{
camera::{Camera, ExtractedCamera},
Expand All @@ -50,12 +56,17 @@ use bevy_render::{
view::ViewDepthTexture,
Extract, ExtractSchedule, Render, RenderApp, RenderSet,
};
use bevy_utils::{nonmax::NonMaxU32, FloatOrd, HashMap};
use bevy_utils::{nonmax::NonMaxU32, tracing::warn, FloatOrd, HashMap};

use crate::{
deferred::{
copy_lighting_id::CopyDeferredLightingIdNode, node::DeferredGBufferPrepassNode,
AlphaMask3dDeferred, Opaque3dDeferred, DEFERRED_LIGHTING_PASS_ID_FORMAT,
DEFERRED_PREPASS_FORMAT,
},
prepass::{
node::PrepassNode, AlphaMask3dPrepass, DepthPrepass, MotionVectorPrepass, NormalPrepass,
Opaque3dPrepass, ViewPrepassTextures, DEPTH_PREPASS_FORMAT, MOTION_VECTOR_PREPASS_FORMAT,
node::PrepassNode, AlphaMask3dPrepass, DeferredPrepass, DepthPrepass, MotionVectorPrepass,
NormalPrepass, Opaque3dPrepass, ViewPrepassTextures, MOTION_VECTOR_PREPASS_FORMAT,
NORMAL_PREPASS_FORMAT,
},
skybox::SkyboxPlugin,
Expand All @@ -69,7 +80,8 @@ impl Plugin for Core3dPlugin {
fn build(&self, app: &mut App) {
app.register_type::<Camera3d>()
.register_type::<Camera3dDepthLoadOp>()
.add_plugins((SkyboxPlugin, ExtractComponentPlugin::<Camera3d>::default()));
.add_plugins((SkyboxPlugin, ExtractComponentPlugin::<Camera3d>::default()))
.add_systems(PostUpdate, check_msaa);

let render_app = match app.get_sub_app_mut(RenderApp) {
Ok(render_app) => render_app,
Expand All @@ -82,6 +94,8 @@ impl Plugin for Core3dPlugin {
.init_resource::<DrawFunctions<Transparent3d>>()
.init_resource::<DrawFunctions<Opaque3dPrepass>>()
.init_resource::<DrawFunctions<AlphaMask3dPrepass>>()
.init_resource::<DrawFunctions<Opaque3dDeferred>>()
.init_resource::<DrawFunctions<AlphaMask3dDeferred>>()
.add_systems(ExtractSchedule, extract_core_3d_camera_phases)
.add_systems(ExtractSchedule, extract_camera_prepass_phase)
.add_systems(
Expand All @@ -92,6 +106,8 @@ impl Plugin for Core3dPlugin {
sort_phase_system::<Transparent3d>.in_set(RenderSet::PhaseSort),
sort_phase_system::<Opaque3dPrepass>.in_set(RenderSet::PhaseSort),
sort_phase_system::<AlphaMask3dPrepass>.in_set(RenderSet::PhaseSort),
sort_phase_system::<Opaque3dDeferred>.in_set(RenderSet::PhaseSort),
sort_phase_system::<AlphaMask3dDeferred>.in_set(RenderSet::PhaseSort),
prepare_core_3d_depth_textures.in_set(RenderSet::PrepareResources),
prepare_prepass_textures.in_set(RenderSet::PrepareResources),
),
Expand All @@ -101,6 +117,15 @@ impl Plugin for Core3dPlugin {
render_app
.add_render_sub_graph(CORE_3D)
.add_render_graph_node::<ViewNodeRunner<PrepassNode>>(CORE_3D, PREPASS)
.add_render_graph_node::<ViewNodeRunner<DeferredGBufferPrepassNode>>(
CORE_3D,
DEFERRED_PREPASS,
)
.add_render_graph_node::<ViewNodeRunner<CopyDeferredLightingIdNode>>(
CORE_3D,
COPY_DEFERRED_LIGHTING_ID,
)
.add_render_graph_node::<EmptyNode>(CORE_3D, END_PREPASSES)
.add_render_graph_node::<EmptyNode>(CORE_3D, START_MAIN_PASS)
.add_render_graph_node::<ViewNodeRunner<MainOpaquePass3dNode>>(
CORE_3D,
Expand All @@ -118,6 +143,9 @@ impl Plugin for Core3dPlugin {
CORE_3D,
&[
PREPASS,
DEFERRED_PREPASS,
COPY_DEFERRED_LIGHTING_ID,
END_PREPASSES,
START_MAIN_PASS,
MAIN_OPAQUE_PASS,
MAIN_TRANSPARENT_PASS,
Expand Down Expand Up @@ -341,12 +369,14 @@ pub fn extract_camera_prepass_phase(
Option<&DepthPrepass>,
Option<&NormalPrepass>,
Option<&MotionVectorPrepass>,
Option<&DeferredPrepass>,
),
With<Camera3d>,
>,
>,
) {
for (entity, camera, depth_prepass, normal_prepass, motion_vector_prepass) in cameras_3d.iter()
for (entity, camera, depth_prepass, normal_prepass, motion_vector_prepass, deferred_prepass) in
cameras_3d.iter()
{
if camera.is_active {
let mut entity = commands.get_or_spawn(entity);
Expand All @@ -361,6 +391,13 @@ pub fn extract_camera_prepass_phase(
));
}

if deferred_prepass.is_some() {
entity.insert((
RenderPhase::<Opaque3dDeferred>::default(),
RenderPhase::<AlphaMask3dDeferred>::default(),
));
}

if depth_prepass.is_some() {
entity.insert(DepthPrepass);
}
Expand All @@ -370,6 +407,9 @@ pub fn extract_camera_prepass_phase(
if motion_vector_prepass.is_some() {
entity.insert(MotionVectorPrepass);
}
if deferred_prepass.is_some() {
entity.insert(DeferredPrepass);
}
}
}
}
Expand Down Expand Up @@ -428,8 +468,7 @@ pub fn prepare_core_3d_depth_textures(
mip_level_count: 1,
sample_count: msaa.samples(),
dimension: TextureDimension::D2,
// PERF: vulkan docs recommend using 24 bit depth for better performance
format: TextureFormat::Depth32Float,
format: CORE_3D_DEPTH_FORMAT,
usage,
view_formats: &[],
};
Expand All @@ -445,6 +484,22 @@ pub fn prepare_core_3d_depth_textures(
}
}

// Disable MSAA and warn if using deferred rendering
pub fn check_msaa(
mut msaa: ResMut<Msaa>,
deferred_views: Query<Entity, (With<Camera>, With<DeferredPrepass>)>,
) {
if !deferred_views.is_empty() {
match *msaa {
Msaa::Off => (),
_ => {
warn!("MSAA is incompatible with deferred rendering and has been disabled.");
*msaa = Msaa::Off;
}
};
}
}

// Prepares the textures used by the prepass
pub fn prepare_prepass_textures(
mut commands: Commands,
Expand All @@ -458,6 +513,7 @@ pub fn prepare_prepass_textures(
Option<&DepthPrepass>,
Option<&NormalPrepass>,
Option<&MotionVectorPrepass>,
Option<&DeferredPrepass>,
),
(
With<RenderPhase<Opaque3dPrepass>>,
Expand All @@ -467,8 +523,12 @@ pub fn prepare_prepass_textures(
) {
let mut depth_textures = HashMap::default();
let mut normal_textures = HashMap::default();
let mut deferred_textures = HashMap::default();
let mut deferred_lighting_id_textures = HashMap::default();
let mut motion_vectors_textures = HashMap::default();
for (entity, camera, depth_prepass, normal_prepass, motion_vector_prepass) in &views_3d {
for (entity, camera, depth_prepass, normal_prepass, motion_vector_prepass, deferred_prepass) in
&views_3d
{
let Some(physical_target_size) = camera.physical_target_size else {
continue;
};
Expand All @@ -489,7 +549,7 @@ pub fn prepare_prepass_textures(
mip_level_count: 1,
sample_count: msaa.samples(),
dimension: TextureDimension::D2,
format: DEPTH_PREPASS_FORMAT,
format: CORE_3D_DEPTH_FORMAT,
usage: TextureUsages::COPY_DST
| TextureUsages::RENDER_ATTACHMENT
| TextureUsages::TEXTURE_BINDING,
Expand Down Expand Up @@ -544,10 +604,56 @@ pub fn prepare_prepass_textures(
.clone()
});

let cached_deferred_texture = deferred_prepass.is_some().then(|| {
deferred_textures
.entry(camera.target.clone())
.or_insert_with(|| {
texture_cache.get(
&render_device,
TextureDescriptor {
label: Some("prepass_deferred_texture"),
size,
mip_level_count: 1,
sample_count: 1,
dimension: TextureDimension::D2,
format: DEFERRED_PREPASS_FORMAT,
usage: TextureUsages::RENDER_ATTACHMENT
| TextureUsages::TEXTURE_BINDING,
view_formats: &[],
},
)
})
.clone()
});

let deferred_lighting_pass_id_texture = deferred_prepass.is_some().then(|| {
deferred_lighting_id_textures
.entry(camera.target.clone())
.or_insert_with(|| {
texture_cache.get(
&render_device,
TextureDescriptor {
label: Some("deferred_lighting_pass_id_texture"),
size,
mip_level_count: 1,
sample_count: 1,
dimension: TextureDimension::D2,
format: DEFERRED_LIGHTING_PASS_ID_FORMAT,
usage: TextureUsages::RENDER_ATTACHMENT
| TextureUsages::TEXTURE_BINDING,
view_formats: &[],
},
)
})
.clone()
});

commands.entity(entity).insert(ViewPrepassTextures {
depth: cached_depth_texture,
normal: cached_normals_texture,
motion_vectors: cached_motion_vectors_texture,
deferred: cached_deferred_texture,
deferred_lighting_pass_id: deferred_lighting_pass_id_texture,
size,
});
}
Expand Down
19 changes: 19 additions & 0 deletions crates/bevy_core_pipeline/src/deferred/copy_deferred_lighting_id.wgsl
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
#import bevy_pbr::utils
#import bevy_core_pipeline::fullscreen_vertex_shader FullscreenVertexOutput

@group(0) @binding(0)
var material_id_texture: texture_2d<u32>;

struct FragmentOutput {
@builtin(frag_depth) frag_depth: f32,

}

@fragment
fn fragment(in: FullscreenVertexOutput) -> FragmentOutput {
var out: FragmentOutput;
// Depth is stored as unorm, so we are dividing the u8 by 255.0 here.
out.frag_depth = f32(textureLoad(material_id_texture, vec2<i32>(in.position.xy), 0).x) / 255.0;
return out;
}

Loading

0 comments on commit a15d152

Please sign in to comment.