StandardMaterial Light Transmission (bevyengine#8015)

# Objective

<img width="1920" alt="Screenshot 2023-04-26 at 01 07 34"
src="https://user-images.githubusercontent.com/418473/234467578-0f34187b-5863-4ea1-88e9-7a6bb8ce8da3.png">

This PR adds both diffuse and specular light transmission capabilities
to the `StandardMaterial`, with support for screen space refractions.
This enables realistically representing a wide range of real-world
materials, such as:

  - Glass; (Including frosted glass)
  - Transparent and translucent plastics;
  - Various liquids and gels;
  - Gemstones;
  - Marble;
  - Wax;
  - Paper;
  - Leaves;
  - Porcelain.

Unlike existing support for transparency, light transmission does not
rely on fixed function alpha blending, and therefore works with both
`AlphaMode::Opaque` and `AlphaMode::Mask` materials.

## Solution

- Introduces a number of transmission related fields in the
`StandardMaterial`;
- For specular transmission:
- Adds logic to take a view main texture snapshot after the opaque
phase; (in order to perform screen space refractions)
- Introduces a new `Transmissive3d` phase to the renderer, to which all
meshes with `transmission > 0.0` materials are sent.
- Calculates a light exit point (of the approximate mesh volume) using
`ior` and `thickness` properties
- Samples the snapshot texture with an adaptive number of taps across a
`roughness`-controlled radius enabling “blurry” refractions
- For diffuse transmission:
- Approximates transmitted diffuse light by using a second, flipped +
displaced, diffuse-only Lambertian lobe for each light source.

## To Do

- [x] Figure out where `fresnel_mix()` is taking place, if at all, and
where `dielectric_specular` is being calculated, if at all, and update
them to use the `ior` value (Not a blocker, just a nice-to-have for more
correct BSDF)
- To the _best of my knowledge, this is now taking place, after
964340c. The fresnel mix is actually "split" into two parts in our
implementation, one `(1 - fresnel(...))` in the transmission, and
`fresnel()` in the light implementations. A surface with more
reflectance now will produce slightly dimmer transmission towards the
grazing angle, as more of the light gets reflected.
- [x] Add `transmission_texture`
- [x] Add `diffuse_transmission_texture`
- [x] Add `thickness_texture`
- [x] Add `attenuation_distance` and `attenuation_color`
- [x] Connect values to glTF loader
  - [x] `transmission` and `transmission_texture`
  - [x] `thickness` and `thickness_texture`
  - [x] `ior`
- [ ] `diffuse_transmission` and `diffuse_transmission_texture` (needs
upstream support in `gltf` crate, not a blocker)
- [x] Add support for multiple screen space refraction “steps”
- [x] Conditionally create no transmission snapshot texture at all if
`steps == 0`
- [x] Conditionally enable/disable screen space refraction transmission
snapshots
- [x] Read from depth pre-pass to prevent refracting pixels in front of
the light exit point
- [x] Use `interleaved_gradient_noise()` function for sampling blur in a
way that benefits from TAA
- [x] Drill down a TAA `#define`, tweak some aspects of the effect
conditionally based on it
- [x] Remove const array that's crashing under HLSL (unless a new `naga`
release with gfx-rs/naga#2496 comes out before
we merge this)
- [ ] Look into alternatives to the `switch` hack for dynamically
indexing the const array (might not be needed, compilers seem to be
decent at expanding it)
- [ ] Add pipeline keys for gating transmission (do we really want/need
this?)
- [x] Tweak some material field/function names?

## A Note on Texture Packing

_This was originally added as a comment to the
`specular_transmission_texture`, `thickness_texture` and
`diffuse_transmission_texture` documentation, I removed it since it was
more confusing than helpful, and will likely be made redundant/will need
to be updated once we have a better infrastructure for preprocessing
assets_

Due to how channels are mapped, you can more efficiently use a single
shared texture image
for configuring the following:

- R - `specular_transmission_texture`
- G - `thickness_texture`
- B - _unused_
- A - `diffuse_transmission_texture`

The `KHR_materials_diffuse_transmission` glTF extension also defines a
`diffuseTransmissionColorTexture`,
that _we don't currently support_. One might choose to pack the
intensity and color textures together,
using RGB for the color and A for the intensity, in which case this
packing advice doesn't really apply.

---

## Changelog

- Added a new `Transmissive3d` render phase for rendering specular
transmissive materials with screen space refractions
- Added rendering support for transmitted environment map light on the
`StandardMaterial` as a fallback for screen space refractions
- Added `diffuse_transmission`, `specular_transmission`, `thickness`,
`ior`, `attenuation_distance` and `attenuation_color` to the
`StandardMaterial`
- Added `diffuse_transmission_texture`, `specular_transmission_texture`,
`thickness_texture` to the `StandardMaterial`, gated behind a new
`pbr_transmission_textures` cargo feature (off by default, for maximum
hardware compatibility)
- Added `Camera3d::screen_space_specular_transmission_steps` for
controlling the number of “layers of transparency” rendered for
transmissive objects
- Added a `TransmittedShadowReceiver` component for enabling shadows in
(diffusely) transmitted light. (disabled by default, as it requires
carefully setting up the `thickness` to avoid self-shadow artifacts)
- Added support for the `KHR_materials_transmission`,
`KHR_materials_ior` and `KHR_materials_volume` glTF extensions
- Renamed items related to temporal jitter for greater consistency

## Migration Guide

- `SsaoPipelineKey::temporal_noise` has been renamed to
`SsaoPipelineKey::temporal_jitter`
- The `TAA` shader def (controlled by the presence of the
`TemporalAntiAliasSettings` component in the camera) has been replaced
with the `TEMPORAL_JITTER` shader def (controlled by the presence of the
`TemporalJitter` component in the camera)
- `MeshPipelineKey::TAA` has been replaced by
`MeshPipelineKey::TEMPORAL_JITTER`
- The `TEMPORAL_NOISE` shader def has been consolidated with
`TEMPORAL_JITTER`

Cargo.toml

@@ -266,6 +266,9 @@ shader_format_glsl = ["bevy_internal/shader_format_glsl"]
 # Enable support for shaders in SPIR-V
 shader_format_spirv = ["bevy_internal/shader_format_spirv"]
 
+# Enable support for transmission-related textures in the `StandardMaterial`, at the risk of blowing past the global, per-shader texture limit on older/lower-end GPUs
+pbr_transmission_textures = ["bevy_internal/pbr_transmission_textures"]
+
 # Enable some limitations to be able to use WebGL2. If not enabled, it will default to WebGPU in Wasm. Please refer to the [WebGL2 and WebGPU](https://github.com/bevyengine/bevy/tree/latest/examples#webgl2-and-webgpu) section of the examples README for more information on how to run Wasm builds with WebGPU.
 webgl2 = ["bevy_internal/webgl"]
 
@@ -806,6 +809,16 @@ description = "Demonstrates transparency in 3d"
 category = "3D Rendering"
 wasm = true
 
+[[example]]
+name = "transmission"
+path = "examples/3d/transmission.rs"
+
+[package.metadata.example.transmission]
+name = "Transmission"
+description = "Showcases light transmission in the PBR material"
+category = "3D Rendering"
+wasm = true
+
 [[example]]
 name = "two_passes"
 path = "examples/3d/two_passes.rs"

crates/bevy_core_pipeline/src/core_3d/camera_3d.rs

@@ -25,6 +25,31 @@ pub struct Camera3d {
     pub depth_load_op: Camera3dDepthLoadOp,
     /// The texture usages for the depth texture created for the main 3d pass.
     pub depth_texture_usages: Camera3dDepthTextureUsage,
+    /// How many individual steps should be performed in the [`Transmissive3d`](crate::core_3d::Transmissive3d) pass.
+    ///
+    /// Roughly corresponds to how many “layers of transparency” are rendered for screen space
+    /// specular transmissive objects. Each step requires making one additional
+    /// texture copy, so it's recommended to keep this number to a resonably low value. Defaults to `1`.
+    ///
+    /// ### Notes
+    ///
+    /// - No copies will be performed if there are no transmissive materials currently being rendered,
+    ///   regardless of this setting.
+    /// - Setting this to `0` disables the screen-space refraction effect entirely, and falls
+    ///   back to refracting only the environment map light's texture.
+    /// - If set to more than `0`, any opaque [`clear_color`](Camera3d::clear_color) will obscure the environment
+    ///   map light's texture, preventing it from being visible “through” transmissive materials. If you'd like
+    ///   to still have the environment map show up in your refractions, you can set the clear color's alpha to `0.0`.
+    ///   Keep in mind that depending on the platform and your window settings, this may cause the window to become
+    ///   transparent.
+    pub screen_space_specular_transmission_steps: usize,
+    /// The quality of the screen space specular transmission blur effect, applied to whatever's “behind” transmissive
+    /// objects when their `roughness` is greater than `0.0`.
+    ///
+    /// Higher qualities are more GPU-intensive.
+    ///
+    /// **Note:** You can get better-looking results at any quality level by enabling TAA. See: [`TemporalAntiAliasPlugin`](crate::experimental::taa::TemporalAntiAliasPlugin).
+    pub screen_space_specular_transmission_quality: ScreenSpaceTransmissionQuality,
 }
 
 impl Default for Camera3d {
@@ -33,6 +58,8 @@ impl Default for Camera3d {
             clear_color: ClearColorConfig::Default,
             depth_load_op: Default::default(),
             depth_texture_usages: TextureUsages::RENDER_ATTACHMENT.into(),
+            screen_space_specular_transmission_steps: 1,
+            screen_space_specular_transmission_quality: Default::default(),
         }
     }
 }
@@ -77,6 +104,37 @@ impl From<Camera3dDepthLoadOp> for LoadOp<f32> {
     }
 }
 
+/// The quality of the screen space transmission blur effect, applied to whatever's “behind” transmissive
+/// objects when their `roughness` is greater than `0.0`.
+///
+/// Higher qualities are more GPU-intensive.
+///
+/// **Note:** You can get better-looking results at any quality level by enabling TAA. See: [`TemporalAntiAliasPlugin`](crate::experimental::taa::TemporalAntiAliasPlugin).
+#[derive(Resource, Default, Clone, Copy, Reflect, PartialEq, PartialOrd, Debug)]
+#[reflect(Resource)]
+pub enum ScreenSpaceTransmissionQuality {
+    /// Best performance at the cost of quality. Suitable for lower end GPUs. (e.g. Mobile)
+    ///
+    /// `num_taps` = 4
+    Low,
+
+    /// A balanced option between quality and performance.
+    ///
+    /// `num_taps` = 8
+    #[default]
+    Medium,
+
+    /// Better quality. Suitable for high end GPUs. (e.g. Desktop)
+    ///
+    /// `num_taps` = 16
+    High,
+
+    /// The highest quality, suitable for non-realtime rendering. (e.g. Pre-rendered cinematics and photo mode)
+    ///
+    /// `num_taps` = 32
+    Ultra,
+}
+
 #[derive(Bundle)]
 pub struct Camera3dBundle {
     pub camera: Camera,

crates/bevy_core_pipeline/src/core_3d/main_transmissive_pass_3d_node.rs

@@ -0,0 +1,148 @@
+use super::{Camera3d, ViewTransmissionTexture};
+use crate::core_3d::Transmissive3d;
+use bevy_ecs::{prelude::*, query::QueryItem};
+use bevy_render::{
+    camera::ExtractedCamera,
+    render_graph::{NodeRunError, RenderGraphContext, ViewNode},
+    render_phase::RenderPhase,
+    render_resource::{
+        Extent3d, LoadOp, Operations, RenderPassDepthStencilAttachment, RenderPassDescriptor,
+    },
+    renderer::RenderContext,
+    view::{ViewDepthTexture, ViewTarget},
+};
+#[cfg(feature = "trace")]
+use bevy_utils::tracing::info_span;
+use std::ops::Range;
+
+/// A [`bevy_render::render_graph::Node`] that runs the [`Transmissive3d`] [`RenderPhase`].
+#[derive(Default)]
+pub struct MainTransmissivePass3dNode;
+
+impl ViewNode for MainTransmissivePass3dNode {
+    type ViewQuery = (
+        &'static ExtractedCamera,
+        &'static Camera3d,
+        &'static RenderPhase<Transmissive3d>,
+        &'static ViewTarget,
+        Option<&'static ViewTransmissionTexture>,
+        &'static ViewDepthTexture,
+    );
+
+    fn run(
+        &self,
+        graph: &mut RenderGraphContext,
+        render_context: &mut RenderContext,
+        (camera, camera_3d, transmissive_phase, target, transmission, depth): QueryItem<
+            Self::ViewQuery,
+        >,
+        world: &World,
+    ) -> Result<(), NodeRunError> {
+        let view_entity = graph.view_entity();
+
+        let physical_target_size = camera.physical_target_size.unwrap();
+
+        let render_pass_descriptor = RenderPassDescriptor {
+            label: Some("main_transmissive_pass_3d"),
+            // NOTE: The transmissive pass loads the color buffer as well as overwriting it where appropriate.
+            color_attachments: &[Some(target.get_color_attachment(Operations {
+                load: LoadOp::Load,
+                store: true,
+            }))],
+            depth_stencil_attachment: Some(RenderPassDepthStencilAttachment {
+                view: &depth.view,
+                // NOTE: The transmissive main pass loads the depth buffer and possibly overwrites it
+                depth_ops: Some(Operations {
+                    load: LoadOp::Load,
+                    store: true,
+                }),
+                stencil_ops: None,
+            }),
+        };
+
+        // Run the transmissive pass, sorted back-to-front
+        // NOTE: Scoped to drop the mutable borrow of render_context
+        #[cfg(feature = "trace")]
+        let _main_transmissive_pass_3d_span = info_span!("main_transmissive_pass_3d").entered();
+
+        if !transmissive_phase.items.is_empty() {
+            let screen_space_specular_transmission_steps =
+                camera_3d.screen_space_specular_transmission_steps;
+            if screen_space_specular_transmission_steps > 0 {
+                let transmission =
+                    transmission.expect("`ViewTransmissionTexture` should exist at this point");
+
+                // `transmissive_phase.items` are depth sorted, so we split them into N = `screen_space_specular_transmission_steps`
+                // ranges, rendering them back-to-front in multiple steps, allowing multiple levels of transparency.
+                //
+                // Note: For the sake of simplicity, we currently split items evenly among steps. In the future, we
+                // might want to use a more sophisticated heuristic (e.g. based on view bounds, or with an exponential
+                // falloff so that nearby objects have more levels of transparency available to them)
+                for range in split_range(
+                    0..transmissive_phase.items.len(),
+                    screen_space_specular_transmission_steps,
+                ) {
+                    // Copy the main texture to the transmission texture, allowing to use the color output of the
+                    // previous step (or of the `Opaque3d` phase, for the first step) as a transmissive color input
+                    render_context.command_encoder().copy_texture_to_texture(
+                        target.main_texture().as_image_copy(),
+                        transmission.texture.as_image_copy(),
+                        Extent3d {
+                            width: physical_target_size.x,
+                            height: physical_target_size.y,
+                            depth_or_array_layers: 1,
+                        },
+                    );
+
+                    let mut render_pass =
+                        render_context.begin_tracked_render_pass(render_pass_descriptor.clone());
+
+                    if let Some(viewport) = camera.viewport.as_ref() {
+                        render_pass.set_camera_viewport(viewport);
+                    }
+
+                    // render items in range
+                    transmissive_phase.render_range(&mut render_pass, world, view_entity, range);
+                }
+            } else {
+                let mut render_pass =
+                    render_context.begin_tracked_render_pass(render_pass_descriptor);
+
+                if let Some(viewport) = camera.viewport.as_ref() {
+                    render_pass.set_camera_viewport(viewport);
+                }
+
+                transmissive_phase.render(&mut render_pass, world, view_entity);
+            }
+        }
+
+        Ok(())
+    }
+}
+
+/// Splits a [`Range`] into at most `max_num_splits` sub-ranges without overlaps
+///
+/// Properly takes into account remainders of inexact divisions (by adding extra
+/// elements to the initial sub-ranges as needed)
+fn split_range(range: Range<usize>, max_num_splits: usize) -> impl Iterator<Item = Range<usize>> {
+    let len = range.end - range.start;
+    assert!(len > 0, "to be split, a range must not be empty");
+    assert!(max_num_splits > 0, "max_num_splits must be at least 1");
+    let num_splits = max_num_splits.min(len);
+    let step = len / num_splits;
+    let mut rem = len % num_splits;
+    let mut start = range.start;
+
+    (0..num_splits).map(move |_| {
+        let extra = if rem > 0 {
+            rem -= 1;
+            1
+        } else {
+            0
+        };
+        let end = (start + step + extra).min(range.end);
+        let result = start..end;
+        start = end;
+        result
+    })
+}