Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WebGPURenderer: Fixed shadows not rendering correctly with logarithmicDepthBuffer #29447

Merged
merged 6 commits into from
Sep 28, 2024
Merged

WebGPURenderer: Fixed shadows not rendering correctly with logarithmicDepthBuffer #29447

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
4ca3e3c
Fixed shadows not rendering correctly when logarithmicDepthBuffer = t…
PoseidonEnergy Sep 19, 2024
c3a5cb6
Further improvements to the logarithmic depth buffer shadows bugfix:
PoseidonEnergy Sep 21, 2024
6a2d716
Further improvements to the logarithmic depth buffer shadows bugfix:
PoseidonEnergy Sep 23, 2024
779d372
Further improvements to the logarithmic depth buffer shadows bugfix:
PoseidonEnergy Sep 23, 2024
fb3e23d
Merge branch 'dev' into pr/29447
sunag Sep 27, 2024
8b09308
cleanup
sunag Sep 27, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 18 additions & 6 deletions src/materials/nodes/NodeMaterial.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ import { normalLocal } from '../../nodes/accessors/Normal.js';
import { instance } from '../../nodes/accessors/InstanceNode.js';
import { batch } from '../../nodes/accessors/BatchNode.js';
import { materialReference } from '../../nodes/accessors/MaterialReferenceNode.js';
import { positionLocal } from '../../nodes/accessors/Position.js';
import { positionLocal, positionView } from '../../nodes/accessors/Position.js';
import { skinningReference } from '../../nodes/accessors/SkinningNode.js';
import { morphReference } from '../../nodes/accessors/MorphNode.js';
import { lights } from '../../nodes/lighting/LightsNode.js';
Expand All @@ -19,8 +19,8 @@ import { float, vec3, vec4 } from '../../nodes/tsl/TSLBase.js';
import AONode from '../../nodes/lighting/AONode.js';
import { lightingContext } from '../../nodes/lighting/LightingContextNode.js';
import IrradianceNode from '../../nodes/lighting/IrradianceNode.js';
import { depth } from '../../nodes/display/ViewportDepthNode.js';
import { cameraLogDepth } from '../../nodes/accessors/Camera.js';
import { depth, perspectiveDepthToLogarithmicDepth, viewZToOrthographicDepth } from '../../nodes/display/ViewportDepthNode.js';
import { cameraFar, cameraNear } from '../../nodes/accessors/Camera.js';
import { clipping, clippingAlpha } from '../../nodes/accessors/ClippingNode.js';
import NodeMaterialObserver from './manager/NodeMaterialObserver.js';

Expand Down Expand Up @@ -215,7 +215,7 @@ class NodeMaterial extends Material {

setupDepth( builder ) {

const { renderer } = builder;
const { renderer, camera } = builder;

// Depth

Expand All @@ -231,9 +231,21 @@ class NodeMaterial extends Material {

} else if ( renderer.logarithmicDepthBuffer === true ) {

const fragDepth = modelViewProjection().w.add( 1 );
if ( camera.isPerspectiveCamera ) {

depthNode = fragDepth.log2().mul( cameraLogDepth ).mul( 0.5 );
// Note: normally we could use "float( camera.near )" and "float( camera.far )" for the near/far arguments, but
// there is currently a bug with TSL/Three Shading Language whereby a "float()" expression using a huge value
// in scientific notation like "float( 1e27 )" will output "1e+27.0" to the shader code, which is causing problems.
// Since it's possible that camera.near/camera.far values may be using huge values like this (such as the logarithmic
// depth buffer examples on threejs.org), we must use the cameraNear/cameraFar nodes for now.
// TODO: can the float() node be fixed to allow for expressions like "float( 1e27 )"?
depthNode = perspectiveDepthToLogarithmicDepth( modelViewProjection().w, cameraNear, cameraFar );

} else {

depthNode = viewZToOrthographicDepth( positionView.z, cameraNear, cameraFar );

}

}

Expand Down
1 change: 0 additions & 1 deletion src/nodes/accessors/Camera.js
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

27 changes: 26 additions & 1 deletion src/nodes/display/ViewportDepthNode.js
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
import Node from '../core/Node.js';
import { nodeImmutable, nodeProxy } from '../tsl/TSLBase.js';
import { log2, nodeImmutable, nodeProxy } from '../tsl/TSLBase.js';
import { cameraNear, cameraFar } from '../accessors/Camera.js';
import { positionView } from '../accessors/Position.js';
import { viewportDepthTexture } from './ViewportDepthTextureNode.js';
Expand Down Expand Up @@ -116,6 +116,31 @@ export const viewZToPerspectiveDepth = ( viewZ, near, far ) => near.add( viewZ )
// maps perspective depth in [ 0, 1 ] to viewZ
export const perspectiveDepthToViewZ = ( depth, near, far ) => near.mul( far ).div( far.sub( near ).mul( depth ).sub( far ) );

export const perspectiveDepthToLogarithmicDepth = ( perspectiveW, near, far ) => {
Copy link
Collaborator

@Mugen87 Mugen87 Nov 4, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In post processing we have to compute the view position of a fragment in various effects. We do that with getViewPosition():

three.js/src/nodes/utils/PostProcessingUtils.js

Line 15 in 9e5528d

export const getViewPosition = /*@__PURE__*/ Fn( ( [ screenPosition, depth, projectionMatrixInverse ], builder ) => {

This method does currently not work if a logarithmic depth buffer is used. How would the reverse operation of perspectiveDepthToLogarithmicDepth() look like that computes a "normal" depth value based on the logarithmic depth?

Copy link
Contributor Author

@PoseidonEnergy PoseidonEnergy Nov 4, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Update 2024-11-13 | This post and info on the logarithmic depth buffer calculation is outdated. Please see the new revisions to the logarithmic depth functions in the pull request here: #29870

####################################################################
####################################################################
####################################################################

@Mugen87 Computing the perspective w value from a logarithmic depth value using the new formula would look like this. Note that this gives you the w value, as in modelViewProjection.w. To get viewZ, just divide z by w (I know you know that already but I'm just saying it for posterity).

image

The TSL node function based on the math shown above would look like this. Like mentioned before, this returns the perspective w value, and not viewZ. To get viewZ, just do z / w:

// TODO: should this function be called "logarithmicDepthToPerspectiveW" instead?
const logarithmicDepthToPerspectiveDepth = (logDepth, near, far) => {
  // NOTE: this returns the "w" component, not "z"
  const exponent = logDepth.mul(log(far.div(near).add(1)));
  return float(Math.E).pow(exponent).sub(1).mul(near);
};

NOTE: I have not tested the above node function above...it looks correct though...needs another set of eyes.

As an aside, should the functions perspectiveDepthToLogarithmicDepth() and logarithmicDepthToPerspectiveDepth() be renamed to perspectiveWToLogarithmicDepth() and logarithmicDepthToPerspectiveW() to make it more clear that the w component is being used?

Desmos graph: https://www.desmos.com/calculator/1e3ttyfe6q

image

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've tried to replace the sampleDepth() function in GTAONode for quick testing but it does not work yet. Here is the code:

const sampleDepth = ( uv ) => {

	const depth = this.depthNode.uv( uv ).x;

	if ( builder.renderer.logarithmicDepthBuffer === true ) {

		const mvp = positionWorld.mul( this.cameraProjectionMatrix );

		return mvp.z.div( logarithmicDepthToPerspectiveDepth( depth, this.cameraNear, this.cameraFar ) );

	} else {

		return depth;

	}

};

Do you see an obvious issue with the above code?

Copy link
Contributor Author

@PoseidonEnergy PoseidonEnergy Nov 4, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can say the following at the moment:

  1. depth in const depth = this.depthNode.uv( uv ).x; should range between 0 and 1 before being passed to logarithmicDepthToPerspectiveDepth( depth, this.cameraNear, this.cameraFar )
  2. Without looking at the rest of the code, this.cameraNear and this.cameraFar should be uniform nodes

In the meantime I will continue verifying that the logarithmicDepthToPerspectiveDepth function is correct.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Both requirements were true when during my tests. I'll revisit the code tomorrow.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As an aside, should the functions perspectiveDepthToLogarithmicDepth() and logarithmicDepthToPerspectiveDepth() be renamed to perspectiveWToLogarithmicDepth() and logarithmicDepthToPerspectiveW() to make it more clear that the w component is being used?

Yes, I think otherwise the names are confusing. Let's use perspectiveWToLogarithmicDepth().

Copy link
Collaborator

@Mugen87 Mugen87 Nov 5, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can't get it to work with the proposed formula. I've also tried a slightly different formula but with no success.

export const logarithmicDepthToPerspectiveW = ( logarithmicDepth, near, far ) => {

	const logRatio = log2( far.div( near ).add( 1 ) );
	const perspectiveW = near.mul( pow( 2, logarithmicDepth.mul( logRatio ).sub( 1 ) ) );
	return perspectiveW;

};

Copy link
Contributor Author

@PoseidonEnergy PoseidonEnergy Nov 5, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Update 2024-11-13 | This post and info on the logarithmic depth buffer calculation is outdated. Please see the new revisions to the logarithmic depth functions in the pull request here: #29870

####################################################################
####################################################################
####################################################################

Perhaps your mvp.z equals (or is very close to) mvp.w? You may be returning 1 after dividing z by w.

Here is a fiddle where mvp.z is extremely close to mvp.w (even at a camera distance of 9,000): https://jsfiddle.net/h7sd13pj

If that is the case, then try returning just the inverse:

return float( 1 ).div( logarithmicDepthToPerspectiveDepth( depth, this.cameraNear, this.cameraFar ) );

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you tell me how I can use GTAONode on my own to help you work on this? Maybe some little example or screenshot of what works when logarithmicDepthBuffer is false compared to when it is true?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is an example for demonstrating the AO: https://threejs.org/examples/webgpu_postprocessing_ao

I suggest you test this locally by turning on logarithmic depth buffer in the demo. You will see the AO breaks.

For testing I have added logarithmicDepthToPerspectiveW() to ViewportDepthNode and then tried to use it in GTAONode (with no success so far).


// The final logarithmic depth formula used here is adapted from one described in an article
// by Thatcher Ulrich (see http://tulrich.com/geekstuff/log_depth_buffer.txt), which was an
// improvement upon an earlier formula one described in an
// Outerra article (https://outerra.blogspot.com/2009/08/logarithmic-z-buffer.html).
// The Outerra article ignored the camera near plane (it always assumed it was 0) and instead
// opted for a "C-constant" for resolution adjustment of objects near the camera.
// Outerra states this about their own formula: "Notice that the 'C' variant doesn’t use a near
// plane distance, it has it set at 0." (quote from https://outerra.blogspot.com/2012/11/maximizing-depth-buffer-range-and.html)
// It was debated here whether Outerra's "C-constant" version or Ulrich's "near plane" version should
// be used, and ultimately Ulrich's "near plane" version was chosen for simplicity, since no "C-constant"
// needs to be worried about.
// Outerra eventually made another improvement to their original "C-constant" formula, but it still
// does not incorporate the camera near plane (for this version,
// see https://outerra.blogspot.com/2013/07/logarithmic-depth-buffer-optimizations.html).
near = near.max( 1e-6 ); // <-- clamp so we don't divide by 0
const numerator = log2( perspectiveW.div( near ) );
const denominator = log2( far.div( near ) );
// The only modification we make to Ulrich's formula is
// adding 1 to the final depth value and dividing by 2.
return numerator.div( denominator ).add( 1 ).div( 2 );

};

const depthBase = /*@__PURE__*/ nodeProxy( ViewportDepthNode, ViewportDepthNode.DEPTH_BASE );

export const depth = /*@__PURE__*/ nodeImmutable( ViewportDepthNode, ViewportDepthNode.DEPTH );
Expand Down
31 changes: 26 additions & 5 deletions src/nodes/lighting/AnalyticLightNode.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@ import { Loop } from '../utils/LoopNode.js';
import { screenCoordinate } from '../display/ScreenNode.js';
import { HalfFloatType, LessCompare, RGFormat, VSMShadowMap, WebGPUCoordinateSystem } from '../../constants.js';
import { renderGroup } from '../core/UniformGroupNode.js';
import { perspectiveDepthToLogarithmicDepth } from '../display/ViewportDepthNode.js';
import { hash } from '../core/NodeUtils.js';

const BasicShadowMap = Fn( ( { depthTexture, shadowCoord } ) => {
Expand Down Expand Up @@ -314,20 +315,40 @@ class AnalyticLightNode extends LightingNode {
const position = object.material.shadowPositionNode || positionWorld;

let shadowCoord = uniform( shadow.matrix ).setGroup( renderGroup ).mul( position.add( normalWorld.mul( normalBias ) ) );
shadowCoord = shadowCoord.xyz.div( shadowCoord.w );

let coordZ = shadowCoord.z.add( bias );
let coordZ;

if ( renderer.coordinateSystem === WebGPUCoordinateSystem ) {
if ( shadow.camera.isOrthographicCamera || renderer.logarithmicDepthBuffer !== true ) {

coordZ = coordZ.mul( 2 ).sub( 1 ); // WebGPU: Convertion [ 0, 1 ] to [ - 1, 1 ]
shadowCoord = shadowCoord.xyz.div( shadowCoord.w );

coordZ = shadowCoord.z;

if ( renderer.coordinateSystem === WebGPUCoordinateSystem ) {

coordZ = coordZ.mul( 2 ).sub( 1 ); // WebGPU: Conversion [ 0, 1 ] to [ - 1, 1 ]

}

} else {

const w = shadowCoord.w;
shadowCoord = shadowCoord.xy.div( w ); // <-- Only divide X/Y coords since we don't need Z

// The normally available "cameraNear" and "cameraFar" nodes cannot be used here because they do not get
// updated to use the shadow camera. So, we have to declare our own "local" ones here.
// TODO: Can we fix cameraNear/cameraFar in src/nodes/accessors/Camera.js so we don't have to declare local ones here?
const cameraNearLocal = uniform( 'float' ).onRenderUpdate( () => shadow.camera.near );
const cameraFarLocal = uniform( 'float' ).onRenderUpdate( () => shadow.camera.far );

coordZ = perspectiveDepthToLogarithmicDepth( w, cameraNearLocal, cameraFarLocal );

}

shadowCoord = vec3(
shadowCoord.x,
shadowCoord.y.oneMinus(), // follow webgpu standards
coordZ
coordZ.add( bias )
);

const frustumTest = shadowCoord.x.greaterThanEqual( 0 )
Expand Down