TSL: Using a custom vertexNode is breaking the normalLocal of a BatchedMesh

[Robpayot]

Description

I'm using TSL with a MeshStandardNodeMaterial on a BatchedMesh. I have a custom vertexNode reusing the logic of BatchNode so I can apply custom bending effect on it later on.

However when doing that the lights of my BatchedMesh are broken (not in the correct direction). When I remove the vertexNode (using the default shader) it fixes it.

Does it mean the vertexNode is overwriting the normalLocal so the fragment doesn't have the correct vNormals anymore? but weirdly I have the normalLocal assigned.

Does it also mean if I want to customise the fragment part I will have to rewrite the whole StandardNodeMaterial fragment shader in the outputNode to get the lights right?

Thank you

Reproduction steps

1. Using a MeshStandardNodeMaterial on a BatchedMesh
2. Apply a custom vertexNode
3. Lights are breaking

Code

const material = new MeshStandardNodeMaterial()
const mesh = new BatchedMesh(50, 4000, 4000, material)
const geometry1 = new BoxGeometry(1.5, 1, .5)
const geometry2 = new ConeGeometry(1, 1, 32)
mesh.addGeometry(geometry1, geometry2)

material.vertexNode = Fn((builder) => {
	// Reusing BatchNode logic
	let batchingIdNode = null

	// WebGL fallback if
	if (batchingIdNode === null) {
		// check if https://github.com/mrdoob/three.js/blob/841ea631018e0bf40c7de1b54811101f77f1e1b3/src/renderers/webgl-fallback/nodes/GLSLNodeBuilder.js#L626
		if (builder.getDrawIndex() === null) {
			batchingIdNode = instanceIndex
		} else {
			batchingIdNode = drawIndex
		}
	}

	const getIndirectIndex = Fn(([id]) => {
		const size = textureSize(textureLoad(mesh._indirectTexture), 0)
		const x = int(id).modInt(int(size))
		const y = int(id).div(int(size))
		return textureLoad(mesh._indirectTexture, ivec2(x, y)).x

	}).setLayout({
		name: 'getIndirectIndex',
		type: 'uint',
		inputs: [{ name: 'id', type: 'int' }]
	})

	const matriceTexture = mesh._matricesTexture

	const size = textureSize(textureLoad(matriceTexture), 0)
	const j = float(getIndirectIndex(int(batchingIdNode))).mul(4).toVar()

	const x = int(j.mod(size))
	const y = int(j).div(int(size))
	const batchingMatrix = mat4(
		textureLoad(matriceTexture, ivec2(x, y)),
		textureLoad(matriceTexture, ivec2(x.add(1), y)),
		textureLoad(matriceTexture, ivec2(x.add(2), y)),
		textureLoad(matriceTexture, ivec2(x.add(3), y))
	)

	const bm = mat3(batchingMatrix)
	const batchPos = batchingMatrix.mul(positionLocal).xyz

	// Now I can apply custom code here
	const transformed = modelViewMatrix.mul(batchPos)
	const mvPosition = vec4(transformed.xyz, transformed.w)

	// Normals
	const transformedNormal = normalLocal.div(vec3(bm[ 0 ].dot(bm[ 0 ]), bm[ 1 ].dot(bm[ 1 ]), bm[ 2 ].dot(bm[ 2 ])))
	const batchingNormal = bm.mul(transformedNormal).xyz
	normalLocal.assign(batchingNormal)

	if (builder.hasGeometryAttribute('tangent')) {
		tangentLocal.mulAssign(bm)
	}

	return cameraProjectionMatrix.mul(mvPosition)
})()

// Positions are good but lights are now broken

Version

0.168.0

[Robpayot]

Sorry I reopened it because I found a fix for InstancedMesh using normalLocal.assign() but not for BatchedMesh it still doesn't work

[Robpayot]

I added a jsfiddle example here, I hope this can help:
https://jsfiddle.net/dfw7hroL/

You can see here the lighting is broken when I use my custom vertexNode (I used a directionnal light), if I comment this line : material.vertexNode = customVertexNode the light is fixed

[Robpayot]

I found a fix using const varNormalLocal = varying(vec3(0)), then I assign the normalLocal to it in the vertexNode, and just apply it to the material.normalNode = transformNormalToView(varNormalLocal). It seems to work now, here is an example:

https://jsfiddle.net/jh4wvyd7/1/

[Makio64]

When adding a point light to the scene, the lights seems totally broken again : https://jsfiddle.net/oc0gkyqa/8/

[RenaudRohlinger]

@Makio64 To handle lights you also need to update the positionView accodingly: positionView.assign(transformed).

https://jsfiddle.net/0Lc2jbg6/10/

[Makio64]

@Makio64 To handle lights you also need to update the positionView accodingly: positionView.assign(transformed).

https://jsfiddle.net/0Lc2jbg6/10/

@RenaudRohlinger Merci! This was bugging me so hard! 🙇‍♂️