Doc : Access to instance positions in NodeMaterial

[Makio64]

Description

I coudnt find the right way to access or use instance properties in TSL from the doc ( Three.js-Shading-Language ) or examples.

Would be nice to get the explanation on how to access to the ID and instanceMatrix / instanceColorMatrix and i would be happy to create simple examples and update the documentation.

Solution

• Update the doc for TSL including accessors for instance / batch
• Add minimal examples : The color of the material change depending of the x y z axis of the instance

Alternatives

Another minimal example : the scale of the instance change depending of the luminosity of the instanceColorMatrix

Additional context

No response

[cmhhelgeson]

Hi @Makio64,

To access the instance of a mesh within a TSL function, you need to use the instanceIndex node. In most current examples, instanceIndex is used within compute shaders to represent the ID of the current compute thread. However, the value of instanceIndex is context-dependent.

• In a compute shader, instanceIndex represents a thread ID.
• However, In a vertex or fragment shader, instanceIndex represents the instance ID that the current vertex belongs to.

Here's a quick example I wrote up demonstrating how to use instanceIndex within a TSL vertex shader.

const geometry = new THREE.BoxGeometry(0.1, 0.1, 0.1);
const texture = new THREE.TextureLoader().load( 'textures/crate.gif' );
texture.colorSpace = THREE.SRGBColorSpace;
const material = new THREE.MeshBasicNodeMaterial( {map: texture});

const uCircleRadius = uniform(1.0);
const uCircleSpeed = uniform( 0.5 );
const instanceCount = 80;
const numCircles = 4;
const meshesPerCircle = instanceCount / numCircles;

material.positionNode = Fn(() => {
        // Multiply elapsed time by speed
	const time = timerLocal().mul( uCircleSpeed );
                
        // Each cube is 1 of 20 within a concentric circle. Get index of cube within a circle
	const instanceWithinCircle = instanceIndex.remainder(meshesPerCircle );

        // Get index of the circle itself
	const circleIndex = instanceIndex.div( meshesPerCircle ).add(1);

       // Radius of each circle increases
	const circleRadius = uCircleRadius.mul(circleIndex);

	// Normalize instanceIndex to range [0, 2*PI]
	const angle = float(instanceWithinCircle).div(meshesPerCirlce).mul(PI2).add( time ); 
	const circleX = sin( angle ).mul( circleRadius );
	const circleY = cos( angle ).mul( circleRadius );

	const scalePosition = positionGeometry.mul( circleIndex );
		
	const rotatePosition = rotate(scalePosition, vec3( timerLocal() , timerLocal(), 0.0));

	const newPosition = rotatePosition.add( vec3( circleX, circleY, 0.0 ));
	return vec4( newPosition, 1.0 );
})(); 

mesh = new THREE.InstancedMesh( geometry, material, instanceCount );
scene.add( mesh );

renderer = new THREE.WebGPURenderer({ antialias: false })
renderer.setPixelRatio( window.devicePixelRatio );
renderer.setSize( window.innerWidth, window.innerHeight );
renderer.setAnimationLoop( animate );
document.body.appendChild( renderer.domElement );
//
	
const controls = new OrbitControls( camera, renderer.domElement );
controls.minDistance = 1;
controls.maxDistance = 20;

const gui = new GUI();
gui.add( uCircleRadius, 'value', 0.1, 3.0, 0.1 ).name( 'Circle Radius' );
gui.add( uCircleSpeed, 'value', 0.1, 3.0, 0.1 ).name( 'Circle Speed' );

And the output:

Vertex.Shader.InstanceIndex.mp4

For modifying the color of an instance, you can simply apply 'instanceColor' as an attribute to your geometry, then access that attribute within your material's fragmentNode or colorNode. This is what I currently do, though I'm not sure if there's a built-in TSL solution for instanceColors.

const instanceColorArray = new Float32Array( instanceCount * 4 );
for ( let i = 0; i < instanceColorArray.length; i++ ) {

	instanceColorArray[i * 4 + 0] = Math.random();
	instanceColorArray[i * 4 + 1] = Math.random();
	instanceColorArray[i * 4 + 2] = Math.random();
	instanceColorArray[i * 4 + 3] = Math.random();

}

geometry.setAttribute( 'instanceColor', new THREE.InstancedBufferAttribute( instanceColorArray, 4 ) );
material.fragmentNode = attribute('instanceColor');

three.js.TSL.Tutorial.Part.1.-.Google.Chrome.2024-08-06.14-53-24.mp4

With this method, you also have the added flexibility of modifying your instanceColor within a compute shader. All you need to do is change your InstancedBufferAttribute to a StorageInstancedBufferAttribute, thus making it a datatype that is accessible as a storage buffer within a compute shader.

let computeColor;

const instanceColorArray = new Float32Array( instanceCount * 4 );
for ( let i = 0; i < instanceColorArray.length; i++ ) {

	instanceColorArray[i * 4 + 0] = Math.random();
	instanceColorArray[i * 4 + 1] = Math.random();
	instanceColorArray[i * 4 + 2] = Math.random();
	instanceColorArray[i * 4 + 3] = Math.random();

}

// Make attribute accessible as a storage buffer within a compute shader.
const instanceColorAttribute = new THREE.StorageInstancedBufferAttribute( instanceColorArray, 4 );
geometry.setAttribute( 'instanceColor', instanceColorAttribute );

material.fragmentNode = attribute('instanceColor');

computeColor = Fn(() => {

	const instanceColor = storage( instanceColorAttribute, 'vec4', instanceCount );

	const r = sin( timerLocal().add(instanceIndex) );
	const g = cos( timerLocal().add(instanceIndex));
	const b = sin( timerLocal() );

	instanceColor.element( instanceIndex ).assign(vec4(r, g, b, 1.0));

})().compute( instanceCount );

function render() {
     renderer.render( scene, camera)
     renderer.compute( computeColor );
}

three.js.TSL.Tutorial.Part.1.-.Google.Chrome.2024-08-06.15-20-07.mp4

[Makio64]

Thanks @cmhhelgeson, nice examples, very instructive !

In my case I still need to access to the : instanceMatrix to know the position of the instance in the worldSpace ( basically to apply position modifier on the vertex depending of the position of the vertex in worldspace )

I tried to access it using {instance} from 'three/tsl' but there is a logic I cant figure out.

The easy solution would be to do a InstancedBufferAttribute or StorageInstancedBufferAttribute with the data I need ( position of the instance ) but as we already have an instanceMatrix system so I would like to understand how to access and use it.

[cmhhelgeson]

Thanks @cmhhelgeson, nice examples, very instructive !

In my case I still need to access to the : instanceMatrix to know the position of the instance in the worldSpace ( basically to apply position modifier on the vertex depending of the position of the vertex in worldspace )

I tried to access it using {instance} from 'three/tsl' but there is a logic I cant figure out.

The easy solution would be to do a InstancedBufferAttribute or StorageInstancedBufferAttribute with the data I need ( position of the instance ) but as we already have an instanceMatrix system so I would like to understand how to access and use it.

I myself wasn't even aware there was an instance node 😅.

I'm not quite sure what the requirements of your application are, but I think a good place to start would be using the positionWorld node (I can get back on whether the modelWorldMatrix used in positionWorld updates to account for instancing). As you mentioned, you could also just replace the current InstanceMatrix InstancedBufferAttribute with a StorageInstancedBufferAttribute, thus making it accessible as a storage node within TSL shaders. Look at webgpu_compute_geometry.html to see how StorageBufferAttributes can easily replace existing buffer attributes.

[Makio64]

@z4122 @sunag do you mind sharing an example of how to use the InstanceNode / access to the instance matrix from the tsl ? 🥺

Thanks !

[sunag]

Currently InstanceNode does not have feature to just export instanceMatrixNode property, maybe you will have to open the source code and use the separate code part for now.

import { buffer, instanceIndex } from 'three/tsl';

// if ( instanceMesh.count <= 1000 )
const node = buffer( instanceMesh.instanceMatrix.array, 'mat4', instanceMesh.count ).element( instanceIndex );

three.js/src/nodes/accessors/InstanceNode.js

Lines 41 to 73 in 410f737

	if ( instanceMatrixNode === null ) {
	const instanceAttribute = instanceMesh.instanceMatrix;
	// Both WebGPU and WebGL backends have UBO max limited to 64kb. Matrix count number bigger than 1000 ( 16 * 4 * 1000 = 64kb ) will fallback to attribute.
	if ( instanceMesh.count <= 1000 ) {
	instanceMatrixNode = buffer( instanceAttribute.array, 'mat4', instanceMesh.count ).element( instanceIndex );
	} else {
	const buffer = new InstancedInterleavedBuffer( instanceAttribute.array, 16, 1 );
	this.buffer = buffer;
	const bufferFn = instanceAttribute.usage === DynamicDrawUsage ? instancedDynamicBufferAttribute : instancedBufferAttribute;
	const instanceBuffers = [
	// F.Signature -> bufferAttribute( array, type, stride, offset )
	bufferFn( buffer, 'vec4', 16, 0 ),
	bufferFn( buffer, 'vec4', 16, 4 ),
	bufferFn( buffer, 'vec4', 16, 8 ),
	bufferFn( buffer, 'vec4', 16, 12 )
	];
	instanceMatrixNode = mat4( ...instanceBuffers );
	}
	this.instanceMatrixNode = instanceMatrixNode;
	}

[Mugen87]

Documentation related issues in context of TSL can be merged into #29829.