WebGLRenderer.customCullCallback and Soft Particles Example

[tsone]

Two-line change in WebGLRenderer to support custom callback function to cull objects in the scene graph. This culling does not replace other culling mechanisms (frustum culling, visibility property), but it performs culling before these. The intent is to eliminate the need to perform unnecessary scene traversals or tweaking of object or material visibility to achieve same effect. With it one can easily perform custom culling for example in complex scenes with the aid of some accelerating structure.

The accompanying Soft Particles [1] [2] example demonstrates the use by culling all transparent objects when the depth texture is rendered.

[tsone]

Forgot to mention that the idea was discussed in the last comments of #5900.

[mrdoob]

This is nice! Maybe you can give a go at the layers approach though? See #5562.

[tsone]

I read that discussion on culling mask/layer and I think that can be easily done using this cull callback. For example right now, one could use Object3D.userData property to store layerBits that tell which layers the object belongs to. Here's how the data would be laid out in that case:

// Set object to belong to layer 0
object.userData.layerBits = 0x01;

// Set object to belong to both layers 0 and 1
object.userData.layerBits = 0x03;

Then the cull callback could check this flag against a mask. Here is how:

function layerMaskCull( layerMask ) {

    return function ( object ) {
        return object.userData.layerBits && (( object.userData.layerBits & layerMask ) === 0 );
    };

};

// Render objects in layer 0
renderer.customCullCallback = layerMaskCull( 0x01 );
renderer.render( scene, camera );

// Render objects in both layer 0 and 1
renderer.customCullCallback = layerMaskCull( 0x03 );
renderer.render( scene, camera );

I think that does what the discussion is about. If bitwise operations are confusing to people, this functionality could be wrapped with a more intuitive API. Also to mention, this does not affect object.visible (or other visibility flags) or frustum culling, but is done before those.

[mrdoob]

Yes I know, but that's not very pretty API-wise.
I think the API proposed in the other thread is prettied and more user friendly.

[tsone]

I'll modify this soft particles example to use culling layers instead.