scene graph based light probe

[richardmonette]

related: #16223

Per #16223 (comment)

I really like @mrdoob's recommendation is that LightProbe is derived from Object3D.

I've changed LightProbe to be Object3D based.

Per #16199 (comment)

Thus I do want baby steps that may not be the perfect solution. This is one of them.

Just change the interpolation to per mesh rather than per vertex for now.

I do agree with that. Per-mesh interpolation, even with a naive interpolation method, should be a great basis to build from.

I've changed from using the lighting system directly to a baby step towards making the interpolated light probe per object.

We will still need to do:

• A better interpolation scheme
• A proper dirty state tracking system which determine when an object needs it's interpolated SH information updated (e.g. either the object or one or more light probes has moved in the previous frame)

This gets us a tiny bit closer to our goal however, as it will at least in some basic way take into account the relative position of the light probe(s) and objects.

@WestLangley @donmccurdy @bhouston

[bhouston]

I believe this is conceptually in the right direction.

[richardmonette]

@WestLangley If you are OK with this, then I will:

1. Resolve conflicts
2. Update the example so the sphere is moving. Even with the super basic hard-cutover nearest neighbour "interpolation", we should be able to see the SH contribution changing as an object moves relative to the light probes.

[donmccurdy]

Maybe this should be interpolatedDiffuseSH rather than a LightProbe instance? 🤔

Once we're actually doing interpolation between probes, we won't need a LightProbe to represent the interpolated result of probes in the scene. For example:

// Update .interpolatedDiffuseSH (instance of SphericalHarmonics3)
computeInterpolatedSH( object, scene, object.interpolatedDiffuseSH );

[donmccurdy]

One suggestion inline, but I agree it's OK to merge this without a more complex interpolation scheme. Per #16223 (comment) I'm willing to work on interpolation, but @richardmonette if you were planning to do that I won't complain. 😇

[WestLangley]

@donmccurdy Do you think the renderer should be interpolating the scene light probes?

To me, that is an app-level responsibility, which can be applied in mesh.onBeforeRender() based on the mesh's position, and at the developers desired update frequency -- not necessarily every frame.

Plus, there are other issues we haven't discussed. One is, how do you set the intensity of the interpolated probe knowing it must be proportional to the intensity of the mesh's environment map? It is the developer's responsibility to do that in a reasonable way.

[WestLangley]

@richardmonette I mean this with all due respect, but I would prefer if you would allow me to respond to feedback from the PRs I file, rather than you doing it. This is leading to file conflicts, and confusion. I hope you understand.

I am, of course, open to your suggestions.

[donmccurdy]

Do you think the renderer should be interpolating the scene light probes?

On one hand – robust and efficient interpolation methods exist, and there's no point in the community maintaining multiple alternative implementations. The proposed method is likely to be a good one for all scenes with multiple probes, and I think it should be provided by three.js – whether as part of the renderer or by some other means.

That said, I agree there are downsides to interpolating in the renderer... In addition to the update frequency and intensity issues you mentioned:

• Managing the light probe volume is complex enough to very easily add 500-1000 LOC to the library.
• If the renderer applies diffuse SH automatically, we'll need new material properties to opt out for static meshes. That could be avoided by just letting the application set mesh.material.lightProbe, or not.

As discussed in another thread, I think an .update( mesh ) pattern may be a good choice:

import { LightProbeManager } from 'three'; // or three/examples/js/...?

var probeManager = new LightProbeManager();
probeManager.setProbes( [ probe1, ..., probeN ] );

mesh.onBeforeRender = function () {

  // Update mesh1.material.lightProbe (or .diffuseSH?)
  probeManager.update( mesh );

}

function animate () {

  renderer.render( scene, camera );

}

I wish that didn't require onBeforeRender. 😕

[donmccurdy]

One is, how do you set the intensity of the interpolated probe knowing it must be proportional to the intensity of the mesh's environment map?

Suddenly I like the idea of using mesh.material.lightProbeIntensity or sh.multiplyScalar(0.5) sh.scale(0.5) and removing lightProbe.intensity 😓

Maybe I don't fully understand your point - the mesh may not have an environment map at all, and I'm not sure why interpolation should need to adjust probe intensity when there is one.

[WestLangley]

I think it should be provided by three.js – whether as part of the renderer or by some other means.

Not as part of the renderer, in my opinion. But definitely as part of the examples.

[WestLangley]

But I'm not sure I fully understand your point - the mesh may not have an environment map at all, and I'm not sure why interpolation should need to adjust probe intensity when there is one.

If the environment map exists, and the mesh environment map intensity is reduced, then for consistency, the light probe intensity (used in the shader) should reduce, too... Well, at least I would hope so, but I guess one would not have to enforce that.

[donmccurdy]

I see, thanks. Perhaps envMapIntensity should just always apply to the light probes' interpolated result (no need for material.lightProbeIntensity) and at some point perhaps we rename it more appropriately.

[bhouston]

This need for matching intensities is temporary. Once both sh and cubemaps are generated from the level data itself hey will just be correct. Also if you have a singular hrdi map both the cubemaps and sh is generated from that single hdri image and again they should have correct intensities. This need for intensity matching is just a temporary thing until we implement everything properly.

…

On Tue, Apr 16, 2019, 9:49 PM Don McCurdy ***@***.***> wrote: One is, how do you set the intensity of the interpolated probe knowing it must be proportional to the intensity of the mesh's environment map? Suddenly I like the idea of using mesh.material.lightProbeIntensity and removing lightProbe.intensity 😓 But I'm not sure I fully understand your point - the mesh may not have an environment map at all, and I'm not sure why interpolation should need to adjust probe intensity when there is one. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#16270 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAj6_YzBP7WTjpT96oyMrhGh5fFxRSQ9ks5vhn2NgaJpZM4czeUQ> .

[richardmonette]

#16228 (comment) looks like a better implementation

[mrdoob]

@richardmonette thanks for giving a push to all this 🙌