Image-Based Lighting

[bagnell]

Roadmap for implementing IBL:

• Decide on format: RGBE, LogLUV, RGBM
• Decide on format container: KTX2, KTX, DDS
• Collaborate with glTF community on storage, e.g., EXT_lights_image_based extension KhronosGroup/glTF#1377
• Load diffuse and specular environment maps plus convoluted mip maps
• Prototype with WebGL2 (requires textureLod in fragment shader)
• Pack specular environment map and convolutions into a texture atlas using octahedral projection.
• Use spherical harmonics for the diffuse irradiance map
• Update default procedural IBL

[bagnell]

For RGBM and octahedral projection, see: http://webglinsights.github.io/downloads/WebGL-Insights-Chapter-16.pdf

For spherical harmonics, see: http://graphics.stanford.edu/papers/envmap/envmap.pdf

[bagnell]

For creating the environment maps: https://github.com/derkreature/IBLBaker
Or: https://github.com/dariomanesku/cmftStudio

[OmarShehata]

This is a nice explanation of actually computing the coefficients for spherical harmonics: https://computergraphics.stackexchange.com/a/5001

And an example from Babylon of getting these coefficients from a cubemap: https://github.com/BabylonJS/Babylon.js/blob/master/src/Tools/HDR/babylon.cubemapToSphericalPolynomial.ts#L94

[TimvanScherpenzeel]

Filament has a tool called cmgen that is able to create environment maps and generate the precomputed spherical harmonics. It recently also added the ability for octahedral projected maps.

I've also been looking into Bash wrapping this tool in order to output all the necessary assets for IBL here.

[OmarShehata]

Oh, this is great @TimvanScherpenzeel ! Thanks for pointing that out. It'll be pretty useful to compare my implementation to theirs. I know I had some issues with artifacts from linear sampling and had to add a 2 pixel border to get around that, so I'm curious if they had to do the same thing.

Additionally, this might be useful to mention in the glTF lights extension (KhronosGroup/glTF#1377). It's still a draft, and you can see they have this note in the spherical harmonics section:

*(this section needs more about how the coefficients are used as well as a brief blurb about what SH are)*
*(also, we should probably just put some shader code in here for using the coefficients)*

So that might be a nice resource to mention there.

[OmarShehata]

Sample model we can use when implementing the glTF loading part:

KhronosGroup/glTF-Sample-Models#201

[OmarShehata]

This was implemented in #7172.

The only remaining thing is using the glTF extension for defining the IBL parameters, which has an issue here: #7252