GLTFLoader: Support KHR_materials_transmission physically-based transparency

[Doerge]

Describe the bug

The extension KHR_materials_transmission does not semi transparent materials from a .glb.

To Reproduce

Steps to reproduce the behavior:

1. Get the reference glb from here https://github.com/KhronosGroup/glTF-Sample-Models/tree/master/2.0/TransmissionTest
2. Load it with threejs gltf loader.
3. The parts of the geometry that are supposed to be semi-transparent are instead 100% opaque. Only 100% transparency is respected. See screenshots below.

Live example

https://jsfiddle.net/jt6ywhs5/12/

Expected behavior

Semi-transparency and transmission works.

Screenshots

Reference:

Threejs:

Platform:

• Device: Desktop
• OS: MacOS
• Browser: Chrome 87.0.4280.88 (Official Build) (x86_64)
• Three.js version: latest stable from https://threejs.org/build/three.module.js

[donmccurdy]

three.js only implements a small part of the KHR_materials_transmission extension today, which is why we haven't listed the extension in the docs yet. There are more transmission features left to implement. See #16977 for a bit more about this. Also note that using an environment map is critical for physically-based reflections and transparency.

[Doerge]

Ah I see. I was a bit too fast then. I landed on the source code through Google, and thought it was already there. Sorry!
Yeah, I didn't bother with an env map, since it was clear the semi transparency wasn't working. Was hoping it was just a setting but alas! Feel free to close this issue if it's already tracked elsewhere.

[donmccurdy]

Let's keep this open — #16977 is a pretty broad issue so I think it's good to have something specifically tracking KHR_materials_transmission.

[yaroslavnikiforov]

Is it possible to implement this in three.js using MeshPhysicalMaterial without extension? (like Rough Transmission yellow ball on the reference image)

[donmccurdy]

No, some changes to MeshPhysicalMaterial would need to be made to support these features.

[yaroslavnikiforov]

Hi, what is the status of implementing this, is it work in progress? Will it be included in the next release? Thanks.

[donmccurdy]

Sorry, this is not being worked on right now. If anyone is interested in working on it please let us know. :)

[patricpalmefjord]

Thank you for the update, I was hoping this enhancement would roll out during March. It would be really useful to have.

[rawwerks]

i would be happy to subsidize the work if someone has the skills and bandwidth to get KHR_materials_transmission fully implemented in three.js . please reach out if you are interested and available in the coming weeks.

[Mandoliiin]

Hi, I'm in real need of this enhancement. Any update when on we can expect this bug to be fixed? Otherwise, is there some workaround I can use to achieve similar effect in the meanwhile? I cannot find anything. Thanks! 🙂

[rawwerks]

if you can think of anyone who can get this done as a contractor, please send them here: https://www.upwork.com/job/Seeking-talented-javascript-developer-enhance-three_~0153b7a387f184d444/

my intention is to prioritize folks who have already made meaningful contributions to three.js - but at this point i really need to find someone who can knock this out.

[whatisor]

The first issue of current transmission implementation is not yet auto-enable Transparent which makes Alpha meaningful.

#ifdef TRANSMISSION diffuseColor.a *= mix( saturate( 1. - totalTransmission + linearToRelativeLuminance( reflectedLight.directSpecular + reflectedLight.indirectSpecular ) ), 1.0, metalness); #endif

However, currently, other issues I found are:

• roughness does not work as expected.
• Alpha map does not work with Metallic
  

[whatisor]

We decided to follow GLTF Viewer Sample to implement this. There is some point which needs to be considered:

• Opaque object is considered as an environment/background of others with transmission. Should it be a forced rule in threejs core or just a flexible option of application where the application can choose background/foreground object?
  @mrdoob

Source

[whatisor]

Improvement points:

• Refraction with thickness
• Dynamic background ( adobe is crazy cool with this stuff for supporting by lazy rendering)
  
  Adobe

[mrdoob]

@whatisor I think for now we should just render the opaque meshes into a renderTarget: #16977 (comment)

[whatisor]

Yes, agreed. @mrdoob
However, I am also considering how it can become a PR with these modifications and next improvements...

[rawwerks]

There's a great sneak peek and explanation of the volume refraction here: https://youtu.be/JMiXv2PgGBY?t=2552

[whatisor]

Hi, is there some source code available to test? Thanks.

@yaroslavkhudchenko
Forked here : https://github.com/3dgraphicsplus/three.js/tree/KHR_materials_transmission
just build and run threejs example ( gltf extension)

[takahirox]

Certainly I'd be excited to see both supported in THREE.MeshPhysicalMaterial. :)

I'm thinking of adding the following properties corresponding to KHR_materials_volume parameters

• thickness
• thicknessMap
• attenuationDistance
• attenuationColor

Note: MeshPhysicalMaterial already has transmission and transmissionMap corresponding to KHR_materials_transmission parameters and ior corresponding to KHR_materials_ior parameter.

[takahirox]

Made a PR #21884

[yaroslavnikiforov]

Hi guys! Will it work out of the box like with Babylon, or it will need some additional setup like in the example? With PMREM env map, that toggleOpaque function and calling render twice in an animation loop. Doesn't look like it would be possible to use it, because of 9-10 fps like in the example.

[donmccurdy]

@yaroslavkhudchenko you can find the three.js example here, the setup is pretty normal:

https://github.com/mrdoob/three.js/blob/dev/examples/webgl_loader_gltf_transmission.html

Physically-based transmission will be more expensive than alpha blending. But your exact framerate will depend on many things. For example, I would expect that using transmission on a smaller area of the viewport will be cheaper than using it on a larger area of the viewport. The best thing to do here would be to test and report issues when this is available.

[donmccurdy]

Fixed in #21884, many thanks @takahirox, @whatisor, @rawwerks and anyone else contributing here. 👏

[takahirox]

So, the transmission support quality got better with the several PRs.

I'm thinking of adding KHR_materials_volume and KHR_materials_ior extensions support to GLTFLoader next.

I would like to know your opinion about the new example.

1. Model

I'm thinking of using TransmissionRoughnessTest model from glTF-Sample-Models repo because it has transmission, volume, and ior extensions

https://github.com/KhronosGroup/glTF-Sample-Models/tree/master/2.0/TransmissionRoughnessTest

Let me know if you know better models.

2. Update the existing example or new Example

I think we have three options.

a. Replacing the model of webgl_loader_gltf_transmission example.

b. Add the new model to webgl_loader_gltf_transmission and allow users to switch models like webgl_loader_gltf_extensions

c. Adding a new example webgl_loader_gltf_volume

I don't have strong opinion. Any preferences?

[donmccurdy]

A few more samples:

• https://github.com/KhronosGroup/glTF-Sample-Models/tree/master/2.0/AttenuationTest
• https://github.com/KhronosGroup/glTF-Sample-Models/tree/master/2.0/DragonAttenuation

/cc @emackey I don't suppose there is a version of https://sketchfab.com/3d-models/real-time-refraction-demo-mosquito-in-amber-37233d6ed84844fea1ebe88069ea58d1 with these extensions added?

No preference on whether, or where, to add a new example.

[takahirox]

• https://github.com/KhronosGroup/glTF-Sample-Models/tree/master/2.0/AttenuationTest
• https://github.com/KhronosGroup/glTF-Sample-Models/tree/master/2.0/DragonAttenuation

I knew those two models but unfortunately they don't use KHR_materials_ior extension although the dragon model looks nicer than TransmissionRoughnessTest.

[donmccurdy]

I'll gladly add IOR to as many models as you want, if you're working on implementing it. 😍

[mrdoob]

@takahirox

b. Add the new model to webgl_loader_gltf_transmission and allow users to switch models like webgl_loader_gltf_extensions

That sounds like the best option to me 👍

[WestLangley]

As I said elsewhere, the webgl_loader_gltf_transmission example is not useful to us because the parameter settings are not accessible. There is no way to determine if the code is working properly because we don't know what we are looking at.

That example is useful for comparing three.js to other renderers.

I would remove it from the examples. We have another transmission example with a GUI.

[mrdoob]

I think it is useful so people can see that we correctly render gltfs that use transmission though.

[WestLangley]

I think it is useful so people can see that we correctly render gltfs that use transmission though.

Do you plan to add all the examples? https://github.com/cx20/gltf-test#extension-test-models

[emackey]

I knew those two models but unfortunately they don't use KHR_materials_ior extension

This is one of those wrinkles in the glTF spec that I wasn't able to avoid. The IOR extension merely sets the IOR, which defaults to 1.5 if you don't use the IOR extension. The actual switch for enabling or disabling the refractions themselves is controlled by the thicknessFactor in KHR_materials_volume being zero or nonzero (thin-walled or thick-volume material). It may not be intuitive at first glance, but it makes sense when you take the time to deeply understand the glTF materials system design.

/cc @emackey I don't suppose there is a version of [the Sketchfab mosquito with the new glTF extensions]

Yes, I had built that a while ago as a Khronos-internal test. But the source model is CC-BY and I've been meaning to post in on the sample repo with some nice documentation and credits. Someday soon I hope, unless someone else beats me to it. Bonus points if the proper IOR for amber is looked up and added to the model.

[takahirox]

I agree with @mrdoob

I think it is useful so people can see that we correctly render gltfs that use transmission though.

We have one example or model per one glTF extension which GLTFLoader support.

[donmccurdy]

In my mind the three purposes of examples are, from most to least important:

1. Help users learn three.js
2. Showcase three.js features
3. Visual regression tests

The current transmission model mostly does (3), i.e. it's a model designed to test a feature and compare across engines. I don't necessarily think we need one model per glTF extension; ideally we'd cover each extension in unit tests instead. If we have a more visually interesting model for the transmission use case (with or without volume and ior) i think it'd be fine to replace the current one too.

[takahirox]

i think it'd be fine to replace the current one too.

I was thinking the similar thing. We have webgl_materials_physical_transmission example which has the parameter GUI then we can test transmission stuffs on it. Instead we can focus on glTF + transmission showcase in webgl_loader_gltf_transmission and we can replace the testing purpose model used in it with a more looking interesting one (maybe that Dragon one? because it's well known by 3D CG folks). Or we can replace the gltf transmission example with a glTF PBR next example (using ToyCar?) as a showcase of glTF PBR next.

[WestLangley]

webgl_loader_gltf_transmission has the transparent property of each sphere to set to false. Are you aware of that? Is that what the loader should be doing?

Even setting transparent to true, only front-faces show the hot-spot. (This may be a known limitation, but @mrdoob just added support for properly rendering backfaces of double-sided meshes.) There should be two hot-spots.

There is a tone mapping problem, somewhere...

[donmccurdy]

webgl_loader_gltf_transmission has the transparent property of each sphere to set to false. Are you aware of that? Is that what the loader should be doing?

In glTF, a model may specify "transmission+alphaMode=BLEND" or "transmission+alphaMode=OPAQUE". The latter is recommended but not required. The loader is interpreting alphaMode=OPAQUE as transparent=false. It is not a perfect analog, since .transparent controls three things: blend mode, sort order, and two-pass double-sided rendering.

I think you are asking if associated behaviors — other than the blend mode — should apply to transmissive materials, too?

[WestLangley]

@donmccurdy Thank you for your reply.

I was trying to determine why, when looking at the transmissive sphere, I do not see two hot-spots. To try to determine why, I had to debug the loaded glTF model, because the model parameter settings are obfuscated, and there is no other way to tell what I am looking at.

That is all I was focusing on, but I think your higher-level questions are important.

[coderextreme]

Hi, I'm in real need of this enhancement. Any update when on we can expect this bug to be fixed? Otherwise, is there some workaround I can use to achieve similar effect in the meanwhile? I cannot find anything. Thanks! slightly_smiling_face

Something like this? https://playcanv.as/p/wQgQBgkE/

[donmccurdy]

@takahirox this might be a helpful sample and discussion, as well: KhronosGroup/glTF-Sample-Models#319

[takahirox]

@donmccurdy

Thanks! I will make an example with it.