How to handle lights?

[bhouston]

It isn't that clear how I get any lighting information. Do I have to hard code lights into the shaders?

[AndrewRayCode]

There is no default light information passed to shaders, and getting this into three.js is definitely a weaker point of shaderfrog. here's an example of a light shader: http://shaderfrog.com/app/view/15 where the uniform "lightPosition" is passed in as a vec3, using an animation helper to preview it. In the end user's runtime, they'd have to manually pass in the light position. and if they had a spotlight, they'd have to compose in a spotlight shader (which isn't in shaderfrog yet, until someone writes it), and then pass in the spotlight position, color, etc, as uniforms. there's a lot of potential optimization here.

something else to consider is multiple lights in a composed shader. if you add multiple light shaders, you'll get multiple uniforms named "lightPosition". you probably want to control all of those individually in your end application, so the uniforms must be combined in a way that makes them unique. (multiple shaders with the time uniform for example would merge time all into one uniform). I'm not sure how to set this up yet but maybe in the compose interface you could mark some uniforms as "merge" and some as "rename" and some as "merge to array" so your end composed shader would have lightPositions[0], 1, etc.

[bhouston]

If one wants to write shaders that are realistic for real materials, like paper, cloth, fabric, wood, etc, one needs lighting information -- e.g. most things (although not all) you see in an Unreal Engine demo. If you do not have lighting information it means that most shaders can only be used for non-photorealistic rendering styles - which while really neat, are of limited value.

[bhouston]

I wonder if a solution is to standardize things for lights or use defines or something that can be replaced when compiling the shader to something.

[bhouston]

Why not just allow one to use all of ThreeJS's light unfiroms in the format they expose it and then have a few light presets you can pick in the viewer like you can with mesh presets (single point, three point, area light, and later HDR IBL)?

[AndrewRayCode]

That's kind of the route I'm experimenting with now. You may be more familiar with it than I am, but I'm looking through how three does lights internally. It's mostly magical, all possible lights in threejs are compiled by shaderchunks into one huge shader with a bunch of ifdefs to turn on / off light types like.

If you add a PointLight or whatever to a scene, Three automatically sends every light uniform to every running light shader (I think).

I've abstracted out most of the point light code as an example to http://shaderfrog.com/app/view/151 . What I'm thinking is if you want lights, you compose this extracted point light shader into yours before export. the runtime will automatically copy all of three's light variables to it, so you can control it with three.pointlight etc instead of manually updating uniforms.

To get multiple lights: If you want more than one point light, you can compose in a second point light, and it will automatically squash the duplicate uniform names into an array, so now your compiled shader has an array of lights (also how three works). this is possible with the compiler but not fully implemented yet.

this method gives a few benefits:

• you can add in whatever light types you need, and only those. If you need two spotlights there will only be an array of length 2 for things like spotlightPosition.
• you can view how each light is calculated in an entirely separate shader. right now trying to read three's shader source code you have to trace one light through dozens of files and multiple ifdefs
• when adding lights for multiple targets (unity, ios?) instead of composing in a three pointlight shaderfrog shader, maybe there will be a unity pointlight shader. or maybe the export step will simply know how to rename three standardized uniform names.

[bhouston]

I think it is awesome you are tackling this.

I think it may be one of those hard problems that you can get stuck in for a while that isn't that hugely beneficial to the project -- just warning you because sometimes I suggest things whose benefits are clearly not worth the effort. Not sure if that is the case here, but if it takes a long time your effort is likely more valuable elsewhere.

Now with that disclaimer out of the way, I'll share an idea I had while letting you judge its value. One could make a few macros and functions to handle this in a clean way while abstracting out the specific ThreeJS light variables.

Something like this:

FOR_EACH_POINT_LIGHT_BEING // macro or something else that auto-expands
  int lightIndex = GET_LIGHT_INDEX();
  int lightIntensity = getPointLightIntensity( lightIndex );
  vec3 lightColor = getPointLightColor( lightIndex );
  float lightDecayExponent = getPointLightDecayExponent( lightIndex );

  // do lighting effect
FOR_EACH_POINT_LIGHT_END // macro

The reason for the above is that the MACRO will create a for loop, single statement, or nothing at all depending on whether there are actually lights in the scene. And it can read the number of point lights from a macro or from a variable or elsewhere. Thus it is flexible. But maybe it can still be functions. I believe ThreeJS excludes the for loop body completely if there are no lights of a certain type, I guess to to reduce shader complexity.

The reason for the functions is that ThreeJS may change its variable definitions (by changing the packing, or move towards data textures) or other engines may use other variable definitions (maybe decay is an int, rather than a float) for the lights and this abstracts that. Also they would inline anyhow.

[bhouston]

Another cool thing about the macro design is that if a certain light type is not supported by an engine, you can just undefined the whole shader block that was about supporting that light type -- for example area lights are very inconsistently supported across engines. I have a PR for that in ThreeJS but it was never merged: mrdoob/three.js#5792

[AndrewRayCode]

For now i'm trying to stay away from macros / inventing any of my own language definitions. The closest thing I have planned is snippet completion in the editor. I don't think most users would manually want to figure out the code they need (even if it is a macro) to add lights to their own shader. The compiler can (hopefully) handle adding all the lights to a shader automatically, through some ui abstraction

[AndrewRayCode]

An even easier option would be to just have a "include threejs lighting information" checkbox on the export form, that automatically composes in three light shaders on export of any shader.

[AndrewRayCode]

You're not the only person to ask about lights, and it makes sense in the context of a 3d scene, so I think it's pretty important.

The groundwork is in place for lights now. http://shaderfrog.com/app/view/151 is reading from an invisible THREE.PointLight light in the scene. you can see for example it has pointLightPosition and pointLightColor but that's not editable on the uniform list. It's coming directly from three's lighting engine.

What this means is that if you compose in this shader to your scene and include the runtime, you can add a three pointlight to your scene and your custom shader will be automatically lit.

One question I'm hoping you can help answer @bhouston is how to model this in the editor. When designing a shader effect, the scene lights aren't the effect you're worrying about. However, it might be nice to preview how certain types of lights interact with your shader. Should the editor have an interface for creating three lights (scene helpers you can drag around) for previewing light effects on shaders? Or should it only have the "export this shader with three lighting" option, which will compose in all the required light data (similar to three's shaderchunks, but using the compiler)?

[AndrewRayCode]

Also re: the macro thing, I have a proof of concept export to unity working, which of course has a different lighting system. However, if I could make an appropriately configured light shader for Unity, it would be as simple as composing in that specific light shader on export to unity.

[bhouston]

Interesting. Be aware that ThreeJS is contemplating adding a dataTexture to optimally specify lights: mrdoob/three.js#7060.