PBR: Shader Compilation Error on Mac OS (GLSL110)

[MeFisto94]

So when running on Mac OS X in the default mode, you only get the compability profile and thus OpenGL 2.0 and GLSL110.
According to the source code, this should not be a problem for PBR, we have our GLSLCompat.glsllib and we have the "Language Switch Statements".

In this case it seems the latter has gone wrong and I need to investigate why.
I might open another issue to track the exact issue, this is mainly to track the defect as is and make it more prominent for searches.

Do note that this also affects the SDK in it's current state and basically all PBR on Mac OS X and probably also every shader wanting to use GLSL120+ language features.

com.jme3.renderer.RendererException: compile error in: ShaderSource[name=Common/MatDefs/Light/PBRLighting.frag, defines, type=Fragment, language=GLSL110]
ERROR: 0:200: GLSL 110 does not allow sub- or super-matrix constructors
ERROR: 0:201: Use of undeclared identifier 'wToLocalRot'
ERROR: 0:201: Use of undeclared identifier 'wToLocalRot'
ERROR: 0:214: Use of undeclared identifier 'wToLocalRot'
ERROR: 0:215: Use of undeclared identifier 'rayLs'
ERROR: 0:218: Use of undeclared identifier 'wToLocalRot'
ERROR: 0:222: Use of undeclared identifier 'rayLs'
ERROR: 0:223: Use of undeclared identifier 'rayLs'
ERROR: 0:224: Use of undeclared identifier 'firstPlaneIntersect'
ERROR: 0:224: Use of undeclared identifier 'secondPlaneIntersect'
ERROR: 0:225: Use of undeclared identifier 'furthestPlane'
ERROR: 0:225: Use of undeclared identifier 'furthestPlane'
ERROR: 0:225: Use of undeclared identifier 'furthestPlane'
ERROR: 0:227: Attempt to use 'distance' as a variable
ERROR: 0:228: Use of undeclared identifier 'intersectPositionWs'

	at com.jme3.renderer.opengl.GLRenderer.updateShaderSourceData(GLRenderer.java:1476)
	at com.jme3.renderer.opengl.GLRenderer.updateShaderData(GLRenderer.java:1503)
	at com.jme3.renderer.opengl.GLRenderer.setShader(GLRenderer.java:1567)
	at com.jme3.material.logic.SinglePassAndImageBasedLightingLogic.render(SinglePassAndImageBasedLightingLogic.java:259)
	at com.jme3.material.Technique.render(Technique.java:166)
	at com.jme3.material.Material.render(Material.java:1026)
	at com.jme3.renderer.RenderManager.renderGeometry(RenderManager.java:614)
	at com.jme3.renderer.queue.RenderQueue.renderGeometryList(RenderQueue.java:266)
	at com.jme3.renderer.queue.RenderQueue.renderQueue(RenderQueue.java:305)
	at com.jme3.renderer.RenderManager.renderViewPortQueues(RenderManager.java:877)
	at com.jme3.renderer.RenderManager.flushQueue(RenderManager.java:779)
	at com.jme3.renderer.RenderManager.renderViewPort(RenderManager.java:1108)
	at com.jme3.renderer.RenderManager.render(RenderManager.java:1158)
	at com.jme3.app.SimpleApplication.update(SimpleApplication.java:272)
	at com.jme3.system.lwjgl.LwjglAbstractDisplay.runLoop(LwjglAbstractDisplay.java:151)
	at com.jme3.system.lwjgl.LwjglDisplay.runLoop(LwjglDisplay.java:197)
	at com.jme3.system.lwjgl.LwjglAbstractDisplay.run(LwjglAbstractDisplay.java:232)
	... 1 more

[MeFisto94]

So I looked into this and can state:

• Materials get parsed and loaded successfully up to the point where the techniques are decided.
• When Techniques are decided we have a "suboptimal" algorithm: it randomly picks one of the multiple supported ones, thus the GLSL Version used could vary. That is no problem, it's just odd.

jmonkeyengine/jme3-core/src/main/resources/Common/MatDefs/Light/PBRLighting.j3md

Line 126 in 0252c1b

VertexShader GLSL300 GLSL110 GLSL150: Common/MatDefs/Light/PBRLighting.vert

Here we state that the shader supports GLSL110 as is but either GLSLCompat.glsllib fails here or we'd need a separate version of the shader, as I guess glslcompat can't do things like

ERROR: 0:200: GLSL 110 does not allow sub- or super-matrix constructors

which might be the only problem GLSL110 has with this shaders.

This defect thus isn't Mac OS only but appears on all devices using the GLSL110 shader (without having a compatibility profile for higher version, which is just uncommon on non-ancient devices)

[stephengold]

Where is the code that randomly picks the technique?

[MeFisto94]

It is not truely random, but since different GLSL Versions don't carry different weights, the algorithm is free to choose them (and probably does so in definition order in j3md, which is random, otherwise we'd have to have GLSL150 before GLSL110):

jmonkeyengine/jme3-core/src/main/java/com/jme3/material/Material.java

Lines 759 to 769 in 0252c1b

	for (TechniqueDef techDef : techDefs) {
	if (rendererCaps.containsAll(techDef.getRequiredCaps())) {
	float techWeight = techDef.getWeight() + (techDef.getLightMode() == renderManager.getPreferredLightMode() ? 10f : 0);
	if (techWeight > weight) {
	tech = new Technique(this, techDef);
	techniques.put(name, tech);
	weight = techWeight;
	}
	}
	lastTech = techDef;
	}

[stephengold]

Can we simply change the definition order in the J3MD? Or do we need a more sophisticated mechanism?

[MeFisto94]

Just to be clear: it doesn't solve this problem and it's also not a real problem, in theory every GLSL Version should be fine, but:
We don't use the highest available GLSL Version and instead potentially add a compat layer and then the GPU Driver has a compat layer again to translate it to the higher version. Probably not desirable, but should work.

Since no one (especially user code) will honor the order of definitions.
It could be a sophisticated problem, but I guess the easiest possible way would be to give every GLSL Version an increasing weight, so the highest language will always have the highest weight.

It will be funny when it's about to weight older GLSL Versions + Extensions versus a more recent version with native support for features.

But as I said, that's an almost philosophical problem instead of an issue.

[pspeed42]

It looks to me like it uses the highest available version that has the required caps... with an additional artificial weight of +10 for if the technique supports the preferred light mode. So Caps.OpenGl45 will beat Caps.OpenGl20 even if 20 has the preferred light mode and 45 doesn't.

Edit: but it's using GLSL version but my point is still the same if translated.

[MeFisto94]

I am sorry, you are right, I didn't pay attention. What actually happens is what I suggested above:

jmonkeyengine/jme3-core/src/main/java/com/jme3/material/TechniqueDef.java

Line 350 in 0252c1b

weight = Math.max(vertCap.ordinal(), fragCap.ordinal());

Here we use the Ordinal of the Shader Version as weight already. Sorry for the confusion!

To come back to topic: PBRLighting.frag is not GLSL110 compatible, though.

[pspeed42]

I guess PBRLighting.j3md says it's GLSL110 compatible, though? If so then that's definitely a bug.

ie: there is a technique that says it works with GLSL110 when it doesn't.

[MeFisto94]

Exactly and that is what this issue is about. See the second comment. The Fragment Shader of the Default technique is not GLSL110 compatible.

Actually that's not entirely true, because the Code Lines don't match as the glsliibs have been imported, so the issue is that:
https://github.com/jMonkeyEngine/jmonkeyengine/blob/master/jme3-core/src/main/resources/Common/ShaderLib/PBR.glsllib#L124

is not GLSL110 compatible.

So we'd have those options:

1. Remove the GLSL110 statement from the j3md
2. Use GLSL110 Syntax
3. Add a PBR110.glslib
4. Could we use #if GLSL < 110? That way we could have both without changing anything but the PBR.glslib

[pspeed42]

(1) seems like the quick and immediate fix... with some combination of the others as things can be tested. I was thinking maybe for a while, PBR just isn't supported on older GL versions.

...but I guess someone is already trying to run PBR in an environment where only GLSL110 is available and so is hitting this issue? In which case, there is already a user trying to use PBR on an older system or something.

[MeFisto94]

Yes, the issue stems from Mac OS X, where the default is OpenGL 2.1. We're also hitting that on our CI for Software-Rendering using MESA, which is just more strict and doesn't seem to start in the compat profile.

It turns out that

jmonkeyengine/jme3-core/src/main/resources/Common/ShaderLib/PBR.glsllib

Line 146 in 0252c1b

mat3 wToLocalRot = mat3(lightProbeData);

this is the source of all evil:
mat3(mat4) is OpenGL 3.1+, iirc.

We've been thinking about GLSLCompat.glslib but there are a few parameter combinations and it's hard to do this with a define.

We could do it in form of mat3(in, out) as a define though, as it's basically just accessing matrices as arrays with the index.

Then #if GLSL >= 310 #define custom_mat3(in, out) out = mat3(in)
So basically, we'd need our custom mat3 function and use it everywhere. But what about the usercode in that case? They will blindly use mat3

[pspeed42]

Then they would be incompatible, eh? To be honest, I've never personally converted from a mat4 to a mat3 in any of my shaders so I don't know how much user code it will affect.

How often do we call mat3(mat4)? Just the once?