Depth pass unnecessarily low precision on Apple iPad Air 2

[bhouston]

Description of the problem

The issue is that the dpeth pass has unecessarily low precision even with RGBA packing on some mobile devices, in particular the Apple iPad Air 2.

This example, set it to use the depth RGBA output.

http://threejs.orgexamples/#webgl_materials_channels

This is what it should look like (all channels are used):

This is what it ends up looking like on the Apple iPad Air 2 device (only blur channel and I think alpha channel are used):

So basically on lowp / mediump devices, depth RGBA pass, which is used by shadows and post effects does not function properly.

Three.js version

• Dev
• r77

Browser

• All of them
• Chrome
• Firefox (Not tested)
• Internet Explorer (N/A)
• Safari

OS

• All of them
• Windows
• Linux
• Android
• IOS

Hardware Requirements (graphics card, VR Device, ...)

Apple iPad Air 2.

[bhouston]

Interesting thing, if I use the old mod approach to packing depth, I get a bunch more precision (at least a couple more bits):

const highp vec4 bit_shift = vec4( 256.0 * 256.0 * 256.0, 256.0 * 256.0, 256.0, 1.0 );
  const highp vec4 bit_mask = vec4( 0.0, 1.0 / 256.0, 1.0 / 256.0, 1.0 / 256.0 );
  highp vec4 res = mod( v * bit_shift * vec4( 255 ), vec4( 256 ) ) / vec4( 255 );
  res -= res.xxyz * bit_mask;
  return res;

The result when using the above packing method:

[mrdoob]

/ping @tschw

[tschw]

highp is not guaranteed to be available and may cause a compile error.

Interesting thing, if I use the old mod approach to packing depth, I get a bunch more precision (at least a couple more bits):

Seems to be about numeric error. Unlike the decoder, the encoder wasn't broken, so I guess it's a good idea to do this revert, as the old implementation appears to be more robust under limited precision. However, I wonder whether there is a way to express it in a more readable way, somehow... Maybe a comment about 255 vs 256 will do - it's rather subtle...

[cnspaha]

any update on this?

[bhouston]

@cnspaha My recommendation is to not use RGBA encoded depth on iOS. Even the revert doesn't actually fix the issue.

[cnspaha]

yeah, but so the whole SAOPass isn't usable as it is on iOS, is there?
So the "solution" would be to copy it and modify the depth part of it until it works?

[bhouston]

I wrote the SAOPass and yeah we disable it by default on iOS. The reason is two fold, one the depth rgba encoding precision issues and also because it is very slow.

[bhouston]

I believe the issue is that the shading calculations are done in FP16 on iOS and our RGBA depth coding assumes FP32 precision or similar. IF you can rewrite the math to work on FP16 precision, it may work. I Suspect it will not fill up all RGBA channels but you may get two bytes of precision in depth. I believe on iOS the z-buffer by default is only 16 bits or 24 bits unsigned.

I look forward to further development here. IT would be great to have a solution.

[bhouston]

@cnspaha Apparently SAO on Sketchfab works well on iOS: https://skfb.ly/6wMJ8 I wonder how they achieve accuracy in the depth pass?

[Mugen87]

FF Shader console shows the following encode/decode functions. Could these be helpful for this issue?

float decodeFloatRGBA( vec4 rgba ) {
    return dot( rgba, vec4(1.0, 1.0/255.0, 1.0/65025.0, 1.0/16581375.0) );
}

vec4 encodeFloatRGBA( float v ) {
    vec4 enc = vec4(1.0, 255.0, 65025.0, 16581375.0) * v;
    enc = fract(enc);
    enc -= enc.yzww * vec4(1.0/255.0,1.0/255.0,1.0/255.0,0.0);
    return enc;
}

vec2 decodeHalfFloatRGBA( vec4 rgba ) {
    return vec2(rgba.x + (rgba.y / 255.0), rgba.z + (rgba.w / 255.0));
}

vec4 encodeHalfFloatRGBA( vec2 v ) {
    const vec2 bias = vec2(1.0 / 255.0, 0.0);
    vec4 enc;
    enc.xy = vec2(v.x, fract(v.x * 255.0));
    enc.xy = enc.xy - (enc.yy * bias);

    enc.zw = vec2(v.y, fract(v.y * 255.0));
    enc.zw = enc.zw - (enc.ww * bias);
    return enc;
}

[bhouston]

Interesting. So I guess we have the option of FP32 into 4 bytes, FP24 into 3 bytes and FP16 into 2 bytes. I Guess we should have the option to pick which resolution we want? I wonder if we can make them compatible with each other, so that FP16 encoded and unpacked by FP24 is still valid, just with reduced precision.

[Mugen87]

decodeFloatRGBA() and encodeFloatRGBA() are similar to our unpackRGBAToDepth() and packDepthToRGBA() functions and I think I understand the parametrization. But why this signature?

vec4 encodeHalfFloatRGBA( vec2 v )

The given half float value is a vec2? Why not just a float?

[bhouston]

IT is encoding two fp16 floats at once into a single vec4 and vice versa. (vec2 of fp16) <-> vec4