Enable Depth Sensing for immersive-ar applications.

[listdo]

Is your feature request related to a problem? Please describe.

In my research project we are looking for a way to use depth sensing and access depth data from immersive ar applications. Currently it seems like it is not possible to enable the optional feature "depth-sensing" provided by the api of modern browsers.

Describe the solution you'd like

I would like to be able to use the new "depth-sensing"-feature provided by the modern chrome browser api described in: https://www.w3.org/TR/webxr-depth-sensing-1/

Describe alternatives you've considered

All alternatives that lead to the same result.

[mrdoob]

@listdo Which modern browsers support this feature? Is there a demo we can try?

[listdo]

@mrdoob
As far as i understood it is currently only supported by Chrome and Chrome for Android. In the following link is the feature status for chrome and there are two examples linked that use the browser api.

https://chromestatus.com/feature/5742647199137792

[julapy]

here is the WebXR sample:
https://storage.googleapis.com/chromium-webxr-test/r1067245/proposals/phone-ar-depth-gpu.html
this feature is available since Chrome 90 and can be enabled session.depthDataFormat != "luminance-alpha"

with a bit of work, this would allow occlusion of threejs scenes with the real world, like this:
https://developers.google.com/ar/develop/unity/depth/quickstart

[zz85]

it looks like this is supported today by passing { 'optionalFeatures': [ 'depth-sensing'] } to XRButton.createButton()

https://github.com/search?q=repo%3Amrdoob%2Fthree.js%20depth-sensing&type=code

[Mugen87]

A first approach of depth sensing has been implemented via #27586 (r161).

[billleap]

Hi, I just tried the Dragging demo: https://threejs.org/examples/webxr_xr_dragging.html

and got the following warning:

XRButton.js:69 Feature 'depth-sensing' is not supported for mode: immersive-ar

[mrdoob]

@billleap Any chance you can share the device you got that error with?

[idrisshah]

@billleap Any chance you can share the device you got that error with?

I am seeing the same error on Magic Leap 2 device. I think the issue is because depth-sensing implementation in threejs does not use XRDepthStateInit, while using depth-sensing feature.

[billleap]

@mrdoob @idrisshah
I was using S21 5G (SM-G991U) Android 13.
I used ADB and 'chrome://inspect/#devices' to see the warning.

[mrdoob]

I am seeing the same error on Magic Leap 2 device. I think the issue is because depth-sensing implementation in threejs does not use XRDepthStateInit, while using depth-sensing feature.

@cabanier Are we missing a check before using depth-sensing?

[cabanier]

I am seeing the same error on Magic Leap 2 device. I think the issue is because depth-sensing implementation in threejs does not use XRDepthStateInit, while using depth-sensing feature.

@cabanier Are we missing a check before using depth-sensing?

It looks like the Quest browser is not requiring that this dictionary is passed. Strange, because I would expect the existing Chromium code to enforce this.
We should fix up three.js and pass an empty dict. I'll file an issue to make this optional.

[idrisshah]

Strange, because I would expect the existing Chromium code to enforce this.

Chromium does seem to enforce it here: