OpenXR: Support composition layers based on Android surfaces

[dsnopek]

This adds support for composition layers that get their 2D content from Android surfaces.

It depends on PR #96182 in order to allow GDScript to get the Android surface as a JavaObject, and then either directly interact with it via JavaClassWrapper, or, more likely, pass it to a JNISingleton registered by a custom Godot Android Plugin.

This could be used to, for example, play a video via Android libraries, and have the output displayed on a composition layer in XR.

You enable it by simply checking "Use Android Surface" on the OpenXRCompositionLayer, which will hide the "Viewport Layer" property, and show the "Android Surface Size" property:

Here's a GDScript example that simply fills the Android surface with red, which can be used to test that it's working:

func _process(delta):
	var android_surface = $OpenXRCompositionLayerQuad.get_android_surface()
	if android_surface:
		var color = JavaClassWrapper.wrap('android.graphics.Color')
		var canvas = android_surface.lockCanvas(null)
		if canvas != null:
			var red = color.rgb(1.0, 0.0, 0.0)
			canvas.drawColor(red)
		android_surface.unlockCanvasAndPost(canvas)

Again, in a real app you'd probably pass the surface to your Godot Android Plugin, but this is much easier to test. :-)

Marking as a DRAFT until PR #96182 is merged.

[dsnopek]

Taking out of draft now that PR #96182 is merged!

[fire]

What does adding compositional layers look like? Can you make a video?

I don't know how to test this.

Edited:

There's a temporary failure, so I'll trigger a Github actions build again.

/usr/share/dotnet/sdk/8.0.401/NuGet.targets(174,5): error : The feed 'nuget.org [https://api.nuget.org/v3/index.json]' lists package 'JetBrains.Annotations.2019.1.3' but multiple attempts to download the nupkg have failed. The feed is either invalid or required packages were removed while the current operation was in progress. Verify the package exists on the feed and try again. [/home/runner/work/godot/godot/modules/mono/editor/GodotTools/GodotTools.sln]

[dsnopek]

@fire:

What does adding compositional layers look like? Can you make a video?
I don't know how to test this.

A composition layer is basically a floating panel with some 2D content on it (although, the panel can be a flat quad, or a slice of a cylinder or sphere which curves).

Testing is fairly simple:

1. Make a 3D scene with all the pieces necessary for an XR app
2. Add an OpenXRCompositionLayerQuad and check the "Use Android Surface" property
3. Add a script to the scene with the code up in the description
4. Run on an Android-based VR headset (should work on any Meta Quest or the Pico 4 per the OpenXR extension support page - I'm not sure about other headsets)
5. If the composition layer is red (via the code in the description), then all is working!

Rather than making a new project, you could also modify the OpenXR composition layer demo.

There's also some good docs on composition layers - the difference here is that rather than getting the content from a SubViewport like those docs show, it comes from an Android surface instead. There's a couple of screenshots in there that show what a composition layer looks like, in case just having a visual would help.

[fire]

Does this work with the HTCVive XR-Elite we have?

[dsnopek]

@fire I just tested it, and, it turns out that it does! However, it only supports quad composition layers (that is the OpenXRCompositionLayerQuad node), so be sure to use one of those.

[devloglogan]

Tested and seems to be working well! Code looks good to me, just have some extremely minor nits.

[dsnopek]

@devloglogan Thanks for the testing and review! I've made all the suggested changes in my latest push

[m4gr3d]

Do we have a capability that depends on chaining or is it for future proofing?

[dsnopek]

Specifically to support XR_FB_android_surface_swapchain_create, which we'll want to be able to add from the vendor extension

[m4gr3d]

Look great!

[akien-mga]

Thanks!