Cannot use custom OpenXR runtime

[Scrumplex]

I am trying to run Beat Saber using Monado. Setting the following three environment variables doesn't work:

# Tell Proton to use OpenComposite instead of OpenVR
export VR_OVERRIDE=/nix/store/wbvv4scl5mn0br0n0kxm60452fi6fwd1-opencomposite-2ab29e49c96c5e446e439a408521a677737c4956/lib/opencomposite
# Help OpenComposite find the OpenXR runtime
export XR_RUNTIME_JSON=/nix/store/fdk7dh4b12kip0pm44prhxgpp5g4nl74-monado-bfa1c16ff9fc759327ca251a5d086b958b1a3b8a/share/openxr/1/openxr_monado.json
# Tell Steam Pressure Vessel to allow access to Monado
export PRESSURE_VESSEL_FILESYSTEMS_RW=$XDG_RUNTIME_DIR/monado_comp_ipc

Interestingly enough, those same environment variables work fine for OpenVR games.

From a PROTON_LOG, I discovered that XR_RUNTIME_JSON is ignored by OpenXR Loader (see https://github.com/KhronosGroup/OpenXR-SDK/blob/49e81bbd919af7c0cf513517a730d12253c28902/src/common/platform_utils.hpp#L329 and log below)

I also tried changing the active runtime by pointing ~/.config/openxr/1/active_runtime.json to my Monado runtime.

I have uploaded a Proton log for Beat Saber here: https://gist.github.com/Scrumplex/49390ce5bea8ce81fb672bfa656e82b8

It is being launched just with those env vars above

[eburema]

Not 100% sure but I think it might have something to with the checks steam_helper does during initialize_vr_data that get checked during get_vulkan_extensions The initialize_vr_data checks get sometimes ignored for OpenVR games (or can be forced to OK by symlinking wineopenxr.dll to /dev/null), but wineopenxr.dll doesn't init unless the check is OK and wineopenxr.dll is present, under monado + opencomposite the check in initialize_vr_data fails due to not being able to open an openxr instance twice (this is a limited by openxrloader).

The result of the the check is stored in the volatile reg key "Software\Wine\VR"

Tried to make a patch to test this, but haven't been able to compile proton to test

[emily-is-my-username]

@eburema

Tried to make a patch to test this, but haven't been able to compile proton to test

This patch successfully worked for me to get around the "Loader does not support simultaneous XrInstances" error. I tested both OpenVR and OpenXR games with Proton + Monado on NixOS.

EDIT:
As a note, using the opencomposite-helper to set VR_OVERRIDE (as @Scrumplex did) does not work for me.
I am using ~/.config/openvr/openvrpaths.vrpath instead, as described here.