Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

VR support #146

Merged
merged 9 commits into from
Nov 28, 2024
Merged

VR support #146

merged 9 commits into from
Nov 28, 2024

Conversation

ninjamode
Copy link
Contributor

Hi,

this adds VR support as mentioned in #17. It is based on the work in https://github.com/ninjamode/Unity-VR-Gaussian-Splatting

Changes are:

Not ported over is the sorting for eye center. It's quite helpful, but not implemented nice enough for me to want to port it over, so maybe should be mentioned in the readme as a optimization that's maybe worth it? Not strictly needed, especially with Desktop VR. Sample implementation can be found here: https://github.com/ninjamode/Unity-VR-Gaussian-Splatting/blob/main/package/Runtime/GaussianSplatRenderer.cs

Anyway, this does not add any Readme changes, but you might want to adjust the VR notice.

Tested and works on: Desktop Steam VR (HTC Vive), Varjo Devices (tested on Aero), Quest 3 and Quest Pro via Link and standalone. Probably also works on other devices, such as mobile phones? It's to keep in mind that GS rendering uses quite some resources, so keep the Gaussian count low on mobile devices, try not to be super close (fillrate issue?), don't sort every frame.

How To:

  1. Use URP (Standard Pipeline has more matrix issues. Can probably be fixed, I didn't see the point)
  2. Enable XR plugin Management
  3. Enable and start OpenXR on init
  4. Set Render Mode to Multi-Pass

Thats it.
If you ever get the chance to try it you should, it's quite impressive. If you feel like it, I'd be happy if you could mention my repository and/or the fact that this was done as a part of a research project that can be cited (see https://github.com/ninjamode/Unity-VR-Gaussian-Splatting/?tab=readme-ov-file#related ).

In any case, thank you for this project, and let me know If you'd like to see any changed. Maybe someone with a VR headset can test this and confirm I didn't accidentally break anything.

@ninjamode
Copy link
Contributor Author

Probably worth mentioning that this does NOT add support for Apple Vision Pro, due to not supporting Single Pass Instanced Rendering.

@aras-p
Copy link
Owner

aras-p commented Nov 28, 2024

This is excellent, thank you! I'll do some non-VR testing here locally and will merge / update the readme soon.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants