Getting the WorldOriginPtr with unity > 2021.2

[Birkenpapier]

Hello petergu684,

thank you very, very much for such a nice project!
Unfortunately I need to continue my development with the unity version 2021.2 and the problem is that the methods for getting the unityWorldOrigin are discontinued from that version.
Are there any replacements I can implement to keep the same functionality that the pointcloud is visualized directly where they got recorded or is it simply not possible from that version to get the alignment process of both done?

Thank you very, very much for your or anyones help!

BR,

Birkenpapier

[petergu684]

var unityWorldOrigin = Microsoft.MixedReality.OpenXR.PerceptionInterop.GetSceneCoordinateSystem(UnityEngine.Pose.identity) as Windows.Perception.Spatial.SpatialCoordinateSystem;

[Birkenpapier]

Thank you @petergu684 very, very much for your quick answer!
I don't have currently access to a Hololens over the weekend so I'll come back to you on Monday with the results.
Thank you once again very, very much!

[Birkenpapier]

Hello again @petergu684,

I've tested now the application with that line and unfortunately the same problem still persists:

The pointcloud is smaller and looks more away from the real scene and also fades away when your move your head.
May this be regarded to the update from unity 2019.4.17 -> unity 2021.3.8?

[petergu684]

Are you by any chance using WLT or similar packages that changes the camera transformation?

[Birkenpapier]

No, nothing like that or similar was installed by me.
I have used your unity sample and opened it with the higher unity version of 2021.3.8.
After that I needed to install OpenXR and setting up the project for whatever reason again for Hololens 3d deployment.
That was all that I needed to change to make it runable on that version.

Could that be maybe the reason why the point cloud visualization gets distorted?
May I ask you for your OpenXR settings if you open the project with that version if that is not to much to ask?

Thank you very, very much in advance for your answer.

BR,

Birkenpapier

[petergu684]

Sorry but I have no idea. One suggestion is to open a clean project in the newer version rather than porting from the old one, although I did not have any issue porting mine from 2019 to 2021 but who knows. The Unity version I am using is 2021.3.11 with OpenXR backend and everything works fine.

I am not sure what OpenXR settings you are looking for, but it should be the standard way. In Project Settings - XR Plugin-in Management, under UWP tab, I have "Initialize XR on Startup" ticked and "OpenXR" and "Microsoft HoloLens feature group" ticked as well beneath. In Project Settings - XR Plugin-in Management - OpenXR, I have Single Pass Instanced, Depth 16 Bit, tick only on "Microsoft Hololens" in OpenXR Feature Groups and selected all the three options (Hand Tracking, Mixed Reality Features and Motion Controller Model) on the right-hand side.

[Birkenpapier]

I see. Thank you very much for answering. May I ask you for sharing your ported and working unity version? That would be very helpful. Thank you very, very much in advance for your answer. BR, Birkenpapier From: Wenhao ***@***.***>Sent: Friday, March 24, 2023 21:25To: petergu684/HoloLens2-ResearchMode-Unity ***@***.***>Cc: Birkenpapier ***@***.***>; Author ***@***.***>Subject: Re: [petergu684/HoloLens2-ResearchMode-Unity] Getting the WorldOriginPtr with unity > 2021.2 (Issue #52)  Sorry but I have no idea. One suggestion is to open a clean project in the newer version rather than porting from the old one, although I did not have any issue porting mine from 2019 to 2021 but who knows. The Unity version I am using is 2021.3.11 with OpenXR backend and everything works fine. I am not sure what OpenXR settings you are looking for, but it should be the standard way. In Project Settings - XR Plugin-in Management, under UWP tab, I have "Initialize XR on Startup" ticked and "OpenXR" and "Microsoft HoloLens feature group" ticked as well beneath. In Project Settings - XR Plugin-in Management - OpenXR, I have Single Pass Instanced, Depth 16 Bit, tick only on "Microsoft Hololens" in OpenXR Feature Groups and selected all the three options (Hand Tracking, Mixed Reality Features and Motion Controller Model) on the right-hand side. —Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you authored the thread.Message ID: ***@***.***>

[petergu684]

As you may find, I don't have much time to maintain this repo regularly, so I don't have updated version of this sample project. I am using it on a different project that I cannot share here, but I will update with you once I got time to update the sample here.

[Birkenpapier]

Thank you very much! Looking forward to hear from you. Thank you very much an BR, Kevin From: Wenhao ***@***.***>Sent: Friday, March 24, 2023 21:55To: petergu684/HoloLens2-ResearchMode-Unity ***@***.***>Cc: Birkenpapier ***@***.***>; Author ***@***.***>Subject: Re: [petergu684/HoloLens2-ResearchMode-Unity] Getting the WorldOriginPtr with unity > 2021.2 (Issue #52)  As you may find, I don't have much time to maintain this repo regularly, so I don't have updated version of this sample project. I am using it on a different project that I cannot share here, but I will update with you once I got time to update the sample here. —Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you authored the thread.Message ID: ***@***.***>

[Birkenpapier]

Hello again @petergu684,

thank you very much for your suggestions.
As you said I've created a clean project, imported all your files and plugin and then created again the solution.
Now it worked that the pointcloud is moving with the camera but there is now a strange offset.
Arount the pose.identity looking forward the PC seems to be behind the real object and turning 180° it seems to be in front of the object.
Video for clarification:

20230329_053359_HoloLens_compressed.mp4

This offset is also visible when you export the PC directly from the C++ application into Open3D ans send it back to the unity scene.

Do you maybe have any clue what is happening here?
Besides the unity version nothing else was changed in your files etc.

BR,

Birkenpapier

EDIT:
Can this be related to the OpenXR coordinate system difference to the Unity coordinte system like described in here:
jdibenes/hl2ss#27

[kookoo9999]

Hello, @Birkenpapier , @petergu684
I'm tryna to run sample(PointCloudSample) built by Unity(2020.3) on Hololens2.
First I add this sample project to Unity Hub, and then I dowloaded "XR Plug-in " on Project Settings, and ticked "Initialize XR on Startup" , "OpenXR" and "Microsoft HoloLens feature group" under UWP tab.
And then at the "XR-Plug-in Management" - "OpenXR" tab, I set "Depth 16 Bit" by "Depth Submission Mode", and add the "Microsoft Hand Interaction Profile" to "Interaction Profiles" , and ticked "OpenXR Feature Groups - Microsoft Hololens - Hand Tracking, Motion Controller Model".
After that, I add a XR-plugin to this project by Microsoft Mixed Reality Feature Tool.
Last, I change the NewXRSDKConfigurationProfile of MixedRealityToolkit's profile.

Actually UWP app which runs on Hololens2 , is running well but it doesn't recognize my hand. so I can't click the button of pannel like a "Toggle Preview" , "Toggle Point Cloud" , "Connect to server" , and etc...

I saw your reply on this post, I checked your porject don't recognize your hand at first your photo , but at last photo, your app is recognizing your hand.

This is my issue.

Can you tell me how fix it?

[flokm]

Hi everyone and thanks for the super useful project!

I have been stuck for three days pretty much at the same point as @Birkenpapier : I can use the project with OpenXR and Unity 2021.3.21f, and everything works with no warnings, including pressing buttons and displaying the pointcloud. But I have an offset of the pointcloud to the real world (and to the scene understanding mesh, which interestingly is correct) that corresponds more or less to the offset between sensor and head center (similar to the video above).

I have tried:

• Different MRTK settings, both for Experience and Camera profiles
• Different transform settings for the PreviewPlanes
• Different initializations of the origin in ResearchModeVideoStream.cs
• Different settings for Mixed Reality Capture, in the Unity Settings and on the HoloLens
• Different versions of the OpenXR package for MRTK
• Different Settings in the XR Plugin Management
• Different Camera and TrackedPoseDriver settings

But nothing seems to even change then offset in the slightest. So I assume there is a transformation happening somewhere, which bypasses all of these different settings, and that transformation probably has a slight difference between Windows MR and OpenXR. I have seen that for hl2ss there seems to have been a fix, but I cannot figure out how to apply that fix here... Has anyone had any luck with this?

Thanks a lot!
Florian

P.S. I have dependencies on newer Unity versions, so 2021.3.21f is unfortunately as low as I can go...