Avoid minor movements of screen in XR glasses when small head movements

[mlell]

When wearing the glasses (Viture One) the projected monitor constantly moves by minor amounts, even if I keep my head still. This is due to minor muscle movements and even pulsating of the skin due to my heartbeat. This is distracting and strainful on my eyes, because the image gets slightly blurry when the glasses move it on the screen to compensate for that minor involuntary movements. It would be good if the projected image would remain fixed unless I move the head more than a predefined threshold.

I would like to try implementing this by myself but I have no experience with this and do not know where to start. Is this something to be implemented in the driver or the GNOME Extension? Could someone point me to a location where to start?

[wheaney]

Filtering out small movements can produce a pretty bad drift, since all movements are important when it comes to understanding your current pose. What I typically recommend is to look into the Movement Look-ahead slider in the advanced settings.

This feature attempts to predict where your head will be based on your current velocity because it takes at least 10-20ms or sometimes more to render the frame based on your last pose, so my driver has to try to predict where you'll be that far ahead to make it appear that the screen is fixed. But the result of this prediction is that it amplifies movements, so when the prediction is wrong like in the case of shaking, it becomes pretty obvious.

Try putting the slider to the second notch (the lowest value above default) and you should see the least amount of shaking, but also a less responsive screen. Move the slider up until you find a balance that is okay for you. Let me know how it goes.

[mlell]

Hello, thank you for your suggestion. Alas, the setting with either value does not help. E.g. when I am typing these words, each keystroke causes the image to become blurry, caused by movement of the image on the glasses' screen. My problem is not to keep the virtual monitor fixed in a position in space. My problem arises when the physical position of the screen on the glasses changes, which causes blur and slight stuttering. For example, I have an easier time to write this text in 0dof mode because in any other mode (either with breezy-desktop or with the native 2dof-mode (head tilt is ignored) each movement of the image on the glasses' screen causes blur. So for each keystroke the vision becomes temporarily blurry.

i don't mind it if it moves in virtual space as long as it does not move on the projected area. Drift could be accountet for in a similar way to the follow mode. For small movements, there would be no attempt to correct the screen location but above a certain threshold it could then move back to its original location.

I do not know if this is enough for me to be able to use this setup productively, that's why I wanted to modify the code to see if it improves things.

[rmiyazaki6499]

I'm not sure if this is related but I'm seeing that my screen drifts from time to time
I'm trying to use the Viture XR Pros while coding and I find myself looking at a certain portion of the screen for a while and then notice that the whole screen has shifted. I find myself re-centering quite often

I'm using GNOME 46 on Ubuntu 24
I have follow mode off

PS
Thank you for building this. I always wanted to try AR glasses as a productivity tool and Breezy is making it possible. So far I'm enjoying using it and with a bit more improvement will be a game changer!

[c42f]

I've noticed this issue with my pulse pushing the glasses around. I think it'd be possible to fix with the right filter, and without drift by separating the pose estimation into two parts:

1. The pose of the glasses in 3D space (rotation matrix R)
2. The small rotation between the glasses and the skull (rotation matrix S)

The rendering currently uses R, but to remove periodic motions due to the wearer's pulse in the skin on the nose, it should use R*inv(S) (or inv(S)*R depending on convention)

We can estimate R using whatever low-drift filter just as you currently do.

The trick, then, is to somehow estimate S. I expect some kind of band-pass filter in the 1-2Hz range could work for this, with update S, state = filter(R, state), keeping some amount of state in the filter?

It seems likely this scheme would have some tradeoffs because estimating S vs R is underdetermined. But I think it's worth trying. If you can point me at the right piece of code I can try make some time to look at this :)

[phooji]

Using the Viture XR Pro as well. I can actually 'see' my heartbeat based on the screen shaking/jittering slightly each time. To be clear: I'm not saying that to exaggerate -- it corresponds exactly to my actual heart rate/beats. The legs of the glasses are relatively tight on my ears.

Adjusting the look-ahead slider doesn't really change much between 'default' and '40ms.' Adjusting the glasses to be slightly higher (above my ears) changes the movement to be mostly vertical rather than horizontal, which helps keep text readable. I agree that some kind of filtering would be helpful for reducing strain.