[Stay Aligned] Add small yaw rotations to keep trackers aligned

[jabberrock]

Add small yaw rotations to keep trackers aligned. Stay Aligned applies small yaw corrects every server tick. It can successfully keep the user aligned in many poses including standing, sitting and lying down.

This feature has two primary ideas:

1. If the player is not moving, the tracker yaws should not be changing.
2. If the player is moving, on average, the player will be in their relaxed pose.

Every server tick, Stay Aligned picked a tracker, calculates an error based on 3 forces (locked trackers, centering force, and neighbor trackers), and uses gradient descent to nudge the tracker's yaw.

Main classes:

1. StayAligned: Entry point of Stay Aligned that is called from HumanSkeleton
2. AdjustTrackerYaw: Performs the gradient descent
3. LockedErrorVisitor, CenterErrorVisitor and NeighborErrorVisitor calculates the errors for gradient descent
4. StayAlignedTrackerState: Tracks the Stay Aligned state per tracker

Supporting classes:

1. TrackerSkeleton: Visits trackers, while providing neighboring trackers
2. TrackerPose and TrackerSkeletonPose: Detects the pose of the player from tracker orientations
3. RestDetector: Detects when a tracker is at rest

TODO: For the actual release, we should make a wizard for detecting the relaxed body angles.

SlimeVR/SolarXR-Protocol#157

[loucass003]

i feel like this feature need deeper integration into the onboarding especially for the detect angles part. we cannot expect people to do it if we do not explain why and better explain how.

Few things that we need to explain to the user precisely.

• When should you perform the auto detect: One time? every session?
• Should it be done before / after autobone / mounting

These questions also make me think that this could be integrated into autobone / done at the same time without user interraction.

Also i would love a list of possible drawbacks: no solution is perfect and because this is basically guessing i suppose there is cases where this could underperfom

[loucass003]

Things like the adjustment rate, depending on the IMU type should be bound to the tracker settings. as people have been mismatching trackers imus in the past. also i feel like this should be auto detected / defaulted based on the IMU type

[jabberrock]

I was thinking of building a wizard for walking the user through setting the angles. This would be a good place to address both of your questions. However, I am not a frontend engineer, so it's going to be challenging for me.

When should you perform the auto detect: One time? every session?

Auto detect angles should be done once, just like Autobone.

Should it be done before / after autobone / mounting

It should be done after full reset and mounting reset. It doesn't need Autobone specifically.

These questions also make me think that this could be integrated into autobone / done at the same time without user interraction.

There is no step in Autobone to collect "relaxed" postures. Maybe this could be added as extra steps to Autobone? I don't have a strong opinion.

Also i would love a list of possible drawbacks: no solution is perfect and because this is basically guessing i suppose there is cases where this could underperfom

In all the situations I have tested, it outperforms the default by a lot, at least on BMI270s and BMI160s. Anecdotally, it also outperforms the default on BNO085s and LSM6DSVs (but I don't have those IMUs).

I am not promising that it is perfect. For example, if you sit at a weird angle, you may see your body drift a little. But the drift ends up being capped to about 20 deg off-center, and the body proportions remain realistic. Compare this to the default where your chest or waist ends up twisted arbitrarily. I'm promising that it's better than the default.

[jabberrock]

Things like the adjustment rate, depending on the IMU type should be bound to the tracker settings. as people have been mismatching trackers imus in the past. also i feel like this should be auto detected / defaulted based on the IMU type

I would recommend just having one setting, chosen based on their worst IMU. Otherwise there'll be settings spread out across trackers, like "Enable drift compensation". In practice, 0.2 works well for all IMUs. I might even consider removing the setting completely.

[jabberrock]

Also i would love a list of possible drawbacks: no solution is perfect and because this is basically guessing i suppose there is cases where this could underperfom

Butterscotch reminded me of a drawback. If the user sits in a significantly different position than what they selected in the relaxed body angles (e.g. they sit with their legs slightly crossed), Stay Aligned will try to undo their position.

[Eirenliel]

Also i would love a list of possible drawbacks: no solution is perfect and because this is basically guessing i suppose there is cases where this could underperfom

Butterscotch reminded me of a drawback. If the user sits in a significantly different position than what they selected in the relaxed body angles (e.g. they sit with their legs slightly crossed), Stay Aligned will try to undo their position.

This is a significant issue which will make it unusable in many social VR situations. Shouldn't it detect that the user is far from the position and just not apply alignment fix?

[Eirenliel]

Things like the adjustment rate, depending on the IMU type should be bound to the tracker settings. as people have been mismatching trackers imus in the past. also i feel like this should be auto detected / defaulted based on the IMU type

I would recommend just having one setting, chosen based on their worst IMU. Otherwise there'll be settings spread out across trackers, like "Enable drift compensation". In practice, 0.2 works well for all IMUs. I might even consider removing the setting completely.

I think the setting should be a multipler: x0.5, x1, x2. That's it. Being, 0.05deg/s, 0.1deg/s and 0.2deg/s.

[jabberrock]

Also i would love a list of possible drawbacks: no solution is perfect and because this is basically guessing i suppose there is cases where this could underperfom

Butterscotch reminded me of a drawback. If the user sits in a significantly different position than what they selected in the relaxed body angles (e.g. they sit with their legs slightly crossed), Stay Aligned will try to undo their position.

This is a significant issue which will make it unusable in many social VR situations. Shouldn't it detect that the user is far from the position and just not apply alignment fix?

I played around with this idea a lot in 3.* and 4.* versions and came to the conclusion that it's better just to always adjust. If I try to limit it to a certain range, it quickly gets out of range due to larger player movements, and Stay Aligned stops working.

[jabberrock]

@Eirenliel would it be OK if we addressed the UI stuff before release, or do I need to address it in this PR?

[ButterscotchV]

Looking good. I'm still not 100% on everything here, so I will do another more definitive review later. I like the structure developed here, should be easy to develop tests for in the future.

[ButterscotchV]

What these values mean should be explained somewhere, now that we have tooltips maybe we can use that?

[ButterscotchV]

This seems like it should be a value class like our other math classes?

[jabberrock]

It becomes very annoying to access from Java if it's a value class. I have to write a separate Java accessor for each angle I expose.

Also, I can't store both rad and deg, which means either:

1. I rewrite the config classes to use raw floats; or
2. I store ugly 0.199999 values in config

I'm happy to change it if you think it has to be a value class.

[ButterscotchV]

It doesn't have to be, but it could be better that way, do we need to care about Java? You should still be able to store both rad and deg in that case? Maybe I'm misunderstanding value classes.

[jabberrock]

https://kotlinlang.org/docs/inline-classes.html An inline class must have a single property initialized in the primary constructor. It cannot store multiple values.

[jabberrock]

Hmm... let me make a separate DegreeAngle class which will be used by config. It gets converted into an Angle class for the rest of the process. Then Angle can be an inline value class.

[ButterscotchV]

This could use a comment explaining its placement at the end of this function, it can seem inconsequential from a glance.

[Eirenliel]

@Eirenliel would it be OK if we addressed the UI stuff before release, or do I need to address it in this PR?

It's okay to address it in the later PR, but it shouldn't be this way on release, the settings are already confusing as is.