Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

AutoBone bone contribution fix & cleanup #1249

Open
wants to merge 8 commits into
base: main
Choose a base branch
from

Conversation

ButterscotchV
Copy link
Member

@ButterscotchV ButterscotchV commented Nov 23, 2024

The bone contribution part of the AutoBone algorithm (getDotProductDiff()) seems to have been broken for a long time... Whoops! For this PR, I fixed contribution and ended up changing it to a metric that makes more sense. This is a fairly major change in the algorithm.

To explain bone contribution works, how this occured, and how this PR changes it:

  1. Originally was to compare the difference in the direction the bone faces in regards to the direction of sliding between frames 1 and 2, this was to try to predict which bones could be causing sliding between two frames and by how much. This was flawed because it was just considering the direction the bone was facing and not really how it affected the tail position.
  2. The bug was introduced in Use Bones #787 by using the bone's global rotation separately from the transform offset, so it was not relative to any reasonable positioning and seems to have caused really poor results (I didn't think it would be any different as we only use the difference in the dot product, but I was wrong).
  3. The way this PR implements this is:
    • slideL/slideR = The difference in the global position of the ankle trackers between frames 1 and 2.
    • boneOffL/boneOffR = The difference in the local tail positon of the bone (with rotation applied, this isolates the bone's effect on the skeleton) between frames 1 and 2.
    • Compute the dot product between slide and boneOff as unit vectors, then apply that as a multiplier for the adjustment value. By computing the dot product between these values, we should get a vague estimate of exactly how much each bone is affecting the overall sliding and whether it needs to be shorter or longer. This sets a very clear target for the algorithm and guides it in its descent to the optimal proportions.
    • Calculations are optimized by a minimum offset, values will not be computed if the offset is insignificantly small.

So far from my limited testing, this seems to make the algorithm perform significantly better. I also plan on working on a bit more cleanup of legacy code, but this is already pretty decent. Legacy code clean-up will be performed in a future PR in order to speed up code reviews.

@ButterscotchV ButterscotchV added Type: Bug Something isn't working Priority: High Important feature or blocks something important Area: AutoBone The AutoBone feature Type: Enhancement Adds or improves a feature Area: Server Related to the server labels Nov 23, 2024
@ButterscotchV ButterscotchV self-assigned this Nov 23, 2024
@ButterscotchV
Copy link
Member Author

ButterscotchV commented Nov 23, 2024

So my current conclusions based on my tests are:

  • New bone contrib performs significantly better over 0.13.2
    • No longer has wide hip bias
    • Handles low quality data without going to extremes
  • Slide error is likely better than offset slide error
    • Not enough evidence yet to be proven
  • Body proportion error is still beneficial to convergence and consistency, though is less necessary
    • Need to find a well balanced default value
  • Scaling bone contrib by the bone length is still beneficial to convergence and consistency
    • Bone length removed: Hip SD 3.07 -> 4.84
  • Performing more epochs may decrease consistency
    • numEpochs: 100 -> 400: Hip SD 3.07 -> 3.34
  • Performing less epochs may increase consistency
    • numEpochs: 100 -> 50: Hip SD 3.07 -> 2.86
    • Confirmed by initializing to equal length proportions, current initial adjust rate of 10 is good
    • Seems decent number-wise, let's test this with other people to validate!
  • Re-using frames in other pairs within the same epoch may decrease consistency
    • cursorIncrement: 2 -> 1: Hip SD 3.07 -> 3.69

Data

Experiments performed using 11 of my decent quality recordings on the latest commit (0baeb32), values are in centimeters. Results from 0.13.2 were either very obviously bad (hips too wide, head shift too short, etc) or produced an error value so bad that it triggered the max error limit, these were tested but not recorded in this section.

Experimental results (very long)

Default

headShift: 10.6 (SD 0.86)
neckLength: 13.09 (SD 0.94)
upperChestLength: 14.07 (SD 0.69)
chestLength: 14.07 (SD 0.69)
waistLength: 20.3 (SD 1.21)
hipLength: 10.52 (SD 3.07)
hipsWidth: 21.79 (SD 1.22)
upperLegLength: 48.12 (SD 2.2)
lowerLegLength: 51.11 (SD 3.12)

Initialized equal

headShift: 11.27 (SD 0.85)
neckLength: 12.91 (SD 0.89)
upperChestLength: 15.37 (SD 0.59)
chestLength: 15.37 (SD 0.59)
waistLength: 13.9 (SD 0.81)
hipLength: 16.4 (SD 2.99)
hipsWidth: 20.84 (SD 1.2)
upperLegLength: 46.05 (SD 2.15)
lowerLegLength: 51.28 (SD 2.93)

Shortened (50)

headShift: 10.52 (SD 0.86)
neckLength: 13.12 (SD 0.94)
upperChestLength: 14.19 (SD 0.66)
chestLength: 14.19 (SD 0.66)
waistLength: 20.34 (SD 1.18)
hipLength: 9.99 (SD 2.86)
hipsWidth: 21.98 (SD 1.18)
upperLegLength: 48.32 (SD 2.14)
lowerLegLength: 51.14 (SD 3.13)

Shortened (50) lower starting rate (1.0) initialized equal

headShift: 11.82 (SD 0.6)
neckLength: 14.37 (SD 0.66)
upperChestLength: 13.95 (SD 0.25)
chestLength: 13.95 (SD 0.25)
waistLength: 13.67 (SD 0.31)
hipLength: 22.45 (SD 1.16)
hipsWidth: 22.37 (SD 0.33)
upperLegLength: 44.52 (SD 1.6)
lowerLegLength: 48.39 (SD 2.05)

Shortened (50) initialized equal

headShift: 11.32 (SD 0.84)
neckLength: 12.92 (SD 0.87)
upperChestLength: 15.31 (SD 0.57)
chestLength: 15.31 (SD 0.57)
waistLength: 13.92 (SD 0.76)
hipLength: 16.63 (SD 2.88)
hipsWidth: 20.88 (SD 1.15)
upperLegLength: 45.92 (SD 2.09)
lowerLegLength: 51.3 (SD 2.9)

Extended (400)

headShift: 10.74 (SD 0.86)
neckLength: 13.04 (SD 0.93)
upperChestLength: 13.9 (SD 0.73)
chestLength: 13.89 (SD 0.72)
waistLength: 20.2 (SD 1.29)
hipLength: 11.36 (SD 3.34)
hipsWidth: 21.56 (SD 1.29)
upperLegLength: 47.78 (SD 2.27)
lowerLegLength: 51.13 (SD 3.07)

More pairs

headShift: 10.96 (SD 0.88)
neckLength: 12.91 (SD 0.92)
upperChestLength: 13.7 (SD 0.8)
chestLength: 13.7 (SD 0.81)
waistLength: 20.01 (SD 1.66)
hipLength: 12.52 (SD 3.69)
hipsWidth: 21.16 (SD 1.37)
upperLegLength: 47.28 (SD 2.38)
lowerLegLength: 51.16 (SD 2.97)

No length

headShift: 11.01 (SD 0.93)
neckLength: 13.01 (SD 0.99)
upperChestLength: 14.92 (SD 1.27)
chestLength: 14.92 (SD 1.27)
waistLength: 16.13 (SD 4.95)
hipLength: 14.62 (SD 4.84)
hipsWidth: 20.81 (SD 1.43)
upperLegLength: 46.68 (SD 2.69)
lowerLegLength: 51.01 (SD 2.89)

No length extended (400)

headShift: 11 (SD 0.89)
neckLength: 13.01 (SD 1.02)
upperChestLength: 15.16 (SD 1.61)
chestLength: 15.17 (SD 1.61)
waistLength: 15.4 (SD 6.06)
hipLength: 14.89 (SD 5.28)
hipsWidth: 20.8 (SD 1.46)
upperLegLength: 46.59 (SD 2.79)
lowerLegLength: 51.08 (SD 2.88)

@ButterscotchV ButterscotchV marked this pull request as ready for review November 24, 2024 00:10
@ImUrX
Copy link
Member

ImUrX commented Nov 29, 2024

good paper, thanks for the full explanation

@ButterscotchV ButterscotchV mentioned this pull request Dec 8, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Area: AutoBone The AutoBone feature Area: Server Related to the server Priority: High Important feature or blocks something important Type: Bug Something isn't working Type: Enhancement Adds or improves a feature
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants