ergoCub 1.0 S/N:000 : Inaccurate Joint Tracking

[Woolfrey]

Robot Name 🤖

ergoCub 1.0 S/N:000

Request/Failure description

The low level joint control on the arms needs tuning to improve trajectory tracking.

Detailed context

The joint tracking on the arms has up to 4 degrees of error on most joints. This is causing issues with the Cartesian control since the pose of the hands cannot be guaranteed. The right wrist has large errors - up to 20 degrees - though this appears to be a recent problem that we didn't observe a month ago.

Additional context

We recorded the tracking error for the joints tracking a joint-level trajectory:

We then recorded the joint tracking when running a Cartesian controller. The errors are much larger. Cartesian feedback is used to try and make sure the hands stay together when grasping with 2 hands, but since the low level joint tracking is innacurate this is causing the robot to overcompensate:

Here is a video of the 2-handed grasping before ICRA 2023 and without Cartesian feedback on the position error of the hands. The ergoCub drops the object because the hands do not move together precisely because of the low-level joint tracking problem:

grasp_failure.mp4

Here is a video after where feedback on the hand positions was used to try and keep them together. The robot overcompensates because the joints are not tracking accurately and the error on the hand poses increases instead of decreasing:

grasp_instability_Trim.mp4

How does it affect you?

Myself @Woolfrey and Vignesh @vigisushrutha23 need accurate Cartesian control to demonstrate research in bimanual manipulation as part of the ergoCub project.

High-level feedback control to correct the pose error of the hands is also not possible unless the underlying joint tracking can be guaranteed (as shown in the video).

@xEnVrE : FYI

[maggia80]

@Woolfrey looking at the plots there are some issues in the wrist roll, right arm. We will investigate and fix it.

For what concern joint tracking accuracy, we used to provide the robot with "default" values of PID parameters. The tuning of the PID since it's application dependent is left to the user. What works for you may not work for some other users and vice-versa.

[AntonioConsilvio]

Hi @Woolfrey, The right wrist roll is not able to reach negative values. Today me and @fbiggi have tightened the screws to reduce the backlash of the wrist, tomorrow we will check the motor belts (although we believe they are already properly tightened).

If the problem on the roll is a mechanical malfunction, then these are the two steps that can help.

cc @maggia80 @sgiraz @Fabrizio69 @fiorisi

[AntonioConsilvio]

Today I found the AMC-BLDC 30B2 damaged.

We tried to recover the board by resoldering a new connector, but tomorrow we will continue with further tests.

cc @sgiraz

[AntonioConsilvio]

Today I replaced the damaged board AMC-BLDC 30B2 (code 14962.C S/N: 013), with a new one (S/N 018).

But that was not enough, because I found the roll motor burned out (this is the motor connected to the damaged AMC-BLDC board).

We will replace the motor in the first days of next week.

cc @maggia80 @sgiraz @Fabrizio69

[Woolfrey]

Hi @Woolfrey, The right wrist roll is not able to reach negative values. Today me and @fbiggi have tightened the screws to reduce the backlash of the wrist, tomorrow we will check the motor belts (although we believe they are already properly tightened).

If the problem on the roll is a mechanical malfunction, then these are the two steps that can help.

cc @maggia80 @sgiraz @Fabrizio69 @fiorisi

Hi @AntonioConsilvio , the plot I gave was the tracking error which is why there are negative values.

I take the joint limits from the motor controllers using the standard YARP devices. My controller enforces these limits at all times.

[AntonioConsilvio]

With the help of @Gandoo and @fbiggi, we replaced the burnt-out motor and now the right wrist works well.

Right.wrist.mp4

The PR updating the configuration files is available here:

• [ergoCubSN000] New right wrist calibration values robots-configuration#550

cc @maggia80 @sgiraz @Fabrizio69 @S-Dafarra

[AntonioConsilvio]

Hi @Woolfrey @vigisushrutha23, after the motor replacement you should have better tracking on the right wrist joints, I don't know if it will completely solve the tracking problems, but you should see an improvement.

[vigisushrutha23]

Hi @Woolfrey @vigisushrutha23, after the motor replacement you should have better tracking on the right wrist joints, I don't know if it will completely solve the tracking problems, but you should see an improvement.

Thanks @AntonioConsilvio . We will run tests and post the results tomorrow so that if there are no further problems, we can close the issue.

[sgiraz]

Hi @vigisushrutha23,

I guess the tests went well, right?
If yes, we proceed to close this issue.

[vigisushrutha23]

Hi @vigisushrutha23,

I guess the tests went well, right? If yes, we proceed to close this issue.

We will be doing them starting from 2pm. Sorry for the delay. We will post the plots ASAP so that the issue can be closed if there are no problems. Thanks.

[Woolfrey]

@vigisushrutha23 @sgiraz here are the new results.

We tested a simple joint trajectory. The arms should be in the exact same position but the right wrist is completely turned around:

video_20230803_142800.2.mp4

Here is a plot of some data we recorded:

Next we tested tracking a Cartesian trajectory for the hands. My controller is trying to compensate for the right wrist error:

video_20230803_145059.2.mp4

Here is a plot of the joint trajectory error when running the Cartesian control:

Here is what happened in the final configuration when the controller was left running:

The arms should be in the exact same configuration:

[AntonioConsilvio]

Hi @Woolfrey, perhaps the right wrist still does not behave correctly due to this issue, which has recently been resolved.

Therefore, I kindly ask you to repeat the test and give us feedback on this, thank you.

cc @sgiraz @vigisushrutha23

[vigisushrutha23]

Hi @Woolfrey, perhaps the right wrist still does not behave correctly due to this issue, which has recently been resolved.

Therefore, I kindly ask you to repeat the test and give us feedback on this, thank you.

cc @sgiraz @vigisushrutha23

Awesome. Thanks!! I have booked the robot for the afternoon tomorrow. We will test it and post the results.

[Woolfrey]

@AntonioConsilvio @vigisushrutha23, much better results now.

The left wrist still has better tracking than the right wrist, but compared to before the magnitude of the tracking error is much smaller.

We are still noticing strange behaviour when grasping with 2 hands in Cartesian mode. Part of this problem is likely due to my controller trying to over-correct the hand positions.

Joint Control Mode

Cartesian Control Mode

[sgiraz]

The left wrist still has better tracking than the right wrist, but compared to before the magnitude of the tracking error is much smaller.

We are still noticing strange behaviour when grasping with 2 hands in Cartesian mode. Part of this problem is likely due to my controller trying to over-correct the hand positions.

Hi @Woolfrey,

In light of the above, can we consider the problem solved from our end?
If yes, I proceed to close this issue.

cc @AntonioConsilvio

[Woolfrey]

Hi @sgiraz, @vigisushrutha23 and I ran some more testing today with the 2-hand grasping.

I removed the grasping constraints in my controller that was causing it to over-correct on the wrist tracking error and the motion is similar to what we had before ICRA:

bimanual_test_september.mp4

The wrists still aren't aligned correctly so this makes it difficult to grasp an object with 2 hands. The tracking error is slightly higher than the last time, but still roughly the same magnitude:

The left wrist has a lot of compliance, which is something we have noticed for several months now:

wrist_compliance.mp4

Here is another test we did with code I had written to optimize the arm configuration in Gazebo:

The robot slowly stretches out the right arm.

When running this code on the real robot the wrist rotates around indefinitely (i.e. it is unstable).

stiffness_test_september.mp4

It is possible that my control code needs some refining to work with the real robot, but given the wrist issues this could also be an explanation.

[vigisushrutha23]

Today @mebbaid and i ran some experiments which used a walking controller receiving references from the @Woolfrey bimanual module. We found four joints having significant tracking errors namely l_shoulder_pitch, l_shoulder_roll, r_shoulder_pitch, r_shoulder_roll. The plots of desired vs measured are below.
L_SHOULDER_PITCH

L_SHOULDER_ROLL

R_SHOULDER_PITCH

R_SHOULDER_ROLL

The somewhat similar nature of the errors this time points to the likely problem being low-level PID tuning which we will attempt to do for these four joints.

[xEnVrE]

Can I ask what is the unit of measurement in the plots? Degs or rad?

[vigisushrutha23]

Can I ask what is the unit of measurement in the plots? Degs or rad?

Radians

[vigisushrutha23]

As of repeated testing last week, the only joints that give rise to errors are the 6 of the shoulders but I believe that will be taken care by re calibration of the PID by @mebbaid later this week or the next . So this issue can be closed. I will be opening a new issue for an issue with the left wrist. Thank you all once again for all the fixes.

[sgiraz]

Hi @vigisushrutha23,

Thanks for your feedback. I proceed to close this issue then.

cc @AntonioConsilvio