Optical Tactile Sensor Plugin #229

mcres · 2020-06-26T12:09:27Z

Optical Tactile Sensor Plugin for GSoC 2020.

It retrieves the contacts given by a contact sensor and computes the normal surfaces of an object given the data returned by a Depth Camera. It also visualizes these forces in a similar way as Gazebo Classic, so this PR partially addresses #112.

Future work will be focused on:

Tutorial explaining the plugin.
Computation performance.
Merging the data returned by contact and camera sensors.

Signed-off-by: Martiño Crespo <marticres@gmail.com>

mcres · 2020-06-26T12:10:32Z

CC @mabelzhang

src/systems/optical_tactile_plugin/OpticalTactilePlugin.cc

mabelzhang

Thank you for the PR! I did a first pass. Mostly performance comments and style. Some of the style comments you should see if you run a linter. colcon test --packages-select ignition-gazebo4 probably has some. The checks at the bottom of the PR should also tell you style problems. If you click on the fail ones, they should take you to Jenkins and give you the diffs of new style errors.

examples/worlds/optical_tactile_sensor_plugin.sdf

src/systems/optical_tactile_plugin/OpticalTactilePlugin.hh

src/systems/optical_tactile_plugin/OpticalTactilePlugin.cc

Signed-off-by: Martiño Crespo <marticres@gmail.com>

chapulina · 2020-10-14T22:51:50Z

@mabelzhang / @azeey : both of you have requests for changes. Would it be possible to do another review? Thanks!

mabelzhang · 2020-10-16T22:23:05Z

Looks like this comment is unresolved: #229 (comment)
Mac CI is still failing.
Those are the only remaining things for this PR.

mabelzhang · 2020-10-16T22:42:10Z

@osrf-jenkins run tests

mabelzhang · 2020-10-16T23:10:46Z

Mac CI has a new warning https://build.osrfoundation.org/job/ignition_gazebo-ci-pr_any-homebrew-amd64/3994/

OpticalTactilePlugin.cc:529
expression result unused

That should be an easy fix.

Test failures:
ServerRepeat/DiffDriveTest.PublishCmdCustomTopics/0
ServerRepeat/DiffDriveTest.SkidPublishCmd/0
INTEGRATION_physics_system.test_ran

@chapulina are these flakey by any chance? The reference build 3993 has a different set of failing tests, so I can't really tell. Looks like the badge linked from osrf/buildfarmer has yet a different set of failing tests ha...

chapulina · 2020-10-17T00:04:46Z

are these flakey by any chance?

We don't have many builds on ign-gazebo4 yet, but luckily we have enough to see these 2 are flaking:

But the NonDefaultCanonicalLink test that segfaulted has been passing on ign-gazebo4:

https://build.osrfoundation.org/job/ignition_gazebo-ci-ign-gazebo4-homebrew-amd64/3/testReport/junit/(root)/PhysicsSystemFixture/NonDefaultCanonicalLink/history/

And ign-gazebo3:

https://build.osrfoundation.org/job/ignition_gazebo-ci-ign-gazebo3-homebrew-amd64/68/testReport/junit/(root)/PhysicsSystemFixture/NonDefaultCanonicalLink/history/

It may be worth rerunning just to see if it wasn't a spurious failure.

mabelzhang · 2020-10-17T00:28:30Z

Ah, cool, took me a few minutes to figure out the correct order to click on History to get to those pages you linked. Nice graphs. Jenkins could use better UI.

Ubuntu CI has one failure,
ContactSystemTest.MultipleCollisionsAsContactSensors, but looks like it's failed once before
https://build.osrfoundation.org/job/ignition_gazebo-ci-pr_any-ubuntu_auto-amd64/4316/testReport/(root)/ContactSystemTest/MultipleCollisionsAsContactSensors/history/

mabelzhang · 2020-10-17T00:28:37Z

@osrf-jenkins run tests

mabelzhang · 2020-10-19T17:30:02Z

Grabbing the results before Jenkins shuts down today.

Ubuntu CI:
https://build.osrfoundation.org/job/ignition_gazebo-ci-pr_any-ubuntu_auto-amd64/4319/
Test Result (2 failures / +1)
ContactSystemTest.MultipleCollisionsAsContactSensors
EntityComponentManagerPerfrormance.Each

I am not sure if the ContactSystem test came from this PR... I saw another failure before 4316, and I think the earlier one is not from this PR. I didn't write down the number.

/var/lib/jenkins/workspace/ignition_gazebo-ci-pr_any-ubuntu_auto-amd64/ign-gazebo/test/integration/contact_system.cc:118
Value of: contactMsgs.size()
  Actual: 3
Expected: 0u
Which is: 0

Mac CI:
https://build.osrfoundation.org/job/ignition_gazebo-ci-pr_any-homebrew-amd64/3997/
There's a seg fault still.
Test Result (2 failures / +1)
PeerTracker.PeerTrackerStale
UNIT_Server_TEST.test_ran

So looks like the ones from 3994 above were flaky.
These new ones did not fail in 3994. PeerTrackerStale failed for both 3997 and 3998. So these look flaky.

Signed-off-by: Mabel Zhang <mabel@openrobotics.org>

azeey

LGTM! Left just a couple of minor comments. Feel free to address them in the follow up PR.

azeey · 2020-11-06T20:39:52Z

src/systems/optical_tactile_plugin/Visualization.cc

+    _normalForce * this->forceLength, normalForceOrientationFromSensor);
+
+  normalForcePoseFromWorld =
+    (normalForcePoseFromSensor + this->depthCameraOffset) + _sensorWorldPose;


Consider using Pose3f::operator* instead of Pose3f::operator+ since the latter is discouraged. If you do, note that you'll have to swap the order of operations

azeey · 2020-11-06T20:41:30Z

src/systems/optical_tactile_plugin/Visualization.cc

+
+//////////////////////////////////////////////////
+OpticalTactilePluginVisualization::OpticalTactilePluginVisualization(
+  std::string &_modelName,


Should these be const refs?

…tactile Fix Mac CI warning

mabelzhang · 2020-12-11T06:39:05Z

ignition_gazebo-ci-pr_any-homebrew-amd64 is still reporting some failures:

The following tests FAILED:
	 79 - INTEGRATION_joint_state_publisher_system (SEGFAULT)
	 83 - INTEGRATION_level_manager (Subprocess aborted)

The first one I don't see anywhere else.

Related to the second one, level manager had some related failures, on
LevelManagerPerfrormance.LevelVsNoLevel https://build.osrfoundation.org/job/ignition_gazebo-ci-ign-gazebo4-homebrew-amd64/4/
and INTEGRATION_level_manager_runtime_performers.test_ran https://build.osrfoundation.org/job/ignition_gazebo-ci-ign-gazebo4-homebrew-amd64/3/

ignition_gazebo-ci-pr_any-ubuntu_auto-amd64's 2 out of 3 failures are expected (LogSystemTest.LogResources and SdfInclude.DownloadFromFuel).
This one failed but shouldn't: ServerRepeat/VelocityControlTest.PublishCmd/0

However, seeing that all the code changes in this PR are strictly within a new directory src/systems/optical_tactile_plugin, I really don't think it should affect any of the tests at all...
Will rerun and see if these fail again.

mabelzhang · 2020-12-11T06:39:19Z

@osrf-jenkins run tests

mabelzhang · 2020-12-11T07:52:53Z

Alright, those two on homebrew-amd64 were flaky. Not failing on the rerun. Rerun fails on INTEGRATION_breadcrumbs which is flaky.

ubuntu_auto-amd64 still failing on ServerRepeat/VelocityControlTest.PublishCmd/0, which is unexpected.
Looks like it could be a precision problem, but it's not comparing >=, it wants >.

123: /var/lib/jenkins/workspace/ignition_gazebo-ci-pr_any-ubuntu_auto-amd64/ign-gazebo/test/integration/velocity_control_system.cc:128: Failure
123: Expected: (poses[i].Pos().X()) > (poses[i-1].Pos().X()), actual: 2.12913e-10 vs 2.13556e-10
123: /var/lib/jenkins/workspace/ignition_gazebo-ci-pr_any-ubuntu_auto-amd64/ign-gazebo/test/integration/velocity_control_system.cc:129: Failure
123: Expected: (poses[i].Pos().Y()) > (poses[i-1].Pos().Y()), actual: 2 vs 2

However, the previous run had 4 unexpected comparison outcomes, the latest one only has 2. So it could just be flaky.
I will rerun one more time to be careful.

mabelzhang · 2020-12-11T07:53:19Z

@osrf-jenkins run tests

mabelzhang · 2020-12-11T18:18:55Z

It looks like ServerRepeat/VelocityControlTest.PublishCmd/0 on ubuntu_auto-amd64 is really failing just for this PR. All other tests are passing. Need to look into it.

mabelzhang · 2020-12-12T12:36:39Z

Huh, that test is failing elsewhere too
https://build.osrfoundation.org/job/ignition_gazebo-ci-pr_any-ubuntu_auto-amd64/4751/testReport/
https://build.osrfoundation.org/job/ignition_gazebo-ci-pr_any-ubuntu_auto-amd64/4752/testReport/
https://build.osrfoundation.org/job/ignition_gazebo-ci-pr_any-ubuntu_auto-amd64/4755/testReport/

I think it's flaky.
That is the last test failure. I'm ready to approve this. Might wait for a few more runs on this test to see if it fails more to be sure.

mabelzhang · 2020-12-12T13:10:07Z

I looked again, the 3 failures of the same test above come from 3 different PRs, so it's definitely flaky. Merging this.

adlarkin · 2020-12-14T15:34:38Z

@mabelzhang was this just a regular merge? It looks like we forgot to squash the PR into one commit before merging. Here's the diff between ign-gazebo4 and a branch I created just before thus was merged: adlarkin/4.1.0...ign-gazebo4

mabelzhang · 2020-12-14T17:44:12Z

Yes just a regular merge, sorry I wasn't sure what was the preferred method of merge and just went with the default. Noted for the future.

mcres added 14 commits June 4, 2020 15:37

Initial commit

0aca10d

Signed-off-by: Martiño Crespo <marticres@gmail.com>

Added plugin files

222d65b

Signed-off-by: Martiño Crespo <marticres@gmail.com>

Initial .sdf file

7be1771

Signed-off-by: Martiño Crespo <marticres@gmail.com>

Pipeline for getting sensor contacts

c3d91fb

Signed-off-by: Martiño Crespo <marticres@gmail.com>

Added visualization of postion and forces

16a268e

Signed-off-by: Martiño Crespo <marticres@gmail.com>

Added plugin for Transform Control

5cd5d5f

Signed-off-by: Martiño Crespo <marticres@gmail.com>

Added initial interpolation

cd58ae6

Signed-off-by: Martiño Crespo <marticres@gmail.com>

Code check

40a87ef

Signed-off-by: Martiño Crespo <marticres@gmail.com>

Update info after specified milliseconds

5b85a48

Signed-off-by: Martiño Crespo <marticres@gmail.com>

Set marker lifetime for better performance

93918eb

Signed-off-by: Martiño Crespo <marticres@gmail.com>

Added Depth Camera

d480c54

Signed-off-by: Martiño Crespo <marticres@gmail.com>

Added callback and unpacking to Depth Camera messages

3774efc

Signed-off-by: Martiño Crespo <marticres@gmail.com>

Compute and visualize normal forces

2bab9c8

Signed-off-by: Martiño Crespo <marticres@gmail.com>

Merge master

6c32ba0

Signed-off-by: Martiño Crespo <marticres@gmail.com>

mcres requested a review from chapulina as a code owner June 26, 2020 12:09

traversaro mentioned this pull request Jun 26, 2020

skin emulation robotology/gazebo-yarp-plugins#55

Open

mabelzhang self-requested a review June 26, 2020 18:12

mcres mentioned this pull request Jun 26, 2020

Visualize contacts #112

Closed

azeey requested changes Jun 29, 2020

View reviewed changes

src/systems/optical_tactile_plugin/OpticalTactilePlugin.cc Outdated Show resolved Hide resolved

chapulina added the 🔮 dome Ignition Dome label Jun 30, 2020

mabelzhang requested changes Jun 30, 2020

View reviewed changes

azeey reviewed Jun 30, 2020

View reviewed changes

src/systems/optical_tactile_plugin/OpticalTactilePlugin.cc Outdated Show resolved Hide resolved

mcres added 3 commits July 9, 2020 11:46

PR Feedback 1

badd572

Signed-off-by: Martiño Crespo <marticres@gmail.com>

Update sdf example to a more realistic environment

70774f4

Signed-off-by: Martiño Crespo <marticres@gmail.com>

Allow moving Depth Camera from model origin

1652081

Signed-off-by: Martiño Crespo <marticres@gmail.com>

mabelzhang mentioned this pull request Oct 28, 2020

Add functionalities for optical tactile plugin #431

Merged

fix mac warning

16807b6

Signed-off-by: Mabel Zhang <mabel@openrobotics.org>

azeey approved these changes Nov 6, 2020

View reviewed changes

mcres and others added 3 commits December 1, 2020 17:33

Merge pull request #2 from ignitionrobotics/mabelzhang/patch_optical_…

4a47c92

…tactile Fix Mac CI warning

Merge branch 'ign-gazebo4' into optical_tactile_plugin

703a916

Merge branch 'ign-gazebo4' into optical_tactile_plugin

af74f4c

mabelzhang approved these changes Dec 12, 2020

View reviewed changes

mabelzhang merged commit 05e2ff6 into gazebosim:ign-gazebo4 Dec 12, 2020

adlarkin mentioned this pull request Dec 14, 2020

4.1.0 #485

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Optical Tactile Sensor Plugin #229

Optical Tactile Sensor Plugin #229

mcres commented Jun 26, 2020 •

edited

Loading

mcres commented Jun 26, 2020

mabelzhang left a comment

chapulina commented Oct 14, 2020

mabelzhang commented Oct 16, 2020

mabelzhang commented Oct 16, 2020

mabelzhang commented Oct 16, 2020

chapulina commented Oct 17, 2020

mabelzhang commented Oct 17, 2020

mabelzhang commented Oct 17, 2020

mabelzhang commented Oct 19, 2020

azeey left a comment

azeey Nov 6, 2020

azeey Nov 6, 2020

mabelzhang commented Dec 11, 2020

mabelzhang commented Dec 11, 2020

mabelzhang commented Dec 11, 2020

mabelzhang commented Dec 11, 2020

mabelzhang commented Dec 11, 2020

mabelzhang commented Dec 12, 2020

mabelzhang commented Dec 12, 2020

adlarkin commented Dec 14, 2020

mabelzhang commented Dec 14, 2020

Optical Tactile Sensor Plugin #229

Optical Tactile Sensor Plugin #229

Conversation

mcres commented Jun 26, 2020 • edited Loading

mcres commented Jun 26, 2020

mabelzhang left a comment

Choose a reason for hiding this comment

chapulina commented Oct 14, 2020

mabelzhang commented Oct 16, 2020

mabelzhang commented Oct 16, 2020

mabelzhang commented Oct 16, 2020

chapulina commented Oct 17, 2020

mabelzhang commented Oct 17, 2020

mabelzhang commented Oct 17, 2020

mabelzhang commented Oct 19, 2020

azeey left a comment

Choose a reason for hiding this comment

azeey Nov 6, 2020

Choose a reason for hiding this comment

azeey Nov 6, 2020

Choose a reason for hiding this comment

mabelzhang commented Dec 11, 2020

mabelzhang commented Dec 11, 2020

mabelzhang commented Dec 11, 2020

mabelzhang commented Dec 11, 2020

mabelzhang commented Dec 11, 2020

mabelzhang commented Dec 12, 2020

mabelzhang commented Dec 12, 2020

adlarkin commented Dec 14, 2020

mabelzhang commented Dec 14, 2020

mcres commented Jun 26, 2020 •

edited

Loading