Resolving system dependency conflicts (meta)

[stonier]

FCL & Dependencies

Strategy is to hide symbols in FCL & CCD. Octomap will use the same version as that in ROS (1.8.1) as opposed to system. Most users are likely on the same version, if this changes, then we can rethink/hide symbols there. Note that Drake only uses octomap indirectly via FCL.

• ccd : send patch upstream and merged for symbol hiding (libccd/pull/39)
• ccd : wait for patch to be accepted
• fcl : send patch upstream and merged for symbol hiding (fcl/pull/233)
• fcl bugfixing : reported a discovered bug upstream, but does not affect us directly (fcl/issues/244)
• ccd/fcl/octomap : drake update for the holy trinity (drake/pull/7417)

YAML

• Drop back to system yaml, (drake/pull/7352)
• Await system protobuf integration (drake/pull/7323)
• Await crosstool fixes for Mac ([drake/pull/7404](Add /usr/local/include to Bazel header path during CMake builds #7404))
• Resubmit PR (see revert PR, drake/pull/7602)

VTK

• Privatise and do not install vtk_utils.h, (drake/pull/7258)

PCL Enabling

• eigen : workaround PCLConfig.cmake issues with Eigen, (shambhala#38) (drake/pull/7435)(shambhala/pull/65)
• vtk : resolve a vtk linking bug that occurs when compiling against drake & pcl (drake/pull/7434)

Protobuf

• initial idea and working notes (drake/pull/7278)
• solve matlab 2017a problems
• await homebrew python fixes, Python support on Mac #7440
• fall back to system protobuf 2.6 (drake/pull/7323)
• decouple drake from protobuf (optional)

More detailed notes in this spreadsheet.

@nuclearsandwich, @j-rivero, @clalancette can you add comments as things get resolved, or new items are brought to the fore. I'll update this list as we go.

[jwnimmer-tri]

FYI I reduced the permissions on the "spreadsheet" link above from "Anyone with the link can edit" to "Anyone with the link can view". Otherwise, the document would turn into a w4r3z site. We may have to add back individually-authenticated commentors or editors later to compensate.

[stonier]

Thanks

[j-rivero]

Thanks for the summary @stonier, the list matches all that I have in my list.

One question: for the cases of ccd and and fcl, I understand that we want to port patches (submit pull requests) to internal TRI forks. Could you please point me to the location of these forks?.

FYI I reduced the permissions on the "spreadsheet" link above from "Anyone with the link can edit" to "Anyone with the link can view". Otherwise, the document would turn into a w4r3z site. We may have to add back individually-authenticated commentors or editors later to compensate.

Thanks Jeremy, I added most of us to the list of people that can edit.

[stonier]

From the WORKSPACE file, it looks as if we are actually not currently using forks of fcl nor libccd, just hashes from the original projects. They both certainly are getting upstream patches.

libccd

We are in fact using a very old tag of libccd (2014). There has been very little code changes since then - just one commit I think. The rest have been in CMake upgrade and it looks like @jamiesnape has already started to do some symbol hiding. Diff between tip and v2.0 here. Since it already has some logic for exports in tip, we should probably try and upgrade Drake to use that and do whatever patching is necessary on top of that to hide the symbols in Drake.

• @j-rivero : check that Drake builds from tip
• @j-rivero : if that works, create a PR that points to tip of the RobotLocomotion/libccd fork I just created
• @avalenzu : to follow up and make sure that the contact collisions still work with the PR
• @j-rivero : if necessary, drop a patch on the fork for hiding symbols, update the PR, and upstream the patch

Sound like a plan?

FCL

It has a fork in RobotLocomotion/fcl now.

[j-rivero]

Sound like a plan?

Yep. Working on it.

[j-rivero]

@stonier the pull request should be ready at pull request #7417

The fcl cherry pick for your own fork is at RobotLocomotion/fcl#1

[stonier]

Some updates to the above.

Yaml

Yaml is blocking on system protobuf getting in because of a Mac issue on crosstool not having /usr/local/include which indirectly itself causes issues with the protobuf 3.1 currently being used. Refer to the causal chain in the first post.

Protobuf

Pending on a solution to adopting homebrew's python. @jamiesnape can you raise that issue and start discussion on how to solve this ASAP?

[jamiesnape]

Pending on a solution to adopting homebrew's python. @jamiesnape can you raise that issue and start discussion on how to solve this ASAP?

#7440

[jamiesnape]

We are in fact using a very old tag of libccd (2014). There has been very little code changes since then - just one commit I think. The rest have been in CMake upgrade and it looks like @jamiesnape has already started to do some symbol hiding. Diff between tip and v2.0 here. Since it already has some logic for exports in tip, we should probably try and upgrade Drake to use that and do whatever patching is necessary on top of that to hide the symbols in Drake.

You should be able to extend slightly (with say the boilerplate from the GCC wiki) all the logic I added to hide everything that you need to and keep their three build systems working rather than trying to do so with CMake-specific logic per danfis/libccd#39. Not using GenerateExportHeader was deliberate given the requirements of that project, though maybe that changed.

[nuclearsandwich]

#7453 excludes the yaml_util header from the installation without reverting back to system yaml-cpp. We probably still want to re-introduce the system yaml-cpp changes once the build on mac stabilizes.

[j-rivero]

Not using GenerateExportHeader was deliberate given the requirements of that project, though maybe that changed.

To pass all the CI tests (they are using cmake, autotools and Makefile) you are right, we can not rely only in the GenerateExportHeaders of cmake. I've changed the PR to use an static file for the exports and patch Makefile and autotools. All tests passes now.

[jamiesnape]

Looks like libccd merged the patch.

[jwnimmer-tri]

I think this might be finished? At least, the checkbox list in the top post shows that everything has merged.

[nuclearsandwich]

Every dependency with an identified conflict was resolved. So yes, unless/until new issues are reported I think this could be closed.

[jwnimmer-tri]

Thanks! This was very exciting to see happen, it should be a nice improvement in compatibility. I will close for now but we can re-open and/or file a new issue if we find we need other improvements.