Tracker: build Drake inside the ros_buildfarm

[j-rivero]

In order to build maliput packages that depends on Drake inside the ROS buildfarm and distribute them through the ROS repositories, first step is being able to build Drake as a ROS package that can be used for other ROS packages.

Considerations #

• Drake version selected at this moment is Drake 1.12.

  • ❔ drake_vendor in maliput uses Drake 0.27 (Feb 2021) Migrates to drake v0.27.0 drake_vendor#37. Rethink if we want to package a modern version or 0.27

• Bazel is required to build Drake even if there is a CMake wrapper that helps with the installation of the build artifacts.

• Bazel is only supported in Ubuntu starting with Ubuntu Jammy. The target of this work would be the ROS versions supported on Jammy and follow.

• Debhelper flags (such as fortify, O3, etc) don't cause any test failures but it does cause changes in floating-point roundoff so that the user's numeric results would change depending on the release channel. Need to avoid debhelper default flags.

• Drake dependencies:

  • Ubuntu packages for building Drake from source are defined in the source code used by install_prereqs.sh.
  • Drake Debian packaging metadata is available in the debian-drake repository.
  • Drake supports a variety of solvers, at least three are commercial optimization solvers dependencies: SNOPT, MOSEK, and GUROBI. Installation is optional. In order to simplify the works of dealing with commercial code and licenses in the starting phase, the commercial solvers installation is disabled at this point.
  • Drake aims to vendor and statically link all of our dependencies using hidden symbols. Usually the version of the dependencies need not match up other uses of the library within the user's ecosystem.
  • The only exceptions to the design in the point above are for the interface libraries: Eigen, fmt, and spdlog. Drake assume that the user will use the host OS version of the libraries.
  • VTK is built using Dockerfile, upload to S3 and Drake build is using the binaries from S3. Need to find a way of building VTK from source to go into the ROS buildfarm.

• ROS buildfarm restrictions:

  • Code inside the ROS Buildfarm needs to be built under a user account with rootless access .
  • Code downloads in the ROS Buildfarm are allowed at building time although other alternatives are preferred.
  • Binary blobs inside a ROS package code or injected at building time are discouraged and needs to be discussed with the ROS team.

Roadmap and tracking #

Prototype building of Drake 1.12 on Ubuntu Jammy with a rootless user

• Determine the required version of Bazel to build Drake selected version #2
  • Need Bazel 4.0 to build drake-1.12
  • Problem with Ubuntu Jammy Bazel version: 3.5.1.
    • Previous Drake version to bump Bazel for 4.0 is Drake v0.31.0. Probably too old to be of interest.
    • Try to bootstrap Bazel 4.0 rootless using Bazel 3.5.1
• Try to build Bazel (4.0) in a rootless environment on Ubuntu Jammy #3
  • 🧪 Success Bootstrap Bazel 4.0 on Ubuntu Jammy using bazel-bootstrap in docker #4
  • ⌚ Around 21 min in Intel(R) Core(TM) i5-4308U CPU @ 2.80GHz old MacMini.
• Check if Drake 1.12 is able to be built without install_prereqs.sh and Bazel 4.0 bootstraped rootless #5
  • ⚒️ WIP: Dockerfile to build drake-1.12 rootless on Jammy #6

Implement Drake 1.12 as a ROS package

• Review and document licenses of Drake components
• Move Bazel 4.0 bootstrap to CMake
• Figure out how to deal with dependencies, specially VTK
• Build the package.xml file
• Bloom locally

[jwnimmer-tri]

Thanks for the write-up!

FYI on some notes:

Drake Debian packaging metadata is available in the debian-drake repository.

Oops! That repository is outdated, but we forgot to mark it as such (see RobotLocomotion/debian-drake#3). Mea culpa.

Our https://drake.mit.edu/apt.html binaries are created using the drake/tools/release_engineering scripts now, following the https://drake.mit.edu/release_playbook.html.

The bottom line is that we make a release archive (tgz) file using the CMake install, and then we run alien on it. Trying to use dpkg-buildpackage was a lot of hassle with little benefit.

Drake supports a variety of solvers, at least three are commercial optimization solvers dependencies: SNOPT, MOSEK, and GUROBI. Installation is optional.

Drake has obtained a license grant to enable MOSEK during the build, and that grant transfers to anyone else redistributing Drake as well.

We're working on doing the same for Gurobi (RobotLocomotion/drake#10804).

(I agree that it makes sense to keep those turned off during the initial testing phase here.)

Drake aims to vendor and statically link all of our dependencies using hidden symbols. Usually the version of the dependencies need not match up other uses of the library within the user's ecosystem. The only exceptions to the design in the point above are for the interface libraries: Eigen, fmt, and spdlog.

Exactly right.

Drake assume that the user will use the host OS version of the libraries.

Drake defaults to using the host OS version, but you can change it to use your own build via CMake options:

    -DWITH_USER_EIGEN:BOOLEAN=ON
    -DWITH_USER_FMT:BOOLEAN=ON
    -DWITH_USER_SPDLOG:BOOLEAN=ON

... in which case Drake will use find_package() for those dependencies.

VTK is built using Dockerfile, upload to S3 and Drake build is using the binaries from S3. Need to find a way of building VTK from source to go into the ROS buildfarm.

As of v1.11.0 when running on Jammy, Drake uses the host OS VTK; see RobotLocomotion/drake#18340.

When we upgrade to VTK 10, we might switch back binary downloads, though. If it helps, we might be able to preserve an option to rebuild VTK from source.

Code downloads in the ROS Buildfarm are allowed at building time although other alternatives are preferred.

Drake fetches most of its externals' source code via downloads (with mirrors), but there are ways to populate that cache ahead of time and/or disable source downloads (in which case pre-population would become mandatory).