Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add build with apt dependencies from Ubuntu 24.04 to docker-build CI #1642

Merged
merged 7 commits into from
May 2, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 13 additions & 5 deletions .ci/install_debian.sh
Original file line number Diff line number Diff line change
Expand Up @@ -18,17 +18,25 @@ source ${SCRIPT_DIR}/../scripts/install_apt_dependencies.sh
# Python
source ${SCRIPT_DIR}/../scripts/install_apt_python_dependencies.sh

# Octave
apt-get install -y liboctave-dev

# Gazebo Classic
lsb_dist="$(. /etc/os-release && echo "$ID")"
dist_version="$(lsb_release -c | cut -d: -f2 | sed s/'^\t'//)"
echo "lsb_dist: ${lsb_dist}"
echo "dist_version: ${dist_version}"

# Octave
if [[ ("focal" == "$dist_version" || "buster" == "$dist_version" || "bullseye" == "$dist_version") ]]; then
apt-get install -y liboctave-dev
else
apt-get install -y octave-dev
fi

# Gazebo Classic
# Just a limited amount of distros are supported by OSRF repos, for all the other we use the
# gazebo packages in regular repos
if [[ ("focal" == "$dist_version" || "buster" == "$dist_version") ]]; then
if [[ ("noble" == "$dist_version")]]; then
# There is no Gazebo Classic package for Noble
echo ""
elif [[ ("focal" == "$dist_version" || "buster" == "$dist_version") ]]; then
mkdir -p /etc/apt/sources.list.d
echo deb http://packages.osrfoundation.org/gazebo/$lsb_dist\-stable $dist_version main > /etc/apt/sources.list.d/gazebo-stable.list
apt-key adv --keyserver keyserver.ubuntu.com --recv-keys D2486D2DD83DB69272AFE98867170598AF249743
Expand Down
7 changes: 7 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -195,6 +195,7 @@ jobs:
docker_image:
- "ubuntu:focal"
- "ubuntu:jammy"
- "ubuntu:24.04"
- "debian:bullseye"

project_tags:
Expand Down Expand Up @@ -256,6 +257,12 @@ jobs:
cd build
cmake -DROBOTOLOGY_USES_MUJOCO:BOOL=OFF .

- name: Disable Gazebo Classic support for distro without Gazebo Classic binaries [Docker ubuntu:24.04]
if: (matrix.docker_image == 'ubuntu:24.04')
run: |
cd build
cmake -DROBOTOLOGY_USES_GAZEBO:BOOL=OFF .

- name: Build [Docker]
run: |
cd build
Expand Down
1 change: 0 additions & 1 deletion CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -66,7 +66,6 @@ include(YCMEPHelper)
# depending on which profiles are enabled
include(RobotologySuperbuildLogic)


if(ROBOTOLOGY_USES_MATLAB)
# The following line is to properly configure the matlab and simulink software installed by the superbuild
configure_file(${CMAKE_CURRENT_SOURCE_DIR}/cmake/template/startup_robotology_superbuild.m.in ${CMAKE_BINARY_DIR}/startup_robotology_superbuild.m)
Expand Down
6 changes: 4 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -128,7 +128,7 @@ If instead you use an older distro in which the default version of CMake is olde

For some [profile](doc/cmake-options.md#profile-cmake-options) or [dependency](doc/cmake-options.md#dependencies-cmake-options) specific CMake option you may need to install additional system dependencies, following the dependency-specific documentation listed in the following. If you do not want to enable an option, you should ignore the corresponding section and continue with the installation process.

Note that the `ROBOTOLOGY_USES_GAZEBO` option is enabled by default, so you should install Gazebo Classic unless you plan to disable this option.
Note that the `ROBOTOLOGY_USES_GAZEBO` option is enabled by default (except on Ubuntu 24.04 when installing with apt dependencies), so you should install Gazebo Classic unless you plan to disable this option.

#### `ROBOTOLOGY_USES_GAZEBO`

Expand All @@ -142,6 +142,8 @@ Otherwise, if you are on other supported Debian/Ubuntu systems, just install the
sudo apt install libgazebo-dev
~~~~

If you are on Ubuntu 24.04, please use conda if you want to install Gazebo Classic, as no Gazebo Classic packages are available via apt.

#### `ROBOTOLOGY_USES_GZ`

To install Modern Gazebo (gz-sim) on Ubuntu Jammy (22.04) and other supported Debian/Ubuntu systems, follow the instructions available at https://gazebosim.org/docs/harmonic/install_ubuntu#binary-installation-on-ubuntu .
Expand All @@ -165,7 +167,7 @@ sudo apt install libpcl-dev

Install octave and the necessary development files using the following command:
~~~
sudo apt-get install liboctave-dev
sudo apt-get install octave-dev
~~~


Expand Down
5 changes: 5 additions & 0 deletions cmake/BuildICUB.cmake
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,11 @@ else()
set(ICUB_PYTHON_INSTALL_CMAKE_ARGS "")
endif()

if(ROBOTOLOGY_SUPERBUILD_USING_LOCAL_SWIG_4_2_1_WORKAROUND_ON_NOBLE)
list(APPEND ICUB_PYTHON_INSTALL_CMAKE_ARGS "-DSWIG_EXECUTABLE=${robotology_superbuild_local_noble_swig_4_2_1_SOURCE_DIR}/bin/swig")
list(APPEND ICUB_PYTHON_INSTALL_CMAKE_ARGS "-DSWIG_DIR=${robotology_superbuild_local_noble_swig_4_2_1_SOURCE_DIR}/share/swig/4.2.1")
endif()

ycm_ep_helper(ICUB TYPE GIT
STYLE GITHUB
REPOSITORY robotology/icub-main.git
Expand Down
10 changes: 10 additions & 0 deletions cmake/BuildYARP.cmake
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,16 @@ if(ROBOTOLOGY_CONFIGURING_UNDER_CONDA)
list(APPEND YARP_OPTIONAL_CMAKE_ARGS "-DYARP_COMPILE_yarpviz:BOOL=OFF")
endif()

if(ROBOTOLOGY_SUPERBUILD_USING_LOCAL_SWIG_4_2_1_WORKAROUND_ON_NOBLE)
list(APPEND YARP_OPTIONAL_CMAKE_ARGS "-DSWIG_EXECUTABLE=${robotology_superbuild_local_noble_swig_4_2_1_SOURCE_DIR}/bin/swig")
list(APPEND YARP_OPTIONAL_CMAKE_ARGS "-DSWIG_DIR=${robotology_superbuild_local_noble_swig_4_2_1_SOURCE_DIR}/share/swig/4.2.1")
endif()

if(LSB_RELEASE_CODENAME STREQUAL "noble" AND NOT ROBOTOLOGY_CONFIGURING_UNDER_CONDA)
# Workaround for https://github.com/robotology/yarp/pull/3108
list(APPEND YARP_OPTIONAL_CMAKE_ARGS "-DYARP_USE_Lua=OFF")
endif()

ycm_ep_helper(YARP TYPE GIT
STYLE GITHUB
REPOSITORY robotology/yarp.git
Expand Down
13 changes: 12 additions & 1 deletion cmake/RobSupPurePythonYCMEPHelper.cmake
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,17 @@ function(ROB_SUP_PURE_PYTHON_YCM_EP_HELPER _name)
set(_PYH_${_name}_PYTHON_PACKAGE_NAME ${_name})
endif()

# We pass --break-system-packages as it is required by Python 3.12 when operating outside a virtual env,
# even if effectively we are kind in a virtual env defined by the superbuild in this case
# We need to make sure not to pass it for Python <= 3.11, as otherwise pip uninstall will fail as --break-system-packages
# is not an option
if(LSB_RELEASE_CODENAME STREQUAL "noble" AND NOT ROBOTOLOGY_CONFIGURING_UNDER_CONDA)
set(BREAK_SYSTEM_PACKAGES_OPTION "--break-system-packages")
else()
set(BREAK_SYSTEM_PACKAGES_OPTION "")
endif()


ycm_ep_helper(${_name} TYPE GIT
STYLE GITHUB
REPOSITORY ${_PYH_${_name}_REPOSITORY}
Expand All @@ -54,6 +65,6 @@ function(ROB_SUP_PURE_PYTHON_YCM_EP_HELPER _name)
# See https://stackoverflow.com/questions/55708589/how-to-pass-an-environment-variable-to-externalproject-add-configure-command
# See https://github.com/robotology/robotology-superbuild/issues/1118
# To avoid the complexity of handling two commands, we just use the build step to uninstall any existing package
BUILD_COMMAND ${CMAKE_COMMAND} -E env PYTHONPATH=${YCM_EP_INSTALL_DIR}/${ROBOTOLOGY_SUPERBUILD_PYTHON_INSTALL_DIR} pip uninstall -y ${_PYH_${_name}_PYTHON_PACKAGE_NAME}
BUILD_COMMAND ${CMAKE_COMMAND} -E env PYTHONPATH=${YCM_EP_INSTALL_DIR}/${ROBOTOLOGY_SUPERBUILD_PYTHON_INSTALL_DIR} pip uninstall ${BREAK_SYSTEM_PACKAGES_OPTION} -y ${_PYH_${_name}_PYTHON_PACKAGE_NAME}
INSTALL_COMMAND ${Python3_EXECUTABLE} -m pip install --upgrade --no-deps --target=${YCM_EP_INSTALL_DIR}/${ROBOTOLOGY_SUPERBUILD_PYTHON_INSTALL_DIR} -VV <SOURCE_DIR>)
endfunction()
27 changes: 27 additions & 0 deletions cmake/RobotologySuperbuildLogic.cmake
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,33 @@ else()
set(ROBOTOLOGY_BUILD_tomlplusplus ON)
endif()

# I (@traversaro) am not particularly happy about this, but Ubuntu 24.04
# got released with a version of swig (4.2.0) that contained several bugs.
# The bugs were reported and fixed by swig authors (a huge thanks), and released
# as swig 4.2.1 but the Ubuntu 24.04 sync with Debian happened exactly when
# swig 4.2.0 was packaged in Debian, so now Ubuntu 24.04 contains a swig
# that does not work with YARP. As swig is a tool just used as build time,
# as workaround just for Ubuntu 24.04 with apt dependencies we download
# a custom build of swig 4.2.1 locally
if(LSB_RELEASE_CODENAME STREQUAL "noble" AND NOT ROBOTOLOGY_CONFIGURING_UNDER_CONDA)
include(FetchContent)
FetchContent_Declare(
robotology_superbuild_local_noble_swig_4_2_1
URL "https://github.com/robotology/robotology-vcpkg-ports/releases/download/storage/swig_4_2_1_ubuntu_24_04.zip")

FetchContent_GetProperties(robotology_superbuild_local_noble_swig_4_2_1)
if(NOT robotology_superbuild_local_noble_swig_4_2_1_POPULATED)
message(STATUS "Downloading robotology_superbuild_local_noble_swig_4_2_1 binaries.")
FetchContent_Populate(robotology_superbuild_local_noble_swig_4_2_1)
message(STATUS "Downloaded a local copy of swig 4.2.1 in ${robotology_superbuild_local_noble_swig_4_2_1_SOURCE_DIR}")
endif()
set(ROBOTOLOGY_SUPERBUILD_USING_LOCAL_SWIG_4_2_1_WORKAROUND_ON_NOBLE ON)
list(APPEND YARP_OPTIONAL_CMAKE_ARGS "-DSWIG_EXECUTABLE=${robotology_superbuild_local_noble_swig_4_2_1_SOURCE_DIR}/bin/swig")
list(APPEND YARP_OPTIONAL_CMAKE_ARGS "-DSWIG_DIR=${robotology_superbuild_local_noble_swig_4_2_1_SOURCE_DIR}/share/swig/4.2.1")
else()
set(ROBOTOLOGY_SUPERBUILD_USING_LOCAL_SWIG_4_2_1_WORKAROUND_ON_NOBLE OFF)
endif()

# Core
if(ROBOTOLOGY_ENABLE_CORE)
find_or_build_package(YARP)
Expand Down
18 changes: 17 additions & 1 deletion cmake/RobotologySuperbuildOptions.cmake
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,23 @@ option(ROBOTOLOGY_USES_CSHARP "Enable compilation of software that depends on CS
mark_as_advanced(ROBOTOLOGY_USES_CSHARP)

## Enable packages that depend on the Gazebo Classic simulator
option(ROBOTOLOGY_USES_GAZEBO "Enable compilation of software that depends on Gazebo Classic" ON)
if(ROBOTOLOGY_CONFIGURING_UNDER_CONDA OR APPLE OR WIN32)
set(ROBOTOLOGY_USES_GAZEBO_DEFAULT ON)
else()
find_program(ROBSUB_LSB_RELEASE lsb_release)
if(ROBSUB_LSB_RELEASE)
execute_process(COMMAND lsb_release -cs
OUTPUT_VARIABLE LSB_RELEASE_CODENAME
OUTPUT_STRIP_TRAILING_WHITESPACE
)
endif()
if(LSB_RELEASE_CODENAME STREQUAL "noble")
set(ROBOTOLOGY_USES_GAZEBO_DEFAULT OFF)
else()
set(ROBOTOLOGY_USES_GAZEBO_DEFAULT ON)
endif()
endif()
option(ROBOTOLOGY_USES_GAZEBO "Enable compilation of software that depends on Gazebo Classic" ${ROBOTOLOGY_USES_GAZEBO_DEFAULT})
option(ROBOTOLOGY_USES_PCL_AND_VTK "Enable compilation of software that depends on PCL and VTK" OFF)
option(ROBOTOLOGY_USES_MUJOCO "Enable compilation of mujoco and software that depends on it" OFF)

Expand Down
4 changes: 2 additions & 2 deletions doc/cmake-options.md
Original file line number Diff line number Diff line change
Expand Up @@ -96,7 +96,7 @@ Not all options are supported on all platforms. The following table provides a r
| `ROBOTOLOGY_USES_PCL_AND_VTK` | ✔️ | ✔️ | ✔️ | ✔️ | ✔️ |
| `ROBOTOLOGY_USES_IGNITION` | ❌ | ❌ | ✔️ | ❌ | ❌ |
| `ROBOTOLOGY_USES_MATLAB` | ✔️ | ❌ | ✔️ | ✔️ | ✔️ |
| `ROBOTOLOGY_USES_OCTAVE` | ✔️ | ❌ | ❌ | ) ❌ | ❌ |
| `ROBOTOLOGY_USES_OCTAVE`<sup id="a3">[3!](#f3)</sup> | ✔️ | ❌ | ❌ | ) ❌ | ❌ |
| `ROBOTOLOGY_USES_PYTHON` | ✔️ | ❌ | ✔️ | ✔️ | ✔️ |
| `ROBOTOLOGY_USES_CFW2CAN` | ✔️ | ❌ | ✔️ | ❌ | ❌ |
| `ROBOTOLOGY_USES_XSENS_MVN_SDK` | ❌ | ✔️ | ❌ | ❌ | ❌ |
Expand All @@ -105,7 +105,7 @@ Not all options are supported on all platforms. The following table provides a r

<b id="f1">1!</b>:`ROBOTOLOGY_USES_MUJOCO` does not support building with apt dependencies on Debian or Ubuntu distributions older than 2022. Furthermore, it does not support build on Windows with Visual Studio 2019, it requires Visual Studio 2022.
<b id="f2">2!</b>:`ROBOTOLOGY_ENABLE_ROBOT_TESTING` and `ROBOTOLOGY_ENABLE_ICUB_HEAD` do not support building with conda-forge dependencies on Apple Silicon.

<b id="f3">3!</b>:`ROBOTOLOGY_USES_OCTAVE` do not support building with apt dependencies on Ubuntu 20.04.


Profile-specific documentation
Expand Down
Loading