Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Gazebo11/Ignition Citadel Debian Buster and Ubuntu Focal #75

Merged
merged 1 commit into from
May 14, 2020

Conversation

j-rivero
Copy link
Contributor

For both, Noetic (ROS1) and Foxy (ROS2) we need to import gazebo11 packages together with all its dependencies. I added two files here to import all the needed files from packages.osrfoundation.org.

Some packages are affected by gazebo-tooling/release-tools#186 and don't have -dbg packages. I think this should not stop us from starting the upload in order to unblock the gazebo_ros_pkgs porting, testing and release.

/cc @chapulina

@ruffsl
Copy link

ruffsl commented May 4, 2020

Can this be merged now? See ros-simulation/gazebo_ros_pkgs#1093 (comment)

@nuclearsandwich
Copy link
Contributor

@tfoote what's the deployment plan for this PR. Are we going to pull these directly into the ROS / ROS 2 repos or pull them into the bootstrap repo and propagate them that way?

@nuclearsandwich
Copy link
Contributor

There is discussion germane to this PR in #72

@ruffsl
Copy link

ruffsl commented May 8, 2020

@nuclearsandwich , I'm woefully ignorant of how the sausage is made here, but from the discussion you linked, the intent of this addition should be redirected to an/the aptly-backed bootstrap repository?

@nuclearsandwich
Copy link
Contributor

the intent of this addition should be redirected to an/the aptly-backed bootstrap repository?

That's what I'm trying to determine. I could see us still reviewing and merging PRs here as documentation but it's not going to affect anything for the ROS build farms.

@sloretz
Copy link

sloretz commented May 12, 2020

I'm woefully ignorant of how the sausage is made here

ditto. @j-rivero what should I be doing to help move this forward?

@tfoote
Copy link
Member

tfoote commented May 14, 2020

We need to manually import these into the bootstrap repository via aptly. I think @nuclearsandwich and I need to simplify that process and document how.

@nuclearsandwich
Copy link
Contributor

Here's what I would propose: If this is the change that we want to make let's merge the PR here as a statement of intent if nothing else, and then I'll affect the change on the aptly-based bootstrap repo with notes that we can turn into documentation.

@tfoote
Copy link
Member

tfoote commented May 14, 2020

That works for me. We should also do the same for #70

@tfoote tfoote merged commit 00b5685 into master May 14, 2020
@tfoote tfoote deleted the import_gazebo11_buster_focal branch May 14, 2020 18:23
@chapulina
Copy link
Contributor

We should also do the same for #70

And for all of Ignition Citadel 😇

@nuclearsandwich
Copy link
Contributor

nuclearsandwich commented May 14, 2020

I needed to create the aptly mirror for focal, the buster one was already there.

aptly mirror create gazebo_ubuntu_stable-focal http://packages.osrfoundation.org/gazebo/ubuntu-stable focal main`

Update the mirrors

aptly mirror update gazebo_debian_stable-buster
aptly mirror update gazebo_ubuntu_stable-focal

aptly's package query format is based on reprepro's but of course keeping it the same would be too difficult. Some sed violence gets us an arg list in the right format.

echo 'Package (% gazebo11-common ) | Package (% gazebo11-dbg ) | Package (% gazebo11-doc ) | Package (% gazebo11-plugin-base ) | Package (% gazebo11 ) | Package (% libgazebo11-dev ) | Package (% libgazebo11 ) | Package (% libignition-common3-av-dev ) | Package (% libignition-common3-core-dev ) | Package (% libignition-common3-events-dev ) | Package (% libignition-common3-graphics-dev ) | Package (% libignition-common3-profiler-dev ) | Package (% libignition-transport8-dev ) | Package (% libignition-transport8-core-dev ) | Package (% libignition-transport8-log-dev ) | Package (% libignition-transport8-log ) | Package (% libignition-transport8-dbg ) | Package (% libignition-fuel-tools4-dev ) | Package (% libignition-fuel-tools4 ) | Package (% libignition-math6-dbg ) | Package (% libignition-math6-dev ) | Package (% libignition-math6 ) | Package (% libignition-msgs5-dbg ) | Package (% libignition-msgs5-dev ) | Package (% libignition-msgs5 ) | Package (% libsdformat9-dbg ) | Package (% libsdformat9-dev ) | Package (% libsdformat9 ) | Package (% sdformat9-sdf ) | Package (% sdformat9-doc ) | Package (% sdformat9-dbg )' | sed -E 's:Package \(% ([^)]+) \):\1:g' > buster-query  
echo 'Package (% gazebo11-common ) | Package (% gazebo11-dbg ) | Package (% gazebo11-doc ) | Package (% gazebo11-plugin-base ) | Package (% gazebo11 ) | Package (% libgazebo11-dev ) | Package (% libgazebo11 ) | Package (% libignition-common3-av-dev ) | Package (% libignition-common3-core-dev ) | Package (% libignition-common3-events-dev ) | Package (% libignition-common3-graphics-dev ) | Package (% libignition-common3-profiler-dev ) | Package (% libignition-transport8-dev ) | Package (% libignition-transport8-core-dev ) | Package (% libignition-transport8-log-dev ) | Package (% libignition-transport8-log ) | Package (% libignition-transport8-dbg ) | Package (% libignition-fuel-tools4-dev ) | Package (% libignition-fuel-tools4 ) | Package (% libignition-math6-dbg ) | Package (% libignition-math6-dev ) | Package (% libignition-math6 ) | Package (% libignition-msgs5-dbg ) | Package (% libignition-msgs5-dev ) | Package (% libignition-msgs5 ) | Package (% libsdformat9-dbg ) | Package (% libsdformat9-dev ) | Package (% libsdformat9 ) | Package (% sdformat9-sdf ) | Package (% sdformat9-doc ) | Package (% sdformat9-dbg )'| sed -E 's:Package \(% ([^)]+) \):\1:g' > focal-query
 aptly repo import gazebo_debian_stable-buster ros_bootstrap-buster "$(cat buster-query)"
Loading packages...
[o] libignition-fuel-tools4-dev_4.1.0-1~buster_arm64 imported
[o] sdformat9-sdf_9.1.0-1~buster_all imported
[o] libignition-common3-profiler-dev_3.5.0-1~buster_amd64 imported
[o] libignition-math6_6.4.0-1~buster_amd64 imported
[o] libignition-msgs5-dev_5.1.0-3~buster_arm64 imported
[o] libignition-math6-dev_6.4.0-1~buster_arm64 imported
[o] libignition-common3-events-dev_3.5.0-1~buster_amd64 imported
[o] libgazebo11-dev_11.0.0-2~buster_amd64 imported
[o] libignition-fuel-tools4_4.1.0-1~buster_amd64 imported
[o] libignition-msgs5_5.1.0-3~buster_arm64 imported
[o] libignition-common3-core-dev_3.5.0-1~buster_amd64 imported
[o] libignition-math6-dbg_6.4.0-1~buster_amd64 imported
[o] libgazebo11_11.0.0-2~buster_amd64 imported
[o] libignition-transport8-dbg_8.0.0-1~buster_arm64 imported
[o] libignition-common3-graphics-dev_3.5.0-1~buster_arm64 imported
[o] sdformat9-doc_9.1.0-1~buster_all imported
[o] libignition-math6-dbg_6.4.0-1~buster_arm64 imported
[o] gazebo11-dbg_11.0.0-2~buster_amd64 imported
[o] libignition-common3-events-dev_3.5.0-1~buster_arm64 imported
[o] libignition-transport8-log-dev_8.0.0-1~buster_amd64 imported
[o] libignition-fuel-tools4-dev_4.1.0-1~buster_amd64 imported
[o] libignition-fuel-tools4_4.1.0-1~buster_arm64 imported
[o] libignition-transport8-dbg_8.0.0-1~buster_amd64 imported
[o] libignition-common3-graphics-dev_3.5.0-1~buster_amd64 imported
[o] libignition-msgs5_5.1.0-3~buster_amd64 imported
[o] libignition-math6-dev_6.4.0-1~buster_amd64 imported
[o] libignition-msgs5-dev_5.1.0-3~buster_amd64 imported
[o] libignition-transport8-log_8.0.0-1~buster_arm64 imported
[o] libignition-math6_6.4.0-1~buster_arm64 imported
[o] libignition-common3-av-dev_3.5.0-1~buster_arm64 imported
[o] libsdformat9_9.1.0-1~buster_amd64 imported
[o] gazebo11-plugin-base_11.0.0-2~buster_amd64 imported
[o] libignition-transport8-dev_8.0.0-1~buster_arm64 imported
[o] gazebo11-common_11.0.0-2~buster_all imported
[o] gazebo11_11.0.0-2~buster_source imported
[o] libignition-common3-core-dev_3.5.0-1~buster_arm64 imported
[o] gazebo11-doc_11.0.0-2~buster_all imported
[o] libsdformat9-dev_9.1.0-1~buster_amd64 imported
[o] gazebo11_11.0.0-2~buster_amd64 imported
[o] libignition-transport8-core-dev_8.0.0-1~buster_amd64 imported
[o] libsdformat9-dbg_9.1.0-1~buster_amd64 imported
[o] libignition-transport8-core-dev_8.0.0-1~buster_arm64 imported
[o] libignition-common3-av-dev_3.5.0-1~buster_amd64 imported
[o] libignition-transport8-log_8.0.0-1~buster_amd64 imported
[o] libignition-transport8-log-dev_8.0.0-1~buster_arm64 imported
[o] libignition-common3-profiler-dev_3.5.0-1~buster_arm64 imported
[o] libignition-transport8-dev_8.0.0-1~buster_amd64 imported
$ aptly repo import gazebo_ubuntu_stable-focal ros_bootstrap-focal "$(cat focal-query)"
Loading packages...
[o] libignition-msgs5_5.1.0-1~focal_amd64 imported
[o] libgazebo11-dev_11.0.0-2~focal_amd64 imported
[o] libignition-msgs5-dev_5.1.0-1~focal_amd64 imported
[o] libignition-transport8-log-dev_8.0.0-1~focal_amd64 imported
[o] libignition-fuel-tools4-dev_4.1.0-1~focal_amd64 imported
[o] gazebo11_11.0.0-2~focal_amd64 imported
[o] libignition-math6-dbg_6.4.0-1~focal_amd64 imported
[o] gazebo11-common_11.0.0-2~focal_all imported
[o] libsdformat9-dev_9.2.0-1~focal_amd64 imported
[o] gazebo11-plugin-base_11.0.0-2~focal_amd64 imported
[o] gazebo11-dbg_11.0.0-2~focal_amd64 imported
[o] libsdformat9-dbg_9.2.0-1~focal_amd64 imported
[o] libignition-transport8-log_8.0.0-1~focal_amd64 imported
[o] libignition-transport8-dbg_8.0.0-1~focal_amd64 imported
[o] gazebo11-doc_11.0.0-2~focal_all imported
[o] libignition-common3-av-dev_3.5.0-1~focal_amd64 imported
[o] libsdformat9_9.2.0-1~focal_amd64 imported
[o] libignition-transport8-dev_8.0.0-1~focal_amd64 imported
[o] libignition-common3-core-dev_3.5.0-1~focal_amd64 imported
[o] libignition-common3-graphics-dev_3.5.0-1~focal_amd64 imported
[o] sdformat9-sdf_9.2.0-1~focal_all imported
[o] libignition-common3-events-dev_3.5.0-1~focal_amd64 imported
[o] libignition-math6_6.4.0-1~focal_amd64 imported
[o] libignition-transport8-core-dev_8.0.0-1~focal_amd64 imported
[o] sdformat9-doc_9.2.0-1~focal_all imported
[o] libignition-common3-profiler-dev_3.5.0-1~focal_amd64 imported
[o] libignition-fuel-tools4_4.1.0-1~focal_amd64 imported
[o] libgazebo11_11.0.0-2~focal_amd64 imported
[o] libignition-math6-dev_6.4.0-1~focal_amd64 imported

Snapshot and publish the ros_bootstrap repositories.

bin/snapshot-and-publish-all ros_bootstrap

Now the import jobs on the buildfarms need to run to pick these up.

@nuclearsandwich
Copy link
Contributor

We should also do the same for #70

I imported the ifcfg packages when that PR was reviewed. Claire and I must have forgotten to merge this PR after discussing the issue on the rosdep key PR.

@nuclearsandwich
Copy link
Contributor

@j-rivero the package list above doesn't appear to produce an installable state for gazebo11 on Focal.

The following packages have unmet dependencies:
 libgazebo11 : Depends: libignition-common3 (>= 3.4.0) but it is not installable
               Depends: libignition-fuel-tools4 (>= 4.1.0) but it is not going to be installed
               Depends: libignition-transport8 (>= 8.0.0) but it is not installable

@chapulina
Copy link
Contributor

the package list above doesn't appear to produce an installable state for gazebo11 on Focal.

@j-rivero , since we'll need to update this to pull some Ignition dependencies, how about just pulling all Citadel at once? I think we may also need to pull DART and Ogre2.

@nuclearsandwich
Copy link
Contributor

I think we may also need to pull DART and Ogre2.

Are either of these libraries upstream and used by ROS packages because pulling in alternates could cause serious conflicts.

@chapulina
Copy link
Contributor

Are either of these libraries upstream and used by ROS packages because pulling in alternates could cause serious conflicts.

@ruffsl
Copy link

ruffsl commented May 14, 2020

Can gazebo use the version of libogre that ships with focal and or buster?

Keeping with system provided dependencies would be nice,

@chapulina
Copy link
Contributor

Can gazebo use the version of libogre that ships with focal and or buster?

Yes, Gazebo 11 works with Ogre 1.9. For Ogre 1.12, see ros/rosdistro#24448.

Ignition Citadel also works with Ogre 1.9, but in addition to that it supports Ogre 2.1. Since Ignition Rendering is plugin based, users can use the Ogre 1 plugin, the Ogre 2 plugin, or develop their own plugin. We've been releasing debs for both Ogre plugins, using this fork for version 2, which has changes that make it possible to install Ogre 1 and 2 side-by-side:

http://packages.osrfoundation.org/gazebo/ubuntu-stable/pool/main/i/ignition-rendering3/

Even though we support the Ogre 1 plugin, the Ogre 2 integration is superior in rendering quality so that's the recommend engine.

@ruffsl
Copy link

ruffsl commented May 14, 2020

Ogre 2.1

I'm a bit lost, that fork looks +2 years old, but 19 days ago v1.12.6 was recently released:
https://github.com/OGRECave/ogre/releases

version 2, which has changes that make it possible to install Ogre 1 and 2 side-by-side

Would any of this affect rviz's vendoring of ogre?

https://github.com/ros2/rviz/tree/ros2/rviz_ogre_vendor

@ruffsl
Copy link

ruffsl commented May 14, 2020

I'm a bit lost, that fork looks +2 years old, but 19 days ago v1.12.6 was recently released:

Oh, they made a new repo. Both repos seem quite active.

https://github.com/OGRECave/ogre-next/releases

@j-rivero
Copy link
Contributor Author

@j-rivero the package list above doesn't appear to produce an installable state for gazebo11 on Focal.

Ugh. Manually created it was not a good idea. I've wrote some bash lines to get the whole dependency string: gazebo-tooling/gazebodistro#4

@j-rivero , since we'll need to update this to pull some Ignition dependencies, how about just pulling all Citadel at once? I think we may also need to pull DART and Ogre2.

Sounds like the best path.

I don't expect problems with Ogre2, packaging modifies path and should not conflict with anything present in Debian/Ubuntu platforms supported by ROS Noetic/Foxy as far as I can say. Packages for Debian Stretch and Ubuntu Focal are available in our repository.

For DART our Ubuntu Focal binaries are being built against Ubuntu's 6.9.2, so we should be safe here. For Debian Stretch I've generated the same version 6.9.2 for it, it is in our repositories..

Let me prepare the whole list for Citadel in the next minutes with the aptly syntax.

@j-rivero
Copy link
Contributor Author

Let me prepare the whole list for Citadel in the next minutes with the aptly syntax.

Ignition Citadel whole suite list:

ignition-tools | libignition-common3 | libignition-common3-av | libignition-common3-av-dev | libignition-common3-core-dev | libignition-common3-dev | libignition-common3-events | libignition-common3-events-dev | libignition-common3-graphics | libignition-common3-graphics-dev | libignition-common3-profiler | libignition-common3-profiler-dev | libignition-fuel-tools4 | libignition-fuel-tools4-dev | libignition-gazebo3 | libignition-gazebo3-dev | libignition-gazebo3-plugins | libignition-gui3 | libignition-gui3-dev | libignition-launch2 | libignition-math6 | libignition-math6-dev | libignition-math6-eigen3-dev | libignition-msgs5 | libignition-msgs5-dev | libignition-physics2 | libignition-physics2-core-dev | libignition-physics2-dartsim | libignition-physics2-dartsim-dev | libignition-physics2-dev | libignition-physics2-mesh-dev | libignition-physics2-sdf-dev | libignition-plugin | libignition-plugin-dev | libignition-rendering3 | libignition-rendering3-core-dev | libignition-rendering3-dev | libignition-rendering3-ogre1 | libignition-rendering3-ogre1-dev | libignition-rendering3-ogre2 | libignition-rendering3-ogre2-dev | libignition-sensors3 | libignition-sensors3-air-pressure | libignition-sensors3-air-pressure-dev | libignition-sensors3-altimeter | libignition-sensors3-altimeter-dev | libignition-sensors3-camera | libignition-sensors3-camera-dev | libignition-sensors3-core-dev | libignition-sensors3-depth-camera | libignition-sensors3-depth-camera-dev | libignition-sensors3-dev | libignition-sensors3-gpu-lidar | libignition-sensors3-gpu-lidar-dev | libignition-sensors3-imu | libignition-sensors3-imu-dev | libignition-sensors3-lidar | libignition-sensors3-lidar-dev | libignition-sensors3-logical-camera | libignition-sensors3-logical-camera-dev | libignition-sensors3-magnetometer | libignition-sensors3-magnetometer-dev | libignition-sensors3-rendering | libignition-sensors3-rendering-dev | libignition-sensors3-rgbd-camera | libignition-sensors3-rgbd-camera-dev | libignition-sensors3-thermal-camera | libignition-sensors3-thermal-camera-dev | libignition-tools-dev | libignition-transport8 | libignition-transport8-core-dev | libignition-transport8-dev | libignition-transport8-log | libignition-transport8-log-dev | libogre-2.1 | libogre-2.1-dev | libsdformat9 | libsdformat9-dev | sdformat9-sdf 

Mostly the whole list of ignition-* packages + sdformat9 packages + ogre2.1. @nuclearsandwich any problem let me know please.

@j-rivero j-rivero changed the title Add gazebo11 Debian Buster and Ubuntu Focal Add Gazebo11/Ignition Citadel Debian Buster and Ubuntu Focal May 15, 2020
@nuclearsandwich
Copy link
Contributor

nuclearsandwich commented May 15, 2020

Mostly the whole list of ignition-* packages + sdformat9 packages + ogre2.1. @nuclearsandwich any problem let me know please.

I'm going to go ahead and try this but can you open a new PR with the full list here. I haven't talked it over with @tfoote yet but I think we're going to convert these config files to a structure a bit more native to aptly and then start building automation around that.... you know, eventually.

@sloretz
Copy link

sloretz commented May 15, 2020

I didn't check all of citadel, but here's a (slooooooow) bash snippet to get libignition-common3-dev dependencies that come from packages.osrfoundation.org. I see all of them in @j-rivero's list ✔️

$ for pkg in $(apt depends --quiet --recurse --important libignition-common3-dev 2>/dev/null | grep --invert-match -e "^[< ]") ; do if apt policy $pkg 2>/dev/null | grep --silent osrfoundation ; then echo "$pkg" ; fi ; done
libignition-common3-dev
libignition-common3-core-dev
libignition-common3-av-dev
libignition-common3-events-dev
libignition-common3-graphics-dev
libignition-common3-profiler-dev
libignition-common3
libignition-common3-av
libignition-math6-dev
libignition-common3-events
libignition-common3-graphics
libignition-common3-profiler
libignition-math6

@nuclearsandwich
Copy link
Contributor

@j-rivero and I are hammering out some technical issues with the latest import offline but as a side note, I notice that the ign-citadel metapackage that @chapulina tells me exists isn't in the import list. Should it be?

@sloretz
Copy link

sloretz commented May 15, 2020

I notice that the ign-citadel metapackage that @chapulina tells me exists isn't in the import list. Should it be?

Looks like libignition-launch2-dev might also be missing.

Using

for pkg in $(apt depends --no-generate --quiet --recurse --important ignition-citadel 2>/dev/null | grep --invert-match -e "^[< ]") ; do if apt policy --no-generate $pkg 2>/dev/null | grep --silent osrfoundation ; then echo "$pkg" ; fi ; done

The only things I don't see in @j-rivero's list are

{'ignition-citadel', 'libignition-launch2-dev'}

@sloretz
Copy link

sloretz commented May 16, 2020

Unable to install libgazebo11-dev on Debian buster

The following packages have unmet dependencies:
 libgazebo11-dev : Depends: libsdformat9-dev but it is not going to be installed
                   Depends: libignition-math6-dev but it is not going to be installed
                   Depends: libignition-transport8-dev but it is not going to be installed
                   Depends: libignition-msgs5-dev but it is not going to be installed
                   Depends: libignition-fuel-tools4-dev but it is not going to be installed
                   Depends: libignition-common3-dev but it is not going to be installed
                   Depends: libignition-common3-graphics-dev but it is not going to be installed
                   Depends: libignition-common3-profiler-dev but it is not going to be installed
E: Unable to correct problems, you have held broken packages.

@sloretz
Copy link

sloretz commented May 16, 2020

@nuclearsandwich @j-rivero I'm not sure why libgazebo11-dev is uninstallable. It seems like it might be because Buster is missing libignition-cmake2-dev

edit: basing that on aptitude install libgazebo11-dev saying the following

The following packages have unmet dependencies:
 libignition-common3-core-dev : Depends: libignition-cmake2-dev which is a virtual package and is not provided by any available package

 libignition-common3-av-dev : Depends: libignition-cmake2-dev which is a virtual package and is not provided by any available package

 libignition-fuel-tools4-dev : Depends: libignition-cmake2-dev which is a virtual package and is not provided by any available package

 libignition-transport8-log-dev : Depends: libignition-cmake2-dev which is a virtual package and is not provided by any available package

 libignition-transport8-core-dev : Depends: libignition-cmake2-dev which is a virtual package and is not provided by any available package

 libignition-common3-graphics-dev : Depends: libignition-cmake2-dev which is a virtual package and is not provided by any available package

 libignition-math6-dev : Depends: libignition-cmake2-dev which is a virtual package and is not provided by any available package

 libignition-common3-profiler-dev : Depends: libignition-cmake2-dev which is a virtual package and is not provided by any available package

 libignition-msgs5-dev : Depends: libignition-cmake2-dev which is a virtual package and is not provided by any available package

 libignition-common3-events-dev : Depends: libignition-cmake2-dev which is a virtual package and is not provided by any available package

gazebo11 itself is installable on Buster.

nuclearsandwich added a commit that referenced this pull request May 18, 2020
Over the course of deploying #75 there were many more packages that
needed to be imported than those in the list and we added all of
ignition-citadel as a project beyond just gazebo11.

This expands the config files from #75 to include everything that was
actually imported.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants