Tracker: job migration to gazebo_libs.dsl (yaml)

[j-rivero]

The issue stores the information on the migration of the different CI jobs from being hardcoded in DSL and generated by ignition.dsl / ignition_collection.dsl and sdformat.dsl to be stored in a permanent configuration place: gz-collections.yaml (still open for future changes but a yaml file by now) and generated via gazebo_libs.dsl

• Yaml

  • Distribution information
    • Gazebo Citadel Include all gazebo releases in gz-collection.yaml #1017
    • Gazebo Fortress Include all gazebo releases in gz-collection.yaml #1017
    • Gazebo Harmonic
    • Gazebo Ionic
    • upcoming Include all gazebo releases in gz-collection.yaml #1017
  • Type of jobs
    • Linux
      • PR testing gz/ign_${lib}-ci-pr_any-ubuntu_${distro} jobs
      • Branch main CI: gz_${lib}-ci-gz-${branch}-${distro} jobs Change daily Linux ci jobs (-ci-${branch}) to be based on gz-collections.yaml #1047
      • ABi checker jobs gz/ign_${lib}-ci-abichecker-${distro} jobs Change abichecker generation to use gz-collection.yaml #1038
      • ASAN jobs Move ASAN jobs to be generated using gz-collections.yaml #1078
      • Install testing gz/ign_${lib}-install-pkg-${distro} jobs Move -install-pkg- jobs to be generated using gz-collections.yaml #1093
    • Brew
      • PR testing Move brew ci-pr_any- to gazebo_libs #1054
      • Branch CI: Change homebrew -ci- daily jobs to use gz-collections.yaml  #1058
      • Install bottle jobs Implement the install_bottle jobs using gazebo_libs #1096
    • Windows
    • PR testing Migrate Windows jobs for -ci-pr_any- and -ci-${branch}- to use gz-collections.yaml #1071
    • Branch CI Migrate Windows jobs for -ci-pr_any- and -ci-${branch}- to use gz-collections.yaml #1071

• DSL (ignition.dsl / sdformat.dsl)

  • Type of jobs
    • Linux
      • ABI checker testing ign_${lib}-abichecker-any_to_any-ubuntu_auto jobs
        (Reduced to fake jobs for migration in Change abichecker generation to use gz-collection.yaml #1038)
      • Debbuilders

New generated jobs and naming

The new source-of-truth for the new generated Gazebo CI jobs is the data in gz-collections.yaml, transformed using gazebo_libs.dsl. The result of using the DSL code and the job names generated can be obtained checked:

• For human reading: In the DSL jenkins run
• For machine consumption: using WRITE_JOB_LOG=1 java -jar tools/jobdsl.jar *.dsl.

The only ign_ prefix maintained (see gz-collections.yaml) is for the libraries corresponding to Gazebo Citadel. Change everything to use gz_ prefixes in #1083 .

Update branch protection branches to new names

Configure in GitHub the new names in branch protection. List of repositories done:

• gz-cmake

[scpeters]

many of the the install_bottle jobs added for harmonic are failing because we did not create ignition-* aliases in homebrew-simulation:

• https://build.osrfoundation.org/view/os_homebrew/job/ignition_physics7-install_bottle-homebrew-amd64/29/

+ brew install --include-test ignition-physics7
Warning: No available formula with the name "ignition-physics7". Did you mean ignition-physics5 or ignition-physics2?

it would be straightforward to add these aliases but I would avoid adding new ignition-* ones if the DSL is going to be migrated away from ignition- sometime soon

[j-rivero]

The -abichecker- for recent branches are know also to fail until the support is implemented. They probably all run on Focal.

[azeey]

Just FYI, the view created from the .yaml file for a new collection (Ionic in my case) only contains the debbuilders, so it's still necessary to add Ionic to ignition_collection.dsl similar to

release-tools/jenkins-scripts/dsl/ignition_collection.dsl

Line 277 in edbffda

'harmonic' : [

[azeey]

The install jobs for the collection install-pkg jobs (eg. gz-harmonic-install-pkg-jammy are failing because the job tries to install lib-<collection>-dev.

Also, it looks like we are trying to run ASAN on gz-tools, but that package doesn't have any tests that would use ASAN.