Add test_metapackages_generation option in generate-conda-packages job #1170

traversaro · 2022-06-14T13:22:35Z

This adds a test_metapackages_generation option to the generate-conda-packages job, so we can check if metapackages are built without the need to create a fake release in a fork.

Furthermore:

Now metapackages are built also on macOS, fixing No robotology-distro and robotology-distro-all metapackages generated for macos and Windows #1158 and robotology-distro-all generation does not work on macOS #1051
It fixes a bug in the metapackages generation. Until now, on releases branches the metapackages were generated using the LatestReleases ROBOTOLOGY_PROJECT_TAGS, instead of using the ROBOTOLOGY_PROJECT_TAGS corresponding to the tag. Instead, the test_metapackages_generation tries to create metapackages using LatestReleases.

traversaro · 2022-06-17T21:24:18Z

First build: https://github.com/robotology/robotology-superbuild/runs/6882624101 .
macOS and Linux are working fine, while windows robotology-distro-all package is failing with error:

2022-06-14T20:23:52.5066539Z Attempting to finalize metadata for robotology-distro-all
2022-06-14T20:23:52.5067302Z   warnings.warn(
2022-06-14T20:23:52.5068219Z INFO:conda_build.metadata:Attempting to finalize metadata for robotology-distro-all
2022-06-14T20:23:53.4192096Z conda-forge/win-64       Using cache
2022-06-14T20:23:53.4262323Z conda-forge/noarch       Using cache
2022-06-14T20:24:00.6978755Z Reloading output folder: C:\Miniconda\envs\test\conda-bld
2022-06-14T20:24:00.8232662Z Reloading output folder: C:\Miniconda\envs\test\conda-bld
2022-06-14T20:24:01.0301704Z BUILD START: ['robotology-distro-all-None-h57928b3_56.tar.bz2']
2022-06-14T20:24:01.0303028Z Reloading output folder: C:\Miniconda\envs\test\conda-bld
2022-06-14T20:24:01.1627778Z Reloading output folder: C:\Miniconda\envs\test\conda-bld
2022-06-14T20:24:02.9208222Z Mamba failed to solve:
2022-06-14T20:24:02.9208600Z  - icub-tests 1.23.0.*
2022-06-14T20:24:02.9208869Z  - funny-things 2.1.0.*
2022-06-14T20:24:02.9209222Z  - bipedal-locomotion-framework 0.6.0.*
2022-06-14T20:24:02.9210477Z  - openvr 1.16.8.1.*
2022-06-14T20:24:02.9210810Z  - matio-cpp 0.2.0.*
2022-06-14T20:24:02.9211102Z  - yarp-openvr-trackers 0.0.1.*
2022-06-14T20:24:02.9211386Z  - human-gazebo 1.0.1.*
2022-06-14T20:24:02.9211633Z  - idyntree-matlab-bindings 5.2.1.*
2022-06-14T20:24:02.9212086Z  - speech 1.2.0.*
2022-06-14T20:24:02.9212518Z  - yarp-cxx 3.7.0.*
2022-06-14T20:24:02.9212838Z  - unicycle-footstep-planner 0.4.1.*
2022-06-14T20:24:02.9213131Z  - whole-body-controllers 2.5.6.*
2022-06-14T20:24:02.9213460Z  - forcetorque-yarp-devices 0.3.0.*
2022-06-14T20:24:02.9213780Z  - blockfactory 0.8.3.*
2022-06-14T20:24:02.9214032Z  - whole-body-estimators 0.7.0.*
2022-06-14T20:24:02.9214353Z  - wearables 1.4.0.*
2022-06-14T20:24:02.9214669Z  - icub-main 1.25.0.*
2022-06-14T20:24:02.9214956Z  - icub-models 1.24.1.*
2022-06-14T20:24:02.9215218Z  - matlab-whole-body-simulator 3.0.0.*
2022-06-14T20:24:02.9215699Z  - robometry 1.1.0.*
2022-06-14T20:24:02.9216398Z  - yarp-telemetry 0.5.1.*
2022-06-14T20:24:02.9216708Z  - icub-basic-demos 1.21.0.*
2022-06-14T20:24:02.9218116Z  - qpoases 3.2.1.*
2022-06-14T20:24:02.9218460Z  - walking-teleoperation 1.3.0.*
2022-06-14T20:24:02.9218924Z  - blocktest 2.3.4.*
2022-06-14T20:24:02.9219180Z  - manif 0.0.4.*
2022-06-14T20:24:02.9219576Z  - robot-testing-framework 2.0.1.*
2022-06-14T20:24:02.9219861Z  - osqp-eigen 0.7.0.*
2022-06-14T20:24:02.9220133Z  - meshcat-python 0.3.2.*
2022-06-14T20:24:02.9220397Z  - openxr-sdk 1.0.23.*
2022-06-14T20:24:02.9221149Z  - ycm-cmake-modules 0.14.2.*
2022-06-14T20:24:02.9221553Z  - walking-controllers 0.5.1.*
2022-06-14T20:24:02.9228335Z  - gazebo-yarp-plugins 4.4.0.*
2022-06-14T20:24:02.9228596Z  - casadi 3.5.5.*
2022-06-14T20:24:02.9228891Z  - idyntree 5.2.1.*
2022-06-14T20:24:02.9229173Z  - icub-firmware-shared 1.25.0.*
2022-06-14T20:24:02.9229642Z  - robot-log-visualizer 0.1.5.*
2022-06-14T20:24:02.9229875Z  - wb-toolbox 5.5.0.*
2022-06-14T20:24:02.9230154Z  - lie-group-controllers 0.1.1.*
2022-06-14T20:24:02.9230465Z  - human-dynamics-estimation 2.5.0.*
2022-06-14T20:24:02.9230769Z  - robots-configuration 1.24.0.*
2022-06-14T20:24:02.9231014Z  - idyntree-yarp-tools 0.0.5.*
2022-06-14T20:24:02.9231299Z  - icub-contrib-common 1.19.0.*
2022-06-14T20:24:02.9231585Z  - yarp-matlab-bindings 3.7.0.*
2022-06-14T20:24:02.9231862Z  - osqp-matlab 0.6.2.3.*
2022-06-14T20:24:02.9232101Z  - casadi-matlab-bindings 3.5.5.2.*
2022-06-14T20:24:02.9232386Z  - pyqtconsole 1.2.2.*
2022-06-14T20:24:02.9232638Z  - qhull 2020.2.*
2022-06-14T20:24:02.9233108Z  - yarp-device-openxrheadset 0.0.1.*
2022-06-14T20:24:02.9233392Z  - blocktest-yarp-plugins 1.1.3.*
2022-06-14T20:24:02.9233674Z  - libosqp 0.6.2.*
2022-06-14T20:24:02.9233930Z  - cppad 20220000.4.*
2022-06-14T20:24:02.9234085Z 
2022-06-14T20:24:02.9234148Z with channels:
2022-06-14T20:24:02.9234386Z  - local
2022-06-14T20:24:02.9234622Z  - conda-forge
2022-06-14T20:24:02.9234861Z  - robotology
2022-06-14T20:24:02.9235005Z 
2022-06-14T20:24:02.9235084Z The reported errors are:
2022-06-14T20:24:02.9235386Z - Encountered problems while solving:
2022-06-14T20:24:02.9236137Z -   - package bipedal-locomotion-framework-0.6.0-py37hc3eccb1_56 requires python_abi 3.7.* *_cp37m, but none of the providers can be installed
2022-06-14T20:24:02.9236555Z - 
2022-06-14T20:24:02.9236749Z conda-bld/win-64          
2022-06-14T20:24:02.9237013Z conda-bld/noarch          
2022-06-14T20:24:02.9237280Z robotology/win-64         
2022-06-14T20:24:02.9237542Z robotology/noarch         
2022-06-14T20:24:02.9237760Z conda-bld/win-64          
2022-06-14T20:24:02.9238243Z conda-bld/noarch          
2022-06-14T20:24:02.9238508Z conda-bld/win-64          
2022-06-14T20:24:02.9238766Z conda-bld/noarch          
2022-06-14T20:24:02.9238980Z conda-bld/win-64          
2022-06-14T20:24:02.9239240Z conda-bld/noarch          
2022-06-14T20:24:02.9239498Z conda-bld/win-64          
2022-06-14T20:24:02.9239758Z conda-bld/noarch          
2022-06-14T20:24:02.9331576Z Traceback (most recent call last):
2022-06-14T20:24:02.9335079Z   File "C:\Miniconda\envs\test\lib\site-packages\boa\cli\mambabuild.py", line 123, in mamba_get_install_actions
2022-06-14T20:24:02.9336180Z     solution = solver.solve_for_action(_specs, prefix)
2022-06-14T20:24:02.9336815Z   File "C:\Miniconda\envs\test\lib\site-packages\boa\core\solver.py", line 234, in solve_for_action
2022-06-14T20:24:02.9337537Z     t = self.solve(specs)
2022-06-14T20:24:02.9338033Z   File "C:\Miniconda\envs\test\lib\site-packages\boa\core\solver.py", line 216, in solve
2022-06-14T20:24:02.9338823Z     raise RuntimeError("Solver could not find solution." + error_string)
2022-06-14T20:24:02.9339401Z RuntimeError: Solver could not find solution.Mamba failed to solve:
2022-06-14T20:24:02.9339890Z  - icub-tests 1.23.0.*
2022-06-14T20:24:02.9340499Z  - funny-things 2.1.0.*
2022-06-14T20:24:02.9340938Z  - bipedal-locomotion-framework 0.6.0.*
2022-06-14T20:24:02.9341415Z  - openvr 1.16.8.1.*
2022-06-14T20:24:02.9341847Z  - matio-cpp 0.2.0.*
2022-06-14T20:24:02.9342303Z  - yarp-openvr-trackers 0.0.1.*
2022-06-14T20:24:02.9342709Z  - human-gazebo 1.0.1.*
2022-06-14T20:24:02.9343377Z  - idyntree-matlab-bindings 5.2.1.*
2022-06-14T20:24:02.9344161Z  - speech 1.2.0.*
2022-06-14T20:24:02.9344543Z  - yarp-cxx 3.7.0.*
2022-06-14T20:24:02.9344964Z  - unicycle-footstep-planner 0.4.1.*
2022-06-14T20:24:02.9345376Z  - whole-body-controllers 2.5.6.*
2022-06-14T20:24:02.9345920Z  - forcetorque-yarp-devices 0.3.0.*
2022-06-14T20:24:02.9346376Z  - blockfactory 0.8.3.*
2022-06-14T20:24:02.9346841Z  - whole-body-estimators 0.7.0.*
2022-06-14T20:24:02.9347224Z  - wearables 1.4.0.*
2022-06-14T20:24:02.9347608Z  - icub-main 1.25.0.*
2022-06-14T20:24:02.9348000Z  - icub-models 1.24.1.*
2022-06-14T20:24:02.9348435Z  - matlab-whole-body-simulator 3.0.0.*
2022-06-14T20:24:02.9348861Z  - robometry 1.1.0.*
2022-06-14T20:24:02.9349392Z  - yarp-telemetry 0.5.1.*
2022-06-14T20:24:02.9349823Z  - icub-basic-demos 1.21.0.*
2022-06-14T20:24:02.9350671Z  - qpoases 3.2.1.*
2022-06-14T20:24:02.9351100Z  - walking-teleoperation 1.3.0.*
2022-06-14T20:24:02.9351504Z  - blocktest 2.3.4.*
2022-06-14T20:24:02.9351915Z  - manif 0.0.4.*
2022-06-14T20:24:02.9352543Z  - robot-testing-framework 2.0.1.*
2022-06-14T20:24:02.9353085Z  - osqp-eigen 0.7.0.*
2022-06-14T20:24:02.9353622Z  - meshcat-python 0.3.2.*
2022-06-14T20:24:02.9353999Z  - openxr-sdk 1.0.23.*
2022-06-14T20:24:02.9354570Z  - ycm-cmake-modules 0.14.2.*
2022-06-14T20:24:02.9354933Z  - walking-controllers 0.5.1.*
2022-06-14T20:24:02.9355342Z  - gazebo-yarp-plugins 4.4.0.*
2022-06-14T20:24:02.9355730Z  - casadi 3.5.5.*
2022-06-14T20:24:02.9356197Z  - idyntree 5.2.1.*
2022-06-14T20:24:02.9356545Z  - icub-firmware-shared 1.25.0.*
2022-06-14T20:24:02.9356959Z  - robot-log-visualizer 0.1.5.*
2022-06-14T20:24:02.9357353Z  - wb-toolbox 5.5.0.*
2022-06-14T20:24:02.9357741Z  - lie-group-controllers 0.1.1.*
2022-06-14T20:24:02.9358165Z  - human-dynamics-estimation 2.5.0.*
2022-06-14T20:24:02.9358550Z  - robots-configuration 1.24.0.*
2022-06-14T20:24:02.9358963Z  - idyntree-yarp-tools 0.0.5.*
2022-06-14T20:24:02.9359364Z  - icub-contrib-common 1.19.0.*
2022-06-14T20:24:02.9359770Z  - yarp-matlab-bindings 3.7.0.*
2022-06-14T20:24:02.9360113Z  - osqp-matlab 0.6.2.3.*
2022-06-14T20:24:02.9360518Z  - casadi-matlab-bindings 3.5.5.2.*
2022-06-14T20:24:02.9360917Z  - pyqtconsole 1.2.2.*
2022-06-14T20:24:02.9361282Z  - qhull 2020.2.*
2022-06-14T20:24:02.9361857Z  - yarp-device-openxrheadset 0.0.1.*
2022-06-14T20:24:02.9362261Z  - blocktest-yarp-plugins 1.1.3.*
2022-06-14T20:24:02.9363092Z  - libosqp 0.6.2.*
2022-06-14T20:24:02.9363794Z  - cppad 20220000.4.*
2022-06-14T20:24:02.9365583Z 
2022-06-14T20:24:02.9366084Z with channels:
2022-06-14T20:24:02.9366526Z  - local
2022-06-14T20:24:02.9366884Z  - conda-forge
2022-06-14T20:24:02.9367341Z  - robotology
2022-06-14T20:24:02.9367642Z 
2022-06-14T20:24:02.9367907Z The reported errors are:
2022-06-14T20:24:02.9368557Z - Encountered problems while solving:
2022-06-14T20:24:02.9369586Z -   - package bipedal-locomotion-framework-0.6.0-py37hc3eccb1_56 requires python_abi 3.7.* *_cp37m, but none of the providers can be installed
2022-06-14T20:24:02.9370507Z - 
2022-06-14T20:24:02.9370930Z 
2022-06-14T20:24:02.9371431Z During handling of the above exception, another exception occurred:
2022-06-14T20:24:02.9371802Z 
2022-06-14T20:24:02.9372209Z Traceback (most recent call last):
2022-06-14T20:24:02.9372771Z   File "C:\Miniconda\envs\test\Scripts\conda-mambabuild-script.py", line 9, in <module>
2022-06-14T20:24:02.9373297Z     sys.exit(main())
2022-06-14T20:24:02.9373792Z   File "C:\Miniconda\envs\test\lib\site-packages\boa\cli\mambabuild.py", line 215, in main
2022-06-14T20:24:02.9374673Z     call_conda_build(action, config)
2022-06-14T20:24:02.9375411Z   File "C:\Miniconda\envs\test\lib\site-packages\boa\cli\mambabuild.py", line 187, in call_conda_build
2022-06-14T20:24:02.9376252Z     result = api.build(
2022-06-14T20:24:02.9376803Z   File "C:\Miniconda\envs\test\lib\site-packages\conda_build\api.py", line 186, in build
2022-06-14T20:24:02.9377510Z     return build_tree(
2022-06-14T20:24:02.9378221Z   File "C:\Miniconda\envs\test\lib\site-packages\conda_build\build.py", line 3088, in build_tree
2022-06-14T20:24:02.9378788Z     packages_from_this = build(metadata, stats,
2022-06-14T20:24:02.9379543Z   File "C:\Miniconda\envs\test\lib\site-packages\conda_build\build.py", line 2128, in build
2022-06-14T20:24:02.9380269Z     create_build_envs(top_level_pkg, notest)
2022-06-14T20:24:02.9380955Z   File "C:\Miniconda\envs\test\lib\site-packages\conda_build\build.py", line 1985, in create_build_envs
2022-06-14T20:24:02.9381728Z     environ.get_install_actions(m.config.test_prefix,
2022-06-14T20:24:02.9382355Z   File "C:\Miniconda\envs\test\lib\site-packages\boa\cli\mambabuild.py", line 125, in mamba_get_install_actions
2022-06-14T20:24:02.9382949Z     conflict_packages = parse_problems(str(e))
2022-06-14T20:24:02.9383544Z   File "C:\Miniconda\envs\test\lib\site-packages\boa\cli\mambabuild.py", line 40, in parse_problems
2022-06-14T20:24:02.9384036Z     assert words[1] == "package"
2022-06-14T20:24:02.9384609Z AssertionError

I guess the problem could be related to

robotology-superbuild/conda/pure_python_recipe_template/meta.yaml

Line 12 in 6596389

    
           # Due to robot-log-visualizer, drop when conda-forge does not uses Python 3.7 anymore

, i.e. for some packages we do not have Python 3.7 packages.

traversaro · 2022-06-17T21:33:55Z

I guess the problem could be related to

robotology-superbuild/conda/pure_python_recipe_template/meta.yaml

Line 12 in 6596389

# Due to robot-log-visualizer, drop when conda-forge does not uses Python 3.7 anymore

, i.e. for some packages we do not have Python 3.7 packages.

I tried to cleanup some "strange" part of the conda_build_config.yml (see 94cab0b), and I launched a new build: https://github.com/robotology/robotology-superbuild/actions/runs/2518055729 .

traversaro · 2022-06-18T09:32:51Z

I guess the problem could be related to

robotology-superbuild/conda/pure_python_recipe_template/meta.yaml

Line 12 in 6596389

# Due to robot-log-visualizer, drop when conda-forge does not uses Python 3.7 anymore

, i.e. for some packages we do not have Python 3.7 packages.

I tried to cleanup some "strange" part of the conda_build_config.yml (see 94cab0b), and I launched a new build: https://github.com/robotology/robotology-superbuild/actions/runs/2518055729 .

This did the trick! The PR is now ready for review and eventually to be cherry-picked on releases/2022.05 branch for the 2022.05.2 release.

traversaro · 2022-06-22T11:59:40Z

Merging to unblock v2022.05.2 release.

#1170)

traversaro · 2022-06-22T12:00:40Z

Backported to releases/2022.05 in 644a5ac .

traversaro added 3 commits June 14, 2022 15:22

Update generate-conda-packages.yaml

a9c082e

Update generate-conda-packages.yaml

39ea24d

Update generate-conda-packages.yaml

598ea77

Update conda_build_config.yml

94cab0b

traversaro requested review from Nicogene and RiccardoGrieco June 18, 2022 09:34

traversaro mentioned this pull request Jun 18, 2022

Do not pin multisheller, mamba and boa versions in Generate Conda Packages CI jobs #1173

Merged

traversaro merged commit 595e04c into master Jun 22, 2022

traversaro deleted the testmetapackages branch June 22, 2022 11:59

traversaro added a commit that referenced this pull request Jun 22, 2022

Add test_metapackages_generation option in generate-conda-packages job (

644a5ac

#1170)

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add test_metapackages_generation option in generate-conda-packages job #1170

Add test_metapackages_generation option in generate-conda-packages job #1170

traversaro commented Jun 14, 2022 •

edited

Loading

traversaro commented Jun 17, 2022

traversaro commented Jun 17, 2022

traversaro commented Jun 18, 2022 •

edited

Loading

traversaro commented Jun 22, 2022

traversaro commented Jun 22, 2022

Add test_metapackages_generation option in generate-conda-packages job #1170

Add test_metapackages_generation option in generate-conda-packages job #1170

Conversation

traversaro commented Jun 14, 2022 • edited Loading

traversaro commented Jun 17, 2022

traversaro commented Jun 17, 2022

traversaro commented Jun 18, 2022 • edited Loading

traversaro commented Jun 22, 2022

traversaro commented Jun 22, 2022

traversaro commented Jun 14, 2022 •

edited

Loading

traversaro commented Jun 18, 2022 •

edited

Loading