[migration-tools] Always release source version with updated release increment instead of last_version. #32112
Conversation
…t_version. When performing the Rolling platform migration several repositories had bloom releases which were not committed to the distribution file at the time of the source ref. As a result the `last_version` entries for those repositories did not match the release version in the source distribution during migration. Since the migration tool was using the last_release in order to handle any releases that were not hard coded to a specific version and/or release tag this created a split between what was performed by git-bloom-release during migration and what was configured in the resulting distribution file. One example is the ros2_control repository which had a 2.2.0-1 release made in ros2-gbp/ros2_control-release@7504ac3 but which never received a rosdistro PR. Thus the migration tool released 2.2.0-2 based on the last_release information in the tracks file but the source distribution was anticipating 2.1.0-2. Of the changes in this commit I am most hesitant about the release_increment handling since this script will eventually generate and force push updated release artifacts and if in a similar situation a 2.1.0-2 was released but not published in the source distribution file its artifacts would be clobbered where they overlapped with the migration-tool's own 2.1.0-2. However I don't have a method of resolving that situation without searching each release repository's artifacts for a full list of release increments associated with a version.
migration-tools/migrate-rosdistro.py
Outdated
|
|
||
| if dest_track['release_tag'] == ':{ask}' and 'last_release' in dest_track: | ||
| # Override the version for this release to guarantee the same version is released. | ||
| dest_track['release_tag_saved'] = dest_track['release_tag'] | ||
| dest_track['release_tag'] = dest_track['last_release'] | ||
| write_tracks_file(tracks, f'Update {args.dest} track to release exactly last-released tag.') | ||
|
|
||
| # Update release increment for the upcoming release. | ||
| release_inc = str(int(source_inc) + 1) |
There was a problem hiding this comment.
Rather than overwriting the tag, I'd opt for a release inc which is "newer than both". It should have the same effect without any force pushing.
(I haven't tested this code)
| release_inc = str(int(source_inc) + 1) | |
| release_inc = str(max(int(source_inc), int(dest_track['release_inc'])) + 1) |
There was a problem hiding this comment.
+1 in case there was a missed push somewhere.
There was a problem hiding this comment.
There are still cases this solution wouldn't cover with multiple uncommitted releases.
Using ros2_control as an example again. If our source distribution was at version 2.1.0-1 and the release repository contained tagged but unpublished in rosdistro releases 2.1.0-2 and 2.2.0-1, then the migration tool would see the last release 2.2.0 and release increment "1" in the tracks file and create a release 2.1.0-2 from
release_inc = str(max(int("1"), int("1")) + 1)which would clobber the upstream 2.1.0-2.
This is what led me to think that the only reliable way to prevent clobbering unreleased tags is to scan for all tags matching the target release version to pick a strictly greater release increment.
There was a problem hiding this comment.
I see - good catch. Well it's still an improvement, no?
There was a problem hiding this comment.
Without a doubt.
The question hidden in my comment was Should we take the improvement and continue the migration or push for a solution that handles the further edge case?
There was a problem hiding this comment.
I don't think that there's any current repositories with that specific scenario so I would be inclined to take your improvement and proceed with the migration. But I'll add a comment documenting the additional edge case and try to revisit it before we pull this out again for Humble.
There was a problem hiding this comment.
I agree with your analysis - we can't do better without scanning the tags, which is a bit more work and IMO sort-of reaches further into the repositories than I think the migration tooling should.
+1 to move forward.
There was a problem hiding this comment.
Updated in d80eb24.
…urrent release increment. Reduce the likelihood that the migration will clobber existing release tags by incrementing the greater between the source distribution increment and the current release increment in the Bloom track. There are still situations where a release could be clobbered which has been documented but left unhandled. Co-authored-by: Scott K Logan <logans@cottsay.net>
nuclearsandwich
changed the title
nuclearsandwich
deleted the
migration-tools/source-distribution-version
branch
Some repositories in this migration were affected by the issue described in #32112 Now that the release version handling has been improved we can retry migrating those repositories by removing the release version as the script would for a failed bloom release.
Some repositories in this migration were affected by the issue described in #32112 Now that the release version handling has been improved we can retry migrating those repositories by removing the release version as the script would for a failed bloom release.
* Remove repositories in preparation for migration. * Update target platforms for Rolling. * Update releases after platform migration. Migration command: ``` python3 migration-tools/migrate-rosdistro.py --source rolling --dest rolling \ --source-ref 7eafe04 \ --release-org ros2-gbp \ ``` Of 299 repositories, 22 failed to bloom successfully. This is most frequently caused by missing rosdep dependencies. The following packages failed to bloom: * aws-robomaker-small-warehouse-world * control_box_rst * fmi_adapter * fmilibrary_vendor * gazebo_ros2_control * gazebo_ros_pkgs * ign_rviz * ignition_cmake2_vendor * ignition_math6_vendor * locator_ros_bridge * moveit * ntpd_driver * ompl * rmw_connextdds * rmw_gurumdds * ros_ign * rosidl_typesupport_gurumdds * rtabmap * sdformat_urdf * usb_cam * warehouse_ros_mongo * webots_ros2 * Pull releases for repositories with mismatched versions. Some repositories in this migration were affected by the issue described in #32112 Now that the release version handling has been improved we can retry migrating those repositories by removing the release version as the script would for a failed bloom release. * Update releases from additional packages after migration tools fix. Using the migration-tool version from 38741a4 Migration command: ``` python3 migration-tools/migrate-rosdistro.py --source rolling --dest rolling \ --source-ref 7eafe04 \ --release-org ros2-gbp \ ``` These packages did not build (as expected from previous runs) * aws-robomaker-small-warehouse-world * control_box_rst * fmi_adapter * fmilibrary_vendor * gazebo_ros2_control * gazebo_ros_pkgs * ign_rviz * ignition_cmake2_vendor * ignition_math6_vendor * locator_ros_bridge * moveit * ntpd_driver * ompl * rmw_connextdds * rmw_gurumdds * ros_ign * rosidl_typesupport_gurumdds * rtabmap * sdformat_urdf * usb_cam * warehouse_ros_mongo * webots_ros2
When performing the Rolling platform migration several repositories had bloom releases which were not committed to the distribution file at the time of the source ref.
As a result the
last_versionentries for those repositories did not match the release version in the source distribution during migration.Since the migration tool was using the last_release in order to handle any releases that were not hard coded to a specifc version and/or release tag this created a split between what was performed by git-bloom-release during migration and what was configured in the resulting distribution file.
One example is the ros2_control repository which had a 2.2.0-1 release made in ros2-gbp/ros2_control-release@7504ac3 but which never received a rosdistro PR.
Thus the migration tool released 2.2.0-2 based on the last_release information in the tracks file but the source distribution was anticipating 2.1.0-2.
Of the changes in this commit I am most hesitant about the release_increment handling since this script will eventually generate and force push updated release artifacts and if in a similar situation a 2.1.0-2 was released but not published in the source distribution file its artifacts would be clobbered where they overlapped with the migration-tool's own 2.1.0-2.
However I don't have a method of resolving that situation without searching each release repository's artifacts for a full list of release increments associated with a version.