Test/scripts/build-info: include runner-distro in info.json (HMS-5350)
#1152
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
While working on replacing manifest-db with images build cache, I've run into a situation when the ability to download the image artifact after the downloading of build info metadata would be useful. Specifically when tests are run in batches, when one needs to first determine the full list of test cases and then download image artifacts only for the relevant test cases. Due to the size of image artifacts, downloading all of them every time is not really feasible as it takes non-trivial amount of time. The idea is to be able to download image artifact from S3 after build metadata have been already downloaded. The `info.json` already contains almost all information necessary to construct the S3 path to download, except for the runner distro. Let's add the `runner-distro` property to `info.json`, so that we can have a script that reads `info.json` and is able to download the complete build cache for the specific build. Lastly, we determine the need to rebuild image on the runner-distro, osbuild-ref and manifest-id values. Therefore it makes sense, that two builds done on different runner distro would have different `info.json`, which was not the case until now. Signed-off-by: Tomáš Hozza <thozza@redhat.com>
schutzbot
changed the title
info.jsoninfo.json (HMS-5350)
achilleas-k
approved these changes
Jan 20, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
While working on replacing manifest-db with images build cache, I've run into a situation when the ability to download the image artifact after the downloading of build info metadata would be useful. Specifically when tests are run in batches, when one needs to first determine the full list of test cases and then download image artifacts only for the relevant test cases.
Due to the size of image artifacts, downloading all of them every time is not really feasible as it takes non-trivial amount of time.
The idea is to be able to download image artifact from S3 after build metadata have been already downloaded. The
info.jsonalready contains almost all information necessary to construct the S3 path to download, except for the runner distro.Let's add the
runner-distroproperty toinfo.json, so that we can have a script that readsinfo.jsonand is able to download the complete build cache for the specific build.Lastly, we determine the need to rebuild image on the runner-distro, osbuild-ref and manifest-id values. Therefore it makes sense, that two builds done on different runner distro would have different
info.json, which was not the case until now.EDIT:
I understand that this change will not ensure that the current build cache
info.jsonfiles will contain the new value. I'd update osbuild-ref once this gets merged to ensure it is there. Determining the runner distro from theSchutzfilecan be still used as a reasonable fallback./jira-epic COMPOSER-2318
JIRA: HMS-5350