Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: remove ocm inception during build CTF aggregation #1065

Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
59 changes: 28 additions & 31 deletions .github/workflows/components.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -89,31 +89,6 @@ jobs:
uses: TooMuch4U/actions-clean@v2.2
- name: Checkout
uses: actions/checkout@v4
- name: Setup Go
uses: actions/setup-go@v5
with:
go-version-file: '${{ github.workspace }}/go.mod'
cache: false
- name: Get go environment for use with cache
run: |
echo "go_cache=$(go env GOCACHE)" >> $GITHUB_ENV
echo "go_modcache=$(go env GOMODCACHE)" >> $GITHUB_ENV
# This step will only reuse the go mod and build cache from main made during the Build,
# see push_ocm.yaml => "ocm-cli-latest" Job
# This means it never caches by itself and PRs cannot cause cache pollution / thrashing
# This is because we have huge storage requirements for our cache because of the mass of dependencies
- name: Restore / Reuse Cache from central build
id: cache-golang-restore
uses: actions/cache/restore@v4 # Only Restore, not build another cache (too big)
with:
path: |
${{ env.go_cache }}
${{ env.go_modcache }}
key: ${{ env.cache_name }}-${{ runner.os }}-go-${{ hashFiles('**/go.sum') }}-${{ hashFiles('**/go.mod') }}
restore-keys: |
${{ env.cache_name }}-${{ runner.os }}-go-
env:
cache_name: ocm-cli-latest-go-cache # needs to be the same key in the end as in the build step
- name: Download CTFs
uses: actions/download-artifact@v4
with:
Expand All @@ -127,15 +102,37 @@ jobs:
mv ${{ github.workspace }}/gen/downloaded-ctfs/ctf-component-${i} ${{ github.workspace }}/gen/${i}/ctf
ls -R ${{ github.workspace }}/gen/${i}
done
- name: Extract OCM Binary from CTF to avoid OCM Inception
id: extract-ocm
run: |
ocm_binary=$(bash ./hack/get_bare_resource_from_ctf.sh \
"ocm.software/ocmcli" \
"" \
"ocmcli" \
$(go env GOARCH) \
$(go env GOOS) \
"application/octet-stream" \
${{ github.workspace }}/gen/ocmcli/ctf)

new_loc=${{ github.workspace }}/bin/ocm
mkdir -p $(dirname $new_loc)
ln -s $ocm_binary $new_loc
chmod +x $new_loc
echo "OCM binary linked to $new_loc"
echo "binary=$new_loc" >> $GITHUB_OUTPUT
- name: Create aggregated CTF
run: |
PATH=$PATH:$(go env GOPATH)/bin \
CTF_TYPE=${{ env.CTF_TYPE }} \
COMPONENTS="${{ env.components }}" \
make plain-ctf
for i in ${{ env.components }}; do
echo "transfering component $i..."
${{ steps.extract-ocm.outputs.binary }} transfer cv \
--type ${{ env.CTF_TYPE }} -V \
${{ github.workspace }}/gen/$i/ctf \
${{ github.workspace }}/gen/ctf
done
- name: Upload aggregated CTF
# only upload the artifact if we are not on a PR
if: needs.pr-check.outputs.number != 'null'
# TODO This is currently permanently disabled,
# until we integrate it with the release build, in which it would be reused
if: false
uses: actions/upload-artifact@v4
with:
if-no-files-found: error
Expand Down
59 changes: 59 additions & 0 deletions hack/get_bare_resource_from_ctf.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
#!/bin/bash

set -e

# This script is used to get a bare resource from a CTF file.
# It can be used in case the OCM CLI is not available to extract the resource from a CTF.
# A typical use case for this is the "OCM Inception" in which a CTF containing the CLI needs to be extracted
# to run the CLI to extract the resource.
#
# In this case one can use this script to extract the correct OCM CLI without having to rely on the CLI being
# already available.
#
# By default the script will look for the OCM CLI component with any version (the first encountered will be used)
# and will extract the resource "ocmcli" for amd64/linux as a filepath. This path can then be used to run the CLI,
# but only after allowing to execute it, e.g with `chmod +x <path>`.

COMPONENT=${1:-"ocm.software/ocmcli"}
COMPONENT_VERSION=${2:-""}
RESOURCE=${3:-"ocmcli"}
ARCHITECTURE=${4:-"amd64"}
OS=${5:-"linux"}
MEDIA_TYPE=${6:-"application/octet-stream"}
PATH_TO_CTF=${7:-"./gen/ctf"}

INDEX=$( \
yq -r ".artifacts | filter(.repository == \"component-descriptors/${COMPONENT}\" and (.tag | contains(\"${COMPONENT_VERSION}\")))[0].digest" \
"${PATH_TO_CTF}"/artifact-index.json | \
sed 's/:/./g' \
)

if [ -z "${INDEX}" ]; then
echo "No index found for ${COMPONENT}"
exit 1
fi

RESOURCE=$( \
yq ".layers | filter(
(
.annotations.\"software.ocm.artifact\" |
from_json |
.[0]
) as \$artifact |
(
\$artifact.identity.name == \"$RESOURCE\" and
\$artifact.identity.architecture == \"$ARCHITECTURE\" and
\$artifact.identity.os == \"$OS\" and
.mediaType == \"$MEDIA_TYPE\"
)
)[0].digest" "${PATH_TO_CTF}"/blobs/"${INDEX}" | sed 's/:/./g' \
)

if [ -z "${RESOURCE}" ]; then
echo "No resource found for ${COMPONENT}"
exit 1
fi

RESOURCE=$PATH_TO_CTF/blobs/$RESOURCE

echo "$RESOURCE"