Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ci: add script to update rootfs URL #507

Closed
wants to merge 1 commit into from
Closed

ci: add script to update rootfs URL #507

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 26 additions & 0 deletions .github/bin/update-rootfs.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
#!/bin/sh
set -euxo pipefail

DEPENDENCY_CLOUDFRONT_URL="https://deps.runfinch.com/"
AARCH64_FILENAME_PATTERN="finch-rootfs-production-arm64-[0-9].*\.tar.zst$"
AMD64_FILENAME_PATTERN="finch-rootfs-production-amd64-[0-9].*\.tar.zst$"

while getopts d: flag
do
case "${flag}" in
d) dependency_bucket=${OPTARG};;
esac
done

[[ -z "$dependency_bucket" ]] && { echo "Error: Dependency bucket not set"; exit 1; }

aarch64Deps=$(aws s3 ls s3://${dependency_bucket}/ | grep "$AARCH64_FILENAME_PATTERN" | sort | tail -n 1 | awk '{print $4}')

[[ -z "$aarch64Deps" ]] && { echo "Error: aarch64 dependency not found"; exit 1; }

amd64Deps=$(aws s3 ls s3://${dependency_bucket}/ | grep "$AMD64_FILENAME_PATTERN" | sort | tail -n 1 | awk '{print $4}')

[[ -z "$amd64Deps" ]] && { echo "Error: x86_64 dependency not found"; exit 1; }

sed -E -i.bak 's|^([[:blank:]]*FINCH_ROOTFS_URL[[:blank:]]*\?=[[:blank:]]*'${DEPENDENCY_CLOUDFRONT_URL}')('${AARCH64_FILENAME_PATTERN}')|\1'$aarch64Deps'|' Makefile
sed -E -i.bak 's|^([[:blank:]]*FINCH_ROOTFS_URL[[:blank:]]*\?=[[:blank:]]*'${DEPENDENCY_CLOUDFRONT_URL}')('${AMD64_FILENAME_PATTERN}')|\1'$amd64Deps'|' Makefile
1 change: 1 addition & 0 deletions .github/workflows/update-deps.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,7 @@ jobs:
- name: update dependencies url
run: |
./.github/bin/update-lima-dep.sh -d ${{ secrets.DEPENDENCY_BUCKET_NAME }} -a {{ secrets.ARTIFACT_BUCKET_NAME }}
./.github/bin/update-rootfs.sh -d ${{ secrets.DEPENDENCY_BUCKET_NAME }}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we have a different bucket/sub-folder for the new artifact?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sure, or how about under respective ${ARCH} folder as lima/qemu is today?

finch/Makefile

Line 38 in 81d8b20

LIMA_URL ?= https://deps.runfinch.com/aarch64/lima-and-qemu.macos-aarch64.1689037160.tar.gz

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think a folder at the same level with aarch64 and amd64 maybe better so that each type of the artifact won't be stored in the same folder which will be cleaner and reduce the risk for artifact remove case for a dir.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

By the way, I think we may only need this change in the Core repo

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

By the way, I think we may only need this change in the Core repo

Yea thanks for pointing that out... There are a couple places where we can deduplicate code / actions between finch and finch-core. Opened #508 to track


- name: create PR
uses: peter-evans/create-pull-request@v5
Expand Down
6 changes: 6 additions & 0 deletions Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -36,12 +36,18 @@ ifneq (,$(findstring arm64,$(ARCH)))
# From https://dl.fedoraproject.org/pub/fedora/linux/releases/37/Cloud/aarch64/images/
FINCH_OS_BASENAME ?= Fedora-Cloud-Base-38-1.6.aarch64-20230713205101.qcow2
LIMA_URL ?= https://deps.runfinch.com/aarch64/lima-and-qemu.macos-aarch64.1689037160.tar.gz

# TODO: Use Finch rootfs in Finch on Windows testing
FINCH_ROOTFS_URL ?= https://deps.runfinch.com/finch-rootfs-production-arm64-1690563031.tar.zst
else ifneq (,$(findstring x86_64,$(ARCH)))
SUPPORTED_ARCH = true
LIMA_ARCH = x86_64
# From https://dl.fedoraproject.org/pub/fedora/linux/releases/37/Cloud/x86_64/images/
FINCH_OS_BASENAME ?= Fedora-Cloud-Base-38-1.6.x86_64-20230713205042.qcow2
LIMA_URL ?= https://deps.runfinch.com/x86-64/lima-and-qemu.macos-x86_64.1689037160.tar.gz

# TODO: Use Finch rootfs in Finch on Windows testing
FINCH_ROOTFS_URL ?= https://deps.runfinch.com/finch-rootfs-production-amd64-1690563027.tar.zst
endif

FINCH_OS_HASH := `shasum -a 256 $(OUTDIR)/os/$(FINCH_OS_BASENAME) | cut -d ' ' -f 1`
Expand Down