Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Publish packages from CircleCI #4026

Merged
merged 7 commits into from
Jun 15, 2023
Merged

Publish packages from CircleCI #4026

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
55bce0a
added debian buster and ubuntu to packages build job
pawlooss1 May 18, 2023
ecea234
added publish package job
pawlooss1 May 18, 2023
5b51903
added otp version info to package name
pawlooss1 May 23, 2023
8262a83
bump otp version for building debian packages
pawlooss1 May 23, 2023
2023845
Add rockylinux and almalinux package build
pawlooss1 May 29, 2023
9522a9f
code review follow-up
pawlooss1 Jun 2, 2023
5c2ed1f
Publishing only on release
pawlooss1 Jun 13, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
51 changes: 48 additions & 3 deletions .circleci/template.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -706,7 +706,7 @@ jobs:
type: executor
platform:
type: enum
enum: [centos_7, debian_bullseye]
enum: [centos_7, rockylinux_9, rockylinux_8, almalinux_9, almalinux_8, debian_bullseye, debian_buster, ubuntu_xenial, ubuntu_focal, ubuntu_bionic]
description: Platform type
otp_package:
type: string
Expand All @@ -722,6 +722,16 @@ jobs:
name: Build package
command: |
./tools/test.sh -p pkg -s false
- when:
condition:
matches:
pattern: /^\d+\.\d+\.\d+/
value: << pipeline.git.tag >>
steps:
- run:
name: Publish package
command: |
./tools/pkg/publish.sh

filters: &all_tags
tags:
Expand All @@ -748,14 +758,49 @@ workflows:
executor: otp_25
platform: centos_7
context: mongooseim-org
otp_package: "25.0.1-1"
otp_package: "25.0.3-1"
filters: *all_tags
- package:
name: rockylinux_8
executor: otp_24
platform: rockylinux_8
context: mongooseim-org
otp_package: "24.1.7-1"
filters: *all_tags
- package:
name: almalinux_8
executor: otp_24
platform: almalinux_8
context: mongooseim-org
otp_package: "24.1.7-1"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

any reason to use an older version?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Currently there is no otp 25 package available for almalinux and rockylinux. I believe that when the new site with packages is running we could expect some newer version published.

filters: *all_tags
- package:
name: debian_bullseye
executor: otp_25
platform: debian_bullseye
context: mongooseim-org
otp_package: "25.0.1-1"
otp_package: "25.3-1"
filters: *all_tags
- package:
name: debian_buster
executor: otp_25
platform: debian_buster
context: mongooseim-org
otp_package: "25.3-1"
filters: *all_tags
- package:
name: ubuntu_focal
executor: otp_25
platform: ubuntu_focal
context: mongooseim-org
otp_package: "25.3-1"
filters: *all_tags
- package:
name: ubuntu_bionic
executor: otp_25
platform: ubuntu_bionic
context: mongooseim-org
otp_package: "25.3-1"
filters: *all_tags
# ======== BASE DOCKER BUILDS ========
- build_in_docker:
Expand Down
5 changes: 1 addition & 4 deletions tools/pkg/Dockerfile_deb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,9 +3,6 @@ ARG dockerfile_platform

FROM $dockerfile_platform AS builder

# Update repositories
RUN sed -i 's/\/updates/-security/g' /etc/apt/sources.list

# Install build deps
ENV DEBIAN_FRONTEND=noninteractive
RUN apt-get update
Expand All @@ -32,7 +29,7 @@ RUN cp -r ./mongooseim/tools/pkg/scripts/deb .
ARG version
ARG revision

RUN ./deb/build_package.sh $version $revision
RUN ./deb/build_package.sh $version $revision $erlang_version

# Create image for sharing and validation of built package
FROM $dockerfile_platform
Expand Down
4 changes: 2 additions & 2 deletions tools/pkg/Dockerfile_rpm
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ RUN yum install -y rpm-build rpmdevtools git make zlib-devel unixODBC-devel gcc
# Fix locale setup
# See https://github.com/CentOS/sig-cloud-instance-images/issues/71#issuecomment-266957519
ARG dockerfile_platform
RUN if [ "$dockerfile_platform" == "centos:8" ]; then \
RUN if [ "$dockerfile_platform" == "rockylinux:8" ] || [ "$dockerfile_platform" == "almalinux:8" ]; then \
dnf install -y glibc-locale-source; \
fi
RUN localedef -i en_US -f UTF-8 en_US.UTF-8
Expand All @@ -45,7 +45,7 @@ RUN cp ./BUILD/mongooseim/tools/pkg/scripts/rpm/mongooseim.service \
ARG version
ARG revision

RUN ./BUILD/mongooseim/tools/pkg/scripts/rpm/build_package.sh $version $revision
RUN ./BUILD/mongooseim/tools/pkg/scripts/rpm/build_package.sh $version $revision $erlang_version

# Create image for sharing and validation of built package
FROM $dockerfile_platform
Expand Down
48 changes: 14 additions & 34 deletions tools/pkg/publish.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,41 +1,21 @@
#!/usr/bin/env sh
#!/usr/bin/env bash

set -e

# Params - override for your env
SHA256SUM="shasum -a 256"
REMOTE_SHA256SUM=sha256sum
PACKAGES_LOCAL=packages
. env/publish
cd tools/pkg/packages
PACKAGE_NAME=$(ls)
PKG_PROFILE=packages
prefix=${pkg_PLATFORM/_//}

# CLI arguments
VERSION=${1:-"(missing)"}
REVISION=${2:-1}

if [ x"$VERSION" = x"(missing)" ]; then
echo "usage: $0 VERSION [REVISION]"
echo
echo Look for package \$VERSION-\$REVISION and publish to $PACKAGES_HOST.
exit 1
if which aws ; then
echo "aws tool ready"
else
curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip"
unzip -q awscliv2.zip
sudo ./aws/install
fi

set +e
PKG=$(ls -1 ${PACKAGES_LOCAL} | grep "$VERSION-$REVISION" | head -1)
if [ -z "$PKG" ]; then
echo "$0: can't find package version $VERSION-$REVISION"
exit 2
fi
set -e
aws configure set aws_access_key_id "$PKG_ACCESS_KEY_ID" --profile "$PKG_PROFILE"
aws configure set aws_secret_access_key "$PKG_SECRET_ACCESS_KEY" --profile "$PKG_PROFILE"

rsync $PACKAGES_LOCAL/$PKG $PACKAGES_HOST:$PACKAGES_PATH
CHKSUM=$($SHA256SUM $PACKAGES_LOCAL/$PKG)
REMOTE_CHKSUM=$(ssh $PACKAGES_HOST "cd $PACKAGES_PATH && $REMOTE_SHA256SUM $PKG")

if [ x"$(echo $CHKSUM | cut -d" " -f1)" != x"$(echo $REMOTE_CHKSUM | cut -d" " -f1)" ]; then
echo "$0: checksum mismatch - try again or check your connection"
exit 3
fi
ssh $PACKAGES_HOST "sed -i /$VERSION-$REVISION/d $PACKAGES_PATH/sha256sum " \
"&& echo $REMOTE_CHKSUM >> $PACKAGES_PATH/sha256sum"
echo Package $PKG is now public at:
echo https://$PACKAGES_HOST/$PACKAGES_PROJECT/$PKG
aws s3 cp "$PACKAGE_NAME" "s3://arn:aws:s3:$PKG_AWS_REGION:$PKG_USER_ID:accesspoint/mim-packages/$prefix/$PACKAGE_NAME" --acl public-read --profile "$PKG_PROFILE" --region "$PKG_AWS_REGION"
3 changes: 2 additions & 1 deletion tools/pkg/scripts/deb/build_package.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@ set -e

version=$1
revision=$2
otp_version=$3

arch="amd64"

Expand Down Expand Up @@ -43,5 +44,5 @@ source /etc/os-release
os=$ID
os_version=$VERSION_CODENAME
package_os_file_name=${os}~${os_version}
mv mongooseim_*.deb ~/mongooseim_${version}-${revision}~${package_os_file_name}_${arch}.deb
mv mongooseim_*.deb ~/mongooseim_${version}_${revision}_otp_${otp_version/-1/}~${package_os_file_name}_${arch}.deb

3 changes: 2 additions & 1 deletion tools/pkg/scripts/rpm/build_package.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@ set -e

version=$1
revision=$2
otp_version=$3

arch="x86_64"
package_name_arch="amd64"
Expand All @@ -20,4 +21,4 @@ os_version=$VERSION_ID
package_os_file_name=${os}~${os_version}

mv ~/rpmbuild/RPMS/${arch}/mongooseim-${version}-${revision}.${arch}.rpm \
~/mongooseim_${version}-${revision}~${package_os_file_name}_${package_name_arch}.rpm
~/mongooseim_${version}_${revision}_otp_${otp_version/-1/}~${package_os_file_name}_${package_name_arch}.rpm
2 changes: 1 addition & 1 deletion tools/test.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -282,7 +282,7 @@ build_pkg () {
local esl_erlang_pkg_vsn=$2
local project_root=$(git rev-parse --show-toplevel)

if [[ $platform == centos* ]]; then
if [[ $platform == centos* ]] || [[ $platform == rockylinux* ]] || [[ $platform == almalinux* ]]; then
local dockerfile_name="Dockerfile_rpm"
elif [[ $platform == debian* ]] || [[ $platform == ubuntu* ]]; then
local dockerfile_name="Dockerfile_deb"
Expand Down