Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Devops ci/add build tools #104

Merged
merged 3 commits into from
Aug 3, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
73 changes: 73 additions & 0 deletions .github/workflows/artifact-release.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,73 @@
name: Create release artifact

on:
workflow_dispatch:

jobs:

cleanup:
runs-on: [self-hosted, linux, x64, myrunner]
container:
image: ubuntu:latest
steps:
- name: Cleaning up the $GITHUB_WORKSPACE as root from a Docker image
run: find /__w/${{ github.event.repository.name }}/${{ github.event.repository.name }}/. -name . -o -prune -exec rm -rf -- {} + || true

build:
needs: cleanup
runs-on: [self-hosted, linux, x64, myrunner]

steps:

- uses: actions/checkout@v3

- name: install dependencies
run: |
echo ~ && sudo apt-get update && sudo apt-get upgrade -y && sudo apt-get install -y docker-ce docker-ce-cli containerd.io unzip curl zip unzip

- name: Configure AWS credentials from Test account
uses: aws-actions/configure-aws-credentials@v1
with:
aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
aws-region: eu-west-2

- name: docker run
run: |
export BUILD_OUTPUT_DIR=~/build
echo $BUILD_OUTPUT_DIR
rm -rf $BUILD_OUTPUT_DIR
mkdir -p $BUILD_OUTPUT_DIR
docker build -f ./build_tools/Dockerfile -t spdb-build .
docker run --rm -v $(readlink -f ${BUILD_OUTPUT_DIR}):/out -v $GITHUB_WORKSPACE:/speedb -e GITHUB_WORKSPACE=$GITHUB_WORKSPACE spdb-build
ls -alFh $BUILD_OUTPUT_DIR
zip --junk-paths "Release-$GITHUB_REF_NAME.zip" $BUILD_OUTPUT_DIR/*

- name: Create Release
id: create_release
uses: actions/create-release@v1
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
with:
tag_name: ${{ github.ref }}
release_name: Release ${{ github.ref_name }}
draft: false
prerelease: false
- name: Upload Release Asset
id: upload-release-asset
uses: actions/upload-release-asset@v1
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
with:
upload_url: ${{ steps.create_release.outputs.upload_url }} # This pulls from the CREATE RELEASE step above, referencing it's ID to get its outputs object, which include a `upload_url`. See this blog post for more info: https://jasonet.co/posts/new-features-of-github-actions/#passing-data-to-future-steps
asset_path: ./Release-${{ github.ref_name}}.zip
asset_name: Release-${{ github.ref_name}}.zip
asset_content_type: application/zip

- name: Upload artifacts to S3
run: |
export BUILD_OUTPUT_DIR=~/build
echo $BUILD_OUTPUT_DIR
aws s3 cp $BUILD_OUTPUT_DIR s3://spdb-github-artifacts/"release-$GITHUB_REF_NAME" --recursive
aws s3 ls s3://spdb-github-artifacts/
rm -rf $BUILD_OUTPUT_DIR
32 changes: 32 additions & 0 deletions build_tools/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
FROM docker.io/centos:7.9.2009

# Install required libraries and utilities for building Speedb
RUN yum install -y centos-release-scl epel-release
RUN yum install -y make devtoolset-9-gcc-c++ \
coreutils wget which git python3 openssl-devel \
libzstd-devel lz4-devel snappy-devel zlib-devel \
java-1.8.0-openjdk-devel
ENV PATH="/opt/rh/devtoolset-9/root/usr/bin:${PATH}"

# Install CMake
RUN wget https://github.com/Kitware/CMake/releases/download/v3.20.1/cmake-3.20.1.tar.gz
RUN tar xf cmake-3.20.1.tar.gz
RUN cd cmake-3.20.1 && ./bootstrap && make -j $(nproc) && make install && cd .. && rm -rf cmake-3.20.1*

RUN echo -e "#!/bin/sh\n\
set -xeo pipefail\n\
java -version\n\
cd speedb\n\
rm -rf build && mkdir build && cd build\n\
cmake .. -DCMAKE_BUILD_TYPE=Release -DPORTABLE=1 -DWITH_GFLAGS=0 -DWITH_SNAPPY=1 -DWITH_LZ4=1 -DWITH_ZLIB=1 -DWITH_ZSTD=1\n\
make -j$(nproc) rocksdb rocksdb-shared\n\
cp librocksdb.{a,so} /out\n\
cd .. && rm -rf build\n\
make clean && LIB_MODE=static DEBUG_LEVEL=0 PORTABLE=1 JAVA_HOME=/usr/lib/jvm/java-openjdk make -j$(nproc) rocksdbjavastatic\n\
cp java/target/rocksdbjni-*.jar /out" > /usr/bin/build-speedb.sh

RUN chmod +x /usr/bin/build-speedb.sh
ENTRYPOINT ["/usr/bin/build-speedb.sh"]

# Declare volumes
VOLUME ["/speedb", "/out"]