Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use Reusable Workflow from everest-ci #926

Merged
merged 17 commits into from
Oct 30, 2024
Merged
9 changes: 9 additions & 0 deletions .ci/build-kit/docker/Dockerfile
Original file line number Diff line number Diff line change
@@ -1,3 +1,12 @@
# syntax=docker/dockerfile:1
ARG BASE_IMAGE_TAG=latest
FROM ghcr.io/everest/everest-ci/build-kit-base:${BASE_IMAGE_TAG}

# Can be used to use an other version of everest-cmake
# ENV EVEREST_CMAKE_PATH=/usr/lib/cmake/everest-cmake
# ENV EVEREST_CMAKE_VERSION=<everest-cmake-verion>
# RUN rm -rf ${EVEREST_CMAKE_PATH} \
# && git clone https://github.com/EVerest/everest-cmake.git ${EVEREST_CMAKE_PATH} \
# && cd ${EVEREST_CMAKE_PATH} \
# && git checkout ${EVEREST_CMAKE_VERSION} \
# && rm -r .git
16 changes: 12 additions & 4 deletions .ci/build-kit/scripts/compile.sh
Original file line number Diff line number Diff line change
@@ -1,16 +1,24 @@
#!/bin/sh

set -e

cmake \
-B "$EXT_MOUNT/build" \
-S "$EXT_MOUNT/source" \
-G Ninja \
-DEVC_ENABLE_CCACHE=1 \
-DISO15118_2_GENERATE_AND_INSTALL_CERTIFICATES=OFF \
-DCMAKE_INSTALL_PREFIX="$EXT_MOUNT/dist" \
-DWHEEL_INSTALL_PREFIX="$EXT_MOUNT/dist-wheels" \
-DWHEEL_INSTALL_PREFIX="$EXT_MOUNT/wheels" \
-DBUILD_TESTING=ON \
-DEVEREST_ENABLE_COMPILE_WARNINGS=ON
retVal=$?
if [ $retVal -ne 0 ]; then
echo "Configuring failed with return code $retVal"
exit $retVal
fi

ninja -j$(nproc) -C "$EXT_MOUNT/build"
ninja -C "$EXT_MOUNT/build"
retVal=$?
if [ $retVal -ne 0 ]; then
echo "Compiling failed with return code $retVal"
exit $retVal
fi
20 changes: 20 additions & 0 deletions .ci/build-kit/scripts/create_integration_image.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
#!/bin/sh

rsync -a "$EXT_MOUNT/source/tests" ./
retVal=$?

if [ $retVal -ne 0 ]; then
echo "Failed to copy tests"
exit $retVal
fi

pip install --break-system-packages \
$EXT_MOUNT/wheels/everestpy-*.whl \

Check warning on line 12 in .ci/build-kit/scripts/create_integration_image.sh

View check run for this annotation

Codacy Production / Codacy Static Code Analysis

.ci/build-kit/scripts/create_integration_image.sh#L12

Double quote to prevent globbing and word splitting.
$EXT_MOUNT/wheels/everest_testing-*.whl \

Check warning on line 13 in .ci/build-kit/scripts/create_integration_image.sh

View check run for this annotation

Codacy Production / Codacy Static Code Analysis

.ci/build-kit/scripts/create_integration_image.sh#L13

Double quote to prevent globbing and word splitting.
pytest-html
retVal=$?

if [ $retVal -ne 0 ]; then
echo "Failed to pip-install"
exit $retVal
fi
11 changes: 6 additions & 5 deletions .ci/build-kit/scripts/install.sh
Original file line number Diff line number Diff line change
@@ -1,8 +1,9 @@
#!/bin/sh

set -e

ninja -C "$EXT_MOUNT/build" install
ninja -C "$EXT_MOUNT/build" everestpy_install_wheel
ninja -C "$EXT_MOUNT/build" everest-testing_install_wheel
ninja -C "$EXT_MOUNT/build" iso15118_install_wheel
retVal=$?

if [ $retVal -ne 0 ]; then
echo "Installation failed with return code $retVal"
exit $retVal
fi
12 changes: 12 additions & 0 deletions .ci/build-kit/scripts/install_wheels.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
#!/bin/sh

ninja -C "$EXT_MOUNT/build" \
everestpy_install_wheel \
everest-testing_install_wheel \
iso15118_install_wheel
retVal=$?

if [ $retVal -ne 0 ]; then
echo "Wheel Installation failed with return code $retVal"
exit $retVal
fi
12 changes: 0 additions & 12 deletions .ci/build-kit/scripts/prepare_integration_tests.sh

This file was deleted.

13 changes: 10 additions & 3 deletions .ci/build-kit/scripts/run_unit_tests.sh
Original file line number Diff line number Diff line change
@@ -1,7 +1,14 @@
#!/bin/sh

set -e
ninja -C "$EXT_MOUNT/build" test
retVal=$?

trap "cp $EXT_MOUNT/build/Testing/Temporary/LastTest.log $EXT_MOUNT/ctest-report" EXIT
# Copy the LastTest.log file to the mounted directory in any case
cp "$EXT_MOUNT/build/Testing/Temporary/LastTest.log" "$EXT_MOUNT/ctest-report"

ninja -C "$EXT_MOUNT/build" test
if [ $retVal -ne 0 ]; then
echo "Unit tests failed with return code $retVal"
exit $retVal
fi

set -e
8 changes: 6 additions & 2 deletions .ci/e2e/scripts/run_integration_tests.sh
Original file line number Diff line number Diff line change
@@ -1,7 +1,5 @@
#!/bin/sh

set -e

cd tests
pytest \
-rA \
Expand All @@ -11,3 +9,9 @@ pytest \
core_tests/*.py \
framework_tests/*.py \
--everest-prefix "$EXT_MOUNT/dist"
retVal=$?

if [ $retVal -ne 0 ]; then
echo "Integration tests failed with return code $retVal"
exit $retVal
fi
202 changes: 12 additions & 190 deletions .github/workflows/build_and_test.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -14,196 +14,18 @@ on:
schedule:
- cron: '37 13,1 * * *'

env:
DOCKER_REGISTRY: ghcr.io
EVEREST_CI_VERSION: v1.3.1

jobs:
lint:
name: Lint
runs-on: ${{ inputs.runner || 'ubuntu-22.04' }}
steps:
- name: Checkout everest-core
uses: actions/checkout@v4.1.6
with:
path: source
- name: Run clang-format
uses: everest/everest-ci/github-actions/run-clang-format@v1.3.1
with:
source-dir: source
extensions: hpp,cpp
exclude: cache

# Since env variables can't be passed to reusable workflows, we need to pass them as outputs
setup-env:
# This job is currently disabled to allow running ci on PRs from forks
if: false
name: Setup Environment
runs-on: ${{ inputs.runner || 'ubuntu-22.04' }}
outputs:
docker_registry: ${{ env.DOCKER_REGISTRY }}
everest_ci_version: ${{ env.EVEREST_CI_VERSION }}
steps:
- id: check
run: |
echo "Setting up environment"
build-and-push-build-kit:
# This job is currently disabled to allow running ci on PRs from forks
if: false
name: Build and Push Build Kit
uses: everest/everest-ci/.github/workflows/deploy-single-docker-image.yml@v1.3.1
needs: setup-env
ci:
name: Build, Lint and Test
uses: everest/everest-ci/.github/workflows/continuous_integration.yml@v1.4.2
permissions:
contents: read
secrets:
SA_GITHUB_PAT: ${{ secrets.SA_GITHUB_PAT }}
SA_GITHUB_USERNAME: ${{ secrets.SA_GITHUB_USERNAME }}
coverage_deploy_token: ${{ secrets.SA_GITHUB_PAT }}
with:
image_name: ${{ github.event.repository.name }}/build-kit-everest-core
directory: .ci/build-kit/docker
docker_registry: ${{ needs.setup-env.outputs.docker_registry }}
github_ref_before: ${{ github.event.before }}
github_ref_after: ${{ github.event.after }}
platforms: linux/amd64
depends_on_paths: |
.ci/build-kit
.github/workflows/build_and_test.yaml
build_args: |
BASE_IMAGE_TAG=${{ needs.setup-env.outputs.everest_ci_version }}

build:
name: Build and Unit Tests
# needs: build-and-push-build-kit
runs-on: ${{ inputs.runner || 'ubuntu-22.04' }}
env:
# Currently the build-kit-base image is used to allow running ci on PRs from forks
# BUILD_KIT_IMAGE: ${{ needs.build-and-push-build-kit.outputs.one_image_tag_long }}
BUILD_KIT_IMAGE: ghcr.io/everest/everest-ci/build-kit-base:v1.3.1
steps:
- name: Format branch name for cache key
run: |
BRANCH_NAME_FOR_CACHE="${GITHUB_REF_NAME//-/_}"
echo "branch_name_for_cache=${BRANCH_NAME_FOR_CACHE}" >> "$GITHUB_ENV"
- name: Setup cache
uses: actions/cache@v3
with:
path: cache
key: compile-${{ env.branch_name_for_cache }}-${{ github.sha }}
restore-keys: |
compile-${{ env.branch_name_for_cache }}-
compile-
- name: Checkout everest-core
uses: actions/checkout@v4.1.6
with:
path: source
- name: Setup run scripts
run: |
mkdir scripts
rsync -a source/.ci/build-kit/scripts/ scripts
- name: Pull build-kit image
run: |
docker pull --quiet ${{ env.BUILD_KIT_IMAGE }}
docker image tag ${{ env.BUILD_KIT_IMAGE }} build-kit
- name: Compile
run: |
docker run \
--volume "$(pwd):/ext" \
--name compile-container \
build-kit run-script compile
- name: Commit compile-container
run: |
docker commit compile-container build-image
- name: Run unit tests
run: |
docker run \
--volume "$(pwd):/ext" \
--name unit-tests-container \
build-image run-script run_unit_tests
- name: Create dist
run: |
docker run \
--volume "$(pwd):/ext" \
--name install-container \
build-image run-script install
- name: Tar dist dir and keep permissions
run: |
tar -czf dist.tar.gz dist
- name: Upload dist artifact
uses: actions/upload-artifact@v4.3.3
with:
path: dist.tar.gz
name: dist
- name: Upload wheels artifact
uses: actions/upload-artifact@v4.3.3
with:
path: dist-wheels
name: wheels
- name: Archive unit test results
if: always()
uses: actions/upload-artifact@v4.3.3
with:
name: ctest-report
path: ${{ github.workspace }}/ctest-report
integration-tests:
name: Integration Tests
needs:
- build
# - build-and-push-build-kit
env:
# Currently the build-kit-base image is used to allow running ci on PRs from forks
# BUILD_KIT_IMAGE: ${{ needs.build-and-push-build-kit.outputs.one_image_tag_long }}
BUILD_KIT_IMAGE: ghcr.io/everest/everest-ci/build-kit-base:v1.3.1
runs-on: ${{ inputs.runner || 'ubuntu-22.04' }}
steps:
- name: Download dist dir
uses: actions/download-artifact@v4.1.7
with:
name: dist
- name: Extract dist.tar.gz
run: |
tar -xzf ${{ github.workspace }}/dist.tar.gz -C ${{ github.workspace }}
- name: Download wheels
uses: actions/download-artifact@v4.1.7
with:
name: wheels
path: wheels
- name: Checkout everest-core
uses: actions/checkout@v4.1.6
with:
path: source
- name: Setup run scripts
run: |
mkdir scripts
rsync -a source/.ci/build-kit/scripts/ scripts
- name: Pull build-kit image
run: |
docker pull --quiet ${{ env.BUILD_KIT_IMAGE }}
docker image tag ${{ env.BUILD_KIT_IMAGE }} build-kit
- name: Create integration-image
run: |
docker run \
--volume "$(pwd):/ext" \
--name prepare-container \
build-kit run-script prepare_integration_tests
docker commit prepare-container integration-image
- name: Run integration tests
run: |
pushd source/.ci/e2e
docker compose run \
e2e-test-server \
run-script run_integration_tests
- name: Upload result & report as artifact
if: always()
uses: actions/upload-artifact@v4.3.3
with:
path: |
${{ github.workspace }}/result.xml
${{ github.workspace }}/report.html
name: pytest-results
- name: Render result
if: always()
uses: pmeier/pytest-results-action@v0.6.0
with:
path: ${{ github.workspace }}/result.xml
summary: True
display-options: fEX
fail-on-empty: True
title: Test results
runner: ${{ inputs.runner || 'ubuntu-22.04' }}
artifact_deploy_target_repo: EVerest/everest.github.io
run_coverage: false
do_not_run_coverage_badge_creation: true
run_install_wheels: true
run_integration_tests: true
Loading