Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CI overhaul #2298

Draft
wants to merge 1 commit into
base: develop
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .gitattributes
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
# GitHub syntax highlighting
pixi.lock linguist-language=YAML

14 changes: 7 additions & 7 deletions .github/workflows/build_docs_gallery.yml
Original file line number Diff line number Diff line change
@@ -1,20 +1,20 @@
name: Build Gallery

on:
pull_request:
pull_request:

jobs:
Test-MSS-docs:
runs-on: ubuntu-latest

container:
image: openmss/testing-develop

steps:
- uses: actions/checkout@v4

- uses: prefix-dev/setup-pixi@v0.5.1
with:
pixi-version: latest
cache: true
environments: dev
- name: Create gallery
timeout-minutes: 5
run: |
cd docs
mamba run --no-capture-output -n mssenv python conf.py
pixi run -e dev python conf.py
25 changes: 5 additions & 20 deletions .github/workflows/testing-all-oses.yml
Original file line number Diff line number Diff line change
Expand Up @@ -22,32 +22,17 @@ jobs:
order: ["normal", "reverse"]
steps:
- uses: actions/checkout@v4
- name: Build requirements.txt file
run: |
sed -n '/^requirements:/,/^test:/p' localbuild/meta.yaml |
sed -e "s/.*- //" |
sed -e "s/menuinst.*//" |
sed -e "s/.*://" > requirements.tmp.txt
cat requirements.d/development.txt >> requirements.tmp.txt
sed -e '/^$/d' -e '/^#.*$/d' requirements.tmp.txt > requirements.txt
rm requirements.tmp.txt
cat requirements.txt
- name: Get current year and calendar week
id: year-and-week
run: echo "year-and-week=$(date +%Y-%V)" >> "$GITHUB_OUTPUT"
- uses: mamba-org/setup-micromamba@v1
- uses: prefix-dev/setup-pixi@v0.5.1
with:
environment-file: requirements.txt
environment-name: ci
cache-environment: true
# Set the cache key in a way that the cache is invalidated every week on monday
cache-environment-key: environment-${{ steps.year-and-week.outputs.year-and-week }}
pixi-version: latest
cache: true
environments: dev
- name: Run tests
timeout-minutes: 20
# The ignored files can somehow cause the test suite to timeout.
# I have no idea yet on why this happens and how to fix it.
# Even a module level skip is not enough, they need to be completely ignored.
# TODO: fix those tests and drop the ignores
run: micromamba run -n ci env QT_QPA_PLATFORM=offscreen pytest -v -n logical --durations=20 --cov=mslib
run: pixi run -e dev env QT_QPA_PLATFORM=offscreen pytest -v -n logical --durations=20 --cov=mslib
--ignore=tests/_test_msui/test_sideview.py --ignore=tests/_test_msui/test_topview.py --ignore=tests/_test_msui/test_wms_control.py
${{ (matrix.order == 'normal' && ' ') || (matrix.order == 'reverse' && '--reverse') }} tests
19 changes: 0 additions & 19 deletions .github/workflows/testing-develop.yml

This file was deleted.

25 changes: 0 additions & 25 deletions .github/workflows/testing-scheduled.yml

This file was deleted.

20 changes: 0 additions & 20 deletions .github/workflows/testing-stable.yml

This file was deleted.

80 changes: 18 additions & 62 deletions .github/workflows/testing.yml
Original file line number Diff line number Diff line change
@@ -1,28 +1,21 @@
name: Pytest MSS

on:
workflow_call:
inputs:
branch_name:
required: true
type: string
event_name:
required: true
type: string
secrets:
PAT:
required: true

env:
mamba-env: mss-${{ inputs.branch_name }}-env
push:
branches:
- develop
- stable
- 'GSOC**'
pull_request:
branches:
- develop
- stable
- 'GSOC**'

jobs:
Test-MSS:
runs-on: ubuntu-latest

container:
image: openmss/testing-${{ inputs.branch_name }}

strategy:
fail-fast: false
matrix:
Expand All @@ -31,57 +24,20 @@ jobs:
steps:
- uses: actions/checkout@v4

- name: Check for changed dependencies
run: cmp -s /meta.yaml localbuild/meta.yaml && cmp -s /development.txt requirements.d/development.txt ||
(echo Dependencies differ && echo "triggerdockerbuild=yes" >> $GITHUB_ENV )

- name: Install pyvirtualdisplay if on stable
if: ${{ inputs.branch_name == 'stable' }}
run: |
source /opt/conda/etc/profile.d/conda.sh
source /opt/conda/etc/profile.d/mamba.sh
mamba install -n mss-${{ inputs.branch_name }}-env pyvirtualdisplay

- name: Always rebuild dependencies for scheduled builds
if: ${{ inputs.event_name == 'schedule' && inputs.branch_name == 'stable' && env.triggerdockerbuild != 'yes' }}
run: echo "triggerdockerbuild=yes" >> $GITHUB_ENV

- name: Reinstall dependencies if changed
if: ${{ env.triggerdockerbuild == 'yes' }}
run: |
cat localbuild/meta.yaml |
sed -n '/^requirements:/,/^test:/p' |
sed -e "s/.*- //" |
sed -e "s/menuinst.*//" |
sed -e "s/.*://" > reqs.txt
cat requirements.d/development.txt >> reqs.txt
cat reqs.txt
mamba env remove -n ${{ env.mamba-env }}
mamba create -y -n ${{ env.mamba-env }} --file reqs.txt

- name: Print conda list
run: mamba run --no-capture-output -n ${{ env.mamba-env }} mamba list
- uses: prefix-dev/setup-pixi@v0.5.1
with:
pixi-version: latest
cache: true
environments: dev

- name: Run tests
timeout-minutes: 10
run: mamba run --no-capture-output -n ${{ env.mamba-env }} xvfb-run pytest
run: pixi run -e dev xvfb-run pytest
-v -n 6 --dist loadfile --max-worker-restart 4 --durations=20 --cov=mslib
${{ (matrix.order == 'normal' && ' ') || (matrix.order == 'reverse' && '--reverse') }} tests

- name: Collect coverage
if: ${{ (inputs.event_name == 'push' || inputs.event_name == 'pull_request') && matrix.order == 'normal' }}
if: ${{ (github.event_name == 'push' || github.event_name == 'pull_request') && matrix.order == 'normal' }}
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
git config --global --add safe.directory /__w/MSS/MSS
mamba install -n ${{ env.mamba-env }} coveralls
mamba run --no-capture-output -n ${{ env.mamba-env }} coveralls --service=github

- name: Invoke dockertesting image creation
if: ${{ !cancelled() && inputs.event_name == 'push' && env.triggerdockerbuild == 'yes' && matrix.order == 'normal' }}
uses: benc-uk/workflow-dispatch@v1.2.2
with:
workflow: Update Image testing-${{ inputs.branch_name }}
repo: Open-MSS/dockertesting
ref: main
token: ${{ secrets.PAT }}
run: pixi run -e dev coveralls --service=github
58 changes: 0 additions & 58 deletions .github/workflows/testing_gsoc.yml

This file was deleted.

38 changes: 38 additions & 0 deletions .github/workflows/update-pixi-lockfile.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
name: Update pixi lockfile

on:
schedule:
# At 04:00 on Monday
- cron: "0 4 * * 1"
# ...or manually
workflow_dispatch:

jobs:
update-pixi-lockfile:
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
base_branch: ["develop", "stable"]
permissions:
contents: write
pull-requests: write
steps:
- uses: actions/checkout@v4
with:
ref: ${{ matrix.base_branch }}
- name: Remove old lockfile
run: rm pixi.lock
- name: Generate new lockfile
uses: prefix-dev/setup-pixi@v0.5.1
with:
pixi-version: latest
cache: false
- name: Create or update pull request
uses: peter-evans/create-pull-request@v6
with:
add-paths: pixi.lock
branch: automation/update-pixi-lockfile
commit-message: Update pixi lockfile
title: Update pixi lockfile
body: ""
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -18,3 +18,6 @@ build/
mss.egg-info/
tutorials/recordings
tutorials/cursor_image.png
# pixi environments
.pixi

5 changes: 0 additions & 5 deletions localbuild/README.rst

This file was deleted.

9 changes: 0 additions & 9 deletions localbuild/bld.bat

This file was deleted.

Loading
Loading