Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Codacy code coverage reporting #443

Open
wants to merge 38 commits into
base: devel
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 26 commits
Commits
Show all changes
38 commits
Select commit Hold shift + click to select a range
3c1ad9b
Added Codecov to CI
kvrigor Apr 24, 2024
f4fff9e
Merge branch 'pyccel:devel' into codecov-test
kvrigor Aug 7, 2024
eecc6f4
Merge branch 'pyccel:devel' into codecov-test
kvrigor Aug 21, 2024
0d2675b
Merge branch 'pyccel:devel' into codecov-test
kvrigor Oct 2, 2024
cd78ac3
coverage: exlude unit tests and pyccelised kernels in the report
kvrigor Oct 2, 2024
1bafc85
CI.yml: added Codacy coverage reporter action
kvrigor Oct 2, 2024
1715bfd
pyproject.toml: deleted redundant pytest-cov
kvrigor Oct 2, 2024
ae33ed0
CI.yml: Codacy reporter should run under pytest folder
kvrigor Oct 2, 2024
ab102b9
CI.yml: Fixed path to coverage.xml
kvrigor Oct 2, 2024
03f605d
CI.yml: Set coverage config file to pyproject.toml
kvrigor Oct 4, 2024
07aa1e1
CI.yml: Printed detailed coverage results
kvrigor Oct 4, 2024
65c27de
Excluded examples dir from coverage
kvrigor Oct 4, 2024
4949d39
'--cov-report term-missing' is redundant
kvrigor Oct 4, 2024
36620af
Dummy change for checking test runtime
kvrigor Oct 8, 2024
3703d5b
CI.yml: Generated coverage report on another step
kvrigor Oct 8, 2024
f175d0b
CI.yml: Run tests via coverage instead of pytest-cov
kvrigor Oct 9, 2024
10d96d4
CI.yml: Specified full path to pyproject.toml
kvrigor Oct 9, 2024
fc6e380
CI.yml: Be explicit about the location of the .coverage file
kvrigor Oct 9, 2024
d2a73f1
CI.yml: Ran 'coverage xml' under ./pytest
kvrigor Oct 9, 2024
e23716e
CI.yml: Run coverage at repo root
kvrigor Oct 9, 2024
6d4aedd
Fixed missing coverage data by setting coverage subprocess hooks
kvrigor Oct 14, 2024
d995440
Fixed typo when unsetting coverage env vars
kvrigor Oct 14, 2024
338aa9a
Made coverage env settings permanent for now
kvrigor Oct 14, 2024
ee41b76
Try running coverage under ./pytest
kvrigor Oct 14, 2024
934a705
Revert to pytest-cov
kvrigor Oct 16, 2024
b7d61a5
Fixed path to coverage.xml
kvrigor Oct 16, 2024
43b35e3
Cached ./pytest_cache
kvrigor Oct 23, 2024
051f1ec
Pytest cache must always be saved regardless of test result
kvrigor Oct 23, 2024
02647ba
Only run coverage tests on macOS
kvrigor Oct 24, 2024
cf024e8
Break test_api_3d_scalar
kvrigor Oct 24, 2024
70eeff4
Pytest cache must be updated every run
kvrigor Oct 24, 2024
45d25b1
Compute cache key based on hash of ./pytest_cache
kvrigor Oct 24, 2024
65742fd
Fixed test_api_3d_scalar.py
kvrigor Oct 24, 2024
6d28291
Break test_2d_biharmonic.py
kvrigor Oct 24, 2024
06cb2f8
Break test_2d_laplace.py
kvrigor Oct 24, 2024
85f869d
Discarded --failed-first idea
kvrigor Oct 24, 2024
5567cf7
Fixed deliberately broken tests
kvrigor Oct 24, 2024
a443720
Declared all pytest packages as optional dependencies; in pyproject.t…
kvrigor Oct 28, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
37 changes: 27 additions & 10 deletions .github/workflows/continuous-integration.yml
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,10 @@ jobs:

name: ${{ matrix.os }} / Python ${{ matrix.python-version }}

env:
PSYDAC_MESH_DIR: ${{ github.workspace }}/mesh
OMP_NUM_THREADS: 2

steps:
- uses: actions/checkout@v4

Expand Down Expand Up @@ -157,7 +161,7 @@ jobs:

- name: Install project
run: |
python -m pip install .
python -m pip install .[test]
python -m pip freeze

- name: Test Pyccel optimization flags
Expand All @@ -169,32 +173,45 @@ jobs:
mkdir pytest
cp mpi_tester.py pytest

# Coverage data is also collected during single process testing.
- name: Run single-process tests with Pytest
working-directory: ./pytest
run: >-
python -m pytest -n auto --cov psydac
--cov-config $GITHUB_WORKSPACE/pyproject.toml
--cov-report=""
--pyargs psydac -m "not parallel and not petsc"

- name: Generate coverage report for Codacy
working-directory: ./pytest
run: |
coverage xml
pytest --cache-show | grep cachedir # To be removed in the next commit

- name: Upload coverage report to Codacy
uses: codacy/codacy-coverage-reporter-action@v1.3.0
with:
project-token: ${{ secrets.CODACY_PROJECT_TOKEN }}
coverage-reports: ./pytest/coverage.xml

- name: Print detailed coverage results
working-directory: ./pytest
run: |
export PSYDAC_MESH_DIR=$GITHUB_WORKSPACE/mesh
export OMP_NUM_THREADS=2
python -m pytest -n auto --pyargs psydac -m "not parallel and not petsc"
coverage report --ignore-errors --show-missing --sort=cover

- name: Run MPI tests with Pytest
working-directory: ./pytest
run: |
export PSYDAC_MESH_DIR=$GITHUB_WORKSPACE/mesh
export OMP_NUM_THREADS=2
python mpi_tester.py --mpirun="mpiexec -n 4 ${MPI_OPTS}" --pyargs psydac -m "parallel and not petsc"

- name: Run single-process PETSc tests with Pytest
working-directory: ./pytest
run: |
export PSYDAC_MESH_DIR=$GITHUB_WORKSPACE/mesh
export OMP_NUM_THREADS=2
python -m pytest -n auto --pyargs psydac -m "not parallel and petsc"

- name: Run MPI PETSc tests with Pytest
working-directory: ./pytest
run: |
export PSYDAC_MESH_DIR=$GITHUB_WORKSPACE/mesh
export OMP_NUM_THREADS=2
python mpi_tester.py --mpirun="mpiexec -n 4 ${MPI_OPTS}" --pyargs psydac -m "parallel and petsc"

- name: Remove test directory
Expand Down
43 changes: 43 additions & 0 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,11 @@ dependencies = [
'igakit @ https://github.com/dalcinl/igakit/archive/refs/heads/master.zip'
]

[project.optional-dependencies]
test = [
"pytest-cov >= 5.0.0",
]

kvrigor marked this conversation as resolved.
Show resolved Hide resolved
[project.urls]
Homepage = "https://github.com/pyccel/psydac"
Documentation = "https://pyccel.github.io/psydac"
Expand All @@ -68,3 +73,41 @@ namespaces = false

[tool.setuptools.package-data]
"*" = ["*.txt"]

[tool.coverage.run]
branch = true
omit = [
# Exclude pyccelised kernels
"*/__psydac__/*",

# Examples don't need to be covered
"*/examples/*",

# Unit tests shouldn't be included
"*/tests/*",
]

[tool.coverage.report]
# Regexes for lines to exclude from consideration
exclude_also = [
# Don't complain about missing debug-only code:
"def __repr__",
"if self\\.debug",

# Don't complain if tests don't hit defensive assertion code:
"raise AssertionError",
"raise NotImplementedError",

# Don't complain if non-runnable code isn't run:
"if 0:",
kvrigor marked this conversation as resolved.
Show resolved Hide resolved
"if __name__ == .__main__.:",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I find the use of the dots . a bit confusing in this regex. You actually want to match just the single and double quote marks, right?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You actually want to match just the single and double quote marks, right?

Exactly; this snippet was based from here: https://coverage.readthedocs.io/en/latest/excluding.html#advanced-exclusion


# Don't complain about abstract methods, they aren't run:
"@(abc\\.)?abstractmethod",
]

# Ignore source code that can’t be found, emitting a warning instead of an exception.
ignore_errors = true

[tool.coverage.html]
directory = "coverage_html_report"