Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add optional CI job for integration tests with cudf.pandas #5881

Merged
merged 11 commits into from
May 6, 2024
Merged

Add optional CI job for integration tests with cudf.pandas #5881

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
11 commits
Select commit Hold shift + click to select a range
278438c
FEA Add optional CI job for integration tests
dantegd May 2, 2024
741f339
FIX forgot test job in needs section of gha file
dantegd May 2, 2024
3655f6e
DBG remove optional flag
dantegd May 2, 2024
4cabd82
DBG add optional flag back
dantegd May 2, 2024
f5f4a2a
DBG remove optional flag
dantegd May 2, 2024
242e0b0
FIX name of script to call integration tests
dantegd May 3, 2024
d76befe
FEA Use new optional jobs functionaluty frok shared-workflows
dantegd May 3, 2024
5ac19ca
FIX remove job from needs section
dantegd May 3, 2024
eaad25d
Empty-Commit
dantegd May 3, 2024
c4a27b0
DBG remove ignored_pr_jobs from checks
dantegd May 3, 2024
d6979c5
ENH add ignore jobs back and switch to branch-24.06
dantegd May 3, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions .github/workflows/pr.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,8 @@ jobs:
uses: rapidsai/shared-workflows/.github/workflows/checks.yaml@branch-24.06
with:
enable_check_generated_files: false
ignored_pr_jobs: >-
conda-python-tests-cudf-pandas-integration
clang-tidy:
needs: checks
secrets: inherit
Expand Down Expand Up @@ -75,6 +77,13 @@ jobs:
with:
build_type: pull-request
script: "ci/test_python_singlegpu.sh"
conda-python-tests-cudf-pandas-integration:
needs: conda-python-build
secrets: inherit
uses: rapidsai/shared-workflows/.github/workflows/conda-python-tests.yaml@branch-24.06
with:
build_type: pull-request
script: "ci/test_python_integration.sh"
conda-python-tests-dask:
needs: conda-python-build
secrets: inherit
Expand Down
7 changes: 7 additions & 0 deletions ci/run_cuml_integration_pytests.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
#!/bin/bash
# Copyright (c) 2024, NVIDIA CORPORATION.

# Support invoking run_cuml_singlegpu_pytests.sh outside the script directory
cd "$(dirname "$(realpath "${BASH_SOURCE[0]}")")"/../python/cuml/tests

python -m pytest -p cudf.pandas --cache-clear --ignore=dask -m "not memleak" "$@" --quick_run .
25 changes: 25 additions & 0 deletions ci/test_python_integration.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
#!/bin/bash
# Copyright (c) 2024, NVIDIA CORPORATION.

# Support invoking test_python_singlegpu.sh outside the script directory
cd "$(dirname "$(realpath "${BASH_SOURCE[0]}")")"/../

# Common setup steps shared by Python test jobs
source ./ci/test_python_common.sh

EXITCODE=0
trap "EXITCODE=1" ERR
set +e

rapids-logger "pytest cuml integration"
./ci/run_cuml_integration_pytests.sh \
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

continued naming nitpick: can we call it run_cudf_pandas_integration_pytests.sh? We are in the cuml repo, so "cuml integration tests" feels like a tautology, while "cudf pandas integration testing" tells me something about what we are testing. Without having to go through the layers of shell scripts to then read and understand the meaning of the option in pytest command. Three weeks ago I am not sure I'd have quickly grasped that -p cudf.pandas is the relevant but in the pytest command line and that it enables integration testing.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's a good point, will rename it

--numprocesses=8 \
--dist=worksteal \
--junitxml="${RAPIDS_TESTS_DIR}/junit-cuml.xml" \
--cov-config=../../.coveragerc \
--cov=cuml \
--cov-report=xml:"${RAPIDS_COVERAGE_DIR}/cuml-coverage.xml" \
--cov-report=term

rapids-logger "Test script exiting with value: $EXITCODE"
exit ${EXITCODE}
Loading