add a simple smoketest for cvxpy integration #199
Conversation
tmckayus
added
feature request
jameslamb
left a comment
jameslamb
left a comment
There was a problem hiding this comment.
Left some comments for your consideration.
|
@jameslamb Thank you for the review, I had addressed few of the review comments and will follow-up on any new ones. |
| RAPIDS_DATASET_ROOT_DIR=./datasets timeout 30m python -m pytest --verbose --capture=no ./python/cuopt/cuopt/tests/ | ||
|
|
||
| timeout 3m ./ci/external/cvxpy_smoketest.sh | ||
| # run cvxpy integration tests |
There was a problem hiding this comment.
@jameslamb We would have more integration tests in future, should we move few items to be partly into a bash script and run it from there to keep the test_wheel_cuopt.sh simple ?
There was a problem hiding this comment.
Sure we can, and to be fair that's how @tmckayus had it originally before I pushed commits here.
Let me tell you why I did it this way. I was mainly concerned about making sure that the pip install that installs cvxpy uses the just-built-in-this-CI-run cuopt packages.
Doing it in 1 script like this makes that easy to guarantee, because I can just reference variables from the same scope and packages downloaded a few lines higher up. Like this:
CUOPT_SH_CLIENT_WHEELHOUSE=$(RAPIDS_PY_WHEEL_NAME="cuopt_sh_client" rapids-download-wheels-from-github python)
python -m pip install \
--extra-index-url=https://pypi.nvidia.com \
"${CUOPT_MPS_PARSER_WHEELHOUSE}"/cuopt_mps_parser*.whl \
"${CUOPT_SH_CLIENT_WHEELHOUSE}"/cuopt_sh_client*.whl \
"${CUOPT_WHEELHOUSE}"/cuopt*.whl \
"${LIBCUOPT_WHEELHOUSE}"/libcuopt*.whl \
'pytest-error-for-skips>=2.0.2' \
"$(echo ./dist/cvxpy*.whl)[CUOPT,testing]"If you want to defer all the cvxpy stuff to a separate script, like Trevor had it before, it'd involve some changes to how packages are downloaded.
I can do that here.
There was a problem hiding this comment.
Since we might be adding few more smoke tests to this, I felt the script might become too huge and it might not be scaled. Yeah, lets move these to a folder "thirdparty-testing" and add a script there to trigger test. Please let me know if your hands are full, I can work on this.
There was a problem hiding this comment.
Ok, I will push changes here.
There was a problem hiding this comment.
I just pushed changes moving cvxpy tests to their own file, as @tmckayus had originally (but using the name thirdparty-testing/ you asked for).
Hopefully this will set up a pattern that can be reliably replicated when other integration tests like this are added.
I will check back in an hour or so to be sure this worked. I especially want to see evidence in the logs that the just-built-in-CI cuopt, cuopt-mps-parser, etc. wheels were used, not packages downloaded from the nightly index.
|
I've removed my blocking review. Sorry @rgsl888prabhu , I wasn't able to get this done today, and I'll be unavailable all of next week. Hopefully I've written enough here to explain my concerns with the original approach and show the direction I want to go. Talk with others in |
|
Python tests are failing in cvxpy because of this issue #233 |
|
@jameslamb May I get your review on this PR |
jameslamb
left a comment
jameslamb
left a comment
There was a problem hiding this comment.
This is looking good! Checked the logs, confirmed that the just-built-in-CI cuopt packages are being installed, so this really is testing that the work with cvxpy:
Processing ./dist/cvxpy-1.8.0-cp310-cp310-linux_x86_64.whl (from cvxpy==1.8.0)
...
Processing /tmp/tmp.YgYbfzhLn4/cuopt_cu12-25.8.0a93-cp310-cp310-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl (from cvxpy==1.8.0->cvxpy==1.8.0)
Requirement already satisfied: nvidia-cuda-runtime-cu12<13.0,>=12.8 in /pyenv/versions/3.10.18/lib/python3.10/site-packages (from cvxpy==1.8.0->cvxpy==1.8.0) (12.9.79)
...
Requirement already satisfied: cudf-cu12==25.8.*,>=0.0.0a0 in /pyenv/versions/3.10.18/lib/python3.10/site-packages (from cuopt-cu12>=25.5->cvxpy==1.8.0->cvxpy==1.8.0) (25.8.0a583)
Processing /tmp/tmp.qP4l2LcWH8/cuopt_mps_parser-25.8.0a93-cp310-cp310-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl (from cuopt-cu12>=25.5->cvxpy==1.8.0->cvxpy==1.8.0)
...
cuopt-cu12 is already installed with the same version as the provided wheel. Use --force-reinstall to force an installation of the wheel.
cuopt-mps-parser is already installed with the same version as the provided wheel. Use --force-reinstall to force an installation of the wheel.
libcuopt-cu12 is already installed with the same version as the provided wheel. Use --force-reinstall to force an installation of the wheel.
...
Installing collected packages: scipy, pycparser, joblib, scs, osqp, hypothesis, cffi, pytest-error-for-skips, clarabel, cvxpy
Successfully installed cffi-1.17.1 clarabel-0.11.1 cvxpy-1.8.0 hypothesis-6.136.6 joblib-1.5.1 osqp-1.0.4 pycparser-2.22 pytest-error-for-skips-2.0.2 scipy-1.15.3 scs-3.2.7.post2
Notice there that cuopt packages don't appear in the "installed" list at the end of that, but instead in earlier log messages saying that the already-installed packages will be used.
I left one small suggestion, do whatever you want with it and I do not need to re-approve.
Co-authored-by: James Lamb <jaylamb20@gmail.com>
|
/merge |
Thank you @jameslamb, @tmckayus most of the work was completed by you |
3 participants
This is a simple smoketest to look for breakage and any abnormal behavior with respect to runtime from cuopt Python API changes. It does not do any exhaustive performance testing or building.
depends on #235