Faster engine compile time #316

Kh4ster · 2025-08-18T15:07:38Z

This PR aims at making the compile time faster by adding several options:

build lp only (extracted MIP files necessary in LP), also meant we needed a skip c and python adapter option
put back no fetch rapids
put back not building test option
added skip routing
added skip write of fatbin
adding ccache to libcuopt target
added two benchmarking targets, one for LP and for MIP

rgsl888prabhu

Thank you @Kh4ster for the PR, I have requested few changes and clarification.

rgsl888prabhu · 2025-08-18T15:14:10Z

build.sh

+SKIP_C_PYTHON_ADAPTERS=0
+SKIP_TESTS_BUILD=0
+SKIP_ROUTING_BUILD=0
+WRITE_FATBIN=0


WRITE FATBIN should be by default enabled, and disabled with option

rgsl888prabhu · 2025-08-18T15:14:54Z

cpp/CMakeLists.txt

 option(BUILD_TESTS "Configure CMake to build tests" ON)
 option(DISABLE_OPENMP "Disable OpenMP" OFF)
 option(CUDA_STATIC_RUNTIME "Statically link the CUDA toolkit runtime and libraries" OFF)
+option(BUILD_LP_ONLY "Build only linear programming components, exclude routing and MIP-specific files" OFF)


Missing BUILD_TESTS
WRITE_FATBIN and BUILD_FATBIN is being used.

Build test is above. We had it already but it was not used and not working in practice

Good catch for the BUILD_FATBIN!

rgsl888prabhu · 2025-08-18T15:15:08Z

build.sh

 BUILD_CI_ONLY=0
+BUILD_LP_ONLY=0
+SKIP_C_PYTHON_ADAPTERS=0
+SKIP_TESTS_BUILD=0


Dow we need to add option to skip BUILD_MIP_BENCHMARKS

I think we can keep this for now and see if someone needs it

rgsl888prabhu · 2025-08-18T15:16:29Z

cpp/CMakeLists.txt

-  PRIVATE "$<$<COMPILE_LANGUAGE:CXX>:${CUOPT_CXX_FLAGS}>"
-  "$<$<COMPILE_LANGUAGE:CUDA>:${CUOPT_CUDA_FLAGS}>"
-)
+if(NOT BUILD_LP_ONLY)


Should this be

Suggested change

if(NOT BUILD_LP_ONLY)

if(BUILD_LP_ONLY)

No I confirm we don't want to have cuopt_cli if we are in lp_only mode since there is a call to the MIP solver

rgsl888prabhu · 2025-08-18T15:17:31Z

cpp/tests/CMakeLists.txt

-add_library(cuopttestutils STATIC
-    routing/utilities/check_constraints.cu
-)
+if(BUILD_TESTS)


Is there an option where we can skip directly from cpp/CMakefiles.txt

I don't know any. That's the only way I found to truely diable the compilation of tests

rg20 · 2025-08-18T17:54:02Z

cpp/libmps_parser/CMakeLists.txt

-}
-]=])
-target_link_options(mps_parser PRIVATE "${MPS_PARSER_BINARY_DIR}/fatbin.ld")
+if(WRITE_FATBIN)


Remove this always?

What do you mean? Remove the fatbin writing completly?

I think he is asking, by default currently this was disabled, so do we need to disable it by default might be the question. Think I also had asked this question.

And you have changed it if I am not wrong.

copy-pr-bot · 2025-08-19T08:53:27Z

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

Kh4ster · 2025-08-19T08:56:15Z

/ok to test 0e3191a

Kh4ster · 2025-08-19T09:13:01Z

/ok to test 3a9bac7

Kh4ster · 2025-08-19T09:37:27Z

/ok to test 7bc2ce4

Kh4ster · 2025-08-20T08:56:46Z

/ok to test 2ad2c73

copy-pr-bot · 2025-08-20T08:56:49Z

/ok to test 2ad2c73

@Kh4ster, there was an error processing your request: E2

See the following link for more information: https://docs.gha-runners.nvidia.com/cpr/e/2/

Kh4ster · 2025-08-20T09:01:20Z

/ok to test 33e9344

cpp/tests/CMakeLists.txt

Kh4ster · 2025-08-20T11:29:58Z

/ok to test 3a3e7c9

rg20 · 2025-08-25T12:53:27Z

benchmarks/linear_programming/utils/benchmark_lp_mittelmann.sh

    instance_name=$(basename $instance)
    echo "Parsing ${instance_name}.mps then solving"
-    ${CUOPT_HOME}/cpp/build/solve_MPS_file --path ${CUOPT_HOME}/benchmarks/linear_programming/datasets/${instance_name}/${instance_name}.mps --time-limit 3600 # --solution-path $CUOPT_HOME/benchmarks/linear_programming/datasets/$instance.sol
+    ${CUOPT_HOME}/cpp/build/solve_LP --path ${CUOPT_HOME}/benchmarks/linear_programming/datasets/${instance_name}/${instance_name}.mps --time-limit 3600 # --solution-path $CUOPT_HOME/benchmarks/linear_programming/datasets/$instance.sol


Let's start using cuopt_cli here? That's what Hans would be using as well.

same comment as below

Here, you are not using any of those additional settings/options from run_pdlp right? We can still use run_pdlp for when benchmarking with those settings.

rg20 · 2025-08-25T12:56:58Z

cpp/CMakeLists.txt

-  target_compile_options(solve_MPS_file
+option(BUILD_LP_BENCHMARKS "Build LP benchmarks" OFF)
+if(BUILD_LP_BENCHMARKS)
+  add_executable(solve_LP ../benchmarks/linear_programming/cuopt/run_pdlp.cu)


For benchmarking, I think we should directly use the cuopt_cli, because that's what everyone will use

The issue I have is run_pdlp has more options that cuopt_cli

Kh4ster · 2025-09-01T07:43:03Z

/ok to test 3a3e7c9

rgsl888prabhu · 2025-09-01T19:12:32Z

/ok to test 50e56b8

Kh4ster · 2025-09-03T07:50:22Z

/ok to test c241602

aliceb-nv

Approving :) Awesome work Nicolas, this is going to be super useful
Is it ready to merge?

rg20 · 2025-09-03T14:00:43Z

/merge

rg20 · 2025-09-03T14:14:16Z

/merge

This PR aims at making the compile time faster by adding several options: - build lp only (extracted MIP files necessary in LP), also meant we needed a skip c and python adapter option - put back no fetch rapids - put back not building test option - added skip routing - added skip write of fatbin - adding ccache to libcuopt target - added two benchmarking targets, one for LP and for MIP Authors: - Nicolas Blin (https://github.com/Kh4ster) Approvers: - Alice Boucher (https://github.com/aliceb-nv) - Rajesh Gandham (https://github.com/rg20) - Ramakrishnap (https://github.com/rgsl888prabhu) URL: #316

several improvements to make the compile time faster

fa32ca4

Kh4ster requested review from a team as code owners August 18, 2025 15:07

Kh4ster requested review from akifcorduk, jakirkham, rg20 and rgsl888prabhu and removed request for jakirkham August 18, 2025 15:07

Kh4ster added non-breaking Introduces a non-breaking change improvement Improves an existing functionality labels Aug 18, 2025

anandhkb added this to the 25.10 milestone Aug 18, 2025

rgsl888prabhu reviewed Aug 18, 2025

View reviewed changes

rg20 reviewed Aug 18, 2025

View reviewed changes

changed fatbin write to be on by default

4baece1

Merge branch 'branch-25.10' into faster-build

0e3191a

disable lp benchmark build by default

3a9bac7

put back correct link library and update lp benchmark script

7bc2ce4

Kh4ster and others added 3 commits August 20, 2025 10:43

Merge branch 'branch-25.10' into faster-build

2ad2c73

update names in argparse

ec33c0f

disable mip benchmark binary building by default

33e9344

akifcorduk reviewed Aug 20, 2025

View reviewed changes

cpp/tests/CMakeLists.txt Show resolved Hide resolved

skip correct tests accoring to flags

3a3e7c9

rg20 reviewed Aug 25, 2025

View reviewed changes

Merge branch 'branch-25.10' into faster-build

50e56b8

Merge branch 'branch-25.10' into faster-build

c241602

aliceb-nv approved these changes Sep 3, 2025

View reviewed changes

rg20 approved these changes Sep 3, 2025

View reviewed changes

rgsl888prabhu approved these changes Sep 3, 2025

View reviewed changes

rapids-bot bot merged commit eac3d69 into branch-25.10 Sep 3, 2025
73 checks passed

Kh4ster deleted the faster-build branch October 22, 2025 07:59

Faster engine compile time #316

Faster engine compile time #316

Uh oh!

Conversation

Kh4ster commented Aug 18, 2025

Uh oh!

rgsl888prabhu left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

copy-pr-bot bot commented Aug 19, 2025

Uh oh!

Kh4ster commented Aug 19, 2025

Uh oh!

Kh4ster commented Aug 19, 2025

Uh oh!

Kh4ster commented Aug 19, 2025

Uh oh!

Kh4ster commented Aug 20, 2025

Uh oh!

copy-pr-bot bot commented Aug 20, 2025

Uh oh!

Kh4ster commented Aug 20, 2025

Uh oh!

Uh oh!

Kh4ster commented Aug 20, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Kh4ster commented Sep 1, 2025

Uh oh!

rgsl888prabhu commented Sep 1, 2025

Uh oh!

Kh4ster commented Sep 3, 2025

Uh oh!

aliceb-nv left a comment

Choose a reason for hiding this comment

Uh oh!

rg20 commented Sep 3, 2025

Uh oh!

rg20 commented Sep 3, 2025

Uh oh!

Uh oh!