[FEA] Add instructions/notebook for using cuOpt from GAMS(Py) and potentially build and deploy the link in CI

[0x17]

At GAMS we are impressed with the significant speed gains from using PDLP on GPUs for solving huge LPs in general and the competitive cuOpt solver implementation from NVIDIA in particular. Hence allowing users of the GAMS modeling language (and the new GAMSPy package for modeling in Python) to solve their extensive collections of LP and MILP optimization models for bigger instances with cuOpt as solver via an efficient link would be something we would love to help enabling.

As a crucial step, we wrote an initial implementation of a GAMS cuOpt solver link that allows GAMS and GAMSPy models to be solved via cuOpt by doing something as simple as setting the LP/MIP solver to cuOpt, e.g., gams trnsport lp=cuopt. The link is written in plain C and uses the cuOpt C-API to communicate with cuOpt, which was refreshingly straightforward to use.

As the link relies on cuOpt and transitively on various shared libraries from the CUDA runtime plus a recent GAMS installation both for building and running, we were wondering if it might be sensible (for us) to create a pull request here that adds building and testing the solver link for cuOpt to the workflows of the cuopt repository and then deploys the resulting binary, so users can download them for insertion into their GAMS installation folder. When including it in a pull request for this repository, we would of course be willing to make the source code of the solver link for cuOpt available under the Apache 2.0 license.

While we could also bundle the link with the GAMS system installer, we are hesitant to do so because it might be easy to get a mismatch of the ABI the link is built against and the version of cuOpt the user has installed and instead bundling all dependencies seems a bit excessive considering there will also be users that cannot take advantage of cuOpt due to not having an appropriate GPU installed on their Linux machine.

When there is an established way to make the link available, it would also be nice to add either an example Python notebook or usage documentation for using cuOpt from GAMS as an additional alternative to the already established interfaces through C, Python, or the REST-API. The instructions will probably mostly be related to setting up the solver link for cuOpt (and having all dependencies for cuOpt itself) as afterwards any GAMS model (of appropriate model type) just needs to set “cuOpt” as choice for the LP, MIP, or RMIP and then the optimization will magically happen with cuOpt.

If the cuOpt team has different ideas for making the GAMS <-> cuOpt integration available for the end user, we are very much open to suggestions. Independent of the specific technical solution, there is a significant portion of GAMS users that would benefit from having cuOpt easily accessible as solver for their large scale linear programs and this would also help stress testing cuOpt with a big pile of models already implemented in the GAMS modeling language.

[afender]

That’s awesome! Excited to see GAMS and GAMSPy models solved via cuOpt. We have been exploring ways to do what you requested. We’ll follow up shortly on how we can best help. @tmckayus @Iroy30 @rgsl888prabhu and @chris-maes for viz.

[tmckayus]

Agree with @afender, this is great! Thank you for your work
cc @chris-maes
cc @bbozkaya
cc @rgsl888prabhu
cc @rg20

We've started discussion about this internally. You're right, ABI stability will be a concern for anyone using the C API via the shared library, not just GAMS. With summer schedules it may take some time to reach a final decision, but this is our proposal currently (subject to change as we discuss, please treat as such):

1. We don't think we can scale to accommodate integrations within the core cuopt repository. As the number of integrations grows, maintaining them here becomes a resource challenge (both human and machine)

2. We agree that ABI stability is very important

3. We will apply best practices and testing (AIBCC? others) to guarantee that the ABI is stable at least within a minor release (ie, it will never change in a patch release so all 25.05.x releases would have the same ABI) . This should be achievable -- the Linux kernel and others do this all the time. We are also looking at adding semantic versioning in addition to the standard "year.month" release versions common at NVIDIA. This might be a second version number, for example "release 25.05, interface version 1.0.0". So for anything with a matching major.minor interface version, ABI compatibility can be assumed. Multiple year.month releases might have the same interface version, for example 25.05, 25.08, and 25.11 might all have 1.0.x interface versions.

4. GAMS and other integrations can list compatible cuopt versions (ie, version Z of GAMS works with cuopt 25.05 - 25.08 or similar). Customers can look up this list, or it could even be handled programmatically (if a compatible cuopt package is installed, then the solver is available otherwise not). This way, GAMS with a supported version of cuopt should always be stable.

5. If GAMS or another integration wants to work with development versions, these should be available from cuopt nightly builds. However, we may from time to time break the ABI in nightlies (this is why we add tests :) ).

6. Testing of the integration should be done outside of the cuopt repo by the implementers

Thanks! I personally will be OOTO for a few days so may be slow to respond, but others may chime in here.

[0x17]

Thank you for sharing the initial proposal. That sounds like a sensible strategy to ensure ABI compatibility.

I have created a new (but currently still private) GitHub repository GAMS-dev/cuoptlink-builder to build and package the GAMS cuOpt link as part of its CI pipeline. In its GitHub actions workflow it fetches the NVIDIA runtime and GAMS and builds the link. It then packages the link together with the shared libraries for cuOpt and the necessary GAMS configuration files into a big zip archive. When the workflow runs for a new tag, it also creates a corresponding release for it and uploads the zip file as artifact. Expanding the resulting archive into a GAMS system directory is enough to make it support gams xyz.gms lp=cuopt. I also adapted the GAMSPy transportation model example notebook to use cuOpt and put that into the new repository.

To also have a testing covered by CI we will look into making our spicy NVIDIA rig available as a GitHub runner to the new repository but I am am optimistic that won't be too difficult after finishing the build step.

Having our own repository to build (and soon test) the solver link then reduces the scope of the issue #188 here. From my understanding, a PR in NVIDIA/cuopt-repo would then just mention our repository for the GAMS solver link in the cuOpt docs markdown. Potentially a PR for the NVIDIA/cuopt-examples-repo could add examples of using GAMS with cuOpt.

[rgsl888prabhu]

Thank you for sharing the initial proposal. That sounds like a sensible strategy to ensure ABI compatibility.

I have created a new (but currently still private) GitHub repository GAMS-dev/cuoptlink-builder to build and package the GAMS cuOpt link as part of its CI pipeline. In its GitHub actions workflow it fetches the NVIDIA runtime and GAMS and builds the link. It then packages the link together with the shared libraries for cuOpt and the necessary GAMS configuration files into a big zip archive. When the workflow runs for a new tag, it also creates a corresponding release for it and uploads the zip file as artifact. Expanding the resulting archive into a GAMS system directory is enough to make it support gams xyz.gms lp=cuopt. I also adapted the GAMSPy transportation model example notebook to use cuOpt and put that into the new repository.

To also have a testing covered by CI we will look into making our spicy NVIDIA rig available as a GitHub runner to the new repository but I am am optimistic that won't be too difficult after finishing the build step.

Having our own repository to build (and soon test) the solver link then reduces the scope of the issue #188 here. From my understanding, a PR in NVIDIA/cuopt-repo would then just mention our repository for the GAMS solver link in the cuOpt docs markdown. Potentially a PR for the NVIDIA/cuopt-examples-repo could add examples of using GAMS with cuOpt.

Agree, adding GAMS example to example repo would be perfect. And the docs will provide details and links to GAMs repository and any instruction or example towards GAM.

[0x17]

I created two pull request drafts to address this issue

1. Mention GAMS and GAMSPy in third-party modeling languages page in documentation in cuopt
2. Adding GAMSPy transportation model example as Jupyter notebook in cuopt-examples

A core requirement for both is making the repository for the solver link cuoptlink-builder public which will happen shortly before the GAMS blog article on the GAMS & cuOpt integration is getting published.

[rgsl888prabhu]

Thank you @0x17 , I will start reviewing these so by the time it is public, we are good to merge.