Gurobipy Environment Handling #588

torressa · 2022-10-03T17:16:44Z

Fix Gurobi: If status not optimal, solution cannot be retrieved #586.
For Gurobi environment handling #571
- Fix issue with pulp.list_solvers. Cheers @simonbowly!
- Added three new parameters to GUROBI that allow users more fine control over the environments used.
  - env: gp.Env users environment to use.
  - envOptions: dict, options to give to the environment if manageEnv is True.
  - manageEnv: bool: whether environments are handled internally or not default is false (as it was before).
- Added a new function to free the environments and models properly close. If users are using their own environments or using manageEnv=True, they should always call this function afterwards.

Sample use:

# Pass environment as a parameter
with gp.Env(params=env_options) as env:
  prob = generate_lp()
  solver = GUROBI(msg=True, env=env, **solver_options)
  prob.solve(solver)
  # Dispose model (required to correctly free env)
  solver.close()

Or,

# Use manageEnv=True and pulp will take care
solver = GUROBI(msg=True, manageEnv=True, envOptions=env_options, **solver_options)
prob = generate_lp()
prob.solve(solver)
# Dispose env + model
solver.close()

CLAassistant · 2022-10-03T17:16:52Z

All committers have signed the CLA.

…odel.addConstr

EwoutH · 2022-12-11T12:15:31Z

This looks quite cool! What's needed to move this further? A review from @pchtsp?

pulp/apis/gurobi_api.py

Tomkourou · 2023-04-26T12:07:35Z

Commenting to move this along. Problems with the gurobipy single-use licenses are still unresolved.

torressa · 2023-05-03T14:50:27Z

@simon-b I've just added some usage explanations to the doc-string, should be clearer now.
(tested building the docs on my machine and was OK).
Also rebased master, hope this is merged soon!

pulp/apis/gurobi_api.py

simon-b

Looks great to me! Test on my side with various options for managing the gurobi env and all seem to work properly.

Suggest a review from @pchtsp and hopefully can get this closed soon !

pulp/tests/test_gurobipy_env.py

simon-b · 2023-07-21T15:48:17Z

pulp/tests/test_pulp.py

-                prob, self.solver, [const.LpStatusOptimal], {x: 4, y: -1, z: 6, w: 0}
-            )
+            if self.solver.name not in [
+                "GUROBI_CMD",  # end is a key-word for LP files


Not clear to me why this was changed, when otherwise the PR doesn't seem to make any changes to GUROBI_CMD 🤔

I think this must've not been tested using GUROBI_CMD. Variable z is called End which is a keyword for LP files hence results in the LP file being solved being pretty different

Minimize 9 Subject To Bounds End

(I double-checked versions of Gurobi back to 8.0.0 with the same result)

Unless this test is supposed to fail, as the name suggests.

- Add decorator to skip `test_measuring_solving_time` as it exceeds the pip-license size limitations. - Add `test_gurobipy_env` to pulptest which will be skipped if `gurobipy` is not installed.

torressa · 2023-08-04T12:03:08Z

The tests should now run using the Gurobi license included when installing gurobipy via pip install (which I've included in the workflow). There is one test where the size limitation is reached, for which I've added a decorator. This test will run fine if a full license is picked up.

Unfortunately, the pip license will not be picked up by gurobi_cl or the GUROBI_CMD interface.

Therefore the best thing would be to generate and use a Gurobi WLS license that can be added to the workflow via a secret.

For now, we can test this by re-running the actions but a maintainer needs to trigger that.

stumitchell · 2023-08-06T23:27:45Z

@torressa we are getting a test fail now,
Also can we remove the restricted licence warning from the test output?

torressa · 2023-08-07T19:04:48Z

Hi @stumitchell! If you mean the tests on the CI, I am not sure why, the errors are:

 File "/opt/hostedtoolcache/Python/3.9.17/x64/lib/python3.9/site-packages/pulp/apis/highs_api.py", line 327, in buildSolverModel
    lp.solverModel.addCol(
ModuleNotFoundError: No module named 'numpy'

This is weird cause it passes on macos and also cause highspy doesn't use numpy.
It seems that it is broken also on all branches

I tried removing the warning from the output (I had forgotten quite a few places).

stumitchell · 2023-08-08T00:37:03Z

Right, I see the issue I have contacted @aphi about it

We probably want to get master fixed before we merge anything new

pchtsp · 2023-08-08T06:55:16Z

Thanks @stumitchell

I was waiting (too) patiently for the numpy issue to resolve in highspy. I did forget to create an issue there : ) so thanks for doing so.

I've currently taken out the installation of the highspy python package from the github CI on the master branch. This should solve this issue.

@torressa can you re-merge master please? Sorry for the lack of response, these months have been very busy.

F.

aphi · 2023-08-08T07:01:19Z

Looks like we both investigated this at the same time 😬.

I have a PR which keeps the testcase for highspy but just includes numpy, which also resolves the issue.

#677

torressa · 2023-08-08T11:00:49Z

No worries @pchtsp! Merged.
I just opened an issue in the HiGHS repo so that they are aware: ERGO-Code/HiGHS#1378

torressa and others added 5 commits October 4, 2022 09:34

Add environment handling and context manager

9930449

Dispose of solvers in listSolvers

f08b8bb

Close environment in Gurobi availability check

691e7e9

Add environment handling and context manager

70c156d

[coin-or/pulp/586] Fix and add test. Remove deprecation message for m…

72af8a1

…odel.addConstr

torressa force-pushed the gurobipy-environments branch from c892b05 to 72af8a1 Compare October 4, 2022 08:35

torressa changed the title ~~Gurobipy updates~~ Gurobipy Environment Handling Jan 11, 2023

simon-b reviewed Apr 20, 2023

View reviewed changes

pulp/apis/gurobi_api.py Outdated Show resolved Hide resolved

torressa added 3 commits May 2, 2023 11:47

Address reviewer, rename tests, fix broken tests

b95e061

Merge branch 'master' into gurobipy-environments

cb2f914

Update docs, add test

3d00c59

torressa commented May 3, 2023

View reviewed changes

pulp/apis/gurobi_api.py Show resolved Hide resolved

torressa requested a review from simon-b July 20, 2023 13:58

simon-b approved these changes Jul 21, 2023

View reviewed changes

torressa added 3 commits August 1, 2023 09:06

Remove unused import

73786c0

Add gurobipy installation to CI

b15e899

Test changes to work with the pip-license.

edba8c2

- Add decorator to skip `test_measuring_solving_time` as it exceeds the pip-license size limitations. - Add `test_gurobipy_env` to pulptest which will be skipped if `gurobipy` is not installed.

torressa added 2 commits August 7, 2023 18:44

Remove output from tests

78b54d3

Merge branch 'coin-or:master' into gurobipy-environments

0925f79

Merge remote-tracking branch 'fork/master' into gurobipy-environments

9001f9f

pchtsp merged commit 42f91ab into coin-or:master Sep 24, 2023

torressa deleted the gurobipy-environments branch September 27, 2023 09:41

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Gurobipy Environment Handling #588

Gurobipy Environment Handling #588

torressa commented Oct 3, 2022 •

edited

Loading

CLAassistant commented Oct 3, 2022 •

edited

Loading

EwoutH commented Dec 11, 2022

Tomkourou commented Apr 26, 2023

torressa commented May 3, 2023

simon-b left a comment

simon-b Jul 21, 2023

torressa Aug 1, 2023 •

edited

Loading

torressa Aug 1, 2023

torressa commented Aug 4, 2023 •

edited

Loading

stumitchell commented Aug 6, 2023

torressa commented Aug 7, 2023 •

edited

Loading

stumitchell commented Aug 8, 2023

pchtsp commented Aug 8, 2023

aphi commented Aug 8, 2023

torressa commented Aug 8, 2023 •

edited

Loading

Gurobipy Environment Handling #588

Gurobipy Environment Handling #588

Conversation

torressa commented Oct 3, 2022 • edited Loading

CLAassistant commented Oct 3, 2022 • edited Loading

EwoutH commented Dec 11, 2022

Tomkourou commented Apr 26, 2023

torressa commented May 3, 2023

simon-b left a comment

Choose a reason for hiding this comment

simon-b Jul 21, 2023

Choose a reason for hiding this comment

torressa Aug 1, 2023 • edited Loading

Choose a reason for hiding this comment

torressa Aug 1, 2023

Choose a reason for hiding this comment

torressa commented Aug 4, 2023 • edited Loading

stumitchell commented Aug 6, 2023

torressa commented Aug 7, 2023 • edited Loading

stumitchell commented Aug 8, 2023

pchtsp commented Aug 8, 2023

aphi commented Aug 8, 2023

torressa commented Aug 8, 2023 • edited Loading

torressa commented Oct 3, 2022 •

edited

Loading

CLAassistant commented Oct 3, 2022 •

edited

Loading

torressa Aug 1, 2023 •

edited

Loading

torressa commented Aug 4, 2023 •

edited

Loading

torressa commented Aug 7, 2023 •

edited

Loading

torressa commented Aug 8, 2023 •

edited

Loading