Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add error message when solve fails due to missing license - Feature Request #116

Closed
mitchphillipson opened this issue May 24, 2024 · 22 comments · Fixed by #118
Closed

Add error message when solve fails due to missing license - Feature Request #116

mitchphillipson opened this issue May 24, 2024 · 22 comments · Fixed by #118

Comments

@mitchphillipson
Copy link
Contributor

Issue

The solver silently fails when a model has more than ~300 variables and no license. Since the PATH license needs to be updated on a yearly basis it's possible it expires unknowingly. This leads to many issues debugging models when the license has silently lapsed.

Proposed Solution

Throw an error when a model does not solve due to a license error.
@odow
Copy link
Collaborator

odow commented May 25, 2024

The termination_status(model) should be OTHER_ERROR, and raw_status(model) should be "License could not be found".

Is it not clear what is happening if you do solution_summary(model)?

@odow
Copy link
Collaborator

odow commented May 25, 2024

If you use the C API, then the return is MCP_LicenseError:

PATHSolver.jl/src/C_API.jl

Line 728 in d91e9b1

return MCP_LicenseError, nothing, nothing

The general approach in MOI is to avoid erroring where possible on optimize! and return useful information via statuses, so I'm hesitant to make a change here.

@mitchphillipson
Copy link
Contributor Author

The issue was in a GitHub test. One of the test cases was too large (which I didn't check, my fault). However, I am receiving a Segmentation Fault error.

image

I know this is in the call to optimize! because I've surrounded it in print statements, just to isolate. This is an example PR with the error. The error occurs in the test_orientation.jl test. I have observed that the test can't be isolated, if it's the only test it fails gracefully, as expected.

Getting a MWE is kind of tricky because it would involve changing my local workflow which is a little dangerous for me.

@odow
Copy link
Collaborator

odow commented May 28, 2024

Hmm. We shouldn't segfault. Let me take a look.

@odow
Copy link
Collaborator

odow commented May 28, 2024

This might actually be because of #114.

Let me make a new release.

@odow
Copy link
Collaborator

odow commented May 28, 2024

Try v1.7.6.

I don't think this is a license error. Even on v1.7.5, we should have returned before we initialized the output:

PATHSolver.jl/src/C_API.jl

Lines 757 to 762 in d6fc14e

if c_api_Path_CheckLicense(length(z), nnz) == 0
return MCP_LicenseError, nothing, nothing
end
@assert length(z) == length(lb) == length(ub)
out_io = silent ? IOBuffer() : stdout
output_data = OutputData(out_io)

@mitchphillipson
Copy link
Contributor Author

mitchphillipson commented May 29, 2024 via email

@odow
Copy link
Collaborator

odow commented May 29, 2024

That still could be consistent. Without the license we return early, and GC happens at a bad time. With the license, we solve, and the GC cleans things up okay. Dunno. I guess see if the new version fixes things first.

@odow
Copy link
Collaborator

odow commented Jun 4, 2024

Any update?

@mitchphillipson
Copy link
Contributor Author

Sorry, I've been traveling. I have time today to test this.

@mitchphillipson
Copy link
Contributor Author

I've updated PATHSolver to 1.7.6 and JuMP to 1.22.1 and still get a segfault here is a link to the output in GitHub, I think you're able to see this.

I'm a little constrained on time at the moment, but I'm going to try to get a local MWE of this issue. This may take me a some time. Luckily, this issue isn't pressing and seems to be a fairly edge case.

I'd be fine with this being a low priority, at least for you, until I can more accurately explain what makes this happen.

@mitchphillipson
Copy link
Contributor Author

I'm back with testing and some strange behavior. For context, everything is up-to-date and I'm running locally without a license. If the first model I run fails due a license error, everything behaves as expected.

using JuMP
using PATHSolver

M2 = Model(PATHSolver.Optimizer)

I = Symbol.("a",1:500)
@variable(M2, x[I])
@constraint(M2, [i=I], x[i]^2-2*x[i]+1 ⟂ x[i])
optimize!(M2)

termination_status(M2)

This shows OTHER_ERROR::TerminationStatusCode = 24, which is expected.

However, if that model is NOT solved first, then we get unexpected behavior

using JuMP
using PATHSolver

M1 = Model(PATHSolver.Optimizer)
@variable(M1, x)
@constraint(M1, x^2-2*x+1 ⟂ x)

optimize!(M1)

M2 = Model(PATHSolver.Optimizer)

I = Symbol.("a",1:500)
@variable(M2, x[I])
@constraint(M2, [i=I], x[i]^2-2*x[i]+1 ⟂ x[i])
optimize!(M2)

termination_status(M2)

This has some variation. Either the model will hang on optimize!, fail to display any issues, or throw a helpful "Demo license: size restriction of 300 variables or 2000 nonzeros exceeded." message.

In the example above, the model hangs. To get the "Demo license" message I need to optimize the M2 model with fewer than 300 variables, then increase to above 300. Further testing shows this to be unreliable.

I also received this error once:

TypeError: in typeassert, expected PATHSolver.OutputData, got a value of type PATHSolver.Options(Ptr{Nothing} @0x0000020352eb3630)

which feels like a kinder segfault.

These errors all go away if I include the license. This could also be an error in PATH itself, perhaps it doesn't explicitly check for a license after the first solve but still fails due to lack of license on subsequent solves.

@odow
Copy link
Collaborator

odow commented Jun 11, 2024

Is this on Linux? Or Windows?

I cannot reproduce on Mac:

julia> using JuMP, PATHSolver

julia> begin
           model = Model(PATHSolver.Optimizer)
           @variable(model, x)
           @constraint(model, x^2 - 2 * x + 1 ⟂ x)
           optimize!(model)
       end
Path 5.0.03 (Fri Jun 26 09:58:07 2020)
Written by Todd Munson, Steven Dirkse, Youngdae Kim, and Michael Ferris


Crash Log
major  func  diff  size  residual    step       prox   (label)
    0     0             1.0000e+00             0.0e+00 ()
    1     1     0     1 2.5000e-01  1.0e+00    0.0e+00 ()
pn_search terminated: no basis change.

Major Iteration Log
major minor  func  grad  residual    step  type prox    inorm  (label)
    0     0     2     2 2.5000e-01           I 0.0e+00 2.5e-01 ()
    1     1     3     3 6.2500e-02  1.0e+00 SO 0.0e+00 6.2e-02 ()
    2     1     4     4 1.5625e-02  1.0e+00 SO 0.0e+00 1.6e-02 ()
    3     1     5     5 3.9062e-03  1.0e+00 SO 0.0e+00 3.9e-03 ()
    4     1     6     6 9.7656e-04  1.0e+00 SO 0.0e+00 9.8e-04 ()
    5     1     7     7 2.4414e-04  1.0e+00 SO 0.0e+00 2.4e-04 ()
    6     1     8     8 6.1035e-05  1.0e+00 SO 0.0e+00 6.1e-05 ()
    7     1     9     9 1.5259e-05  1.0e+00 SO 0.0e+00 1.5e-05 ()
    8     1    10    10 3.8147e-06  1.0e+00 SO 0.0e+00 3.8e-06 ()
    9     1    11    11 9.5367e-07  1.0e+00 SO 0.0e+00 9.5e-07 ()

Major Iterations. . . . 9
Minor Iterations. . . . 9
Restarts. . . . . . . . 0
Crash Iterations. . . . 1
Gradient Steps. . . . . 0
Function Evaluations. . 11
Gradient Evaluations. . 11
Basis Time. . . . . . . 0.000022
Total Time. . . . . . . 0.001435
Residual. . . . . . . . 9.536743e-07

julia> begin
           model_2 = Model(PATHSolver.Optimizer)
           @variable(model_2, x[1:500])
           @constraint(model_2, [i in 1:500], x[i]^2 - 2 * x[i] + 1 ⟂ x[i])
           optimize!(model_2)
           termination_status(model_2)
       end
OTHER_ERROR::TerminationStatusCode = 24

julia> using JuMP, PATHSolver

julia> begin
           model = Model(PATHSolver.Optimizer)
           @variable(model, x)
           @constraint(model, x^2 - 2 * x + 1 ⟂ x)
           optimize!(model)
       end
Path 5.0.03 (Fri Jun 26 09:58:07 2020)
Written by Todd Munson, Steven Dirkse, Youngdae Kim, and Michael Ferris


Crash Log
major  func  diff  size  residual    step       prox   (label)
    0     0             1.0000e+00             0.0e+00 ()
    1     1     0     1 2.5000e-01  1.0e+00    0.0e+00 ()
pn_search terminated: no basis change.

Major Iteration Log
major minor  func  grad  residual    step  type prox    inorm  (label)
    0     0     2     2 2.5000e-01           I 0.0e+00 2.5e-01 ()
    1     1     3     3 6.2500e-02  1.0e+00 SO 0.0e+00 6.2e-02 ()
    2     1     4     4 1.5625e-02  1.0e+00 SO 0.0e+00 1.6e-02 ()
    3     1     5     5 3.9062e-03  1.0e+00 SO 0.0e+00 3.9e-03 ()
    4     1     6     6 9.7656e-04  1.0e+00 SO 0.0e+00 9.8e-04 ()
    5     1     7     7 2.4414e-04  1.0e+00 SO 0.0e+00 2.4e-04 ()
    6     1     8     8 6.1035e-05  1.0e+00 SO 0.0e+00 6.1e-05 ()
    7     1     9     9 1.5259e-05  1.0e+00 SO 0.0e+00 1.5e-05 ()
    8     1    10    10 3.8147e-06  1.0e+00 SO 0.0e+00 3.8e-06 ()
    9     1    11    11 9.5367e-07  1.0e+00 SO 0.0e+00 9.5e-07 ()

Major Iterations. . . . 9
Minor Iterations. . . . 9
Restarts. . . . . . . . 0
Crash Iterations. . . . 1
Gradient Steps. . . . . 0
Function Evaluations. . 11
Gradient Evaluations. . 11
Basis Time. . . . . . . 0.000017
Total Time. . . . . . . 0.001030
Residual. . . . . . . . 9.536743e-07

julia> begin
           model_2 = Model(PATHSolver.Optimizer)
           @variable(model_2, x[1:500])
           @constraint(model_2, [i in 1:500], x[i]^2 - 2 * x[i] + 1 ⟂ x[i])
           optimize!(model_2)
           termination_status(model_2)
       end
OTHER_ERROR::TerminationStatusCode = 24

julia> using JuMP, PATHSolver

julia> begin
           model = Model(PATHSolver.Optimizer)
           @variable(model, x)
           @constraint(model, x^2 - 2 * x + 1 ⟂ x)
           optimize!(model)
       end
Path 5.0.03 (Fri Jun 26 09:58:07 2020)
Written by Todd Munson, Steven Dirkse, Youngdae Kim, and Michael Ferris


Crash Log
major  func  diff  size  residual    step       prox   (label)
    0     0             1.0000e+00             0.0e+00 ()
    1     1     0     1 2.5000e-01  1.0e+00    0.0e+00 ()
pn_search terminated: no basis change.

Major Iteration Log
major minor  func  grad  residual    step  type prox    inorm  (label)
    0     0     2     2 2.5000e-01           I 0.0e+00 2.5e-01 ()
    1     1     3     3 6.2500e-02  1.0e+00 SO 0.0e+00 6.2e-02 ()
    2     1     4     4 1.5625e-02  1.0e+00 SO 0.0e+00 1.6e-02 ()
    3     1     5     5 3.9062e-03  1.0e+00 SO 0.0e+00 3.9e-03 ()
    4     1     6     6 9.7656e-04  1.0e+00 SO 0.0e+00 9.8e-04 ()
    5     1     7     7 2.4414e-04  1.0e+00 SO 0.0e+00 2.4e-04 ()
    6     1     8     8 6.1035e-05  1.0e+00 SO 0.0e+00 6.1e-05 ()
    7     1     9     9 1.5259e-05  1.0e+00 SO 0.0e+00 1.5e-05 ()
    8     1    10    10 3.8147e-06  1.0e+00 SO 0.0e+00 3.8e-06 ()
    9     1    11    11 9.5367e-07  1.0e+00 SO 0.0e+00 9.5e-07 ()

Major Iterations. . . . 9
Minor Iterations. . . . 9
Restarts. . . . . . . . 0
Crash Iterations. . . . 1
Gradient Steps. . . . . 0
Function Evaluations. . 11
Gradient Evaluations. . 11
Basis Time. . . . . . . 0.000019
Total Time. . . . . . . 0.001248
Residual. . . . . . . . 9.536743e-07

julia> begin
           model_2 = Model(PATHSolver.Optimizer)
           @variable(model_2, x[1:500])
           @constraint(model_2, [i in 1:500], x[i]^2 - 2 * x[i] + 1 ⟂ x[i])
           optimize!(model_2)
           termination_status(model_2)
       end
OTHER_ERROR::TerminationStatusCode = 24

julia> using JuMP, PATHSolver

julia> begin
           model = Model(PATHSolver.Optimizer)
           @variable(model, x)
           @constraint(model, x^2 - 2 * x + 1 ⟂ x)
           optimize!(model)
       end
Path 5.0.03 (Fri Jun 26 09:58:07 2020)
Written by Todd Munson, Steven Dirkse, Youngdae Kim, and Michael Ferris


Crash Log
major  func  diff  size  residual    step       prox   (label)
    0     0             1.0000e+00             0.0e+00 ()
    1     1     0     1 2.5000e-01  1.0e+00    0.0e+00 ()
pn_search terminated: no basis change.

Major Iteration Log
major minor  func  grad  residual    step  type prox    inorm  (label)
    0     0     2     2 2.5000e-01           I 0.0e+00 2.5e-01 ()
    1     1     3     3 6.2500e-02  1.0e+00 SO 0.0e+00 6.2e-02 ()
    2     1     4     4 1.5625e-02  1.0e+00 SO 0.0e+00 1.6e-02 ()
    3     1     5     5 3.9062e-03  1.0e+00 SO 0.0e+00 3.9e-03 ()
    4     1     6     6 9.7656e-04  1.0e+00 SO 0.0e+00 9.8e-04 ()
    5     1     7     7 2.4414e-04  1.0e+00 SO 0.0e+00 2.4e-04 ()
    6     1     8     8 6.1035e-05  1.0e+00 SO 0.0e+00 6.1e-05 ()
    7     1     9     9 1.5259e-05  1.0e+00 SO 0.0e+00 1.5e-05 ()
    8     1    10    10 3.8147e-06  1.0e+00 SO 0.0e+00 3.8e-06 ()
    9     1    11    11 9.5367e-07  1.0e+00 SO 0.0e+00 9.5e-07 ()

Major Iterations. . . . 9
Minor Iterations. . . . 9
Restarts. . . . . . . . 0
Crash Iterations. . . . 1
Gradient Steps. . . . . 0
Function Evaluations. . 11
Gradient Evaluations. . 11
Basis Time. . . . . . . 0.000020
Total Time. . . . . . . 0.001310
Residual. . . . . . . . 9.536743e-07

julia> begin
           model_2 = Model(PATHSolver.Optimizer)
           @variable(model_2, x[1:500])
           @constraint(model_2, [i in 1:500], x[i]^2 - 2 * x[i] + 1 ⟂ x[i])
           optimize!(model_2)
           termination_status(model_2)
       end
OTHER_ERROR::TerminationStatusCode = 24

julia> using JuMP, PATHSolver

julia> begin
           model = Model(PATHSolver.Optimizer)
           @variable(model, x)
           @constraint(model, x^2 - 2 * x + 1 ⟂ x)
           optimize!(model)
       end
Path 5.0.03 (Fri Jun 26 09:58:07 2020)
Written by Todd Munson, Steven Dirkse, Youngdae Kim, and Michael Ferris


Crash Log
major  func  diff  size  residual    step       prox   (label)
    0     0             1.0000e+00             0.0e+00 ()
    1     1     0     1 2.5000e-01  1.0e+00    0.0e+00 ()
pn_search terminated: no basis change.

Major Iteration Log
major minor  func  grad  residual    step  type prox    inorm  (label)
    0     0     2     2 2.5000e-01           I 0.0e+00 2.5e-01 ()
    1     1     3     3 6.2500e-02  1.0e+00 SO 0.0e+00 6.2e-02 ()
    2     1     4     4 1.5625e-02  1.0e+00 SO 0.0e+00 1.6e-02 ()
    3     1     5     5 3.9062e-03  1.0e+00 SO 0.0e+00 3.9e-03 ()
    4     1     6     6 9.7656e-04  1.0e+00 SO 0.0e+00 9.8e-04 ()
    5     1     7     7 2.4414e-04  1.0e+00 SO 0.0e+00 2.4e-04 ()
    6     1     8     8 6.1035e-05  1.0e+00 SO 0.0e+00 6.1e-05 ()
    7     1     9     9 1.5259e-05  1.0e+00 SO 0.0e+00 1.5e-05 ()
    8     1    10    10 3.8147e-06  1.0e+00 SO 0.0e+00 3.8e-06 ()
    9     1    11    11 9.5367e-07  1.0e+00 SO 0.0e+00 9.5e-07 ()

Major Iterations. . . . 9
Minor Iterations. . . . 9
Restarts. . . . . . . . 0
Crash Iterations. . . . 1
Gradient Steps. . . . . 0
Function Evaluations. . 11
Gradient Evaluations. . 11
Basis Time. . . . . . . 0.000026
Total Time. . . . . . . 0.001290
Residual. . . . . . . . 9.536743e-07

julia> begin
           model_2 = Model(PATHSolver.Optimizer)
           @variable(model_2, x[1:500])
           @constraint(model_2, [i in 1:500], x[i]^2 - 2 * x[i] + 1 ⟂ x[i])
           optimize!(model_2)
           termination_status(model_2)
       end
OTHER_ERROR::TerminationStatusCode = 24

(path) pkg> st
Status `/private/tmp/path/Project.toml`
  [4076af6c] JuMP v1.22.1
  [f5f7c340] PATHSolver v1.7.6

@odow
Copy link
Collaborator

odow commented Jun 11, 2024

TypeError: in typeassert, expected PATHSolver.OutputData, got a value of type PATHSolver.Options(Ptr{Nothing} @0x0000020352eb3630)

I don't understand this at all.

PATH is passing the pointer of the options struct to the print method, instead of the output data.

That seems like a bug in PATH, not in PATHSolver.jl

@mitchphillipson
Copy link
Contributor Author

mitchphillipson commented Jun 11, 2024 via email

@odow
Copy link
Collaborator

odow commented Jun 11, 2024

This doesn't explain your CI failure, but I guess it's related. I assume the answer is just don't forget the license.

@mitchphillipson
Copy link
Contributor Author

I'll email Michael Ferris this thread with a bit more info. He might be interested.

@WalterLu3
Copy link

WalterLu3 commented Jun 12, 2024 •
edited
Loading

Hello. I am Michael's student. I will create an issue about this in the internal PATH codebase for you. But first, could you try something for me? I saw that you call c_api_Path_CheckLicense in the solve_mcp routine, and I was wondering if you could make this check be done after you set the c_api_Output_SetInterface. This is just a wild shot because c_api_Path_CheckLicense will print out messages for the demo license limit if violated, but it requires the output_interface to be set already. I think maybe the fact that the pointer to the print function PATH calls in the output_interface is not set can lead to some weird behaviors.

@mitchphillipson mitchphillipson mentioned this issue Jun 12, 2024
Merged
@mitchphillipson
Copy link
Contributor Author

This does appear to solve the issues I've been having. I've opened PR and linked this issue.

@odow
Copy link
Collaborator

odow commented Jun 12, 2024

because c_api_Path_CheckLicense will print out messages for the demo license limit if violated, but it requires the output_interface to be set already.

Ahhhhhh. I didn't consider this. This makes sense

@odow
Copy link
Collaborator

odow commented Jun 12, 2024

Hah. I obviously ran into this at some point, and then forgot about it:

PATHSolver.jl/test/C_API.jl

Lines 23 to 45 in cbb4b8a

# Calling this test twice seems to lead to segfaults, so test it
# only if we are running on CI.
# manual_test_CheckLicense()
end
end
return
end
function _check_info_sanity(info)
@test info.residual < 1e-5
@test iszero(info.restarts)
@test info.function_evaluations > 0
return
end
function test_License_SetString()
@test PATHSolver.c_api_License_SetString("bad_license") != 0
return
end
function manual_test_CheckLicense()
@test PATHSolver.c_api_Path_CheckLicense(1, 1) == 1
@test PATHSolver.c_api_Path_CheckLicense(1_000, 1_000) == 0

@odow odow mentioned this issue Jun 12, 2024
Merged
@odow odow closed this as completed in #118 Jun 12, 2024
@odow
Copy link
Collaborator

odow commented Jun 12, 2024

I should have asked Michael at the time, but I guess I forgot about this.

Fixed the test: #120

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Place c_api_Path_CheckLicense below c_api_Output_SetInterface mitchphillipson/PATHSolver.jl
3 participants
@odow @mitchphillipson @WalterLu3