indicator to SOS bridge #877

matbesancon · 2019-09-11T13:31:33Z

Based on the comment on indicator constraints in the SCIP documentation:

This constraint is equivalent to a linear constraint ax−s ≤ b and an SOS1 constraint on z and s (at most one should be nonzero). In the indicator context we can, however, separate more inequalities.

codecov-io · 2019-09-11T14:54:38Z

Codecov Report

Merging #877 into master will decrease coverage by 0.06%.
The diff coverage is 100%.

@@            Coverage Diff             @@
##           master     #877      +/-   ##
==========================================
- Coverage   95.22%   95.15%   -0.07%     
==========================================
  Files          84       86       +2     
  Lines        8978     9167     +189     
==========================================
+ Hits         8549     8723     +174     
- Misses        429      444      +15

Impacted Files	Coverage Δ
...c/Bridges/Constraint/indicator_activate_on_zero.jl	`100% <ø> (ø)`
src/Test/UnitTests/basic_constraint_tests.jl	`98.21% <ø> (ø)`	⬆️
src/Bridges/Constraint/Constraint.jl	`96.15% <100%> (+0.15%)`	⬆️
src/Bridges/Constraint/indicator_sos.jl	`100% <100%> (ø)`
src/constraints.jl	`88.88% <100%> (ø)`	⬆️
src/Bridges/Objective/bridge.jl	`30% <0%> (-30%)`	⬇️
src/Bridges/Constraint/bridge.jl	`75% <0%> (-25%)`	⬇️
src/Bridges/Constraint/norm_to_lp.jl	`90.78% <0%> (-9.22%)`	⬇️
src/Bridges/debug.jl	`96.74% <0%> (-3.26%)`	⬇️
src/Bridges/Variable/free.jl	`96.96% <0%> (-3.04%)`	⬇️
... and 5 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 1d4f0d3...5d3ab8c. Read the comment docs.

matbesancon · 2019-09-11T15:47:10Z

this should be good for review

matbesancon · 2019-09-12T15:12:54Z

ping @blegat @mlubin

odow · 2019-09-12T15:38:14Z

ping @blegat @mlubin

I believe they're on vacation.

Have you tried this with some solvers? E.g., GLPK?

matbesancon · 2019-09-12T15:42:14Z

thanks that's why it's all quiet around here!

Not yet I wanted to be sure the whole thing was not off, I'll give it a try

odow · 2019-09-12T15:50:13Z

Seems okay to me. The only style question is whether to include it in bridges/constraint/indicator or in a separate file.

matbesancon · 2019-09-12T15:54:34Z

GLPK does not seem to support SOS constraints. The other indicator bridge defined is for activated on 0 => activated on 1, it's not awfully big

odow · 2019-09-12T16:01:11Z

GLPK does not seem to support SOS constraints.

Oh. I always forget that. Gurobi then?

matbesancon · 2019-09-12T16:02:21Z

Cbc seems ok, I'm trying to use only the FOSS ones for tests

matbesancon · 2019-09-12T16:35:44Z

So, it seems tricky, I'm not sure this is supposed to behave as ignoring the bridge, needs more investigation

matbesancon · 2019-09-13T14:52:07Z

There seems to be a problem, but I haven't figured out what exactly. I still haven't figured how to check exactly the newly formulated problem

odow · 2019-09-13T15:39:52Z

I still haven't figured how to check exactly the newly formulated problem

Try saving to MathOptFormat?

matbesancon · 2019-09-13T16:57:04Z

Ah yes thanks, I was looking for an excuse to try it :)

…

On Fri, Sep 13, 2019, 17:39 Oscar Dowson ***@***.***> wrote: I still haven't figured how to check exactly the newly formulated problem Try saving to MathOptFormat? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#877>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AB2FDMVX3RBFWJGXXFUKCWTQJOX4TANCNFSM4IVUBPFQ> .

src/Bridges/Constraint/indicator.jl

src/Bridges/Constraint/Constraint.jl

blegat · 2019-09-15T10:26:00Z

src/Bridges/Constraint/indicator.jl

+`EqualTo` constraints are handled without a bound constraint:
+``z \\in \\mathbb{B}, z == 1 \\implies f(x) == b`` is reformulated as:
+``z \\in \\mathbb{B}, w \\text{ free}, f(x) + w == b, SOS1(w, z)``.
+


Why not always add a free w so that it works for any indicator constraint (also MOI.Interval, ...) ?

Can we have a free fallback with all constraints but LessThan, GreaterThan, otherwise we add a free, which has to be re-bounded by two variables for LP& MILP solvers

Not sure I understood "which has to be re-bounded by two variables"

Sorry yes, if we create the variable w free, it will be replaced by w+ >= 0, w- >= 0, w = w+ + w- instead of creating a bounded w right away

That's also the case for SDP solvers with the Variable.FreeBridge. Which LP solver does not support free variables ? This transformation is done internally ?
For a solver supporting free variables such constraints may increase the size of the problem, we should probably have informative constraints but that's out of the scope of this PR.

Which LP solver does not support free variables ? This transformation is done internally ?

All solvers (that I am aware of) support free variables, but the transformation needs to be done internally somehow. The simplex algorithm does require positive variables

If you want to use the standard form you have equality constraints and nonnegative variables but then the dual has free variables. Some solvers might solve the dual using the simplex in which case they need all variables to be free. Although that's the textbook simplex so I don't know whether solvers might use a more sophisticated one supporting mixing free and non-free ones.

If you want to use the standard form you have equality constraints and nonnegative variables but then the dual has free variables.

My understanding is that the standard form is a form A x <= b, x >= 0 transformed in Ax + s = b, x, s >= 0. You need to have basic columns in the equality, so starting from a generic A x = b, it will need to change the variable system in [A; -A] x <= [b; -b] and then add slacks, in order to have as many basic columns as constraints.

test/Bridges/Constraint/indicator.jl

blegat · 2019-09-16T13:14:10Z

test/Bridges/Constraint/indicator.jl

@@ -53,3 +53,92 @@ include("../utilities.jl")
        @test t.coefficient ≈ 1.0
    end
 end
+


We should also have a test with basic_constraint_test (see #750) and a test against indicator1, indicator2 or indicator3 (https://github.com/JuliaOpt/MathOptInterface.jl/blob/98a0e472e1757db14b849bbba8fe84597856def2/src/Test/intlinear.jl#L609-L611) with test_model_equals (see #820)

I didn't manage to make that work (yet)

matbesancon · 2019-10-03T17:03:50Z

@blegat I can't figure out what is off here. Any tips on a debugging process for that?

blegat · 2019-10-04T08:29:21Z

I can't figure out what is off here. Any tips on a debugging process for that?

You mean for making basic_constraint_tests working ? What is the issue you are facing ? Can you share an error message ?

matbesancon · 2019-10-04T08:34:50Z

The constraints defined in the bridge seem not to be activated.

blegat · 2019-11-14T21:55:59Z

I guess the fallback assumes 0?

Yes :)

test/Bridges/Constraint/indicator_sos.jl

src/Bridges/Constraint/indicator_sos.jl

blegat · 2019-11-20T11:03:58Z

src/Bridges/Constraint/indicator_sos.jl

+    if attr.N == 1
+        return MOI.get(model, MOI.VariablePrimal(), bridge.z_variable_index)
+    end
+    wv = MOI.get(model, MOI.VariablePrimal(), bridge.w_variable_index)


This should be MOI.VariablePrimal(attr.N)

how come? w is a scalar variable

For the Nth result, you have VariablePrimal(N), ConstraintPrimal(N) and ConstraintDual(N). The value of ConstraintPrimal(N) is the value of the function when the variables have values VariablePrimal(N)

ok, that's not really intuitive, I thought ConstraintPrimal(N) would be the primal of the N-th function in the vector function. I'll update that

src/Bridges/Constraint/indicator_sos.jl

matbesancon · 2019-11-25T14:45:12Z

@blegat after thoughts, not sure MOI.ConstraintPrimal should be implemented. You cannot test it without a value set, and you cannot support setting a ConstraintPrimal without other things (see error below)

Getting primal attributes: Error During Test at /home/mbesancon/.julia/dev/MathOptInterface/test/Bridges/Constraint/indicator_sos.jl:179
  Got exception outside of a @test
  ArgumentError: `supports` is not defined for MathOptInterface.ConstraintPrimal(1), it is only defined for attributes such that `is_copyable` returns `true`.
  Stacktrace:
   [1] supports(::MathOptInterface.Utilities.Model{Float64}, ::MathOptInterface.ConstraintPrimal, ::Type{MathOptInterface.ConstraintIndex{MathOptInterface.ScalarAffineFunction{Float64},MathOptInterface.LessThan{Float64}}}) at /home/mbesancon/.julia/dev/MathOptInterface/src/attributes.jl:197
   [2] set(::MathOptInterface.Utilities.UniversalFallback{MathOptInterface.Utilities.Model{Float64}}, ::MathOptInterface.ConstraintPrimal, ::MathOptInterface.ConstraintIndex{MathOptInterface.ScalarAffineFunction{Float64},MathOptInterface.LessThan{Float64}}, ::Float64) at /home/mbesancon/.julia/dev/MathOptInterface/src/Utilities/universalfallback.jl:356

blegat · 2019-11-25T18:49:22Z

If you set the variable values, the mock optimizer uses them to compute ConstraintPrimal using get_fallback. I dont't why this bridge should not support it.

matbesancon · 2019-11-25T20:56:44Z

Alright ConstraintPrimal re-implemented and passing

src/Bridges/Constraint/indicator_sos.jl

matbesancon added 3 commits September 11, 2019 15:30

indicator to SOS bridge

7506209

fix type signatures

bf829d9

register bridge

f4a32ef

matbesancon added 2 commits September 11, 2019 17:43

added tests

11f5032

ignore benchmarks

b2be67d

blegat reviewed Sep 15, 2019

View reviewed changes

src/Bridges/Constraint/indicator.jl Outdated Show resolved Hide resolved

blegat reviewed Sep 15, 2019

View reviewed changes

src/Bridges/Constraint/Constraint.jl Show resolved Hide resolved

blegat reviewed Sep 15, 2019

View reviewed changes

matbesancon added 2 commits September 16, 2019 12:54

remove equalto constraint

62ae22b

remove ambiguity

94665e7

blegat reviewed Sep 16, 2019

View reviewed changes

test/Bridges/Constraint/indicator.jl Outdated Show resolved Hide resolved

blegat reviewed Sep 16, 2019

View reviewed changes

matbesancon added 2 commits September 24, 2019 16:51

file reformatting

23fd06b

split files

3e54c05

matbesancon added 2 commits November 14, 2019 16:21

add primal start

f7c2e85

add supports

a01b091

blegat reviewed Nov 14, 2019

View reviewed changes

test/Bridges/Constraint/indicator_sos.jl Show resolved Hide resolved

added get tests

2fb4b59

matbesancon closed this Nov 18, 2019

matbesancon reopened this Nov 18, 2019

matbesancon added 3 commits November 18, 2019 18:12

more tests

3603585

fix test types

007e23e

added test for mock bridged

970d801

blegat reviewed Nov 20, 2019

View reviewed changes

src/Bridges/Constraint/indicator_sos.jl Outdated Show resolved Hide resolved

blegat reviewed Nov 20, 2019

View reviewed changes

src/Bridges/Constraint/indicator_sos.jl Outdated Show resolved Hide resolved

matbesancon added 3 commits November 21, 2019 12:06

use attr

7938dd5

replace attr with N

4d4a8f0

variable attr propagation

9a7aaca

remove ConstraintPrimal

dbe41e2

reset test VariablePrimal

0c8ad66

blegat reviewed Nov 25, 2019

View reviewed changes

src/Bridges/Constraint/indicator_sos.jl Outdated Show resolved Hide resolved

replace rebuilt attribute with attr to propage result number

ec89295

blegat reviewed Nov 26, 2019

View reviewed changes

src/Bridges/Constraint/indicator_sos.jl Outdated Show resolved Hide resolved

no primal start N

5d3ab8c

blegat approved these changes Nov 26, 2019

View reviewed changes

blegat added this to the v0.9.8 milestone Nov 26, 2019

blegat merged commit 8618ca0 into jump-dev:master Nov 26, 2019

odow mentioned this pull request May 20, 2020

[WIP] Evaluation distance to set #1023

Closed

odow mentioned this pull request Sep 5, 2021

[breaking] [Bridges] Refactor IndicatorSOS1Bridge #1585

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

indicator to SOS bridge #877

indicator to SOS bridge #877

matbesancon commented Sep 11, 2019 •

edited

Loading

codecov-io commented Sep 11, 2019 •

edited

Loading

matbesancon commented Sep 11, 2019

matbesancon commented Sep 12, 2019

odow commented Sep 12, 2019

matbesancon commented Sep 12, 2019

odow commented Sep 12, 2019

matbesancon commented Sep 12, 2019

odow commented Sep 12, 2019

matbesancon commented Sep 12, 2019

matbesancon commented Sep 12, 2019

matbesancon commented Sep 13, 2019

odow commented Sep 13, 2019

matbesancon commented Sep 13, 2019 via email

blegat Sep 15, 2019

matbesancon Sep 16, 2019

blegat Sep 16, 2019

matbesancon Sep 16, 2019

blegat Sep 16, 2019

matbesancon Sep 16, 2019

blegat Sep 16, 2019

matbesancon Sep 16, 2019

blegat Sep 16, 2019

matbesancon Sep 25, 2019

matbesancon commented Oct 3, 2019

blegat commented Oct 4, 2019

matbesancon commented Oct 4, 2019

blegat commented Nov 14, 2019

blegat Nov 20, 2019

matbesancon Nov 21, 2019

blegat Nov 21, 2019

matbesancon Nov 21, 2019

matbesancon commented Nov 25, 2019

blegat commented Nov 25, 2019

matbesancon commented Nov 25, 2019

indicator to SOS bridge #877

indicator to SOS bridge #877

Conversation

matbesancon commented Sep 11, 2019 • edited Loading

codecov-io commented Sep 11, 2019 • edited Loading

Codecov Report

matbesancon commented Sep 11, 2019

matbesancon commented Sep 12, 2019

odow commented Sep 12, 2019

matbesancon commented Sep 12, 2019

odow commented Sep 12, 2019

matbesancon commented Sep 12, 2019

odow commented Sep 12, 2019

matbesancon commented Sep 12, 2019

matbesancon commented Sep 12, 2019

matbesancon commented Sep 13, 2019

odow commented Sep 13, 2019

matbesancon commented Sep 13, 2019 via email

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

matbesancon commented Oct 3, 2019

blegat commented Oct 4, 2019

matbesancon commented Oct 4, 2019

blegat commented Nov 14, 2019

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

matbesancon commented Nov 25, 2019

blegat commented Nov 25, 2019

matbesancon commented Nov 25, 2019

matbesancon commented Sep 11, 2019 •

edited

Loading

codecov-io commented Sep 11, 2019 •

edited

Loading