WIP/DNMY: add linear matrix inequality cone

[chriscoey]

Hypatia supports this cone thanks to an SC barrier. It is parametrized by a vector of symmetric matrices of equal size. The dimension of the cone is the number of such matrices, which is potentially and in many practical cases much smaller than the dimension of the PSD cone that would be needed in the SDP reformulation (which I will add as a Bridge after the definition is settled).

[blegat]

The cone [x, y] in LMICone(A, B) can be written as Ax + By in PSD so the LMI cone seems to be a way to rewrite a VAF-in-PSD into a VOV constraint.
Why is it needed and why would we do this for the PSD cone and not for other cones ?
We could also have a LinearVectorSOC for which [x, y] in LVSOC(A, B) is equivalent to xA + yB in SOC

[chriscoey]

I agree that there are many many potential cone definitions we could add to MOI, and that we don't want that sort of proliferation. So the goal is to add common ones that are convenient and provide efficiency advantages over their equivalent extended formulation representations.

It's obvious that this LMI set has a SDP extended formulation. There are several advantages to being able to use this set, in order from least convincing to most convincing in my mind:

1. This LMI set is a very commonly-used set, and it is convenient to have it because it is slightly easier to construct than the equivalent SDP extended formulation.
2. This representation in MOI is more efficient than the equivalent SDP representation. Think about the case where the side dimension of the symmetric matrix parameters is large, but the number of such parameters n (the dimension of the variable vector x) is small. It will be faster and more memory-efficient to construct the LMI function than the PSD function.
3. The SC barrier evaluations for this set are generally much more efficient than barrier evaluations for the PSD set, especially in the case of large side dimension of the parameter matrices, because we have knowledge of the structure of the set that is lost when the SDP formulation is used. The algorithmic efficiency gain is analogous to that of e.g. WSOS cone formulations vs PSD formulations for WSOS problems.

We'll provide computational results in future that should support the 3rd argument. Until then let's leave this PR unfinished and unmerged.

[blegat]

It will be faster and more memory-efficient to construct the LMI function than the PSD function.

Why is that ? The VAF-in-PSD and VOV-in-LMI are represented the same way in memory if the matrices of LMI are sparse matrices, aren't they ?
Since the LMI set of this PR uses dense matrices instead of sparse matrices, it will in fact use more memory than VAF-in-PSD.

[chriscoey]

We are planning to relax the current type restriction on the components, to allow for a mixture of identity, diagonal, block/sparse/dense symmetric, etc etc. For each component matrix, we just need it to have symmetry, even if it is not a Symmetric type. Then we can dispatch to whatever is most efficient for each component in the barrier evaluations. I'll let you know when we make that generalization in Hypatia.

[chriscoey]

@blegat see some of the combinations of sparse/dense/structured matrix types that work as parameters to this cone in Hypatia: https://github.com/chriscoey/Hypatia.jl/blob/f1070b6d49b236de07f85feef65c19e7a5cb12f4/test/nativeinstances.jl#L1027

We are working on applied examples to demonstrate computational efficiency gains and I'll update you when we have results

[blegat]

I understand that there may be benefits with structured matrices since it allows to communicate the structure while VAF-in-PSD forces the structure to be SparseMatrixCSC.
However, we could say the same for other cones, a solver might be able to exploit A * x in SOC where A has some specific structure.
The faith of this PR seems to be linked to this discussion: #863
Until we find an agreement on how we go with #863, this set could start as a solver specific one.

[chriscoey]

That's reasonable. I'll close the PR and keep it in mind for the future.