SYMMLQ improvements

[amontoison]

I modified SYMMLQ to be able to compute x_lq when x_cg doesn't exist.
ζ = ζbar * c is equal to Inf if ζbar = Inf / γbar = 0.
The η variable has been introduce to solve this problem.

[dpo]

Could you provide more information on these two changes?

[dpo]

Should we check if gammabar = 0 ?

[amontoison]

I rebase this pull request.

Contrary to BiLQ or USYMLQ, I don't check that γbar ≠ 0 when I compute ζbar. We need it to compute CG residual norms and error norms bounds that are returned in a SymmlqStats at the end. Nevertheless I still check that γbar ≠ 0 iN solved_cg.

I also add test problems where CG, BiCG and USYMCG don't exist at the first iteration and verify that our implementations / LQ factorizations handle it even if T_k is singular (γbar = 0).

[coveralls]

Coverage increased (+0.07%) to 96.277% when pulling cb6a961 on amontoison:symmlq_singular into f5419ff on JuliaSmoothOptimizers:master.

[codecov]

Codecov Report

Merging #138 into master will decrease coverage by 0.11%.
The diff coverage is 93.33%.

@@            Coverage Diff             @@
##           master     #138      +/-   ##
==========================================
- Coverage   97.16%   97.05%   -0.12%     
==========================================
  Files          26       26              
  Lines        2155     2176      +21     
==========================================
+ Hits         2094     2112      +18     
- Misses         61       64       +3

Impacted Files	Coverage Δ
src/Krylov.jl	100% <ø> (ø)	⬆️
src/symmlq.jl	96.95% <92.72%> (-2.37%)	⬇️
src/krylov_utils.jl	95% <95%> (-5%)	⬇️
src/krylov_aux.jl	96.25% <0%> (+2.5%)	⬆️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update f5419ff...cb6a961. Read the comment docs.

[dpo]

I'm not thrilled about having Infs in the code when gammabar = 0. I think we should test like in the other solvers?!

[amontoison]

My problem is what should we do with rcgNorm and errorscg if gammabar = 0?
A solution could be to not update them, but the drawback is that we loose the link between iter and rcgNorm[iter]. We don't know anymore at which iteration errors norm or residuals norm have been computed.

[dpo]

How do you do it in BiLQ? We could insert a missing value for the CG residual and error, which would avoid using Infs and NaNs.

[amontoison]

In BiLQ and USYMLQ, residual norms are computed only if CG point exists (δbarₖ ≠ 0). With those methods, a SimpleStats is returned at the end. So it's not a problem if rcgNorms are not computed because we don't need to store them.
With SYMMLQ, more elaborated statistics are returned. They include residuals and errors norms of CG points. Using missing value if a CG point doesn't exist is a good idea.

[dpo]

In BiLQ and USYMLQ, residual norms are computed only if CG point exists (δbarₖ ≠ 0). With those methods, a SimpleStats is returned at the end. So it's not a problem if rcgNorms are not computed because we don't need to store them.

BiLQ and USYMLQ could (should) also return CG residuals, even if they don't form the CG iterate until the end.

[amontoison]

I think we could change SymmlqStats to LQStats in an other pull request for all LQ methods (LSLQ, LNLQ, SYMMLQ, BiLQ, USYMLQ).

[dpo]

sounds good.

[dpo]

Thank you.