deprecate orthogonal decomposition keyword "thin" to "full"

[Sacha0]

This pull request deprecates the thin keyword argument to orthogonal factorizations in favor of square, addressing JuliaLang/LinearAlgebra.jl#472. Best!

[fredrikekre]

([#24279])

[Sacha0]

Fixed. Thanks! :)

[Sacha0]

Absent objections or requests for time, I plan to merge these changes tomorrow. Best!

[StefanKarpinski]

which has the opposite meaning: thin=true if and only if square=false.

[Sacha0]

Good call! Revised accordingly. Thanks!

[Jutho]

square=false sounds a bit strange given the fact that, if the input matrix is square, the thin output matrices will also be square.

[StefanKarpinski]

Something about the square terminology here does feel a bit funny.

[Sacha0]

Agreed, square lacks something :). Apart from simply retaining "thin" and further documenting that name's oddity in e.g. the LQ context, "reduced" is the remaining possibility that comes to mind. Thoughts? Thanks!

[StefanKarpinski]

"Reduced" feels a bit better since it's a reduced dimension factorization. Could you humor me and explain in words what this argument means in full generality?

[Jutho]

For the SVD of a size (m,n) matrix, the most compact form has U of size (m,min(m,n)), S a diagonal of length min(m,n) and Vt of size (min(m,n),n). Matlab has this strange thing where an argument 0 means that U will be reduced, but V is still guaranteed square (and thus S is rectangular if n>m), whereas the argument econ gives both U and V in reduced form.

That as an aside to propose the argument econ with some precedence. However, personally I prefer (and would therefore also propose) compact over econ, reduced or thin.

[StefanKarpinski]

I thought the whole point of this terminology change was to use consistent terminology across different factorizations, not just SVD. So what's the explanation for the other ones? The point of this exercise is that if you find the common language to naturally describe what this means in a way that applies to all of the usages, then whatever words you use in that explanation are what you should use for the keyword name. If there is no way to succinctly describe what this does across many usage contexts then we should reconsider whether it's actually the same concept or not.

[Jutho]

It is the same concept, and on second thought I am as happy with reduced as with compact. It seems that in the case of QR, reduced at least has some precedence in Trefethen and Bau as alternative to thin:
https://en.wikipedia.org/wiki/QR_decomposition

[StefanKarpinski]

It is the same concept

Which I'm asking for evidence of in the form of a common explanation.

[Jutho]

You can always compute a reduced SVD of an (m,n) matrix A by performing the following steps:

• compute a reduced QR of A -> Q1 of size (m,min(m,n)), R of size (min(m,n), n)
• compute a reduced LQ of R -> L of size (min(m,n), min(m,n)), Q2 of size (min(m,n), n)
• compute a regular SVD of L -> U1 , S and V2' , all of size (min(m,n), min(m,n))
  Set U = Q1*U1
  Set V' = V2'*Q2

[Sacha0]

Each of the common orthogonal(/unitary) decompositions comes in several forms. In the canonical form of each, the orthogonal factors are square and thus proper orthogonal matrices (as opposed to rectangular matrices that contain orthonormal columns). For example, in a canonical SVD decomposition USV of m-by-n matrix A, U and V respectively are m-by-m and n-by-n orthogonal (square) matrices while S is m-by-n. Similarly, in a canonical QR decomposition of m-by-n matrix A, Q is an m-by-m orthogonal (square) matrix while R is m-by-n. This canonical form is often called the "full" form.

When A is rectangular, this canonical form is wasteful in practice: Computing part of the orthogonal factors is unnecessary, burning both decomposition time and storage. Hence in practice another form is common, wherein only the necessary, rectangular part of the "orthogonal" factors is computed and stored. For example, in this form of SVD decomposition USV of m-by-n matrix A, U is an m-by-n (rather than m-by-m) matrix with orthonormal columns, S is n-by-n (rather than m-by-n), and V is again n-by-n. Similarly, in this form of QR decomposition of m-by-n matrix A, Q is an m-by-n (rather than m-by-m) matrix with orthonormal columns, and R is n-by-n (rather than m-by-n). This form has two common names, seemingly propagated by two classic numerical linear algebra textbooks: Golub & van Loan call such decompositions "thin", while Trefethen & Bau call such decompositions "reduced". Those taught from one tend to prefer the one, and the other the other.

There are also distinct "compact" (for efficient decomposition of low-rank matrices) and "truncated" (for approximate low-rank decomposition) forms, separate from those forms at issue here.

Does the above clarify? :)

[StefanKarpinski]

This canonical form is often called the "full" form.

Perhaps this is the answer? Have a full::Bool keyword argument to request the full form of a factorization. Since the full form seems like it's usually not what you want, we could have full default to false. The name full avoids the problem with square=false when the non-full factorization happens to be square, since you're not explicitly asking for something not to be square.

Or in other words: "full" is dead, long live "full"!

[Sacha0]

Cheers, full is much better than square :). Either of full or reduced for the keyword sounds great on this end. Independent of which we choose for the keyword, I propose we use full and reduced consistently in the docstrings. Thoughts? Thanks!

[StefanKarpinski]

The term "full" seems clearer to me and if the default is not full, then I think it's even better since the full factorization is something that you need to explicitly ask for. The opposite of "full" could be reduced, but that could just be a matter of terminology rather than the name of a keyword.

[StefanKarpinski]

Let's give other linalg types like @Jutho and @andreasnoack a chance to chime in.

[Sacha0]

To be clear, I agree that full seems like the best choice for the keyword at present. Best!

[Jutho]

Or in other words: "full" is dead. Long live "full"!

Hail to "full"!

[StefanKarpinski]

Ok, seems like we have a consensus: everyone seems to like a full::Bool=false keyword argument for this – defaulting to false since you usually don't want the full factorization. Edit: @Sacha0 is way ahead of me and has already updated this :)

[Sacha0]

Revised with keyword full. (This pull request will leave the docs and comments ripe for some smoothing, but that can happen at a future point.) Thanks all!

[Sacha0]

Additional comments? :) Absent objections or requests for time, I plan to merge these changes tomorrow. Best!

[Sacha0]

Thanks all! :)