make linear algebra math functions work for scalars

[SamChill]

I was writing an optimization algorithm and wanted it to work with scalars and vectors. I noticed that norm and dot aren't defined for scalar numbers.

julia> norm(1.0)
no method norm(Float64,)
 in method_missing at base.jl:60

julia> dot(1.0,2.0)
no method dot(Float64,Float64)
 in method_missing at base.jl:60

The definitions would be simple:

norm(x::Number) = x                                                                                                                     
dot(x::Number, y::Number) = x*y

There might be more functions to consider defining for scalars as well.

[mvanderkolff]

I'd suggest that your definition for norm is wrong: it should be norm(x::Number) = abs(x)

[SamChill]

I completely agree.

On Apr 26, 2012, at 8:04, Michael van der Kolffreply@reply.github.com wrote:

I'd suggest that your definition for norm is wrong: it should be norm(x::Number) = abs(x)

---

Reply to this email directly or view it on GitHub:
#763 (comment)

[StefanKarpinski]

What's the argument for having norm(1.0) and dot(1.0,2.0) work? If you write norm(v) you are expecting v to be vector. Likewise, the dot product of two scalars doesn't make sense — only vectors have dot products.

[mvanderkolff]

One of the main ones would be people coming from octave - people expect a 1x1 matrix to act as a scalar.

[StefanKarpinski]

That's a major misfeature of Matlab (and Octave by extension). Fixing mistakes like that is one the major reasons for a new language.

[JeffBezanson]

The way I'd put it is that we're not going to be compatible with matlab or octave anyway, so we might as well fix little behaviors like that along the way.

[StefanKarpinski]

Yes, that's a better way to put it :-). Of course, we're not just creating a new language to fix little misfeatures like that.

[stevengj]

I disagree with your conclusion this thread. Scalars are vectors (they are vector spaces of dimension 1, and tensors of rank 0 for that matter). Norms and inner products of real and complex scalars are perfectly sensible operations, that (with the usual definitions) should give the absolute value and (conjugated) product respectively.

Scalars are also perfectly good linear operators on the vector space of scalars, so operations like eig on scalars are perfectly sensible too.

This is not a "misfeature" of Matlab. Linear algebra is not just for matrices in parentheses.

[StefanKarpinski]

I'm starting to come around to the view of identifying scalars with 0-tensors this way, but I still have some concerns about scalars that are also iterable, like strings. I'll have to sit down and figure out what whether they are actually a problem or not. For things like norm and dot, handling scalars is harmless.

[johnmyleswhite]

It's probably worth separating the Iterableness of things from their tensor rank. Strings aren't iterated using their indexing rules, for example.

-- John

On Dec 31, 2012, at 12:12 PM, Stefan Karpinski notifications@github.com wrote:

I'm starting to come around to the view of identifying scalars with 0-tensors this way, but I still have some concerns about scalars that are also iterable, like strings. I'll have to sit down and figure out what whether they are actually a problem or not. For things like norm and dot, handling scalars is harmless.

—
Reply to this email directly or view it on GitHub.

[toivoh]

@stevengj: That's a perfectly valid mathematical argument for regular linear algebra, but it's not the whole story.

What are scalars? The elements of arrays? Or non-arrays? (let's call those pure scalars)
Consider the most general case: an array A::Array. A might contain entries of any kind, including both pure scalars, e.g. ::Number, and another array B. For operations on such arrays, it's not meaningful to distinguish between scalar and array arguments per se, and we can definitely not do so just by looking at their types. Instead, we have to distinguish what kind of role an argument plays in the context of the operation in question, which might e.g. be as a scalar or an array.

I would hate to introduce something into Julia that does exactly what you want most of the time, and then breaks things/introduces idiosyncrasies/hard to find bugs in less common cases; in this case arrays of arrays. That's the matlab fallacy to me.

That said, I guess that a lot of the stuff that goes under the heading of treating scalars as 0-d arrays really comes down to cases when a function that always expects an array argument gets a non-array argument (pure scalar). Then it doesn't seem so dangerous to promote that pure scalar to be treated as a 0-d array. I guess that your suggestion about indexing into scalars goes under this heading; if you are indexing into it, you must have been expecting an array.

Of course, I still didn't even say anything about indexable non-arrays like strings...

[stevengj]

@StefanKarpinski, when I say "scalar" I'm only talking about Numbers. I'm not discussing whether norm should work for strings. @johnmyleswhite, the question here is not whether something is iterable, it is whether norm and dot are meaningful for numbers (they are).

The argument is simple: if a documented Julia function foo has a standard, unambigous, meaningful definition for an argument of type T, and it is practical to implement it, then it should be implemented. This is certainly the case for norm and dot, as well as for a large number of other linear-algebraic functions, from eig to inv.

@toivoh, I'm not sure what you're getting at here. I'm not arguing that all operations should work on both scalars and arrays, just the ones where it makes sense. Where is the "fallacy" or "idiosyncracy" introduced by defining norm, dot, and similar functions for scalar numeric arguments? The existence of a norm does not require "arrays" (or tensors).

(In fact, norm and dot are actually more ambiguous for arrays: for scalars, the norm and inner product are uniquely determined up to a scale factor, but for multidimensional vector spaces there are many possible definitions. Defining them for arrays but not for scalars is therefore especially perverse. Nor is your choice of a definition for arrays inconsistent with, or preclude using, the standard definition for scalars.)

[johnmyleswhite]

Oh, I definitely agree that we should have norm and dot for numbers. I find it really frustrating that norm(5) produces an error.

[andreasnoack]

@stevengj Can you give an example of the advantage of having norm, dot, inv and eig to work for Number? Personally, I think it makes good sense to distinguish between vectors and scalars and therefore let linear algebra functions work on Arrays only. If in your code norm sometimes has length one then why couldn't it be a Vector? For example, randn(10)[1:1] is a Vector, so there wouldn't be a problem. Also, if you work with scalars only there is no reason to use the linear algebra functions as abs, *, / would be more efficient (and cleaner).

[toivoh]

@stevengj: Ok, it seems that we're on the same page after all. I don't have anything in particular against norm and dot being defined for arguments of type Number. I'm just a bit uncomfortable with sweeping statements like "scalars are vectors".

[stevengj]

@andreasnoackjensen, the advantage is that it makes it easier to write polymorphic routines that operate on either vectors or scalars. The more functions work for both and have consistent meanings, the fewer special cases you have to write manually. (Note that [], length, etctera already work for numeric scalars; see #1843 .) The other advantage, of course, is that it is mathematically the right thing to do.

@toivoh, in the mathematical sense, real and complex scalars are vector spaces (and Hilbert spaces, and...); this viewpoint is totally standard and widespread. In the data-structure/computer-sciencey sense ("vector" == "array"), the two are somewhat different (although still very closely related). That's why it makes sense for nearly all linear-algebra functions to apply to numeric scalars as well as to arrays, whereas there might be other non-algebraic functions that make less sense or are ambiguous for scalars.

[stevengj]

Fixed by #1871.