make .+= et al. mutating

[StefanKarpinski]

We're now quite consistent about + being "mathematical" addition while .+ is "container" addition. In a sense + treats its arguments as conceptually immutable mathematical objects – numbers, vectors; .+ treats its arguments as containers of values. In that view, it would make sense for x += y to mean x = x + y as it does now, while x .+= y could modify x in-place.

[IainNZ]

Would this mean you could write a function Base.(.+=)?

[nalimilan]

So that would mean that x .+= y is not equivalent to x = x .+ y?

[lindahua]

I guess this implies a = a .+ b and a .+= b have different behavior?

Then what about when a and b are both scalars -- in such cases, it should be inplace or not?

[StefanKarpinski]

Yes, that's what it would mean. x .+= y would desugar as x = (.+=)(x, y) and there would be a default definition of (.+=)(x,y) = x .+ y. This means it would keep working for immutable numbers, but you could add an in-place implementation for arrays and other mutable containers.

[IainNZ]

Can we get that for :(+=) or is that still a no-go?

[JeffBezanson]

I don't like the kind of behavior where f(x,y) might or might not mutate x depending on its type.

see also JuliaLang/julia#249, JuliaLang/julia#1115

[lindahua]

Yes, this kind of inconsistency would be a problem. What about something like:

.+!(x, y)

Or, perhaps let's just do it using named functions:

add!(x, y)

[porterjamesj]

what about just not defining x .+= y, etc. unless it makes sense to mutate x and documenting that these "in-place" operators should only be defined in cases where in place operations make sense?

[stevengj]

I'm not sure I see the rationale for this. The main reason for doing element-wise operations in-place is for performance and to avoid memory allocation. However, I would think that in most performance-critical real applications you aren't doing something as simple as performing a single binary operation on two arrays, and so these semantics won't solve the problem.

For example, in ODE solvers you often do things like a = α*b + γ*c + ζ*z + ..., and for performance reasons you want to devectorize this into a single loop where a is pre-allocated.

Who is helped by .+= etc. enough to be worth the complications these semantics will cause?

[stevengj]

I'd rather have something like @lindahua's Devectorize package, but one in which I can write:

@devec! a = b + c

and have it work in-place on a with a single loop if possible but fall back to a = b + c for types that are not arrays or iterables. (That's currently the problem with Devectorize ... if we use it in e.g. ODE.jl then we sacrifice generality.)

[johnmyleswhite]

I really like the simplicity of having a OP= b be sugar for a = a OP b.

[StefanKarpinski]

Ok, it's good to get feedback on this. There's been a lot of pushback about this not being in-place so I thought maybe we could have it both ways with this. Complaints tend to be louder than affirmations of the status quo.

[porterjamesj]

For me it's always seemed like there's a tension here between simplicity (as John says, knowing that a OP= b always translates to a = a OP b is simple, as is not having the compiler try to do some magic to avoid allocating temporaries) and easiness (e.g. lots of people complain on the mailing list about not just being able to write a += b and have it not allocate a temporary array). Finding the right balance is tricky because a lot of the options for making things easier (this proposal, making the compiler "sufficiently smart" to avoid allocating temporaries, etc.) involve sacrificing simplicity in some way.

[tknopp]

Could someone give me an example where one can determine from outside whether a += b is inplace or not?

While it would be really cool to have something like @devec in base, I don't see this as an argument not to make += as fast as possible.

[lobingera]

For me (and my scope of writing programs) it's pretty clear: a = a + b and a+= b is the same operation: you update (if inplace or by a temporary) a by adding b. If a and b qualify for multiple elements, then the respective + is used - which is in most cases elementwise.

What do we loose here, if a = a + b and a+=b is not the same operation?

[nalimilan]

I agree with @stevengj. While the fact that f(x) = x .+= 1 will have no effect on the original x may certainly be impractical/unexpected, I think the same issue exists with x += A, x = x .+ 1 and x = anything more complex. So I'm not sure it makes sense to do something special for .+=.

What do other languages do in that regard (though most do not have the notion of element-wise operations)?

[IainNZ]

Your example there @nalimilan, of f(x) = x .+= 1 having no effect on the original x, is great, because I find the behavior actively confusing, especially in a language that doesn't provide an easy way to do the more obvious thing. This may be because I'm coming from C++.
Given @stevengj's point about more than one operation, I think it suggests

• A Base.@devec + keep the existing behavior (probably least cognitive overload)
• Let people overload the += and .+= families.

[JeffBezanson]

• We can debate whether things like += should be mutating or not, but what I really oppose is having it be either/both, depending on types.
• It makes me uneasy for discussions of in-place-ness to center on operators. That's only a mostly-fixed set of functions with special names. I would hope for something more general. In the issues I linked above, .= and := are discussed. Maybe something like @devec could be incorporated into base using those as syntax.

[JeffBezanson]

Note for example that .+ is a shortcut for calling broadcast with +. This is the kind of thing I like :)

[GlenHertz]

@JeffBezanson I'm curious why you like .+ broadcasting to +. I find this confusing as I'm asking it for an element operation and then it gets converted to a different operation for each element. Shouldn't that be a bug?

[JeffBezanson]

I don't know, but that's not what I was referring to. The only point of my comment is that broadcasting is not a special feature of a few dot operators, but has a general version via the broadcast function. Special syntax should ideally be sugar for compositions of more general operations.

[nalimilan]

It seems there are three related issues here:

• mutation
• element-wise operation
• devectorization

The two latter are logically strongly related, and could maybe be merged together (this is what @devec does). But the question of making .+= mutating conflates mutation and element-wise operations. As discussed in JuliaLang/julia#249, one could perfectly have a .= assignment operator which would be mutating but not element-wise; thus . is misleading, but since != is already taken... let's call it :=. Following this path, a mutating += could be written :+=, with an element-wise version called... .:+=. Just kidding, but what I'm trying to say is that there are many possible combinations of these notions, and it's not obvious that we always want to apply all of them.

So maybe using [:] or a macro like @in! (à la https://github.com/simonbyrne/InplaceOps.jl) would be better than modifying operators. I would love to hear a syntactically more natural solution, though.

[tknopp]

This is a very good observation of @nalimilan. Athough element-wise /devectorized operation will likely be implemented in a mutating manner it is less clear that . implies mutation.

To the point of @JeffBezanson of mutating behavior of +=: Would it be possible to make this operator mutating for value types (e.g. Float64). I somehow have hard times to understand what the difference would be to the non-mutating +=. Would one reuse the binding to make it mutating?

[JeffBezanson]

Mutating means the following:

a = b
println(b)
a += x
println(b)  # shows something different than the first println

[tknopp]

Ok, but sorry that I still not get it. I thought that Float64 is immutable and thus a = b makes a bitwise copy so that one always has value semantics and not reference semantics. (I absolutely understand your example for mutable types like arrays)

[JeffBezanson]

Yes, Float64 is immutable so the behavior I described above is not possible with it.

[tknopp]

Thanks for the clarification. So making += mutable would infact mean that we would have to drop this operator for immutables (speaking this out loud of course makes it pretty obvious).

One could think about deprecating += for vectors as it can be confusing that it creates temporaries.

[vtjnash]

-1

I've grown to like that x+=1 can't modify the callee's environment. I would rather get the speedup from optimizing the GC, rather than changing the semantics of this operation.

[JeffBezanson]

I think this is covered by JuliaLang/julia#249 and JuliaLang/julia#1115. Probably will not do something special for dot operators.

[StefanKarpinski]

Fair enough. Good to have the discussion in any case.

[brendano]

Hi, I realize this issue is closed, but I ran into this just now. I was suprised this didn't mutate the vector i passed in:

function f!(xs)
  xs /= 2.0
  nothing
end

I'm not sure whether my expectations are somehow justified, or if it's just because I'm used to how numpy does it. If it's policy that in-place doesn't mutate, it might be nice to document somewhere (unless I missed it!).

[JeffBezanson]

http://docs.julialang.org/en/latest/manual/mathematical-operations/#updating-operators

Also please read the above discussion; it should explain our reasoning pretty well. xs /= 2.0 is always equivalent to xs = xs / 2.0. It's not an in-place version of the operator, just syntactic sugar for this common pattern.

[brendano]

Thanks! Sorry, I was just trying to contribute a data point. Maybe github issue comments are not a good way to do this.

[stevengj]

Fixed by JuliaLang/julia#17510, although we won't get the full power of fusing in-place assignment until .+ etcetera turn into fusing calls ala JuliaLang/julia#16285.