proposal: Go 2: define += operator for appending to slices

[rillig]

For strings the operator += is defined to append one string to another. For slices, the same operator could be defined with the same meaning. This would shorten code that appends to a slice.

Example

var strs []string
strs = append(strs, "first")
strs = append(strs, "second")

This could instead be written as:

var strs []string
strs += "first"
strs += "second"

Details

This pattern avoids the syntactic overhead of a function call and the repetition of the variable name. Most occurrences of the append function follow this pattern of appending a single value to a slice and then assigning to that same slice again.

The proposed operator would work only for appending a single element. Appending two or more elements would still require to use the strs = append(strs, "first", "second") form.

Appending a slice to a slice might look like strs += other..., though this would require an extension of the Go syntax. Besides that, it would create an ambiguity for slices of interfaces. In such a case the += could either mean to append the slice as a single element or to append each element of the slice like in the ... operator. Therefore this proposal is only about defining the += operator to append a single element.

[randall77]

It would be weird to be able to += a slice and an element, but not +.

[dotaheor]

If this proposal is approved, I would it could expand to support one of my proposal: #23905

For example:

var s1, s2, s3 = make([]int, 100), []int{1, 2, 3}, []int{4, 5, 6}
s1 += s2...                // will try to use s1's free capacity slots
s1 = s1... + s2... + s3... // will always allocate the underlying element slots.

The last line is equivalent to my merge proposal.

[kybin]

It creates ambiguity that means I cannot tell the variable is a string or a slice of string when I see += sign.

I don't think the current overhead is big.

[Tolsee]

When we use append() which uses value semantic, my underlying thinking is the result is another slice and we assign it the slice we have(If I have understood this correctly). This clearly shows implementation details in the sense that we always return another value where we use value semantics in functions.

+= in other hands, seems to hide the implementation details(For me). I feel like the same thing was changed, and then I would think it should not be possible because how the backing array is created/changed.

[mvdan]

Perhaps this should be considered together with operator overloading. Without operator overloading, I think this particular use case isn't important enough to warrant making the language more complex.

[dana321]

How about push(slice,value) which takes a reference to the slice instead, the perl way of doing these things i've always found a good way, adding push, unshift, shift and pop for slices would make everyone's lives easier. See here: https://perlmaven.com/manipulating-perl-arrays

[urandom]

@kybin

It creates ambiguity that means I cannot tell the variable is a string or a slice of string when I see += sign.

I don't think the current overhead is big.

How do you tell right now if the variables are ints or strings?

[bcmills]

It's bad enough that + conflates addition and string concatenation, but at least in both of those cases the operands remain unchanged. Adding slice-mutation to the mix seems far too subtle, especially if the benefit is only to avoid writing out the word append.

[deanveloper]

The spec on op= assignment operators:

An assignment operation x op= y where op is a binary arithmetic operator is equivalent to x = x op (y) but evaluates x only once. The op= construct is a single token. In assignment operations, both the left- and right-hand expression lists must contain exactly one single-valued expression, and the left-hand expression must not be the blank identifier.

This would mean that in order to allow slice += elem we would want to allow slice = slice + elem, in order to not pollute the spec with exceptions to rules and end up with it being 100 pages long. Go's spec is very short and I'd like to keep it that way.

In terms allowing slice = slice + elem, I also don't like the idea of allowing two separate types to be added together.

[deanveloper]

How do you tell right now if the variables are ints or strings?

I think they are saying that if you have foo += bar and you know bar is a string, you don't know if foo is a string or []string

[randall77]

I think they are saying that if you have foo += bar and you know bar is a string, you don't know if foo is a string or []string

Sure, but I don't think that is a showstopper. If I have:

    x + 1

Is that a floating-point add or an integer add? How wide is that add? It depends on the type of x, and that's ok.

What foo += bar (or foo + bar) means depends on the types of foo and bar. The fact that string+string and []string+string do different things is fine.

It is a bit weird to have + be non-commutative. But that's already true for string+string.

[deanveloper]

Is that a floating-point add or an integer add? How wide is that add? It depends on the type of x, and that's ok.

Right, but 1 is untyped. foo + bar is a typed operation, both foo and bar have a type and they are separate from each other.

It is a bit weird to have + be non-commutative. But that's already true for string+string.

Not just non-commutative, but also non-associative. ([]T + T) + T and []T + (T + T) are two very different things.

EDIT - this would also mean that foo += bar + baz and foo = foo + bar + baz have two separate meanings

[randall77]

That's already true for floats. (x+y)+z and x+(y+z) aren't the same thing.
Although they are close.

I think it is understood that the RHS is evaluated before doing the x += y expansion. In other words, x += y expands to x = x + (y). Otherwise, things like x -= y + z wouldn't expand correctly.

[deanveloper]

Although they are close.

I think that's the main thing that separates it. Being off by an ulp or so isn't too bad, but []string + string + string would add two elements to the list, which I know was something the proposal wasn't initially going for. My main concern would be code like:

var foo []int
// lots of code ...
// ...
// ...
foo = foo + 3 + 8 + 5

We are currently talking about this kind of stuff, so it may be easy to realize what's going on in the code above. But seeing something like that in a random codebase somewhere would be extremely confusing.

We already have append( ... ), I think the verbosity really helps illustrating what's really going on.

[dotaheor]

Yes, it is best to guarantee the types of all operands are slices and their element types are identical
Otherwise there will be ambiguity if the target slice is an interface slice:

var x, y = []interface{}{9}, []interface{}{1, 2, 3}
x = x + y // this is ambiguous 
// two possibilities here:
// 1.
//    x = []interface{}{9, 1, 2, 3}
// 2.
//    x = []interface{}{9, []interface{}{1, 2, 3}}

@bcmills

..., especially if the benefit is only to avoid writing out the word append.

This proposal also avoids unnecessary resetting all elements to zeros in a make call. Please see my proposal for details: #23905

[martisch]

This proposal also avoids unnecessary resetting all elements to zeros in a make call.

I think this can also be achieved by allowing append to have multiple ... arguments foo = append(foo, l1..., l2...). Which doesn't add an alternative way to append slices but expands the existing way. #18605

I am also working on CL to allow make+copy not to do unnecessary zeroing https://go-review.googlesource.com/c/go/+/146719.

[dotaheor]

@martisch
It would be great if compilers can optimize some make+copy+copy+copy cases. But I feel it is hard for compilers to find all such cases. And it doesn't solve the code verbose problem.

I just made an alternative proposal for my original proposal.

[deanveloper]

I don't think verbosity is really a problem in this case. I think the verbosity makes the code much more readable.

Also what's all this talk about how this proposal eliminates a make+copy pattern? From what I understand this proposal is just syntactic sugar to convert slice += elem to slice = append(slice, elem)

[deanveloper]

While I don't interact with the Kotlin community much anymore, they define operator functions on their data structures. In my experience, mathematical operator functions on data structures are avoided because they make code less readable and are really more of a gimmick than anything else. Bracket notation on custom structures is commonly used, though. Just not addition/subtraction.

[martisch]

It would be great if compilers can optimize some make+copy+copy+copy cases. But I feel it is hard for compilers to find all such cases.

If the copy(s) follow(s) right after the make it is not hard and I think covers all the cases that issues #29186 covers with make([]T, aSlice0, aSlice1, aSlice2, ...). If the go frontend moves to use kind of an SSA like representation to find the make+copy pattern then it would actually cover more cases than a enhanced make call could. Internally in both cases the compiler can emit the same code. The difference is how the pattern is parsed and detected. If it works on exiting language constructs all go code that already uses make+copy benefits and does not need to write makes in another way to gain the performance benefits.

[deanveloper]

There are definitely cases where compilers can optimize where a make(...) call couldn't, especially for something like where you'd want a slice to be the same slice repeated 10 times. Sure, you could do make([]T, slice..., slice..., slice..., ...), but it would be much more elegant to do make([]T, len(slice)*10) and then put a for loop with a copy in it afterward. I personally think that a compiler optimization would make much more sense.

[ianlancetaylor]

We don't have to have += without +. And having + mean append puts us in the odd situation that the addition operation can change its operands. That seems like a recipe for confusion. People new to Go already get confused about append; let's not let them get confused about +.

Currently Go operators operate on immutable operands; this proposal would change this property. That doesn't necessarily mean this is something we might not want to consider, but if we do consider it it should be in the broader context of operator overloading. This proposal just adds syntactic sugar for something we can already do in a more verbose way.

[bradfitz]

Not a real proposal, but another way common append operations could be made more brief is if you didn't need to repeat the slice name twice.

If append's first argument took a pointer to a slice instead of a slice, the pointer could mean to append to *ptr.

In your example,

var strs []string
strs = append(strs, "first")

Would be:

var strs []string
append(&strs, "first")

And that's something you could do with the generics proposal yourself too.

[DeedleFake]

I actually like the separation of the return from the argument. It simplifies some neat little tricks, such as

t := []int{1, 2, 3}
t = append(t[:2], t[1:]...)
t[1] = 0
// t: []int{1, 0, 2, 3}

or

t := []int{1, 2, 3}
t = append(t[:1], append([]int{0}, t[1:]...)...)
// t: []int{1, 0, 2, 3}

for insertions and

t := []int{1, 0, 2, 3}
t = append(t[:1], t[2:]...)
// t: []int{1, 2, 3}

for deletions.

Of course, I suppose it could just allow either syntax, but that just seems like more complication than it's worth. It doesn't need a second syntax.