proposal: Go 2: a proposal to support final values and read-only parameters/results in Go

[zigo101]

(The latest version of this proposal is here.)

Please note, the following many comments are based on the initial proposal.

[zigo101]

I forget an important thing. The rules in the proposal don't apply to basic values. Or we need to introduce a new keyword fixed to replace the const keyword to support both basic and composite values.

[zigo101]

OK, it looks the fixed keyword is essential, I changed all const uses to fixed in the proposal.

[zigo101]

Updated the "assign immutable values to mutable values" part.

[zigo101]

Two use cases which become possible with this proposal:

	fixed bs = []byte(aString) // no underlying bytes are copied.
	
	fixed pb = &aString[0] // string bytes are addressable.

[zigo101]

I have an interesting idea to extend this proposal but will make this proposal more complex.

The fixed and fixed! can be unified as fixed:N, which N is a non-negative integer.
fixed! == fixed:0, fixed == fixed:1 and var == fixed:∞.

An example:

fixed:0 p = new(int)
pp := &p // <=> fixed:1 pp = &p <=> fixed pp = &p
ppp := &pp // <=> fixed:2 ppp = &pp
pppp := &ppp // <=> fixed:3 pppp = &ppp

// A value declared with fixed:N can only be dereferenced N times
// when its dereference is used as a destination value.

pp = new(**int) // ok
*pp = new(*int) // ok
**pp = new(int) // error: immutable

ppp = new(***int) // ok
*ppp = new(**int) // ok
**ppp = new(*int) // ok
***ppp = new(int) // error: immutable

pppp = new(****int) // ok
*pppp = new(***int) // ok
**pppp = new(**int) // ok
***pppp = new(*int) // ok
****pppp = new(int) // error: immutable


// a fixed:N value can be assigned to a fixed:M value if M <= N + 1.
// [update]: SORRY, the above old line is wrong. The following line is right.
// a fixed:N value can be assigned to a fixed:M value if M <= N.
fixed:2 pppp2 = pppp // ok
fixed:3 pppp3 = pppp // ok
fixed:4 pppp4 = pppp // error
fixed:5 pppp5 = pppp // error

[zigo101]

Ah! If we replace all the fixeds in my last comment with var, then we can save a keyword, and this propsoal can be Go 1 compatible!

[zigo101]

the comment previous to last is updated and corrected.

[zigo101]

The N in var:N means the mutable depth.

The following example shows some var:N uses:

type T *T
var t T
t = &t

var:0 x0 = t
x0 = t // error: "x0" is immutable.

var:1 x1 = t
x1 = t  // ok. "x1" is mutable.
*x1 = t // error: "*x1" is immutable.

var:2 x2 = t
x2 = t   // ok. "x2" is mutable
*x2 = t  // ok. "*x2" is mutable
**x2 = t // error: "**x2" is immutable.

var:3 x3 = t
x3 = t    // ok. "x3" is mutable
*x3 = t   // ok. "*x3" is mutable
**x3 = t  // ok. "**x3" is mutable
***x3 = t // error: "***x3" is immutable.

// var <=> var:∞
var x = t
x = t      // ok
*x = t     // ok
**x = t    // ok
***x = t   // ok
****x = t  // ok
*****x = t // ok
*******************************x = t

[deanveloper]

How would this interact with things already in Go currently such as const? Especially if we accept a proposal to allow const on all types.

[zigo101]

It has not any relations to (and any impacts on) the current constant things.

It is a competitor proposal to many other immutable type proposals, for this proposal and those proposals all try to solve the same problems.

[deanveloper]

I understand a bit more about how this works. It is not illustrated anywhere how this interacts with function parameters, which is the main use-case for read-only variables. It looks like fixed would be a keyword similar to var and const, but we don't currently allow var and const in function parameters.

[zigo101]

We can think that, currently, func (T1) (T2) is a simplified form of func (var T1) (var T2).

I have mentioned immutable function parameters and results in the starting comment. I re-post it here (by adding two vars)

A func(fixed T) value can be assigned to a func(var T) value, not vice versa. A func()(var T) value can be assigned to a func()(fixed T) value, not vice versa.

Or the revised version:

A func(var:1 T) value can be assigned to a func(var T) value, not vice versa. A func()(var T) value can be assigned to a func()(var:1 T) value, not vice versa.

In fact, argument passing and result returning are just general value assignments.

[zigo101]

adjusted my last reply comment.

[zigo101]

The general value assignment rule is: a var:N value can be assigned to a var:M value if M <= N.
[update]: With one exception: a var:0 value can be assigned to a var:1 value.

[zigo101]

A BTW idea: maybe we can merge the proposal described by @niemeyer in this comment into this read-only value proposal.

In short, we can specify the mutable depth for new declared variables in short variable declarations as:

	N:newV0, M:newV1, oldV = x, y, z

The mutable depth prefixes of a new declared variable can be omitted, which means its mutable depth is infinite. (But the : can't be omitted.)

Similarly, we can use the mutable depth prefixes in standard variable declarations. One difference to short variable declaration is, the : can be omitted in standard variable declarations. This design makes that the var keyword can continue being absent in function parameter and result declaration lists.

[update]:

mutable depth conversions:

type T [][]int
var a = [][]int{{1, 2, 3}, {7, 8, 9}}
var:1 b = a // type of b is also [][]int
            // <=> var 1:b = a
c := (T:1)(a) // type of c is T, it is a var:1

The following function prototypes are equivalent.

func(1:a, 2:b T1, 1:c T2) (1:x, 1:y T3)
func(1:T1, 2:T1, 1:T2) (x, y 1:T3)

[zigo101]

Arbitrary mutable depth may be an interesting idea, but it brings some complexities to Go, and it might be not much useful in practice. Maybe only supporting mutable depth 1 and 0 is enough.

I have another interesting (but some verbose) idea to extend the genres of values:

var.var // self is modifiable, references are also modifiable. 
        // It can be shortened as the current used "var".
        
var.fixed // self is modifiable, references are unmodifiable.
          // It is equivalent to the "fixed" in the starting comment.
          // It can be shortened as "fixed". 
 
fixed.var // self is unmodifiable, references are modifiable.
          // The usefulness of this value genre is limited.

fixed.fixed // self is unmodifiable, references are also unmodifiable.
            // It is equivalent to the "fixed!" in the starting comment.

[update]: one shortcoming of these denotations is they make the function prototype literals look verbose (but more readable) than the N:varName denotation. For example:

func(fixed a, b T1, fixed c T2) (fixed x, y T3)
func(fixed T1, fixed T1, fixed T2) (fixed T3, fixed T3)

[update 2]: another shortcoming of these long denotations is they make short variable declarations look verbose (but might be also more readable) than the N:varName denotation. For example:

newV0, newV1, oldV := fixed(x), fixed(y), z

[zigo101]

Updated the first comment for the channel section. Updated several other comments.

[zigo101]

Updated the first comment again by adding rules for values of more kinds of types.

[funny-falcon]

I believe, it is better to go with types and type conversion.

This proposal any way could not be implemented without types (because of interface{}.

Therefore, why don't go with types in first place?

[zigo101]

Therefore, why don't go with types in first place?

I made one: #29392
But I think this one is better for its simplicity.
And there are already many immutble-type proposals.

This proposal any way could not be implemented without types (because of interface{}.

why?

[funny-falcon]

fixed:0 p = new(int)
pp := &p // <=> fixed:1 pp = &p <=> fixed pp = &p
ppp := &pp // <=> fixed:2 ppp = &pp
pppp := &ppp // <=> fixed:3 pppp = &ppp
var val interface{} = pppp // have to put type into interface{}

[zigo101]

The last line fails to compile.

[networkimprov]

Could I ask you not to use the issue comments as a changelog? It obscures the discussion, and generates unnecessary notifications to anyone subscribed to the thread. You can keep a changelog at the end of the main text. Thanks!

[zigo101]

Sorry, here are so few discussions that I think not many people subscribed to this thread. :)
Thanks for having interests on this thread.

[alvaroloes]

TBH, I like all the comments go101 is posting. I am looking forward for good immutability support in Go and reading the progress he/she is doing on this matter excites me!

[vincent-163]

Hello. I briefly looked at your proposal, not quite thoroughly, but may I ask some questions so I get a more complete idea of the proposal?
It seems like you suggested that 'fixed' be a compile time constraint rather than type constraint, and in order to enforce immutability, fixed variables may not be copied into mutable regular variables, and all its fields are also fixed. Will this force all functions defined on that type to take a fixed parameter to work for fixed types? What is the fixed value's method set if only a subset of the methods support fixed receivers? It would really confuse things if the interface must have two different method sets depending on whether it is fixed.

[zigo101]

It seems like you suggested that fixed be a compile time constraint rather than type constraint, and in order to enforce immutability,

I would say fixed is a type constraint. And yes, most of the rules are enforced by compilers. Only the reflection rules are enforced by runtimes.

fixed variables may not be copied into mutable regular variables, and all its fields are also fixed.

More accurately speaking, all the values referenced by it are also fixed.

Will this force all functions defined on that type to take a fixed parameter to work for fixed types?

Mutable arguments can be passed to fixed parameters.

It would really confuse things if the interface must have two different method sets depending on whether it is fixed.

The interface part is really the most complicated part of the proposal, though I still think the rules are natural.

(BTW, I just modified the method set and interface sections of the proposal a little.)

The key point of the interface rules design is to ensure type T.fixed also implements I.fixed if a mutable typeT implements a mutable interface type I.

For the 4 types declared below,

• A has three methods: M0, M1 and fixed.M1. A.fixed has one method: fixed.M1.
• B has two methods: M0 and M1. B.fixed has no methods.
• X has three methods: M0, M1 and fixed.M1. X.fixed has one method: fixed.M1.
• Y has two methods: M0 and M1. Y.fixed has no methods.

The following implementation relations exist:

• A implements X, A.fixed implements X.fixed, A implements X.fixed
• B implements Y, B.fixed implements Y.fixed, B implements Y.fixed
• A implements Y, A.fixed implements Y.fixed, A implements Y.fixed
• X implements Y, X.fixed implements Y.fixed, X implements Y.fixed

type X interface {
	M0()
	fixed.M1()
}

type Y interface {
	M0()
	M1()
}

type A struct{}
func (A) M0() {}
func (A.fixed) M1() {}

type B struct{}
func (B) M0() {}
func (B) M1() {}

[vincent-163]

@go101 Thanks for the explanation.

If I didn't get it wrong, every interface has a fixed counterpart and some of the methods has a fixed modifier in front of it to indicate that the method takes a fixed receiver.

I see two problems with the design:

1. In Go, the type interface{} is often used to store a value of any Go type. Does an immutable value fit into an interface{}? I believe it does since every type implements an empty method set. But as you wrote in your proposal:

For this reason, the xyz ...interface{} parameter declarations of all the print functions in the fmt standard package should be changed to xyz ...interface{}.fixed instead.

It suggests that interface{}.fixed is different than interface{}, which is contradictory to the proposal. Did you get yourself confused?

2. Interface is the Go way to achieve polymorphism. When you call an interface method, you don't really care what the object is, you care about what it can do. In most cases, you don't even care if the object is immutable or not. By giving every interface a fixed counterpart, people have to worry about whether the object is immutable.

[zigo101]

Does an immutable value fit into an interface{}?

The answer is surely NO. I think the rules are specified clearly in the proposal.

Immutable values can only be boxed in immutable interfaces. Mutable values can also boxed into immutable interfaces, but we can think the mutable values are implicitly converted to fixed ones before boxing.

... In most cases, you don't even care if the object is immutable or not. ...

This is not true any longer by introducing the proposal. When you call a method on an immutable interface value, you DO expect the dynamic receiver value will not be modified. (Edit: expect the values referenced by the dynamic receiver value will not be modified.)

[vincent-163]

... you DO expect the dynamic receiver value will not be modified. ...

I am not convinced that such property can be expected from an immutable value.

Immutable values are not read-only, and vice versa, in the following ways:

1. Immutable values are not necessarily read-only in that it could be a pointer to another storage where immutability is not enforced. For example:

type Object [16]byte
var database = OpenMyDatabase()
func (id Object.fixed) Get() string {
  return database.Get(id)
}
func (id Object.fixed) Set(s string) {
  database.Set(id, s)
}

2. Read-only values are not necessarily immutable. Some data structures write even during reads, usually to guarantee amortized time/space cost. Check out the implementation of sync.Map. There's also a data structure called splay tree where every read is followed by a "splay operation", which must acquire a write mutex.

[zigo101]

Immutable values are not necessarily read-only in that it could be a pointer to another storage where immutability is not enforced.

You can't modify the values referenced by a fixed value through the fixed value, this is the core principle of this proposal. All other rules are derived from the core principle.

So the immutability of another storage must be enforced through the immutable values. The other storage may be modified through other mutable values, but it can't through fixed values.

In your example, the types of the first arguments of the methods of database.Set and database.Get must be both Object.fixed.

Read-only values are not necessarily immutable.

I'm confused by the your definition of read-only and immutable. It looks, by your definitions, both read-only and immutable values can be modified. And I don't understand what are the relations to sync.Map and the splay tree algorithm.

[vincent-163]

You can't modify the values referenced by a fixed value through the fixed value, this is the core principle of this proposal. All other rules are derived from the core principle.

The Object in my example simply stores a 16-byte identifier, which is used by the database to lookup values. The reference here is an indirect reference, and I can't see how such immutability can be enforced on a language level (since it's always possible to "launder" an immutable value and turn it mutable by copying it). It's also possible that the actual value is stored on a remote server outside runtime's control.

I'm confused by the your definition of read-only and immutable. It looks, by your definitions, both read-only and immutable values can be modified. And I don't understand what are the relations to sync.Map and the splay tree algorithm.

The thing here is that sync.Map and splay tree can have a read-only interface even though the underlying struct is modified during read-only operations. For the sync.Map case, its Range and Get functions both modify the data in certain cases. Splay tree is a bit complex, but the core idea is that every read operation needs to be followed by a write operation in order to guarantee amortized O(log n) time complexity.

Such data structures can't fit into an immutable value even if they are "read-only" as a programmer normally expects from an immutable value.

[zigo101]

It looks we are talking about two different concepts of "immutability". This proposal only makes some immutability guarantees for Go values in the language level, not application logic specified level.

The Object in my example simply stores a 16-byte identifier, which is used by the database to lookup values. The reference here is an indirect reference,

I'm sorry, the "reference" concept used in this proposal is not the same as your mentioned "reference". In this proposal, a "reference` means a pointer pointing to another value. It is not the "reference" in logic.

The thing here is that sync.Map and splay tree can have a read-only interface even though the underlying struct is modified during read-only operations.

This may be true. A sync.Map Go value has many internal primitive fields. It is ok to read one field and write another field at the same time. But a read and a write should never happens on such the same primitive field at the same time.

[vincent-163]

It looks we are talking about two different concepts of "immutability". This proposal only makes some immutability guarantees for Go values in the language level, not application logic specified level.

One concept is the concept mentioned in the proposal, the other concept is what I believe programmers normally expect from "immutability" or the term "read-only". The examples illustrate the difference between the two concepts. And it is subtle.

The thing here is that sync.Map and splay tree can have a read-only interface even though the underlying struct is modified during read-only operations.

Maybe I didn't talk about this point clearly. It means that the read-only operation itself writes to the struct as part of the operation. It cannot read without doing the write.

The problem with such two different concepts is that it confuses programmers and add unnecessary complexity to the code, since most functions expect a "read-only" parameter but it differs from the "immutable" property in the proposal. Take fmt package as an example, since most of its functions expect "read-only" parameters. As you have mentioned, all the interface{} in the fmt package would have to be changed into interface{}.fixed. But what fmt expects is not really an immutable value. It just needs to be "read-only". This prevents any objects that are read-only but cannot be made immutable from being an input to fmt.

[zigo101]

What the "fixed" described in the proposal is just the "read-only" described in your last comment. Maybe it is a fault for the proposal to use the "immutable" word in descriptions. Please note, a fixed value itself might be modifiable.

The proposal does support another "immutable" concept. Values declared with final can never be modified through any ways. But the values referenced by a final value are possible to be modified.

[zigo101]

I modified the proposal to try to avoid using the "immutable" word.

[zigo101]

@HEWENYANG about the sync.Map, do you mean the Load method should not call the sync.Map.mu.Lock method if the sync.Map receiver value is fixed?

This is really a problem. A simpler case:

type CounterA struct {
	mu sync.Mutex
	n uint64
}

func (c *CounterA.fixed) Get() uint64 {
	c.mu.Lock() // error: the receiver of Lock is not fixed.
	v := c.n
	c.mu.Unlock()
	return v
}

The current rules are over-strict here. Here, we expect the n field will not be modified in the Get method, but the rules also prevent the Lock method of the mu field from being called.

[zigo101]

How about to add one more notation? T.unfixable, which means a type whose values can never be converted to T.fixed. So that, we can declare CounterA as

type CounterA struct {
	mu sync.Mutex.unfixable
	n uint64
}

The T.unfixable notation can only be used in field specifications.

All unsafe pointer types are viewed as such unfixable types.

Values of type CounterA can be converted to CounterB, but not vice versa. (Edit: Not convertible in both directions)

type CounterB struct {
	mu sync.Mutex
	n uint64
}

Edit 2: T.unfixable values can be assigned to type T.

[zigo101]

As this proposal has mentioned, each value has two properties, self_modifiable and ref_modifiable.
However, up to now, the proposal hasn't make a way to specify the two properties for struct fields.
In fact, currently, a struct field t.x will inherit the two properties from the struct value t.

To make it possible to fully control the properties of struct fields, the full literal form of an unnamed struct type should be like

struct {
	var   x []int
	final y []int.unfixable
	      z []int.fixable // the ".fixable" can be ommited here,
	                      // just like it in the "x" declaration.
}

As we can see, there are three prefix options:

1. var: this field will be always modifiable.
2. final: this field will never be modifiable (after the parent struct value initialized).
3. blank: this field will inherit the self_modifiable property from its parent struct value.

And there are two suffix options:

1. unfixable: the ref_modifiable property of this field will be always true.
2. fixable or blank: this field will inherit the ref_modifiable property from its parent struct value.

However, with all of these prefixes and suffixes being supported, the language will become much complicated. So I suggest to only support the prefixes, and (for struct field declarations) let

1. a var prefix mean var.unfixable
2. a final prefix mean final.fixable
3. a blank prefix mean inherit.fixable

So, for the CounterA struct example shown the previous comments, it can be declared as

type CounterA struct {
	var mu sync.Mutex
	n uint64
}

func (c *CounterA.fixed) Get() uint64 {
	c.mu.Lock() // OK: no problems now
	v := c.n
	c.mu.Unlock()
	return v
}

Is it an acceptable design? (Personally, I think this design is a bit ugly, but I haven't found a better one yet.)

[zigo101]

I just re-read this proposal thread. It looks the fixed concept of my proposal is much like @jba's ro keyword, though there are many detailed differences.

[zigo101]

Does anyone think taking addresses of final values is meaningful?
I'm planning to add a rule to forbid taking addresses of final values, so that we can unify some explanations. We can view all intermediate results as final values, including function returns, operator operation evaluation results, explicit value conversion results, etc.

Edit: calling intermediate results as final values may be good, but unaddressable values and final values should be two different concepts.

[ianlancetaylor]

Thanks. While there are differences from #22876, this is similar enough that we are going to group them together. They both center around the same idea of qualifying types to mark them as read-only. Both require more thought, and I hope we can combine the ideas into something that will work.

[zigo101]

@ianlancetaylor I do agree the fixed concept is like @jba's ro keyword. However, this proposal has so many other ideas that the proposal #22876 is almost a subset (about 1/2) of this proposal.

• this proposal also proposes another important concept: final values. Without the important concept, it is hard to implement proposal: spec: allow constants of arbitrary data structure type #6386. The two concepts fixed values and final values are not independent to each other, there are some interactions between them.
• this proposal mentions reflection modifications
• this proposal clarifies the method sets of fixed values
• this proposal clarifies how to box fixed values in interfaces
• this proposal clarifies we can send values to fixed channels
• and this proposal is still trying to find a way to specify the properties of struct fields (yes, this proposal is still not done.).

I also put much effort on improving the syntax. Comparing to the old revisions of this proposal, the current syntax is much simpler and more readable.

In fact, this proposal is not just an early-age idea, I have put much effort to evolve it into a full practical solution. Currently, it specifies the detailed rules for almost every corner.

And I'm still updating and improving this proposal constantly, almost everyday. This proposal is still not stabilized yet, some new elements are still in pending status. If it should be closed, I really expect another reason, such as there are serious flaws in it.

So I appeal to reopen this issue.

[ianlancetaylor]

Frankly, this proposal is long and with so many revisions that I find it very difficult to understand. And the fact that you are updating and improving the proposal constantly is not helpful for proposal review. We can't evaluate a proposal that is constantly being changed.

If you really think that you need to discuss this separately from #22876 then I recommend that you bring this proposal to a final and complete state that you are happy with, perhaps with discussion on golang-dev or elsewhere with other interested people, and then open a new issue with the final, complete, proposal that we can reasonably examine.

That said, I have to say that I am not optimistic about any type qualifier based approach to read-only parameters. Perhaps there is a way to make it work nicely in Go, but as has been discussed in many places I haven't seen one that avoids the problems with the const type qualifier in C.

Thanks.

[zigo101]

I can't make a final proposal I'm happy with without collecting feedbacks.
I also posted this proposal at many other places, but sadly, I got not any useful feedbacks from those places. (Yes, I haven't tried it on golang-dev.)

Although the feedbacks I got here are also not many, but they really helped me improving the initial proposal much.

But OK, I think I'm almost happy with this proposal now (except the struct field part). I will try to post it at the last place, golang-dev., where I might get more feedbacks.

I will try to create a new issue again when all are done.

[zigo101]

Just remind some new benefits of declared final values (or any true immutable values): #30528, #30449, #30529