proposal: spec: add kind-specific nil predeclared identifier constants

[ianlancetaylor]

A common error for people new to Go is misunderstanding that a interface that is not nil can contain a nil pointer. This is one of the most commonly cited entries in the Go FAQ: https://golang.org/doc/faq#nil_error. A quick search shows at least 24 threads on golang-nuts discussing this, even though it is already in the FAQ.

It is not new to observe that one of the causes of this common mistake is that nil is overloaded. Since changing that would not be backwards compatible, I propose the following changes.

1. We add six new predefined identifiers: nilptr, nilinterface, nilslice, nilchan, nilfunc, nilmap. These new identifiers are untyped constants that only resolve to a certain kind, much as 1.1 is an untyped constant that only resolves to a float or complex type. nilptr may be used as the zero value for any pointer type, nilinterface may be used as the zero value for any interface type, and so forth. An attempt to use, for example, nilptr in an assignment or comparison with a variable or value that is not a pointer type is invalid.

2. We add a vet check that warns about any comparison of a value of any type with plain nil. We encourage people to change x == nil to x == nilinterface or x == nilptr or whatever is appropriate. Since initially this vet check will trigger all the time, we can not turn it on when running go test. It could be on by default when running go vet.

3. At this point people who run go vet will no longer make the common mistake. If v is a value of interface type, then writing v == nilptr will be a compilation error. Writing v == nilinterface will not cause people to erroneously think that this is testing whether v contains a nil pointer.

4. In some later release, we turn on the vet check when running go test.

5. If we are ever willing to make a backward incompatible change, we can make v == nil a compilation error rather than simply being a vet error.

Something to consider is that one could imagine permitting v == nilptr when v has an interface type, and having this be true if v is not nilinterface, if v holds a value of pointer type, and if the pointer value is nilptr. I don't know that this is a good idea, and I'm not proposing it. I'm only proposing the above.

[cespare]

One consequence of this would be that the extremely common if err != nil { ... } would be significantly longer: if err != nilinterface { ... }.

[ianlancetaylor]

@cespare Very valid point.

[bradfitz]

I'd be happier with something in the middle: only adding nilinterface, but naming it none or something shorter. I don't think nilmap, nilslice, etc generate enough confusion.

[ianlancetaylor]

@bradfitz It sounds like you are suggesting that we move toward a different name for the zero value of an interface type, but keep the general overloading of nil for non-interface types. Does that sound right?

I don't really see none as conveying the right idea, but I'm having trouble coming up with a good name. We could try nili or inil, but those might be too close to nil. Maybe empty? I hope someone else has a better idea.

(C++ has a similar confusion in which 0 is the zero value for both integers and pointers, with the preprocessor macro NULL often being used in code. In C++ this has now been resolved by preferring the new keyword nullptr as the pointer zero value.)

[bradfitz]

It sounds like you are suggesting that we move toward a different name for the zero value of an interface type, but keep the general overloading of nil for non-interface types. Does that sound right?

Yes.

Maybe empty?

I like empty.

Seeing lots of if err != empty { ... } is still pretty verbose, though.

[cznic]

Maybe empty?

empty is nice, and it would be enough to predeclare it in the universe scope without breaking any existing Go1 program.

package main

import (
	"fmt"
	"io"
)

var empty interface{}

func main() {
	var w io.Writer
	fmt.Println(w == empty)
}

Playground

[cznic]

it would be enough to predeclare it in the universe scope without breaking any existing Go1 program.

Which is actually what the proposal says, but I noticed that only now, mea culpa 👎

[davecheney]

@ianlancetaylor thank you for raising this issue. I think this proposal adds a lot of complexity to solve a single issue, returning a typed nil. What about just making a typed nil equal to nil? It is after all what the author of the check intended.

[bradfitz]

What about just making a typed nil equal to nil?

That breaks valid code.

[neild]

If interface{}((*T)(nil)) == nil, how do you test to see if an interface value is zero? I also think changing this equality might be superficially more intuitive, but is more confusing when you dig into it.

Instead of changing the name of a zero-valued interface, it would also be possible to change the name of a zero-valued pointer. null, nullptr, or maybe just 0.

[davecheney]

I don’t doubt that, but the situation now is equally error prone. Anyway, this was always in the context of a Go 2 major version bump.

…

On 15 Nov 2017, at 09:32, Brad Fitzpatrick ***@***.***> wrote: What about just making a typed nil equal to nil? That breaks valid code. — You are receiving this because you commented. Reply to this email directly, view it on GitHub, or mute the thread.

[faiface]

This issue of nil vs nil comes up all the time.

I think it's time to reconsider what the real issue is. I personally think the current implementation is sound, useful, and when someone understands how it works, they're not going to make any mistakes with it. Not any more than with any other feature.

Maybe the only real problem is education around the issue. People come to Go with expectations. And, when they are not explicitly and repeatedly told about this behavior, they assume it works differently than it actually does. But, that doesn't mean it works bad, it just means that people's expectations are bad. Maybe the only thing we need to fix is people's expectations.

[neild]

Perhaps the problem is less interface equality and more the ease of accidentally assigning a nil pointer to an interface value when you didn't intend to.

[ianlancetaylor]

@davecheney Even for Go 2, I don't think it's OK to remove the ability to test whether an interface is the zero value for interfaces.

@faiface You're not wrong, but if people consistently get something wrong it's worth exploring ways to change it to make it less error prone.

@neild suggested in conversation that we should simply change the zero value of pointers to null. A comparison of an interface value with null would be a compilation error. Assigning nil to a pointer, or comparing a pointer with nil, would be a vet error at least for now. Another option would be to copy C++ and use nullptr. The advantage of these approaches is that all the err != nil code would still be fine. The disadvantage is that all the p != nil code, where p is some pointer type, would want to change to p != null.

[jimmyfrasche]

I'm concerned that this would just push the misunderstanding around while breaking everyone else's muscle memory—but another alternative:

Accept #19642 for making _ a universal zero value and then (initially via a vet check) disallow checking an interface (and possibly other things) against untyped nil.

All the err == nil would change but they'd shorten to err == _.

This could of course expand the problem by allowing one to think it possible to see if whatever is in the interface is zero, but that would come with allowing a universal zero value anyway.

[griesemer]

I'm not a fan of introducing so much nomenclature (nilptr, nilmap, etc.) for relatively little benefit. I also think that we should keep nil as the zero value for pointers, it seems to me that nil is more commonly used with pointers than interfaces (to be verified).

It also would be a mistake to equate a test of (interface) x == nil as a test that ignores typed nils (as @davecheney suggested). @bradfitz already pointed out that it would not be backward-compatible. But it's actually important that we have this facility in the first place: A common idiom in Go is to define data types whose zero values are values that are ready to use. It can make a lot of sense to define a pointer type whose zero value (nil) is a ready-to use value. It may have methods that work with nil receivers. Such a type may even implement the error interface and thus represent a valid error (and we might not know about it if such a type is defined in some external library). It is imperative that in such a case the test err == nil doesn't yield true. Perhaps far-fetched a scenario for an error type, but not so much for other situations.

But there is another side to this problem, and that is that there is no easy way (*) to test if a variable of interface type holds a (typed) nil pointer (or map, or channel) without knowing the actual type. (This does come up, e.g., in iterators walking over graphs such as the Go syntax tree in go/ast.)

If the language had a facility to test for such non-nil interfaces x holding nil values (perhaps via a builtin function isnilptr(x)), presented squarely next to the x == nil test, as a programmer one might be more acutely aware of the two possibilities and automatically think twice.

(*) One way of doing it is using reflection

func isnilptr(x interface{}) bool {
	v := reflect.ValueOf(x)
	return v.IsValid() && v.Kind() == reflect.Ptr && v.IsNil()
}

[jimmyfrasche]

@griesemer why single out pointer-y things for such a builtin? Why not a more general holdsZero?

func holdsZero(i interface{}) bool {
	v := reflect.ValueOf(i)
	if !v.IsValid() { // maybe drive the point home further and let this panic if i == nil
		return false
	}
	z := reflect.Zero(v.Type()).Interface()
	return i == z
}

(I wouldn't be surprised if that's missing some edge cases)

In either case, if it were a builtin it should fail to compile unless it's parameter is an interface, to avoid confusion like holdsZero(0) being true.

[neild]

Why single out testing for the presence of a zero value?

An interface type specifies the desired properties of implementations of that interface; if zero-ness is a significant property of an interface, then shouldn't it be part of the type definition?

type T interface {
  IsZero() bool
}

It feels to me that interface equality is approaching the problem from the wrong direction; I think that in cases where someone is trying to test for a typed nil value in an interface they almost always really wanted an untyped nil interface value instead. e.g., problem the nil error FAQ entry describes is not that error((*myerror)(nil)) != nil, but that the error contains a typed nil in the first place.

[griesemer]

@jimmyfrasche Maybe. I chose the pointer case because this is the one that I've actually seen in practice.

@neild You have a point with using a method.

[earthboundkid]

I love this proposal.

I think it's worth considering null as the name to add. It's a very commonly used keyword. Semantically, it would make more sense for everything but interface to use null, and interface would continue to use nil. OTOH, if we use none, then interface could use that and everything else could continue using nil. In either case, there's an easy transition using go vet from Go 1 to Go 2.

[ugorji]

What about making nil a variable OR a function that has a context i.e.

• nil // current interpretation of nil
• nil(map)
• nil([])
• nil(func)
• nil(interface)
• nil(*)
• nil(chan)

This way, there's no new keyword, but we expand the use of the nil keyword (like we expanded the use of make).

If we combine this with a builtin iszero(x) function where x is any value, and accompanying reflect.IsZero(reflect.Value) function, we should capture most of the use cases without introducing many new keywords.

[griesemer]

@ugorji Just for the record: nil is not a keyword. It's a predeclared identifier. There's a difference.

[faiface]

I would like to point out one thing. Having interfaceValue == nil or even interfaceValue == nilptr is inconsistent with current Go anyway. Let me explain why. It basically assumes, that the nilptr value will get automatically converted to the concrete type of interfaceValue and the true comparison will happen afterwards.

This is not how it happens in Go. Let's take a look at numbers for a similar example:

var x interface{}
x = float64(0)
if x == 0 {
    fmt.Println("this doesn't happen!")
}

Did you guess correctly what happens in the above example? It is similarly counter-intuitive at first, but it makes sense. The 0 value gets converted to its default type, which is int and comparison proceeds and results in false, because we are comparing different types.

With interfaces it's similar. Does it even make any sense whatsoever, to ask whether an interface value is a nil value of an arbitrary pointer type?

[ianlancetaylor]

@faiface I agree: it doesn't. But the evidence is clear that that is what many people new to Go think is happening. It's a FAQ, and it's visibly a stumbling block for new Go programmers. I think it's worth examining whether there is something we can do to cause fewer people to stumble here.

[faiface]

@ianlancetaylor How about the numbers? Would you solve those too? If not, being able to check if interface contains arbitrarily typed nil pointer, but not being able to check if it contains an arbitrarily typed zero, or one, would be an inconsistency that could cause more confusion than clarification.

[ianlancetaylor]

@faiface This proposal would not fix the comparisons of an interface value with 0. Current experience is that that is a much less significant problem: it is rarely reported as a bug, and there has been no need to create a FAQ entry for it. I'm not particularly worried about that inconsistency myself.

(Note that this specific proposal is not for a way to check whether an interface contains a nil pointer of arbitrary type. I mentioned that as "something to consider," but it is not part of the proposal.)

[pcostanza]

@alanfo You seem convinced that interface nil vs. pointer nil is just a matter of teaching and better understanding of the language. I don't agree with that. I rather think it's objectively confusing. Consider:

n = nil
e = n
if e != nil {
   fmt.Println(e)
}

This code fragment prints <nil> depending on the involved types. It's a pity you have to know the involved types in a language that otherwise (rightfully) prides itself of being particularly easy to read and understand.

The problem is that interface nil and pointer nil are conceptually two vastly different concepts, and it was not a good idea in the original design of Go to give them the same name.

I believe this particular problem should not anymore be the rite of passage for Go programmers that it currently is, because it's unnecessary and doesn't serve a purpose. (Backwards compatibility is not a particularly convincing purpose in a language that is currently seeing some dramatic changes anyway.)

Just my 0.02€... ;)

Pascal

[alanfo]

@pcostanza

The problem is that interface nil and pointer nil are conceptually two vastly different concepts, and it was not a good idea in the original design of Go to give them the same name.

They are different but presumably, back in v1.0 days, the Go team didn't consider they were different enough to justify the use of another predefined identifier (such as empty or void) to indicate that an interface contained no value at all. So we are where we are.

Although we are likely to see a dramatic change to the language in the introduction of generics (but probably not now error handling) the current generics proposal is still backwards compatible (or, at any rate, can be) and I know the Go team regard this as a very important consideration.

As I don't really see how anyone can use interfaces successfully without being able to distinguish between an interface variable and what (if anything) it currently contains, then education must play an important part and, whilst I don't object to a language change being made to reinforce this, I do think such a change should be simple, proportionate and (preferably) backwards compatible.

[ianlancetaylor]

Reading the proposal again, your alternative idea of allowing x == nilptr for interfaces would be equivalent to isnilptr(x) though I prefer the latter as the former would (in effect) be overloading the == operator.

This proposal has a series of five steps aimed to fix the problem over time. Now maybe it's a bad proposal--in fact, it probably is--but it's more than just one idea.

[alanfo]

When I referred to your 'alternative idea' I was alluding to your final paragraph:

Something to consider is that one could imagine permitting v == nilptr when v has an interface type, and having this be true if v is not nilinterface, if v holds a value of pointer type, and if the pointer value is nilptr. I don't know that this is a good idea, and I'm not proposing it. I'm only proposing the above.

As you made clear, this wasn't part of the actual proposal but an idea to consider.

[ianlancetaylor]

Ah, sorry.

[andig]

This is a little off-topic, but since https://golang.org/doc/faq#nil_error was mentioned I feel it's not precise enough:

func returnsError() error {
	var p *MyError = nil
	if bad() {
		p = ErrBad
	}
	return p // Will always return a non-nil error.
}

func returnsError() error {
	if bad() {
		return ErrBad
	}
	return nil
}

It's a good idea for functions that return errors always to use the error type in their signature (as we did above) rather than a concrete type such as *MyError, to help guarantee the error is created correctly.

It seems that the bad (first) example is "use(ing) the error type in their signature" as advised, yet still broken. It should not only use the error type but also return the nil interface instead of a differently-typed nil (if that's the correct wording).

[rogpeppe]

@griesemer wrote:

It also would be a mistake to equate a test of (interface) x == nil as a test that ignores typed nils (as @davecheney suggested). @bradfitz already pointed out that it would not be backward-compatible. But it's actually important that we have this facility in the first place: A common idiom in Go is to define data types whose zero values are values that are ready to use. It can make a lot of sense to define a pointer type whose zero value (nil) is a ready-to use value. It may have methods that work with nil receivers. Such a type may even implement the error interface and thus represent a valid error (and we might not know about it if such a type is defined in some external library). It is imperative that in such a case the test err == nil doesn't yield true. Perhaps far-fetched a scenario for an error type, but not so much for other situations.

I've been thinking about this for a long time, and while I buy the theoretical argument, I'm still not entirely convinced.
I haven't yet seen many significant cases where this property is used to advantage.

One point in favour of disallowing typed nils as valid receivers: the pointer method set also contains the value method set, but if you happen to call a value method when the pointer is nil, you'll get a panic, so that property isn't as useful as it could be. Passing the zero value is one possible way of fixing that, but disallowing nil receivers would be another. If you disallow nil receivers, then perhaps it's reasonable to change a typed nil pointer into an untyped nil when converting to an interface. The problem is much less acute for slices, channels and maps, so perhaps they could be left as is.

But as @ianlancetaylor points out in the above-mentioned issue, doing this would change the behaviour of existing code, which probably rules it out as a language change.

[tandr]

Would completely disallowing a naked (uncasted) nils in comparisons solve this? Like, enforce a rule that nil cannot be naked, it has to be dressed in a type wrapper. (if only for "complex" type comparisons at the beginning) go vet should be able to help with transition.

Sorry, I don't remember if it was suggested and declined as "too much existing code affected".

[ianlancetaylor]

@tandr Anything that requires us to rewrite if err != nil is a non-starter. (That is also a problem with this proposal.)

[pcostanza]

@ianlancetaylor You could have nil take over the meaning of your nilinterface and drop all other current uses of nil. nil would then only be valid for interface types, and for nothing else anymore. (At least after a transitional period.)

[earthboundkid]

The options are:

• Add a new identifier for interfaces (+slice +map +channel?) and leave nil pointers alone
• Add a new identifier for pointers (+slice +map +channel?) and leave nil interfaces alone
• Add two new identifiers (or more with slice/map/channel)

Whichever is done, for backwards compatibility reasons p == nil and err == nil can probably never be changed, but you could add a lint check + rewriter to used the preferred one.

On identifiers, my two cents are that nilinterface is too long to type regularly. (interface{} is already long enough, and you don't have to type it that often.) null is good on its own, but probably too close to nil if we have both in the language. none seems fine. So, I lean towards, just add none for interfaces and get a rewriter for err == nil to err == none. You could also make none valid for map/slice/chan, but I don't think that solves a real problem, so I would lean against it.

[andig]

To pick up thw comment by @tandr: why to we need the ability to invoke methods on a nil value at all? I think I understand why its technically possible and permitted by the language as-is, but is it necessary for any purpose?

[pcostanza]

@andig It’s useful for exactly the same reason that dynamic dispatch is useful in other cases: You can implement the if receiver == nil case in a central place, and don’t have to spread it all over the rest of the code. (No language constructs are strictly speaking “necessary,” it’s all about convenience.)

[xuanswe]

I did not read all comments in this very long topic, sorry for that.
I am also new to this language, so sorry if I don't understand the details.

But is that the same story as Dart?

Dart 1: no null-safety
Dart 2: null-safety feature is optional. migration-tool is provided.
Dart 3: null-safety feature is a must

100% sound null safety
Dart 2.12 introduced null safety more than two years ago. In Dart 2.12, users needed to enable null safety with a pubspec setting. In Dart 3, null safety is built in; you cannot turn it off.

Why don't we apply this successful story to Go?

[ianlancetaylor]

@xuan-nguyen-swe This issue doesn't have anything to do with null safety. That is a completely different topic. For null safety see, for example, #28133.

[xuanswe]

@ianlancetaylor thanks for redirecting me to the correct topic!
I am new to Go, so please apologies to me if I misunderstood something.