proposal: spec: add sum types / discriminated unions

[DemiMarie]

This is a proposal for sum types, also known as discriminated unions. Sum types in Go should essentially act like interfaces, except that:

• they are value types, like structs
• the types contained in them are fixed at compile-time

Sum types can be matched with a switch statement. The compiler checks that all variants are matched. Inside the arms of the switch statement, the value can be used as if it is of the variant that was matched.

[ianlancetaylor]

This has been discussed several times in the past, starting from before the open source release. The past consensus has been that sum types do not add very much to interface types. Once you sort it all out, what you get in the end is an interface type where the compiler checks that you've filled in all the cases of a type switch. That's a fairly small benefit for a new language change.

If you want to push this proposal along further, you will need to write a more complete proposal doc, including: What is the syntax? Precisely how do they work? (You say they are "value types", but interface types are also value types). What are the trade-offs?

[rsc]

See https://www.reddit.com/r/golang/comments/46bd5h/ama_we_are_the_go_contributors_ask_us_anything/d03t6ji/?st=ixp2gf04&sh=7d6920db for some past discussion to be aware of.

[griesemer]

I think this is too significant a change of the type system for Go1 and there's no pressing need.
I suggest we revisit this in the larger context of Go 2.

[rogpeppe]

Thanks for creating this proposal. I've been toying with this idea for a year or so now.
The following is as far as I've got with a concrete proposal. I think
"choice type" might actually be a better name than "sum type", but YMMV.

Sum types in Go

A sum type is represented by two or more types combined with the "|"
operator.

type: type1 | type2 ...

Values of the resulting type can only hold one of the specified types. The
type is treated as an interface type - its dynamic type is that of the
value that's assigned to it.

As a special case, "nil" can be used to indicate whether the value can
become nil.

For example:

type maybeInt nil | int

The method set of the sum type holds the intersection of the method set
of all its component types, excluding any methods that have the same
name but different signatures.

Like any other interface type, sum type may be the subject of a dynamic
type conversion. In type switches, the first arm of the switch that
matches the stored type will be chosen.

The zero value of a sum type is the zero value of the first type in
the sum.

When assigning a value to a sum type, if the value can fit into more
than one of the possible types, then the first is chosen.

For example:

var x int|float64 = 13

would result in a value with dynamic type int, but

var x int|float64 = 3.13

would result in a value with dynamic type float64.

Implementation

A naive implementation could implement sum types exactly as interface
values. A more sophisticated approach could use a representation
appropriate to the set of possible values.

For example a sum type consisting only of concrete types without pointers
could be implemented with a non-pointer type, using an extra value to
remember the actual type.

For sum-of-struct-types, it might even be possible to use spare padding
bytes common to the structs for that purpose.

[bcmills]

@rogpeppe How would that interact with type assertions and type switches? Presumably it would be a compile-time error to have a case on a type (or assertion to a type) that is not a member of the sum. Would it also be an error to have a nonexhaustive switch on such a type?

[josharian]

For type switches, if you have

type T int | interface{}

and you do:

switch t := t.(type) {
  case int:
    // ...

and t contains an interface{} containing an int, does it match the first case? What if the first case is case interface{}?

Or can sum types contain only concrete types?

What about type T interface{} | nil? If you write

var t T = nil

what is t's type? Or is that construction forbidden? A similar question arises for type T []int | nil, so it's not just about interfaces.

[rogpeppe]

Yes, I think it would be reasonable to have a compile-time error
to have a case that can't be matched. Not sure about whether it's
a good idea to allow non-exhaustive switches on such a type - we
don't require exhaustiveness anywhere else. One thing that might
be good though: if the switch is exhaustive, we could not require a default
to make it a terminating statement.

That means that you can get the compiler to error if you have:

func addOne(x int|float64) int|float64 {
    switch x := x.(type) {
    case int:
        return x + 1
    case float64:
         return x + 1
    }
}

and you change the sum type to add an extra case.

[rogpeppe]

For type switches, if you have

type T int | interface{}

and you do:

switch t := t.(type) {
case int:
// ...
and t contains an interface{} containing an int, does it match the first case? What if the first case is case interface{}?

t can't contain an interface{} containing an int. t is an interface
type just like any other interface type, except that it can only
contain the enumerated set of types that it consists of.
Just like an interface{} can't contain an interface{} containing an int.

Sum types can match interface types, but they still just get a concrete
type for the dynamic value. For example, it would be fine to have:

type R io.Reader | io.ReadCloser

What about type T interface{} | nil? If you write

var t T = nil

what is t's type? Or is that construction forbidden? A similar question arises for type T []int | nil, so it's not just about interfaces.

According to the proposal above, you get the first item
in the sum that the value can be assigned to, so
you'd get the nil interface.

In fact interface{} | nil is technically redundant, because any interface{}
can be nil.

For []int | nil, a nil []int is not the same as a nil interface, so the
concrete value of ([]int|nil)(nil) would be []int(nil) not untyped nil.

[bcmills]

The []int | nil case is interesting. I would expect the nil in the type declaration to always mean "the nil interface value", in which case

type T []int | nil
var x T = nil

would imply that x is the nil interface, not the nil []int.

That value would be distinct from the nil []int encoded in the same type:

var y T = []int(nil)  // y != x

[jimmyfrasche]

Wouldn't nil always be required even if the sum is all value types? Otherwise what would var x int64 | float64 be? My first thought, extrapolating from the other rules, would be the zero value of the first type, but then what about var x interface{} | int? It would, as @bcmills points out, have to be a distinct sum nil.

It seems overly subtle.

Exhaustive type switches would be nice. You could always add an empty default: when it's not the desired behavior.

[rogpeppe]

The proposal says "When assigning a value to a sum type, if the value can fit into more
than one of the possible types, then the first is chosen."

So, with:

type T []int | nil
var x T = nil

x would have concrete type []int because nil is assignable to []int and []int is the first element of the type. It would be equal to any other []int (nil) value.

Wouldn't nil always be required even if the sum is all value types? Otherwise what would var x int64 | float64 be?

The proposal says "The zero value of a sum type is the zero value of the first type in
the sum.", so the answer is int64(0).

My first thought, extrapolating from the other rules, would be the zero value of the first type, but then what about var x interface{} | int? It would, as @bcmills points out, have to be a distinct sum nil

No, it would just be the usual interface nil value in that case. That type (interface{} | nil) is redundant. Perhaps it might be a good idea to make it a compiler to specify sum types where one element is a superset of another, as I can't currently see any point in defining such a type.

[ianlancetaylor]

The zero value of a sum type is the zero value of the first type in the sum.

That is an interesting suggestion, but since the sum type must record somewhere the type of the value that it currently holds, I believe it means that the zero value of the sum type is not all-bytes-zero, which would make it different from every other type in Go. Or perhaps we could add an exception saying that if the type information is not present, then the value is the zero value of the first type listed, but then I'm not sure how to represent nil if it is not the first type listed.

[jimmyfrasche]

So (stuff) | nil only makes sense when nothing in (stuff) can be nil and nil | (stuff) means something different depending on whether anything in stuff can be nil? What value does nil add?

@ianlancetaylor I believe many functional languages implement (closed) sum types essentially like how you would in C

struct {
    int which;
    union {
         A a;
         B b;
         C c;
    } summands;
}

if which indexes into the union's fields in order, 0 = a, 1 = b, 2 = c, the zero value definition works out to all bytes are zero. And you'd need to store the types elsewhere, unlike with interfaces. You'd also need special handling for the nil tag of some kind wherever you store the type info.

That would make union's value types instead of special interfaces, which is also interesting.

[shanemhansen]

Is there a way to make the all zero value work if the field which records the type has a zero value representing the first type? I'm assuming that one possible way for this to be represented would be:

type A = B|C
struct A {
  choice byte // value 0 or 1
  value ?// (thing big enough to store B | C)
}

[edit]

Sorry @jimmyfrasche beat me to the punch.

[jimmyfrasche]

Is there anything added by nil that couldn't be done with

type S int | string | struct{}
var None struct{}

?

That seems like it avoids a lot of the confusion (that I have, at least)

[jimmyfrasche]

Or better

type (
     None struct{}
     S int | string | None
)

that way you could type switch on None and assign with None{}

[bcmills]

@jimmyfrasche struct{} is not equal to nil. It's a minor detail, but it would make type-switches on sums needlessly(?) diverge from type-switches on other types.

[jimmyfrasche]

@bcmills It wasn't my intent to claim otherwise—I meant that it could be used for the same purpose as differentiating a lack of value without overlapping with the meaning of nil in any of the types in the sum.

[jimmyfrasche]

@rogpeppe what does this print?

// r is an io.Reader interface value holding a type that also implements io.Closer
var v io.ReadCloser | io.Reader = r
switch v.(type) {
case io.ReadCloser: fmt.Println("ReadCloser")
case io.Reader: fmt.Println("Reader")
}

I would assume "Reader"

[bcmills]

@jimmyfrasche I would assume ReadCloser, same as you'd get from a type-switch on any other interface.

(And I would also expect sums which include only interface types to use no more space than a regular interface, although I suppose that an explicit tag could save a bit of lookup overhead in the type-switch.)

[jimmyfrasche]

@bcmills it's the assigment that's interesting, consider: https://play.golang.org/p/PzmWCYex6R

[rogpeppe]

@ianlancetaylor That's an excellent point to raise, thanks. I don't think it's hard to get around though, although it does imply that my "naive implementation" suggestion is itself too naive. A sum type, although treated as an interface type, does not have to actually contain direct pointer to the type and its method set - instead it could, when appropriate, contain an integer tag that implies the type. That tag could be non-zero even when the type itself is nil.

Given:

 var x int | nil = nil

the runtime value of x need not be all zeros. When switching on the type of x or converting
it to another interface type, the tag could be indirected through a small table containing
the actual type pointers.

Another possibility would be to allow a nil type only if it's the first element, but
that precludes constructions like:

var t nil | int
var u float64 | t

[rogpeppe]

@jimmyfrasche I would assume ReadCloser, same as you'd get from a type-switch on any other interface.

Yes.

@bcmills it's the assigment that's interesting, consider: https://play.golang.org/p/PzmWCYex6R

I don't get this. Why would "this [...] have to be valid for the type switch to print ReadCloser"
Like any interface type, a sum type would store no more than the concrete value of what's in it.

When there are several interface types in a sum, the runtime representation is just an interface value - it's just that we know that the underlying value must implement one or more of the declared possibilities.

That is, when you assign something to a type (I1 | I2) where both I1 and I2 are interface types, it's not possible to tell later whether the value you put into was known to implement I1 or I2 at the time.

[jimmyfrasche]

If you have a type that's io.ReadCloser | io.Reader you can't be sure when you type switch or assert on io.Reader that it's not an io.ReadCloser unless assignment to a sum type unboxes and reboxes the interface.

[jimmyfrasche]

Going the other way, if you had io.Reader | io.ReadCloser it would either never accept an io.ReadCloser because it goes strictly right-to-left or the implementation would have to search for the "best matching" interface from all interfaces in the sum but that cannot be well defined.

[Splizard]

@meln5674
Go already supports union types through generics and struct-reflection. Including support for marshaling and struct tags (see some of my earlier replies). What practical benefits does a language change have?

[meln5674]

Performance, type-safety, and improved developer experience. "We don't need X because we have reflection" is a bit of a cop-out, don't you think? Go's reflection is certainly powerful, but that doesn't mean it should be the hammer for every nail. The same argument could easily be (and was) applied to generics themselves. As I discussed in the linked proposal, yes, it is currently possible to emulate tagged unions in Go, but all possible mechanisms have downsides which render Go a poor choice for problems that are best solved with them, and it would be possible to add with minimal changes.

[Splizard]

@meln5674
Have a look at this https://go.dev/play/p/Pp06ahQrt5-
(https://pkg.go.dev/github.com/splizard/tagged)

type Float tagged.Union[[8]byte, struct {
	Bits32 tagged.As[Float, float32]
	Bits64 tagged.As[Float, float64]
}]
var FloatWith = tagged.Fields(Float{})

var pi Float
pi = FloatWith.Bits32.New(math.Pi)
pi = FloatWith.Bits64.New(math.Pi)

switch tagged.FieldOf(pi) {
	case FloatWith.Bits32.Field:
		var f32 float32 = FloatWith.Bits32.Get(pi)
		fmt.Println(f32)
	case FloatWith.Bits64.Field:
		var f64 float32 = FloatWith.Bits64.Get(pi)
		fmt.Println(f64)
}

This has good performance, it is type-safe and IMO has a good developer experience. Best of all, unlike any of the proposals here, it is supported today. You can use it. You can customise it. It's only a couple hundred lines. I haven't included reflection support or marshalling in this implementation but it is easy to add (I've done it before). If the unsafe code in this particular implementation makes you nervous, you can implement unions with an underlying interface/any. Depends on your performance use-case.

[rami3l]

@Splizard Thanks for the library!

However, I do want to point out that there is a distinction between having something available as a library API and having it natively supported by the language (or its stdlib). In the former case, a tagged union will be much less likely to appear in public API surfaces.

[meln5674]

While that's an interesting approach, and perhaps this isn't the appropriate place for a code review, I can see a number of things that would steer me away from using this in production code.

First, this isn't type-safe. I suspect that you and I are using that word to mean different things. For example, if you were to mix up the cases on a switch, that would not be a compiler error, but a runtime panic. Compare this to a switch t := x.(type). Similarly, if one incorrectly guesses the largest size of any field and supplies the wrong buffer type parameter, then that is also a runtime panic. If it is your position that it is the programmer's responsibility, not the language's, to catch such errors, I would suggest you investigate a new and upcoming language by the name of FORTRAN77, which aligns with such philosophy. Reading the source code, unless I'm misunderstanding, it appears some methods panic if a field is a struct with a pointer, which would be a complete non-starter for a huge amount of Go I've seen.

Please don't interpret this post as "this is everything wrong with your code, your library is bad, and you should feel bad", that is absolutely not what I'm trying to say, this is quite a clever piece of code, and I imagine it was a useful exercise to write it, but rather, these are the things that are an inevitable consequence of implementing tagged unions using workarounds, because the language lacks appropriate facilities to implement them correctly. The entire purpose of this, my, and other similar proposals is to add those facilities so that they don't need to be workarounds, plus all the benefits of a unified ecosystem that rami3l touched on.

If you genuinely believe that this is the correct approach, then you should propose adding something based on it to the standard library, and if you do, I wish you the best of luck.

[Merovius]

@meln5674 @Splizard I don't think this discussion is particularly helpful. In particular, I'll note that there is a certain amount of snark entering here. Let's just note that 1. there are certain ways to work around the lack of sum/union types in Go, but 2. this issue is about adding a first-party, language-level support for them - which is a feature request that has a place (even though I'm personally not that on board with its usefulness either).

[gophun]

Your proposal introduces a new keyword, union, and is therefore not backwards compatible.

Adding a new keyword does not seem to be completely ruled out, as noted by @rsc. So if we made unions separate with the simple "zero value based on first type" rule:

type foo union { a | b }

Then these

interface { a | b }
interface { a | b ; c | d }

could be short form for:

interface { union{ a | b } }
interface { union{ a | b } ; union{ c | d } }

Within interfaces the order of types in the unions wouldn't matter.

There would be only one source of type options (union), and it would integrate nicely into other places, such as interfaces as demonstrated above.

We could still generalize interfaces as proposed in #57644 later, if we feel that we want to have nilable unions, or we could leave them the way they are today.

[Merovius]

@gophun The main argument against adding a separate union concept like that (regardless of whether or not that requires a new keyword - see #54685 for a design that doesn't require a new keyword) is the conceptual overlap with interfaces. That is, the main reason not to do it, is that we'd then have both unions in general interfaces and unions-as-types, which serve very similar functions, but are different concepts, which is not very orthogonal.

(Also, obligatory note that all of this has been discussed above, at length)

[gophun]

@gophun The main argument against adding a separate union concept like that [...] is the conceptual overlap with interfaces

I had hoped that I demonstrated that there is no conceptual overlap with interfaces, but a conceptual composition.

They are orthogonal concepts, but they can be composed, given these short-hand rules:

interface { union { a | b } } <=> interface { a | b } <=> a | b (in type parameter lists)

[Merovius]

The conceptual overlap is that both represent a list of concrete types. Preventing them from be used interchangeably doesn't prevent that overlap in concept - it's what makes it confusing.

[gophun]

int and interface { int } also both represent a list of concrete types (of size one), but I don't see them as conceptual overlap. In my mind unions are a building block, and everything can come together in interface as a big composing brace.

[gophun]

Ideally, I would prefer just a | b instead of union{ a | b }, as it would drive home the idea that the a | b in interface{ a | b } is just another component of the interface and not a direct feature of interface (as it is currently), but that's probably a parsing ambiguity (although I'm not sure). It would also reduce nicely to a in the single type case (e.g. int).

[gophun]

but that's probably a parsing ambiguity (although I'm not sure)

Can anybody help me find a potential parsing ambiguity? These seem to be possible to distinguish from the bitwise | operator:

type foo a | b		// union type definition

type foo a		// single type case

var x a | b		// variable declaration

interface { a | b }	// component of the interface (not a direct feature of 'interface', unlike today)

struct {
	F a | b		// used as type for struct field
}

foo.(a|b)		// type assertion

func f(x a|b)		// function parameter

func f[T a|b]()		// type parameter

[Merovius]

@gophun I agree that int and interface{ int } do not have a lot of conceptual overlap. I disagree that both represent a list of types. int is a type - a set of values. interface{ int } is a set of types. A bag containing an apple is not the same thing as an apple.

FTR you might well disagree with the judgement or not find the argument of conceptual overlap persuasive. But that is the reason we have not accepted any proposal for a separate union/sum-type construct. And if we accepted using a separate concept, I'd be strongly in favor of making it actual sum types, not union types - if we have to pay the cost of overlap and a new concept anyways, we might as well make it the better one.

As for the ambiguity, I don't believe there is an ambiguity per se, but that syntax might require more lookahead than Go has traditionally been comfortable with.

[gophun]

@Merovius
In the place where interfaces with type lists are accepted today, which is type parameter constraints, [T interface{int}] and [T int] are identical. There is no difference between bag and apple here, both represent a type set.

[Merovius]

Fair enough. I still don't think that invalidates the argument that having to separate ways to express a union of types is a downside of that idea. Note that union-elements where not introduced to be able to write interface{ int }, but to write interface{ a | b }. That there is some overlap in a corner-case does not mean interface{ a | b } does not carry its own weight. It also doesn't remove the overlap between interface{ a | b } and union { a | b }.

Again, it's fine to not find these arguments persuasive. But they are the reason we haven't done this yet and it might be beneficial to try to understand them, instead of trying to argue them away.

[gophun]

It also doesn't remove the overlap between interface{ a | b } and union { a | b }.

As you know, since you have followed my train of thought, I actually prefer the variant without the union keyword. So it would be the overlap between interface{a | b} and a | b, which already exists today as the overlap between [T interface{a | b}] and [T a | b].

[Merovius]

@gophun Okay. I can see that you are not interested in understanding the context of this discussion.

[Merovius]

To recap, from my perspective:

1. Over on proposal: spec: sum types based on general interfaces #57644 you suggested that the zero value of interface{ a | b } should be the zero value of the first type mentioned. I explained that this is an issue, as currently interface{ a | b } and interface{ b | a } are equivalent and because interface{ a | b ; b | a } is valid and has no clear "first type".
2. To that, your response was to suggest a new thing, which you called union, in which the order does matter. Note that the actual syntax and name does not matter here - the point is, you suggested to invent a new thing, in which order matters, which would thus have to be different from general interfaces, to solve this problem.
3. When pointing out that this new thing would have significant overlap with general interfaces and union elements, your response was to say "no it doesn't". And then to pivot towards "I actually mean a | b, which is just a building block in interface{ a | b } - which brings us back to the ordering problem.

The point is, either a) you are suggesting a new thing, which has significant conceptual overlap with union elements in interfaces, or b) you are suggesting to assign a type-meaning to union elements in interfaces, in which case you have the zero-value and/or ordering problem.

This isn't a syntax problem. It's a problem with the concepts.

[gophun]

It's true that my thoughts are evolving, that's what a discussion is good for. My goal is to find a solution that has the desired non-nilable property of a union type, integrates well with the existing language, and that does not have more conceptual overlaps than what already exists.

And then to pivot towards "I actually mean a | b, which is just a building block in interface{ a | b } - which brings us back to the ordering problem.

No, it does not bring us back to that.

a | b would be a standalone type. When used in a variable context like var x a | b the ordering matters (for the zero value). When this type is embedded in an interface declaration like interface { a | b } the ordering does not matter, because the interface is either used in a type parameter context to describe a type set for a constraint where the ordering does not matter, or if #57644 would be adopted at some later point in time, which I do not necessarily propose, but which would be possible, the zero value of interface { a | b } would be nil, just like the zero value of interface { int } would be nil).

[Merovius]

Fair enough, we are talking about inventing a new thing with conceptual overlap.

TBQH it is quite confusing that you are choosing to focus on a syntax that gives the new thing also syntactical overlap with general interfaces. For example, if a|b is a type, I would expect interface{ a | b } to be "the constraint of a type to being a|b, just like interface{ int } is "the constraint of a type to be int". Not something else - and yes, it means something else, because a|b and b|a are different types, but interface{ a|b } and interface{ b|a } are the same constraint. But really, that's a problem with the syntax - syntax can be solved after we have an idea of the semantics.

I really don't believe you can escape the dichotomy I lined out above. Either union types are the same as general interfaces - in which case you can't rely on order to answer the zero value problem - or they are not - in which case there now are two very similar, but distinct concepts in the language. This isn't really a thing to "solve". Instead, if we are to add union (or sum) types to Go, we'll just have to accept the downsides of one of those branches.

[gophun]

if a|b is a type, I would expect interface{ a | b } to be "the constraint of a type to being a|b, just like interface{ int } is "the constraint of a type to be int". Not something else

This is true; I hadn't seen this. So consider the idea scrapped.

[StevenM2002]

Can we just get optionals please, I know it is a special case for enums but no enums is fine can we just get optionals.

[melodyogonna]

What does it take to make a final decision on an issue like this?
Will a decision ever be made or are we doomed to eternal bikeshedding?

[DmitriyMV]

@melodyogonna currently the proposal that has most chances of being adopted is #57644 But there are still unanswered question: what to do with nil?

[alecgargett]

For what it's worth, my own personal favorite alternative to #57644 is @zephyrtronium's #54685. It is a pretty well-written proposal that has managed like few others to fit sum types into Go, in my opinion. It still faces the same basic issues, but if we would want to add some sort of union/sum type and we would not re-use the union-element syntax from general interfaces, it would definitely be what I'd put my weight behind. So it might be useful to contrast with #57644, to see opposite ends of the decision spectrum here.

Would either of these approaches lend themselves to the type of enums used for Option --> None/Some and Result --> Ok/Err in Rust? I get the impression that's what advocates of enums in Go are really after (understandably).