proposal: spec: multi-type type parameters #56462
Comments
seankhliao
added
LanguageChange
|
Is this meaningfully different from #41148 ? |
|
As far as channels specifically go, no I don't think so, but I think the generic usage is the most important part of this one. I was primarily inspired by #56102. I would be fine, given #41148, with removing everything except for the generic-specific part of this proposal. Edit: I updated the original post to explain that the parts of this proposal that are not related to generics should be ignored. |
DeedleFake
changed the title
|
If I understand correctly, this is called |
|
Yes, I believe so. See https://en.wikipedia.org/wiki/Variadic_template. The Go draft proposal for generics also lists
At least to me, I think that the time that Go has now had generics for has shown that the way that the language handles multiple returns, and their usage in the error handling model in particular, makes the addition of variadic generics quite attractive. |
|
I think it could be pretty confusing to have a construct like We also need to specify exactly what is permitted with |
|
I've been thinking about what functionality would be required at a minimum for
Also, if a In terms of access to struct fields, I don't think it would ever be an issue outside of For Edit: Let me try to explain what I'm saying above about struct fields more clearly. Given type Example[T ...] struct {
v T
}the following would be allowed func Something[T ...](e E[T]) {
a := e.v // a would be of type T.
// ...
}but the following would not func Something(e E[(int, string)]) {
i, str := e.v // Can't destructure from a variable. Has to be multiple returns from a function or method.
// ...
}Instead, you'd have to use a wrapper function func Get[T ...](e E[T]) T {
return e.v // Legal because it's just an assignment from T to T with no destructuring.
}This is why I think it should be limited to unexported fields only. It wouldn't actually be a problem in terms of usage by other packages, but it could be confusing that in some unlikely situations, you wouldn't actually be able to access the values of the field directly and would instead have to write a wrapper function to decompose it. I think that avoiding the confusion outweighs the oddity of disallowing exporting of variadic generic fields. |
It never would, though. In all cases outside of function arguments are returns, and possible type Optional[T ...] struct {
v T // Looks like a single, normal struct field.
ok bool
}
func NewOptional[T ...](v T, ok bool) Optional[T] {
return Optional[T]{
v: v, // Acts like a single variable here.
ok: ok,
}
}
func (opt Optional[T]) Or(v T) T {
if opt.ok {
return opt.v // Again, acts like a single variable.
}
return v
}But then in usage, it could do something like this: opt := NewOptional[(int, string)](3, "example", true) // At this end, it acts exactly like the T-typed stuff is more than one argument.
i, str := opt.Or(1, "something") // And the same here in the returns.So inside of the definition, stuff of 'type' Edit: Another example, based on #56102: package sync
// Adding in arguments just for fun.
func Lazy[A ...,R ...](f func(A) R) func(A) R {
var once Once
var r R
return func(args A) R {
once.Do(func() { r = f(args) })
return r
}
} |
|
I can think of a number of functions of general forms like |
|
The func Thunk[I various, O various](func(I) O) func(I) O
func FoldMap[I various, A various](func(I) A, ...func(A) A) AThere's nothing particularly strange about these functions signatures. They're not long or particularly complex. |
|
Just to check, would this permit writing a single |
|
In the current design, yes, but only the variant mentioned that uses a comparison function, not the one with the type Metric[T ...] struct {
mu sync.Mutex
eq func(T, T) bool
// ...
}
func NewMetric[T ...](eq func(T, T) bool) *Metric[T] {
return &Metric[T]{
eq: eq,
// ...
}
}
func (m *Metric[T]) Add(vals T) {
m.mu.Lock()
defer m.mu.Unlock()
// Do whatever is necessary to add the data, calling m.eq to compare.
}This does have some weirdness, though, including the fact that comparison function will wind up with two arguments for each type in It might be possible to allow constraints on variadics, but it would be quite limited. Essentially, it comes down to whether or not it makes sense to define operations on variadics. For example, comparability could be defined much like it is for a struct: func Example[T ...comparable]() {
var v1, v2 T
v1 == v2 // This just makes sense.
}In this case, a more broad variadic would be written as func Add[T ...constraints.Integer](v1, v2 T) T {
return v1 + v2
}
func main() {
i1, i2 := Add[(int, float64)](1, 2, 3, 4)
}makes sense, but I'm not sure if it makes sense to allow it or not. Edit: The constrained version also provides the benefit of allowing structs to maintain comparability. For example, Edit 2: I'm looking over the metrics example some more and I'm wondering if this is a good fit for variadic generics in the first place. It seems to make more sense to me to just define a Edit 3: I realized that I didn't demonstrate how the metric example would work with constraints. Here's the definition working for any number of types: type Metric[T ...comparable] struct {
mu sync.Mutex
m map[key[T]]int
}
func (m *Metric[T]) Add(vals T) {
m.mu.Lock()
defer m.mu.Unlock()
if m.m == nil {
m.m = make(map[key[T]]int)
}
m.m[v]++
}
// Intermediary type necessary for use as a map key.
type key[T ...comparable] struct {
v T
} |
|
Putting on hold for discussion of future generics changes. |
|
This would be so incredibly helpful for designing more typesafe middleware, so each middleware function could attach data as a new parameter instead of just shoving everything into a In order for it to actually be useful in this context though, you do need the ability to pop/push values from/to the parameter list - preferably from/to either side. E.g.: func PanicOnError[T ...](f func() (T, error)) T {
...t, err := f()
if err != nil {
panic(err)
}
return t
}( I'm not sure you'd need full tuple types, though, since as far as I can tell the only "variable-like" operations needed are:
which is a far cry from what was proposed in e.g. #64457, and are all things that should be possible to support without a dedicated tuple type. For example, a very basic solution could be to have this: func A[T ...](f func() (int, T, string)) (float64, T) {
i, ...t, s := f()
fmt.Println(s)
return float64(i) / 2, t
}be expanded to this when instantiating func A(f func() (int, X, Y, Z, string)) (float64, X, Y, Z) {
i, __x, __y, __z, s := f()
fmt.Println(s)
return float64(i) / 2, __x, __y, __z
}and since you can't do anything with the individual values without explicitly naming the types of the values to pop, there's no inference or bounds checking to be done for As a motivating example of what I mentioned at the start, using this you would be able to write a handler like func MyHandler(
r *http.Request,
dbConn *Connection,
user *User,
) (result *Result, err error) {
// ...
}which could be transformed into a standard var Handler http.HandlerFunc = LogError(WriteResult(WithUser(WithDB(MyHandler))))where type GenericHandler[In ..., Out ...] func(In) Out
// ErrorHandler is any function that can return an error.
type ErrorHandler[In ..., Out ...] GenericHandler[In, (Out, error)]
// RequestHandler is any function that takes an *http.Request
// and can return an error.
type RequestHandler[In ..., Out ...] ErrorHandler[(*http.Request, In), Out]
func WithDB[In ..., Out ...](
inner RequestHandler[(*Connection, In), Out],
) RequestHandler[In, Out] {
return func(r *http.Request, params In) (Out, error) {
conn, err := OpenConnection()
if err != nil {
var zero Out
return zero, err
}
defer conn.Close()
return inner(r, conn, params)
}
}
func WithUser[In ..., Out ...](
inner RequestHandler[(*User, In), Out],
) RequestHandler[In, Out] {
return func(r *http.Request, params In) (Out, error) {
user, err := Authenticate(req)
if err != nil {
var zero Out
return zero, err
}
return inner(r, user, params)
}
}
func WriteResult[Result any, In ..., Out ...](
inner RequestHandler[In, (Result, Out)],
) ErrorHandler[(http.ResponseWriter, *http.Request, In), Out] {
return func(w http.ResponseWriter, r *http.Request, params In) (Out, error) {
result, ...out, err := inner(r, params)
if err != nil {
w.WriteHeader(http.StatusInternalServerError)
return out, err
}
err = json.NewEncoder(w).Encode(result)
return out, err
}
}
func LogError[In ..., Out ...](
inner ErrorHandler[In, Out],
) GenericHandler[In, Out] {
return func(params In) Out {
...out, err := inner(params)
if err != nil {
log.WithError(err).Error("error while handling request")
}
return out
}
}allowing for amazing composability without losing any type safety. Right now I think the closest you could really get to something like this is keeping a stack of all pushed data using something like // Bottom represents the bottom of the stack.
type Bottom struct{}
type Stack[ThisType, ParentType any] struct {
Value *ThisType
Parent ParentType
}
func MyHandler(
r *http.Request,
dbConn Stack[Connection, Stack[User, Bottom]],
) (*Result, error) {
user := dbConn.Parent.Value
// ...
}
func WithDB[In any, Out any](
inner func(*http.Request, Wrapper[Connection, In]) (Out, error),
) func(*http.Request, In) (Out, error) {
return func(r *http.Request, params In) (Out, error) {
conn, err := OpenConnection()
if err != nil {
var zero Out
return zero, err
}
defer conn.Close()
wrapped := Stack[Connection, In]{Value: conn, Parent: params}
return inner(r, wrapped)
}
}...which isn't particularly pretty. |
On what basis? Because AFAICT it would be pretty useful to also have a |
Absolutely, I'm sure it would be useful for a lot of different use cases! But I was trying to think of what the minimum requirements would be specifically for the kind of function wrapping that I'd like to see, as in my middleware example. (If it wasn't obvious already, it's heavily inspired by decorators and how I was also trying to keep the requirements as simple as possible to avoid accidentally veering into tuple type territory. With the rules I listed it's relatively straightforward to see how the parameter list could simply be expanded to multiple parameters at function instantiation time (assuming that the Go compiler creates a separate copy for each new combination of generic arguments, which I'm not actually sure if it's the case). I'm not sure
"Pop" would only work if you've explicitly assigned the elements you're popping to a separate, non-variadic type paramater. So this would work: func GetTail[Head any, Tail ...](f() (Head, Tail)) Tail {
_, ...tail := f()
return tail
}in which case calling Meanwhile this would not work: func GetTail[Tail ...](f() Tail) ???? {
_, ...tail := f()
return tail
}Note how if this was allowed there's no way to even specify the return type without some additional special syntax. |
|
Although your note about recursively applying functions with variadic arguments did get me thinking about different edge cases... func A[Head any, Tail ...](head Head, tail Tail) Tail {
return A(tail)
}This will fail because the recursive call will have a different return type, so not worried. func PrintAll[Head any, Tail ...](head Head, tail Tail) {
fmt.Println(head)
PrintAll(tail)
}Scary! This would lead to a call chain like I'm happy for this to not work at this stage since this is starting to veer into pattern matching territory instead, which would be cool to have to be clear, but certainly not within scope of this feature! But how would you present a sensible error to the user? func AddOneArg[T ...](items T) {
AddOneArg(1, items)
}Oops. Now we're suddenly statically generating infinitely many function copies. Well that's no good. Either my idea of statically generating a new copy for each instantiation is a no-go, or we'd have to disallow recursive calls like this. func AddOneReturnValue[T ...](f func() T) func() (int, T) {
return func() {
return 1, f()
}
}
Limiting variadic types to only be used as return values wouldn't be a huge loss as far as my use case is concerned, but I'm sure there are others who would like to see it. For one, it would mean you couldn't write functions like: func RunWithLogs[Args ..., Rets ...](f func(Args) Rets, args Args) Rets {
fmt.Println("Running function")
return f(args)
}
result, err := RunWithLogs(myFunc, 1, 2, "test")though you could still write: func RunWithLogs[Rets ...](f func() Rets) Rets {
fmt.Println("Running function")
return f()
}
result, error := RunWithLogs(func() (int, error) {
return myFunc(1, 2, "test")
})Would be interested to see how other languages with variadic types solve this issue. [*] Edit: OK yeah that's just me not being creative enough: func AddOneReturnValue[T ...](f func() T) {
AddOneReturnValue(func() (int, T) {
return 1, f()
})
}Presumably just forbidding (mutual) recursion in variadically-typed functions would be enough to fix this. Are there any major use cases that would break that would be otherwise supported? |
ISTM you are trying to side-step my question. Why is "the kind of function wrapping you'd like to see" the right notion of "the kind of function wrapping we'd like to support". Why is this the right level of support for this feature?
My question was what would happen if you called it with e.g.
That was, if I understand you correctly, the "edge case" I was talking about.
I don't find anything about this statement obvious. Why do you use "certainly"? Note that the title of this issue is "multi-type type parameters". That seems certainly open enough to encompass recursive generic code. And note, that even without recursivity, you get into issues: func F[T ...](f() T) {
...x := f()
fmt.Printf("%T", x)
}
func main() {
F(f())
}I can't assign meaningful behavior to this program. So it should probably not be allowed? But that seems a pretty severe restriction. For example, one of the main use-cases I would see for this concept is to allow something like this: func Compose[A, B, C ...](f func(context.Context, A) (B, error), g func(context.Context, B) (C, error)) func(context.Context, A) (C, error) {
return func(ctx context.Context, a A) (C, error) {
b, err := f(ctx, a)
if err != nil {
return *new(C), err
}
return g(ctx, b)
}
}It seems unfortunately, to disallow calling this with an empty I think this sort of "working with arbitrary function types" stuff is exactly what this is about and adding a "as long as the function types have some arguments and returns" restriction seems - to me - to require some justification. |
It's literally just my subjective and intuitive interpretation of the intention of this proposal. For instance there are no examples of recursive variadically-typed functions prior to my comment, which I took to mean that it's not something people are primarily concerned with here. Other basic operations (like
Calling func GetTail[Head any, Tail ...](f() (Head, Tail)) Tail {
_, ...tail := f()
return tail
}as func GetTail(f() int) {
_ = f()
return
}which looking at this I now realised might require special handling of
I might have been a bit quick to dismiss it, but it does feel like opening up a huge can of worms, because the only way I can see recursive variadically-typed functions being usable - at least with recursive calls that change the number of type parameters - is allowing multiple function definitions so you can specify a base case: func PrintAll() {}
func PrintAll[Head any, Tail ...](head Head, tail Tail) {
fmt.Println(head)
PrintAll(tail)
}My reasoning is that it would not be possible to compile something like the following: func PrintAll[Head any, Tail ...](head Head, tail Tail) {
fmt.Println(head)
if len(tail) > 0 {
PrintAll(tail)
}
}because when called as (Or at least it would not be possible to compile it unless the And if we allow multiple function definitions for different type parameters we're basically heading into Haskell-style pattern matching territory. Which is a laudable goal to be sure, but not something I'm sure we'd want to tackle here!
For the record, this already works in Go: func GetNumbers() (int, int, int) {
return 1, 2, 3
}
func PrintValues(values ...any) {
fmt.Printf("%T", values...)
}
func main() {
PrintValues(GetNumbers())
}(It prints So I don't think there's a need to disallow your example, even if it's not necessarily useful. If we wanted to make it more useful for the sake of this feature, perhaps a new format verb could be added that repeats itself (separated by spaces) for every unprocessed argument, or something along those lines. But either way the user could just write their own auxiliary function to do this: func F[T ...](f() T) {
...x := f()
PrintTypes(x)
}
func PrintTypes(vs ...any) {
for _, v := range vs {
fmt.Printf("%T\n", v)
}
}
Maybe I'm missing something, but shouldn't this be fine? (Using my ad-hoc func Compose[A, B, C ...](
f func(context.Context, A) (B, error),
g func(context.Context, B) (C, error),
) func(context.Context, A) (C, error) {
return func(ctx context.Context, a A) (C, error) {
...b, err := f(ctx, a)
if err != nil {
return *new(C), err
}
return g(ctx, b)
}
}
func F(ctx context.Context) (string, int, error) {
return "test", 0, nil
}
func G(ctx context.Context, _ string, _ int) error {
return nil
}
var f func(context.Context) error = Compose(F, G)Note that this doesn't break my suggested "pop" rule, because the argument being popped (
func Compose(
f func(context.Context) (string, int, error),
g func(context.Context, string, int) error,
) func(context.Context) error {
return func(ctx context.Context) error {
__b1, __b2, err := f(ctx)
if err != nil {
// special handling needed if new(C)
// should be supported
return *new(string), *new(int), err
}
return g(ctx, __b1, __b2)
}
} |
Yes. Note that the My question was "what is the type of a variable that is derived from a variadic type parameter list?". Making variadic type parameters useful requires us to be able to declare such variables. And probably from lists derived from them by various means (like your "push" and "pop" operations). |
Yes, I didn't mean for the Essentially my suggestion is that a variadic type parameter list doesn't have a type of its own. I.e. if we have: func F[T ...](f() T) {
...x := f()
fmt.Printf("%T", x)
}then we get: // Printf("%T") -> %!T(MISSING)
F(f() {})
// where F[()] is...
func F(f()) {
f()
fmt.Printf("%T")
}
// Printf("%T", 0) -> int
F(f() (int) { return 0})
// where F[(int)] is...
func F(f() int) {
__x1 := f()
fmt.Printf("%T", __x1)
}
// Printf("%T", 0, "test", true) -> int%!(EXTRA string=test, bool=true)
F(f() (int, string, bool) { return 0, "test", true})
// where F[(int, string, bool)] is...
func F(f() (int, string, bool)) {
__x1, __x2, __x3 := f()
fmt.Printf("%T", __x1, __x2, __x3)
} |
The way that I'm picturing it, the type of the variable is func UselessExample[T ...](v T) T {
v2 := v
// v2 is T. The actual types are determined at compile-time and have no impact inside of the function.
return v2
}
r1, r2 := UselessExample(3, "example")
// r1 and r2 are int and string. The pseudo-type T is inaccessible outside of the function.
type UsefulExample[T ...] struct {
v T // Again, is type T.
}
func NewUsefulExample[T ...](vals T) UsefulExample[T] {
return UsefulExample[T]{v: vals} // Treated like a single value inside of the function.
}
func (ex UsefulExample[T]) Get() T {
return ex.v
}
r := usefulExample.Get() // Accessible via method call.Because of that, somethiing like |
|
One thing that would be incredibly cool is if you could do is something like: type Wrapper[T any] struct {
Value T
}
// here the return value would be a variadic type list, i.e.
// WrapAll[(int, string)] has the type:
// (int, string) -> (Wrapper[int], Wrapper[string])
func WrapAll[T ...](values T) Wrapper[T] {
// very jank ad-hoc syntax, but the idea is to be able to
// define a transformation that's run for each individual value
// where the transformation is passed the type of the value
// currently operated on as a type parameter (here U)
return map(values, func[U](value U) Wrapper[U] {
return Wrapper[U]{Value: value}
})
}
// could be compiled to something like:
func WrapAll(__values1 int, __values2 string) (Wrapper[int], Wrapper[string]) {
__map1 := Wrapper[int]{Value: __values1}
__map2 := Wrapper[string]{Value: __values2}
return __map1, __map2
}I doubt this is realistic (and it would definitely need a better syntax proposal), but if it was possible I bet you could create all sorts of very cool, expressive and ergonomic APIs using this. Especially useful for more declarative APIs, like builders and DSLs. Imagine being able to pass in a list of column definitions to a SQL |
|
Sorry, I completely forgot about this one. I wrote a similar but more precise version at #66651. |
ianlancetaylor
added
LanguageChangeReview
seankhliao
changed the title
Update
As pointed out by @seankhliao, the pieces of this involving channels were already covered in #41148. However, I think that the generics-related code is still worth discussing. Therefore, this proposal is now only for the
...constraint detailed below. By limiting the proposal to this constraint, that also partially fixes thereflectissue, aswould be illegal. A variable of a type constrained by
..., such asT ..., would not be allowed to be assigned to anything not also of that exact typeT, meaning that even passing it to something that takes anany, such asreflect.ValueOf(), would be illegal.However, this doesn't fix it completely. In order for this proposal to be useful, things like
type Future[T ...] struct { val T }would have to be allowed, which would mean thatreflectcould still gain access to the 'variable' via the struct. I'm unsure of how to handle this, though I have a few ideas. One of them is touched on briefly in the original proposal below.Original Proposal
Author background
Related proposals
Proposal
In Go, functions can return multiple values. This is handled directly, rather than via tuples, the way that a lot of other languages, such as Python and Rust. handle it. Because of this, some expressions in which it would be useful are not capable of yielding multiple 'returns' directly. For example, a user can declare a
chan int, but not achan (int, error). Due to the convention of returningerrorfrom functions directly as their own value, this can lead to some complication when trying to handle errors in other cases, such as when sent from a channel. The common solutions are to either use two channels, one for values and one for errors, or to define a top-level type that can hold the value for you. The first solution is better suited to specific situations, while the second solution mostly solves the problem, but can lead to namespace pollution and other annoyances, especially when the channel operation is happening across package boundaries.This proposal is not a proposal for general tuples. Instead, this proposal is to extend the usage of multiple 'returns' to other places in which types are specified. These 'tuples' would not be possible to store in a single variable except for in one specific situation, but would instead require that they be destructured immediately, just like how multiple returns from a function works. Unfortunately, this can lead to some ambiguities, so these would have to introduce a small new syntax change to fix, but it would only apply in those caes.
The biggest complication that I can think of with these changes is how to handle them with
reflect. I think the simplest approach is to introduce the idea directly intoreflect.Typewith some new methods that would allow you to pull out the types one by one, much like how it currently works for multiple returns to a function. This is potentially a backwards-compatibility issue, as existing code that didn't expect, for example, the element type of a channel to itself have multiple types would break when then given a channel that does have multiple types.It would be quite useful for a number of recent proposals. For example, in #56102 the current thinking is to add both
Lazy[T any]andLazy2[T1, T2 any]just to allow people to use either functions that do or functions that do not return an error. This has happened in a number of other places as well. This proposal would allow that to be replacedLazy[T ...]instead to handle not only those cases but any number of returns from the wrapped function. #56461 is another that would benefit.reflectissue detailed above.Costs
goplsandgofmtwould definitely be affected because they would need to support the new syntax, but it should be relatively minimal.The text was updated successfully, but these errors were encountered: