-
Notifications
You must be signed in to change notification settings - Fork 17.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
proposal: Go 2: returnFrom() #54361
Comments
It's quite common in Go to have anonymous function literals. f := func() { returnFrom(f) } What happens if the label is ambiguous, due to a recursive call? Do we return from the innermost instance? That seems potentially quite confusing. In general the notion of labelling a function call by naming the function being called seems confusing and non-orthogonal. What about variables of function type? Recall that Go explicitly does not permit using |
Thanks for taking the time to read through this! I don't think For recursive functions I think the inner-most call would be the most obvious choice (it seems possible to further extend
Definitely agree that naming a function call can be confusing (it looks like you are passing a function, but you aren't); but requiring every function call to be labelled seemed not great either... |
From reading the proposal I understand this as more-or-less some syntactic sugar over Do you anticipate that every function call would generate code to approximate the effect of calling The details of how
This suggests that there are two fields of this type: some sort of identifier for the function, and some return values. Your definition of I'm also curious about the subsequent arguments representing return values. If the compiler knows from the first argument exactly which function we're returning from then I suppose it can statically check that there's the appropriate number of additional arguments and that they are assignable to the function's return types, in which case we wouldn't need to consider the situation where the types mismatch at runtime. In the implementation details, is there a field inside Stepping away from the implementation and into the programmer-visible effect, it seems like a key consideration of this proposal is that now any function call can potentially be an early return. That is admittedly already true in the presence of I have some difficult-to-substantiate concerns about that situation. I personally currently use If you haven't seen it already, it might interest you that there's an active discussion about defining a standard iterator interface that might integrate with |
No, I expect only calls that are statically determined to need it will get it (because we're re-using panic/recover this is determinable by looking lexically locally; there's no need for this information to move between compilation units).
Either way; one of the reasons the prototype doesn't work well with generics right now is exactly this ambiguity. As a user of the feature, I'd prefer to use
In the prototype version this is a hidden type per call-site; I imagine it depends what's easiest to compile, but I agree there will be no runtime-type mismatches as the compiler will check.
Yeah; there's some discussion related to this on making errgroup "panic-safe". In general one thing that go is not good at is expectations around panics() (other than that they might crash your program). In the kind of code I write, I think it's important that even unhandled programmer errors do not terminate programs (one web request shouldn't be able to kill others in flight for example), so I am careful that panics are handled appropriately and error logging is tied to the request that was made. This is not uniformly how the go community works (I've heard people express the ideal that panics should always cause programs to exit), so there's some tension there. It gets particularly gnarly when using go-routines, but we have internal libraries that let you start a new goroutine and ensures that panics are propagated correctly.
Thanks for the pointer! I'll wade in in a bit – in general (as described in this proposal) I find the idea of an iterator interface described there to be a bit backwards. It may let you make the syntax on the consumer side shorter in the common case, but it puts a lot of burden on the implementor to do things right; I think the iterator proposal from #47707 makes a lot more sense, and if we had |
It seems very difficult to make this fully orthogonal. Consider func F() func() {
return func() { returnFrom(F) }
}
func G() {
f := F()
f() // what happens here?
} |
As written right now it would panic at runtime: "panic: call to returnFrom(F) outside of F" (or similar). I'd expect Similarly to a send/recv on a closed channel; trying to return from a function that has already returned is a programmer error. |
OK, but what we aim for in Go is that, as much as possible, language features simply work. That's why, for example, it's fine (today) to return a function closure out of a function. Consider also code like this: var C1 = make(chan func())
var C2 = make(chan bool)
func F() {
go G()
C1 <- func() { returnFrom(F) }
<-C2
}
func G() {
f := <-C1
f()
C2 <- true
} What happens here? Presumably |
Thank you for continuing to explore the ideas here with me! Your example would also panic with "panic: call to returnFrom(F) outside of F". G is running on a different goroutine, so when I think this is "as much as possible, the feature just works" – it doesn't make sense to allow a function call to return more than once. There's definitely an editorial question of whether the benefits brought by the feature make up for the confusion – but that's why this is a proposal not a pull request after all :D. As mentioned in the proposal I think the benefits are surprisingly far reaching for a comparatively small language change:
One example from nested in the proposal that is related to the example you just gave, it is possible (with the proposal as written currently) to do some amount of go-routine work, for example (if contrived):
|
I don't understand how to implement that in the general case. |
Sorry about that, it took a while to think through, and I'm not sure I'm explaining it clearly. In order for returnFrom to work you need a way to identify a particular stack-frame. There are two cases to consider:
In both cases the stack-frame we need to identify is the stack-frame created by the call to The simplest approach to identifying stack frames is a global counter, but that is not ideal because you have to make all access to that counter atomic. A slightly more lock friendly version (proposed above) would be to use a counter attached to the current go-routine, as a go-routine can (presumably) not race with itself no locks are needed. Using syntax rewriting as a way to try and explain this, if you start with a function that has an early return inside it:
You end up compiling something that behaves like:
Handling the second case is a little trickier to demonstrate as it can't be expressed easily using the current semantics of defer; but you can use the same mechanism to identify the correct frame, but then resume execution at the right point instead of returning. In the prototype I resolved the problem by adding an intermediate function and then using the first case solution on that, so I'm convinced that it is possible to implement it (and also convinced that this is not the most efficient way!)
In reality I don't think an implementation in the runtime would need to add extra functions, a better solution would be to customize how stack unwinding works so you're not restricted by the current semantics of defer. The other subtle point to reinforce is that it is always possible to statically identify which frames may targeted by I hope that answers the question, and please let me know if I can clarify further. |
OK, thanks. I still don't see this as being particularly orthogonal, but I believe that it could be implemented. Another issue that comes to mind is code like func F() int {
fns := []func() {
func() { returnFrom(fns[0], 0) },
func() { returnFrom(fns[1], 1) },
}
fns[0], fns[1] = fns[1], fns[0]
fns[0]()
return 2
} I want to be clear that I'm poking at the concept, but personally I think it's very unlikely that we will add this to the language. I think there are too many oddities about it, and it doesn't seem to add any functionality that is not already available via |
Thanks! Your example here would be a compile error because It'd be hard to be orthogonal I think. The idea was to provide an ergonomic way to break out of user-defined iterators: as today, you can either use a boolean return value as in #47707, or do some messy panic/recover stuff yourself, or (as happens today) you can just discourage custom iteration. I'm glad you're looking at this (regardless of the ultimate decision) because I think passing a callback is a significantly better approach to solving the problem you're tackling in the iterator discussion. As I mentioned there: allowing the author of the iteration to own the control flow makes it much easier to create custom iterators (implementors can themselves use I do see from your feedback that the approach of labeling a function call site by passing the function name seems to be a non-starter! If we still did want something to eliminate the boolean return value, one (maybe less confusing?) version would be to use a different syntax for this:
When I wrote the original version of the proposal though, that seemed to make the semantics much less clear; so I doubt that this is an overall improvement in direction. |
Based on the discussion above, and the emoji voting, this seems like a likely decline. Leaving open for four weeks for final comments. -- for @golang/proposal-review |
No further comments. |
Author background
Would you consider yourself a novice, intermediate, or experienced Go programmer?
Experienced
What other languages do you have experience with?
Significant amounts of Ruby, Javascript/Typescript and Python. Small amounts of many others.
Related proposals
Has this idea, or one like it, been proposed before?
Yes, #35093 is the clearest other version of this (and a few other linked issues are also similar)
If so, how does this proposal differ?
It limits the change to lexically visible places, and shows how this might interact with other Go features in more detail.
Does this affect error handling?
Somewhat
If so, how does this differ from previous error handling proposals?
It is similar to #35093. The primary focus is a new control flow mechanism. While it is not an error-handling proposal per-se, it would help with error handling in currently hard-to-support contexts.
Is this about generics?
Not as such, but generic code would benefit from this change particularly.
Proposal
What is the proposed change?
Go should add a new keyword/builtin
returnFrom(fn, ...retArgs)
that lets you return early from a surrounding function or function call.Who does this proposal help, and why?
The primary case it would be helpful is for iterating over custom collections.
Iteration in go is today handled using a
for
loop that looks something like this:It is currently necessary in go to use a for loop to iterate if you know in advance that you may not want to iterate over the entire collection; because
break
andreturn
let you end iteration early.This has two problems:
for
loops. As each call toNext()
happens on a different stack, and implementors must manually track and update progress.i.Close()
).Both of these problems could be solved by allowing the implementor of the iteration to own the control flow. That would mean that the lifetime of the iterator is tied to a function, and so implementors could use
for
loops, anddefer
statements to clean up as necessary. (This was proposed in the context of extending for range loops to custom types at #47707). WithreturnFrom()
this proposal is easier to implement, and doesn't suffer the "forget to check the return value" problem.Two examples of this API:
With this support in the language, the implementor of
collection.Range()
can defer the cleanuprequired itself; and if the collection used something like a slice under the hood, the implementation of the iterator would be very simple (something like this):
This would also pave the way for generic helper functions like
slices.Map
that are common in otherlanguages (particularly those that use exceptions for error related control flow), but which are hard to implement in go because there is currently no idiomatic way to end iteration early if something goes wrong.
That said, there are a few other cases it would also help with. You could use it to clarify returning from the parent function in a deferred function.
There are also a few cases where a nested function is used outside of the context of iteration. One example is
errgroup
(though until #53757 is fixed, this will not work).In terms of prior art, a similar keyword return-from exists in lisp. The problem is solved in Ruby by differentiating between methods and blocks (
return
returns from the enclosing method even when called in a block; andbreak
returns control to the next statement in the enclosing method when called in a block). The distinction between methods and blocks seems unnecessarily subtle for a language like go. In Python this problem was solved by adding generators to the language, but that seems very heavy-handed for a language like go.Please describe as precisely as possible the change to the language.
A new builtin would be added
returnFrom(label, ...retArgs)
. The first argumentlabel
identifies a funtion call, that is either the call to the function named
label
that lexically encloses the function literal containingreturnFrom
, or a call to a function namedlabel
that contains the function literal in its argument list.A new type would be added to the runtime package,
type EarlyReturn { }
which contains unexported fields; and which implements theerror
interface.When
returnFrom
is called, it creates a new instance ofruntime.EarlyReturn
that identifies which function call to return from, and with which arguments; and then callspanic
with that instance.When
panic
is called with an instance ofruntime.EarlyReturn
it unwinds the stack, calling deferred functions as normal until it reaches the identified function call. At that point, stack unwinding is aborted and execution is resumed as if the called function had executedreturn ...retArgs
.If the identified function call is not found on the stack, then the stack is unwound and the program terminates as it would with an unhandled panic. This would happen if the closure in which
returnFrom
is called was returned from the function it was defined in, or run on a different goroutine. The error message would be:panic: call to returnFrom(<label>) outside of <label>
.If
recover
is called while unwinding in this way, then the instance ofruntime.EarlyReturn
is returned. If thedeferred
function calls does notpanic
then execution is resumed as it is when recovering from a panic; and thereturnFrom
behavior is abandoned. Similarly if a deferred function panics with a different value, then thereturnFrom
is abandoned and the panic happens as normal. If the instance ofruntime.EarlyReturn
is later passed topanic
, then it works as thoughreturnFrom
had been called.If
returnFrom
is called in a deferred function while the current goroutine is panicking (either with an un-recovered panic, or anotherreturnFrom
) it does not do an early return, but instead continues panicking as though you'd runpanic(recover())
.What would change in the language spec?
A new section under
Builtins
Return from
The built in function
returnFrom(label, ...args)
takes the name of a function as the first argument and the remaining arguments correspond to the return values of that function.returnFrom
can be called in two places: within a nested function literal defined inside a function namedlabel
, or in a function literal in the argument list of a call to a function namedlabel
.When
returnFrom(label, …)
is called, it creates an instance ofruntime.EarlyReturn
that identifies which function call to return from and with what values, and then callspanic()
with the instance ofruntime.EarlyReturn
.When
panic()
is called with an instance ofruntime.EarlyReturn
it runs all deferred functions on the current stack that were added after the function call, and then acts as thoughreturn …args
was run infn
immediately returning to the caller offn
. If any of those deferred functions callsrecover()
then the panic is aborted, and execution resumes as it normally would. IfreturnFrom
is called in a deferred function while the current goroutine is panicking, it instead resumes stack unwinding with the current panic.Please also describe the change informally, as in a class teaching Go.
When using loops you can use
break
orreturn
to exit early. When using nested functions usereturnFrom
instead.If you are implementing a function that accepts a callback, then you must be aware that it could
panic()
orreturnFrom()
. As such, any cleanup that must happen after you run the callback should be scheduled using adefer
statement.Is this change backward compatible?
Yes
Orthogonality: how does this change interact or overlap with existing features?
This feature iteracts heavily with
panic()
andrecover()
, and introduces a new case for functions that defer a callrecover()
across a user-provided callback to consider. In most cases this will not be an issue as code should not swallow unexpected panics; though it could exacerbate a problem if this is not the case (e.g. encoding/json before 2017).This feature also interacts heavily with iteration. Although not in scope for this proposal, an obvious extension would be to allow
for x := range collection
loops ifcollection
implements eitherThe compiler would convert break/return statements within the loop to calls to
returnFrom()
as appropriate.Given that, I considered proposing that the syntax was
break fn, ...retArgs
; but I think that is more confusing in the case you're trying to return from the current function, and it is less obvious that this feature interacts withpanic()
andrecover()
.This feature also interacts with
defer
as it introduces a new way for deferred functions to set the return arguments of the enclosing function. Instead of using named arguments, code can now just usereturnFrom(enclosingFn, ...retArgs)
.This feature also interacts with function literals. Currently in go it is not possible to name function literals; and this proposal does not suggest changing that. This means that
returnFrom
will not be able to return from an enclosing function literal directly. I don't think this is likely to be a problem in practice, as if you want to name a function you can move it's definition to the top level.Is the goal of this change a performance improvement?
No.
Costs
Would this change make Go easier or harder to learn, and why?
Harder, it adds surface area to the language. Although the behaviour is easy to describe in the common case, it is still more than not having it.
What is the cost of this proposal? (Every language change has a cost).
The primary cost is an additional powerful keyword that programmers are unlikely to have encountered before. That said, it worked for
go
statements :).How many tools (such as vet, gopls, gofmt, goimports, etc.) would be affected?
All tools would need to learn about the new builtin. As the syntax resembles a function call (albeit one that handles the first argument specially), and the semantics resemble a panic() (control flow is aborted) I think the changes would be minor. Generics may complicate things, as this would be a place you're allowed to "pass" a generic function with no type parameter.
Go vet could be updated to add handling for obviously broken calls to
returnFrom
as discussed below.What is the compile time cost?
The compiler must identify which calls can be returned from early, and ensure that there's a mechanism to do that. This would need two parts: one to identify which stack frame to unwind to, likely by inserting a unique value onto the stack in a known place in the frame; and secondly a mechanism to resume execution at that point; possibly with a code trampoline.
As any modifications are scoped within a single function, this change would not slow down linking, or require changes to functions that do not lexically enclose
returnFrom()
statements.What is the run time cost?
I would imagine that it has similar performance to a
panic
/defer
, as the stack must be unwound explicitly instead of using normal returns.Can you describe a possible implementation?
To identify which call
returnFrom
targets, stack-frames that may be targetted byreturnFrom
will need a unique identifier. I had initially hoped to reuse the frame pointer, but that may not be sufficient as a given frame pointer may be re-used for later calls after the current call has returned. The combination of a frame pointer and a per-goroutine counter value stored at a known offset in the frame should be sufficient.To actually implement
returnFrom
it may be necessary to add a small trampoline, either at the end of the function if the enclosed function is identified; or after the function call if a function call is identified, that copies the values passed toreturnFrom
into the right place. That said, if go's scheduler is able to resume a goroutine at any point, it may be possible to set up the state of the stack and registers and then just jump back into the existing code.Do you have a prototype? (This is not required.)
Yes, I have implemented a version of this by syntax rewriting at https://github.com/ConradIrwin/return-from
Examples
To help understand how this could work:
The text was updated successfully, but these errors were encountered: