-
Notifications
You must be signed in to change notification settings - Fork 789
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[WIP] Tail recursion warning and attribute #1976
Conversation
Merge branch 'master'
Merge branch 'master'
merge from dev
merge from master
merge from master
merge from master
|
This is a wonderful addition to F#! Writing either We may need a keyword if we are to allow this feature on local functions:
|
Mhm since many many rec functions are local I think the introduction of a new tailrec keyword makes a lot of sense. If we decide on attributes then we should go with "TailRecursionAttribute". This is very similar to the existing RequireQualifiedAccessAttribute". Regarding the error message. I think we should make it really verbose and explain as much as possible. |
Forgot to mention that this is just wonderful. One of the best additions to the compiler in quite some time. Kudos for tackling this. |
There is also the possibility of taking a more radical approach and to always check for tail recursion. This has a couple of advantages.
Tbh I think this should be really considered. @dsyme what do you think? |
@forki suggestion to check tail recursion systematically seems a good idea. |
+1 to check always |
Thanks for everyone I will wait for @dsyme to decide about the details |
I'm in favor of checking always, with one caveat: beginning users will be stymied by this warning, it requires a working knowledge of too many concepts to even understand the gist of what it means, let alone knowing how to fix it. Either we should make it a L4 warning (are levels currently used?) or we should make sure that the warning includes a "ignore me" statement, for instance:
Either way, I would prefer this as an option for experienced programmers. In 9 out of 10 cases it doesn't matter, as recursion won't go too deep anyway and performance isn't critical. If L3 warnings is the default, making it an L4 warning would suffice (though I am uncertain whether or not the levels are actually adhered to, or used, see for instance this uservoice post, which wasn't converted?) Warnings like these should either be leveled (L4, L5), or be allowed to be switched on/off on a more granular level than per-file (but that's OT for this issue, I know). Even better yet: if someone feels like it, this could be switched on/off on the property page of a project (an option "analyze for tail-recursion"). |
Idea of detecting calls in non-tail position definitely sounds useful, however I think that implementation should be different. Reason: currently it is done at the codegen phase which is IDE never does (because there is no point to generate actual binary on every keystroke) - so this error will never be reported in editor which to me significantly decreases its value. |
My (completely made up) statistics suggest the exact opposite. ;-) |
@forki, I came to this idea from Brian on SO suggesting, esp. too beginners or moderately advanced programmers to stay away from even trying to use tail recursion: http://stackoverflow.com/a/1797302/111575. From my own experience: looking at the my largest F# project I see some 50 or so Thinking of which: I also encountered a few mutual recursive members. Would these be detected as tailcall with this fix? |
I absolutely agree with the "prefer higher order functions" part, but that
is actually not really related to this here.
Am 10.12.2016 6:20 nachm. schrieb "Abel Braaksma" <notifications@github.com
…:
@forki <https://github.com/forki>, I came to this idea from Brian on SO
suggesting, esp. too beginners or moderately advanced programmers to stay
away from even trying to use tail recursion: http://stackoverflow.com/a/
1797302/111575.
From my own experience: looking at the my largest F# project I see some 50
or so let rec statements. Only a handful of which would benefit from
this. That is not to say, I also have a few while-statements which I didn't
dare to make recursive for fear of getting it wrong... (shame on me, I
know, kudos to this fix, it can be a life saver!)
Thinking of which: I also encountered a few mutual recursive members.
Would these be detected as tailcall with this fix?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#1976 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AADgNMzK8ZTV7iP5S9sP069xBLQMhSajks5rGt9KgaJpZM4LI8dW>
.
|
My train of thought was: if the tendency is to suggest beginning users to stay away from recursion, and if they don't, then stay away from tailcalls (makes sense, they can be hard, even for experienced programmers), then I think we ought to make sure this warning either doesn't throw them off, helps them, or is only available as an extra option for advanced users. |
stay away from tail-calls is a bad advice. really |
I'm just trying to follow what I believe is consensus here: keep it simple for beginners, but give power tools for advanced users. Saying "stay away" is perhaps too strong, but my point is simply: don't overwhelm beginning users with trying to explain what TCO is by means of warnings. We are trying to get more users, not less. Regardless of your opinion on my ideas of tutoriability of warnings, this warning is a very valuable addition, just don't get my comments the wrong way, I am not arguing against implementing it, or against TCO in general (apologies if it looked that way). |
We already have the |
I'm for always checking as well, without any special keywords or decorations. Recursion is so important in functional programming that one should be able to consider it "safe" by default, so the compiler telling you when it isn't is a good thing. |
I would vote for a |
I'm for |
Is the suggestion here to have let tailrec foo() = ... |
That is one of three proposed versions.
Am 12.12.2016 16:30 schrieb "Phillip Carter" <notifications@github.com>:
… Is the suggestion here to have tailrec be a replacement for rec? e.g.
let tailrec foo() = ...
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#1976 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AADgND3RrUjxX5Z5bS2VWPwoesbgMW2cks5rHWipgaJpZM4LI8dW>
.
|
@forki I can bring back the check in the code gen |
I think that would be good, not sure how others feel about it. |
@@ -1332,3 +1332,4 @@ tcTupleStructMismatch,"One tuple type is a struct tuple, the other is a referenc | |||
3211,DefaultParameterValueNotAppropriateForArgument,"The default value does not have the same type as the argument. The DefaultParameterValue attribute and any Optional attribute will be ignored. Note: 'null' needs to be annotated with the correct type, e.g. 'DefaultParameterValue(null:obj)'." | |||
tcGlobalsSystemTypeNotFound,"The system type '%s' was required but no referenced system DLL contained this type" | |||
3213,typrelMemberHasMultiplePossibleDispatchSlots,"The member '%s' matches multiple overloads of the same method.\nPlease restrict it to one of the following:%s." | |||
3212,chkNotTailRecursive,"The member or function '%s' has the 'TailCallAttribute' attribute, but is not being used in a tail recursive way." |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Maybe "was marked with"? Do we already have similar texts?
Ideally yes. This would be an F# 4.2 feature, and I think we would progress by first having one or two people try out the feature, e.g. try to use apply it to the F# codebase. Having both the intrinsic and the attribute implemented would help. Note
Checking in codegen does make sense, to make sure the optimizer hasn't screwed things around. |
So intrinsics feature should be a separate PR? |
merge from master
I've been thinking -- introducing a Given this is a rather complex piece of analysis with many edge cases to consider, what if we went the route of implementing it within something like VFPT for now? There, it could be enabled optionally while we work on getting the analysis correct; once it's ready, we'd roll the analysis code into the compiler itself and implement the keyword/attribute if it still seems useful. If VFPT isn't a good fit, what about implementing the analysis as a standalone tool (again, for now) on top of FCS -- we could provide it as a NuGet package which would install the tool into a project (so no manually configuration would be needed, and it'd still be easy to iterate quickly). |
Jack,
I think an IDE analyzer rather than language keyword is the way to go.
It could just kick out a warning that
Let rec foo =
Blah …
Blah …
Is not be tail-recursive. The il spec regards the .tail instruction as a hint to the jitand it is up to the implementor of the run-time to implement support or just ignore it.
I know jit-devs who believe and state quite loudly that if we have to use the .tail command to get the right behavior at all it is a jit bug.
So … in short try to think of ways not to bake this into the language, instead make it an IDE analyzer. We may have to figure out ways to make it easy to switch on and off … but that is probably better than baking it in.
Thanks
Kevin
From: Jack Pappas [mailto:notifications@github.com]
Sent: Tuesday, January 17, 2017 11:49 AM
To: Microsoft/visualfsharp <visualfsharp@noreply.github.com>
Cc: Subscribed <subscribed@noreply.github.com>
Subject: Re: [Microsoft/visualfsharp] [WIP] Tail recursion warning and attribute (#1976)
I've been thinking -- introducing a tailrec keyword or [<TailRecursive>] attribute doesn't seem strictly necessary. Unless there's a something I'm missing, it'd only serve to augment the analysis; for example, when the keyword/attribute is applied, the analysis pass might emit an error instead of a warning.
Given this is a rather complex piece of analysis with many edge cases to consider, what if we went the route of implementing it within something like VFPT for now? There, it could be enabled optionally while we work on getting the analysis correct; once it's ready, we'd roll the analysis code into the compiler itself and implement the keyword/attribute if it still seems useful. If VFPT isn't a good fit, what about implementing the analysis as a standalone tool (again, for now) on top of FCS -- we could provide it as a NuGet package which would install the tool into a project (so no manually configuration would be needed, and it'd still be easy to iterate quickly).
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FMicrosoft%2Fvisualfsharp%2Fpull%2F1976%23issuecomment-273278947&data=02%7C01%7CKevin.Ransom%40microsoft.com%7Cf37606d53da44733108508d43f11ed57%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636202793629281664&sdata=KXE5uIlltpBkMgliUZzjVapftHiQ%2FR7uClbGn%2FZw%2Fig%3D&reserved=0>, or mute the thread<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAE76FiPKvnjSxjOYUVplvXyVBkTtxCQMks5rTRs8gaJpZM4LI8dW&data=02%7C01%7CKevin.Ransom%40microsoft.com%7Cf37606d53da44733108508d43f11ed57%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636202793629281664&sdata=y%2B3gPvnMv0aEwdD9dJ1Mwlilah9wQWWkxZF2RR9ybyM%3D&reserved=0>.
|
Analyzer are roslyn based. This feature is something that would useful in the whole ecosystem. We should really think about that and make the compiler smart instead of relying on analyzers. I'm not saying that we have to introduce a keyword or attribute - just that we should use analyzers only as last resort. |
The C# compiler runs analyzers. We should make that work.
From: Steffen Forkmann [mailto:notifications@github.com]
Sent: Tuesday, January 17, 2017 2:48 PM
To: Microsoft/visualfsharp <visualfsharp@noreply.github.com>
Cc: Kevin Ransom <Kevin.Ransom@microsoft.com>; Comment <comment@noreply.github.com>
Subject: Re: [Microsoft/visualfsharp] [WIP] Tail recursion warning and attribute (#1976)
Analyzer are roslyn based. This feature is something that would useful in the whole ecosystem. We should really think about that and make the compiler smart instead of relying on analyzers. I'm not saying that we have to introduce a keyword or attribute - just that we should use analyzers only as last resort.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FMicrosoft%2Fvisualfsharp%2Fpull%2F1976%23issuecomment-273325695&data=02%7C01%7CKevin.Ransom%40microsoft.com%7C744dbe59529b4581573c08d43f2af602%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636202901122980774&sdata=xVQGmEdxtaocBGg4%2F4W5Qu28wcBr7qRNYI2t2ExYdgk%3D&reserved=0>, or mute the thread<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAE76FloGvHUHgLu1_4EcW43uHBni1aHsks5rTUU9gaJpZM4LI8dW&data=02%7C01%7CKevin.Ransom%40microsoft.com%7C744dbe59529b4581573c08d43f2af602%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636202901122980774&sdata=TmslPm%2Bc%2BzjoKRe5m%2F2t%2B3Fh0j7SrKcGUI8ovgrS6UU%3D&reserved=0>.
|
I can already run as many analyzers as I want during my FAKE build. That's not the point. |
Btw it's nice that people say it's a jit bug if you have to emit tail calls. Are these bugs fixed? |
Even if the jitter could detect tail calls (without explicit tail call), then we still have to write it tail recursive. And the whole idea of this feature is to detect this somehow in the compiler. |
@forki I believe they would consider fixing the Jit to improve code generation where they believe tail recursion should occur and it isn't. Provide a repro of where the jit does the wrong thing and they will improve it. |
There seems to be a belief going around that F# function calls are normally tail calls and we can just warn if they aren't. This isn't the case - a huge number of F# calls (the majority of syntactic call sites) are not tail recursive - even within the recursive scope of the function or member - even more so when larger recursive scopes are being used in object-oriented programming or So some kind of attribute is needed to indicate that tail recursion is intended. Otherwise we end up flagging almost everything. |
Everyone - his is in the context of the approved language RFC FS-1011. We should probably put design feedback there. @KevinRansom I'm not opposed to any particular analyzer of course, likewise I wouldn't be opposed to an FSharpLint rule. (FSharpLint rules are active in Ionide by defaut). Writing a code fix to make something tail recursive would be hard, as there are quite a few ways to do it (see Expert F# 4.0 chapter ~15) |
The thing is even with analyzer we need to commit that decision with the
code or we will flag everything.
Am 18.01.2017 15:55 schrieb "Don Syme" <notifications@github.com>:
… Everyone - his is in the context of the approved language RFC FS-1011
<https://github.com/fsharp/fslang-design/blob/master/RFCs/FS-1011-warn-on-recursive-without-tail-call.md>.
We should probably put design feedback there.
@KevinRansom <https://github.com/KevinRansom> I'm not opposed to any
particular analyzer of course, likewise I wouldn't be opposed to an
FSharpLint rule. (FSharpLint rules are active in Ionide by defaut).
Writing a code fix to make something tail recursive would be hard, as
there are quite a few ways to do it (see Expert F# 4.0 chapter ~15)
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#1976 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AADgNJBK2O315l8U73g7WgDci4-5c99cks5rTifNgaJpZM4LI8dW>
.
|
@dsyme Agreed, we should move discussion about the design for this feature over to fsharp/fslang-design#82. |
@dsyme I also realized after your comment today that we may be talking about two different things -- tail calls in general (which don't necessarily imply any recursion) and functions designed to be (mutually-)recursive (where any call sites to functions in that set of mutually-recursive functions should be tail-calls to avoid overflowing the stack). I think there's an easy way to get some low-hanging fruit on the mutually-recursive functions, and I've outlined the analysis algorithm in fsharp/fslang-design#82. |
Thanks for this PR and lively discussion but it doesn't seem that we are anywhere close to having the necessary investigations and designs complete yet for a tail recursion feature. I'm going to close this PR for now, the discussion needs to occur on user voice. When consensus or an approach has been determined, then please feel free to resubmit this or an alternate PR. Thank you, Kevin |
@KevinRansom please reopen. This is one of the most interesting pull requests in recent history |
The language design is This interesting thread is not being captured at the right spot. I will add a link from : back to this thread. Please consider continuing the discussion at the RFC. |
@KevinRansom I think we could leave this open - any efforts to make progress on this particular issue are really valuable, even half-way. It's a really tricky feature to get right, and the thing we need most is a series of prototypes for people to use. A WIP PR is one way to get that (and get eyes on it)? |
Can the FSSF help with triggering this discussion again @dsyme? |
Start working on fsharp/fslang-suggestions#147