Syntax Strawmen

[bterlson]

One big question is what the syntax is. I think there are two primary proposals floating around, both modifications of the return keyword - statements starting with:

• !return
• return continue

(Note that return! doesn't work due to conflicting with existing negation syntax)

One concern with a statement form is that arrow function concise bodies couldn't contain a tail call. That could argue for an expression form.

Another issue is the ternary operator, though my sense is that syntactic opt-in at the statement level works here too - !return x ? Y : Z could only allow calls in tail position in the expressions Y and Z.

[bterlson]

I would hope that any syntax we arrive at will make it a syntax error for opting into tail calls and subsequently writing code that involves calls not in tail position. The exact mechanism for this is yet to be determined but it seems possible and desirable in my opinion.

[littledan]

FWIW I ran a Twitter poll on a few syntaxes, and return continue came out the strong winner: https://twitter.com/littledan/status/716850181409353728 . Most people who responded argued for a contextual keyword (e.g., goto), but that would face the issue that goto (f(x)) would parse as a function call, and break the invariant that you can put parentheses around things that come after return-like constructs. @getify suggested marking the entire function as having all returns be subject to tail call semantics.

[getify]

getify suggested marking the entire function as having all returns be subject to tail call semantics.

First, I don't want any syntax. But operating for the moment on the assumption that it may end up having to happen, let me elaborate: I am imagining some sort of "decoration" on the syntax of a function (and potentially also a generator) declaration that indicates all PTCs from inside that function should be optimized as tail calls. For example:

#function foo() { .. }
#function *bar() { .. }
let x = #function *baz() { .. }
x = {
   #bam() { .. },
   #["qux"]() { .. }
};

I originally liked the #, but I understand there are other syntactic plans for it. Not sure if they'd be incompatible here, but I also think the ^ could work too, if moved to between the function and the lexical name or (.

Others suggested words similar to async, like:

rec function foo() { .. }
tail function foo() { .. }
function rec foo() { .. }
function tail foo() { .. }

I can't really imagine practical use-cases for wanting PTC forms in an => arrow, so I don't think this syntax would need to support that.

[getify]

But syntax bikeshedding aside, my main reasoning for wanting to use a function-level syntax instead of a call-site syntax is the idea that call-site syntax creates a footgun if I annotate one of my PTCs but forget to annotate another intended one. For the record, in my recursive coding, it's quite common that I have two or more PTCs in a function. That's especially true when you consider the various rewritten forms (with inner funcs, etc) that you have to do to turn a non-PTC recursive call into a PTC.

In general, I think there will always be more (and maybe many more) PTCs than function declarations. But if we just went with the assumption of an average of 2 PTCs per function, then I'd really prefer to have half as many places to remember to add explicit syntax.

What if I mark a function as PTC-desired, but there's a function call in there that I don't want to be optimized as a tail call? The "opt-out" is very easy. You just simply do anything that makes it not a PTC. There's several ways to imagine doing that fairly unobtrusively, such as let x; return x = fn();.

My biggest "footgun" concern is thinking about a pattern for a self-adjusting form of recursive coding that I've advocated, which uses try..catch around PTCs. The idea is sort of meta-programming in that you assume a PTC will be optimized so recursion can keep going, and if so, it just runs to completion. But if the stack size limitation is hit, the code simply catches that error then resumes the "loop" to keep going.

Example:

"use strict";

function foo(x) {
    function _foo() {
        if (x > 1) {
            acc = acc + (x / 2);
            x = x - 1;
            return _foo();
        }
    }

    var acc = 1;

    while (x > 1) {
        try {
            _foo();
        }
        catch (err) { }
    }

    return acc;
}

foo( 123456 );          // 3810376848.5

This approach is the best I've conceived for writing recursive code right now during the bridge from non-PTC engines to PTC engines. It allows code to "work" everywhere, but get better and faster as newer engines come about (aka progressive enhancement).

The footgun comes in that if I have to do explicit syntax on each PTC, and I get a few of them but forget one, this adaptive approach will mask that problem. I'd only be able to "catch" the mistake indirectly through profiling. It's much easier to catch (especially with linting rules, for example) that I missed the syntax opt-in on a function declaration than on PTCs sprinkled throughout a function's code.

[littledan]

@getify There is no spec which says that a stack overflow will throw any particular kind of recoverable error, as opposed to crashing the page (as out-of-memory conditions do on many platforms), or allocating all of the memory in the system for the stack which could cause another OOM at any time. Therefore, in a pre-PTC world (e.g., the world of stable browsers), this pattern would not be guaranteed to work per-spec.

This pattern as written also seems dangerous for catching all errors, which could cause issues if an unexpected error is thrown. Further, because the extra driving while loop is needed, the terminating condition needs to be written in two places, the function doesn't have the flexibility to tail-call into other functions reliably (without an additional tracking variable), and acc and x need to be held in mutated variables, rather than parameters of _foo, making the pattern imperative and limited rather than functional and flexible. I don't see the advantage of this idiom over simply putting lines 6-7 in the body of the while loop, even if PTC does ship in some browsers.

[getify]

Would it be possible to accomplish the explicit syntax opt-in via existing valid syntax rather than new syntax? For example, in the vein of how ASM.js took existing (but uncommon) syntactic patterns as annotations of desired optimizations, I'm imagining something like:

function foo(x) {
   if (x < 1) return x;
   { return foo(x-1); }    // <-- explicit block containing only the PTC `return` statement.
}

function foo(x) {
   if (x < 1) return x;
   return foo, foo(x-1);   // <-- comma separated list where first element is the PTC function, second is the PTC call
}

// etc

[littledan]

Let's keep this thread on, what should the sigil be if it's per-return, and discuss function-wide syntaxes in #5 . thanks for filing that, @getify .

That return foo, value syntax seems like it would work, but do users find it intuitive?

[ljharb]

return foo, value is already valid syntax, due to the comma operator - I don't think that could work.

[getify]

return foo, value is already valid syntax

@ljharb my intent is to do exactly that, just like ASM.js did. I'm likening the "optimization" of PTC to the compilation optimizations that ASM did, by inferring that expressions with | 0 are intended to be ints, or whatever.

[ljharb]

aha, interesting - makes sense. wouldn't that run the (tiny) risk of opting a non-tail-call function into PTC? Surely somebody somewhere is using that syntax.

In the case of asm.js, "use asm" is the thing that enables those optimizations, so there's no compat hazard.

[bterlson]

I don't see the value of either of the subset proposals over an explicit syntax personally. The syntax is also highly confusing (can I remove this useless block? Hmm....) and difficult to read.

[getify]

wouldn't that run the (tiny) risk

Yes, but that only means that this function call could run slower than it used to (which btw is already possible... engines regress performance from time to time). If you discovered the performance regression impacting your code, all you'd need to do to opt-out is slightly alter your pattern to avoid the specific narrow PTC pattern.

Whatever it would be, would need to be a narrow enough slice as to be fairly unlikely to hit much code.

[bterlson]

I'm coming to like return continue despite its verbosity. One key reason is arrow functions: an ergonomic usage for arrow functions with concise bodies seems necessary. () => { return continue expr } could be improved by () => continue expr. => still can be thought of as a shorthand for { return which seems nice a nice property. Another option is recur () => expr but I like that less as the tail call sigil is far removed from the expression.

[getify]

I don't see the value of either of the subset proposals

I didn't argue the "why" behind this in this thread, because there's another thread I started to explain why I'm objecting to the new syntax in the first place. TLDR: I think new syntax makes the migration/adoption story for tail call based coding much less friendly. The advantage of using existing syntax is that it's easier to bridge to it (see my several other references to adaptive recursion meta programming).

[littledan]

@getify thanks for splitting off the 'why' into a separate thread; let's leave this to discuss syntax ideas.

We have current draft spec text using return continue. However, in private conversation with @wycats , Yehuda expressed some reservations about reusing the continue keyword in expression context for this totally different feature, partly because of concerns that it may not have very wide appeal/utility, that it would be difficult for users to understand, and difficult to Google/search Stack Overflow for.

One piece of open syntactic space is keyword.property, as pioneered by new.target and championed by @allenwb . As a possible alternative, Yehuda and I discussed options like return.tail and function.tail. These might act like return continue in statement context and continue in expression context. Or, maybe function.tail could be continue, and return.tail could be shorthand for return function.tail. Any thoughts?

On the topic of these options, @erights mentioned to me that it would be not so great if we ended up with something excessively verbose, e.g., return function.tail f(x) might be too verbose.

Whatever the alternative is, in my opinion it would be great if we can preserve the expression (rather than statement) syntax that we have now.

[erights]

Definitely against anything too verbose. Back when I think it was still reserved, I suggested "goto" for this purpose, but too late now.

The difficult of searching for this use of continue is a new argument for me, and the strongest argument I've heard so far. I can't say it changes my mind yet though.

I like the idea of using keyword.tail and then using return.tail expr; as shorthand for return keyword.tail expr;, but only if we use a short enough keyword. I hate function.tail. Looking over the list or keywords and future reserved words at http://www.ecma-international.org/ecma-262/6.0/#sec-keywords , the only decent choices I see are do.tail and with.tail. Although I do not like in.tail, I'll mention that it can be explained: "Compiler, please verify for me that the following expression is indeed in tail position."

[erights]

We could also consider keyword.goto rather than keyword.tail. But it no longer fills me with enthusiasm.

[erights]

After just a few minutes, in.tail expr is growing on me because of the reading "expr, which is in tail position". If STC co-exists with PTC, then it means "Compiler, please verify for me that expr is in tail position" as above. If we have STC only, then it means "Compiler, please consider expr to be in tail position."

[bterlson]

@erights, I also like in.tail. What are your thoughts on the following alternatives in statement context:

return in.tail foo();
return.tail foo();
something else?

[isheludko]

If it's going to be "return.tail" then how will we write a tail call inside complex expression like return a || continue b();?
I also like in.tail.

[bterlson]

With only return.tail, it would be allowed as both an expression and a statement depending on context. With the in.tail proposal, the best thing IMO is to have return.tail work only where return works today (ie, as a statement), and have in.tail for expression-level tail calls.

[littledan]

@bterlson If I understand you correctly, sounds like in.tail would have identical syntax to the current continue, solving @isheludko 's issue, and we'd also add an additional shorthand return.tail => return in.tail for brevity. Right? If so, sounds good to me. I like the parallelism between the two .tail constructs.

[bterlson]

@littledan that is what I'm suggesting, yeah. Wonder if this aligns with what @erights is thinking?

[erights]

@bterlson @littledan @isheludko y

[isheludko]

I like the .tail idea. And the combination return.tail f() / do.tail f() also looks good to me.

[concavelenz]

I want to understand what is being proposed for:

function f() {
g();
}

With "in.tail", am I correct in understanding this would be:

function f() {
in.tail g();
}

I don't know what this looks like with only "return.tail". I wish we could
get away without a "ignore the return value of the function" version so
that the only thing available was:

function f() {
return.tail g();
}

or in the alternate proposal:

function f() {
continue g();
}

I'm not really clear on where the real value is of not returning the result
of the tail called function and not supporting it would reduce the surface
area of the language.

On Mon, May 9, 2016 at 2:36 PM, ishell notifications@github.com wrote:

If it's going to be "return.tail" then how will we write a tail call
inside complex expression like return a || continue b();?
I also like in.tail.

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub
#1 (comment)

[isheludko]

The idea is the following:

function f() {
  return in.tail g();
}
function f() {
  return.tail g();  //  Shorter form of the above.
}

function f() {
  // Use in.tail when you need to tail call from the middle of expression.
  return a || in.tail g();
}

let f = () => in.tail g();

[claudepache]

function f() {
g();
}

The g() is not in tail position (because it will return undefined, not whatever g() returns).

function f() {
in.tail g();
}

Syntax error: in.tail g() is not in tail position (see above).

function f() {
return.tail g();
}

A shortcut for return in.tail g(), previously known as return continue g().

[rossberg]

Okay, I gonna say it: in.tail is terrible. So far, the keyword.property pattern has only been used or proposed for names of pseudo variables, denoting values. That at least made some tiny amount of sense. But as random contextual keywords it's just "Wat?" I doubt that's gonna win over skeptics of the STC proposal. ;)

[littledan]

@rossberg-chromium can you present an alternative?

[rossberg]

@littledan, the current proposal is a perfectly sensible alternative AFAIAC. I don't buy the searchability complaint. Lots of other syntax is far less searchable, so how is that a requirement all of a sudden?

[erights]

used or proposed for names of pseudo variables, denoting values

That's a very good point that had not occurred to me. I agree that it is a good design rule to impose in general for the use of keyword.name.

[wycats]

@littledan, the current proposal is a perfectly sensible alternative AFAIAC. I don't buy the searchability complaint. Lots of other syntax is far less searchable, so how is that a requirement all of a sudden?

The problem is not with new syntax in general, it's with repurposing existing syntax, especially for power-user features that most users won't know about a priori.

What I'd like to optimize for here is:

1. the first time a user sees an STC, it's quickly identifiable as a special form.
2. when they go to search for it, they will find some results

Because return and continue are both keywords, there are already many results for them:

• Stack Overflow
• GitHub
• Google (with verbatim search!)

Because this feature is relatively niche, it also may not come to quickly dominate these searches.

TLDR: I prefer terminology that is clearly identifiable to users the first time they use it, and likely to eventually dominate searches for that term.

[wycats]

I don't buy the searchability complaint.

For what it's worth, I think we'll generally make more progress here by accepting (but asking for details and clarification) the constraints other people have, rather than dismissing them.

The only plausible result of dismissing constraints that other people have (that they feel strongly about) is deadlock. Let's start by assuming we can find a solution within the constraints, and only if we've fully explored the space and can't find any, then we can start pushing on people to defend their strongly-held constraints more aggressively.

[claudepache]

As a precedent, I recall that the static keyword has three completely different uses in PHP; but it didn't hurt the search of its meaning for me (by contrast, I have more difficulty when searching for the semantics of the + binary operator applied to arrays).

Because return and continue are both keywords, there are already many results for them:

There are also already many results for in.tail, from which I wouldn't draw any conclusion. Whatever syntax we choose, the search results for one or for the other will become more and more relevant, as blog posts will be written, documentation will be updated, and Stack Overflow will be filled.

[littledan]

I think "return continue" will become more searchable as the language feature comes into existence and use. Although there are results already, search engines can use heuristics like downranking for intervening punctuation. GitHub search already fails to get good results for my queries which I think are reasonable.

[bobzhang]

@bterlson would you have a look at my syntax proposal in #19, I think it is a very good candidate without inventing a new syntax.

function even(x){
  tailcall: return odd(x-1);
}
function odd(x){
   tailcall: return even(x-1)
}

[rossberg]

@wycats, generators have repurposed *, destructuring has repurposed literal syntax, do-expressions repurpose do, etc. All of these are much harder to search for. Also, for a programmer facing unfamiliar syntax, what difference does it make whether it's new or "repurposed"?

[mintern]

How does return do f() measure up against return continue f() in your opinions?

[littledan]

do is proposed for do-expressions, so I don't know if that would be such a good idea.

[mintern]

Ahh, of course! Thanks.

[Pauan]

@littledan My understanding is that do-expressions must always have {} braces, so using do foo() is unambiguous.

I like it the best out of all the proposals I've seen:

• It is very short, far shorter than continue

  This is especially important for functional languages: because functional languages contain so many tail calls, when they are transpiled to JavaScript they will contain a lot of continue, which bloats the file size. Using do is a lot more acceptable.

• When speaking out loud (or mentally), "do foo" sounds a lot nicer than "continue foo"

• It works in expression position

• It works with function and =>

So my vote goes toward do foo()

[ljharb]

Because of gzip, I don't think the length of the keyword will have any meaningful impact. The other points stand, ofc.

[Pauan]

@ljharb Not all JavaScript code is gzipped: Chrome/Firefox extensions, Node.js, phone apps, embedded devices (IoT), etc.

[ljharb]

@Pauan filesize isn't as much of a concern in those arenas; but either way, I would object strenuously to any proposal that used "smaller filesize" as its argument, as I feel that the existence of that argument weakens every other argument used alongside it. Optimizing for keyword length (for filesize; as opposed to for readability) will just lead to illegible hard-to-maintain code, and I wouldn't want that argument given even implied approval.

[Pauan]

@ljharb Sorry, guess I shouldn't have said anything then.

[bterlson]

@Pauan not true, thank you for taking the time to share your thoughts and keep them coming. It's very helpful for us!

But this proposal is dead and PTC continues to be the consensus position. It's interesting to think about what could have been, though :)

[ljharb]

@Pauan I didn't mean to give you that impression; I like the rest of your points :-)

[iddan]

I see no reason why rec should be used like async: Great languages like OCaml (and now ReasonML) use it and seems very readable

[littledan]

@iddan We discussed alternatives like rec in #5.

[iddan]

I know, I just think that now at 2018 when function*, async function and async function* are part of the language using the same syntax pattern with rec makes more sense.

[joshnuna]

Why not *implement it now and use a directive like "use strict"

That way people that know what tco is and dont want to do a huge refactor to their code to remove all the treampolines when this FINALLY lands can stop waiting for people to flame each other and get on with their lives????

"use tco"

const overNineThousand = (i) => {
  if (i > 9000) {
    return i;
  }
  return overNineThousand(i+1);
}

const nineThousandOne = overNineThousand(0);