Function literal selectors

[eernstg]

[Edit May 13th 2019: Closing this issue; cf. this comment for details.]

In response to #343, this issue is a proposal for function literal selectors, which is a mechanism for passing a function literal as the last actual argument, including abbreviated function literals (#265). This is a tiny syntactic optimization. It can be worthwhile because it could be used frequently (for instance, in code similar to Kotlin type-safe builders).

[Edit May 9th 2019: Note that #342 has a very similar topic. That issue describes a larger number of situations, and this issue contains a concrete syntax proposal which has been tested using Dart.g. May 10th: Resolved disambiguation issue.]

Consider the following tiny Kotlin example of a type-safe builder from here:

// Kotlin example of a type-safe builder.

class HTML {
    fun body() { ... }
}

fun html(init: HTML.() -> Unit): HTML {
    val html = HTML()  // create the receiver object

    html.init()        // pass the receiver object to the lambda
    return html
}

There's nothing special about expressing this in Dart:

// Dart version.

class HTML {
  void body() { ... }
}

HTML html(void Function(HTML) init) {
  final html = HTML();
  init(html);
  return html;
}

However, when we use the builder in Kotlin we get to use the special syntax where a lambda (Dart lingo: function literal) is passed as the last argument to a function invocation without parentheses:

// Kotlin expression using the `html` builder.
html { body() }

In Dart, we have more syntactic noise:

// Dart expression using the `html` builder.
html((x){ x.body(); })

Using a function literal selector we would get this:

// Dart expression using the `html` builder, using this proposal.
html { body(); }

In general, function literal selectors allow for the last parameter to be passed outside the parentheses if it is a function literal, and the parentheses can be omitted entirely in the case where no other parameters are passed (as in the example above). Here's an example with two parameters:

// Expression using current syntax.
foo(42, (x){ print("Received $x!"); })

// Expression using a function literal selector.
foo(42) { print("Received $this!"); }

This difference matters when we consider more complex expressions. For instance, this example would have many occurrences of ({ ... }); in Dart (assuming #265, and we'll have a lot more noise without that). This proposal will allow us to drop those parentheses.

Syntax

This proposal is intended to allow abbreviated function literals as well as regular function literals as selectors. Hence, the grammar changes in #265 are assumed here.

In addition to that, the grammar is modified as follows in order to allow for function literal selectors:

<selector> ::= <assignableSelector>
    | <WHITESPACE> <functionPrimary>
    | <argumentPart>
    | <functionPrimary>

The full grammar is available as Dart.g in this CL.

The Dart grammar has always relied on rule ordering for disambiguation. For function literal selectors this becomes visible, because we need to disambiguate expressions like f<X>(){ g(); }: If we do not handle that issue then it could be parsed as <primary> <argumentPart> <nonEmptyBlock> and it could also be parsed as <primary> <functionPrimary>.

The only change is that <selector> now includes <functionPrimary>. It occurs as an alternative before <argumentPart>, but in that case it requires some whitespace, and it occurs after <argumentPart> with no extra requirements.

This means that a selector after whitespace will be parsed as a <functionPrimary> if possible; otherwise it will be parsed as an <argumentPart>. A selector that doesn't come right after some whitespace will be parsed as an <argumentPart> if possible, otherwise as a <functionPrimary>. In other words, you include that space, as in f <X>() { g(); } rather than f<X>() { g(); }, when you want to indicate that <X>() should be considered as a parameter part for the following function body. And you omit the space (as is common) when it should be considered as an argument part.

Here is an example using the new syntax:

void f0(Function g) => g();

void f1(Function g) => g(42);

class C {
  get h0 => f0;
  get h1 => f1;
}

main() {
  var c = C();

  // Pass an abbreviated function literal as an actual argument.
  f1 {
    c.h1 { print(this); };
  };

  // Pass a regular function literal as an actual argument.
  f0 (){};
  f1 (x){};
  c.h0 (){};
  c.h1 (x){};

  // Disambiguation relies on whitespace before the <argumentPart>; if there is no space
  // then the parse of `()` followed by `{}` as <argumentPart> followed by <selector> is
  // chosen, and then the block can't be empty.
  f0(){}; // 01: syntax error
  f1(x) {}; // 02: syntax error
  c.h0() {}; // 03: syntax error
  c.h1(x){}; // 04: syntax error

  // However, anything that makes the `(...)` fail as an <argumentPart>
  // will force the parse as a <functionPrimary>, with or without the
  // space.
  f1 (var x) {};
  c.h1 ({int x}){};
  f1([int x]) {};
  c.h1({int x = 42}){};
}

Static Analysis and Dynamic Semantics

This feature is a piece of syntactic sugar, which means that the static analysis and dynamic semantics is determined by the desugaring step.

The desugaring step transforms a sequence of terms derived from <selector>, iterating over it from left to right, as follows. Consider a sequence of selectors s₁ s₂ ... s_k, and assume that desugaring has reached index j for some j in 1 .. k.

• If s_j is derived from <argumentPart> and s_j+1 is derived from <nonEmptyBlock> then replace s_j s_j+1 by s1_j, which is obtained from s_j by adding s_j+1 as the last actual argument, adding a comma if needed.

• Otherwise, if s_j is derived from <nonEmptyBlock> then replace s_j by (s_j).

The second case applies when j = k as well as when s_j+1 is not derived from <nonEmptyBlock>.

It is a compile-time error if the desugaring step yields an argument part where a positional argument is passed after a named argument.

[natanfudge]

f1 (x){};
Isn't this ambiguous between "f1 is called with a function literal with a parameter as an argument" and "f1 is called with x as the first argument, and a function literal as the second argument"?
See #342 in Passing a function literal to the last parameter.

[eernstg]

@natanfudge, thanks for pointing out #342, I hadn't seen that! I added an 'Edit:' paragraph to this issue, pointing out that #342 and this issue have very similar topics.

[Edit May 10th: deleted two paragraphs about an ambiguity in the grammar: It has been resolved.]

[eernstg]

Followup on ambiguity: We haven't had any such elements in the Dart syntax before, but it actually comes out quite naturally to disambiguate terms like f(){} in the following manner:

If there is any whitespace immediately before the syntax which could be either an <argumentPart> or a <parameterPart> then the parse as <functionPrimary> will be chosen if possible, otherwise <argumentPart>; when there is no such whitespace the parse as <argumentPart> will be chosen if possible, otherwise <functionPrimary>. I think it's a rather natural choice to put a space (as in f (){}) when we want to associate the parentheses with the following function body, and omit the space (as in f(){...} or f() {...}) when we want to associate the parentheses with the preceding function or selector.

The proposal and the grammar update CL have been updated accordingly.

[natanfudge]

This definitely resolves the ambiguity. However, I think it is mandatory to offer some criticism. It might be quite confusing to have a syntax error resolved with a space - as you said, there aren't such elements in Dart. Additionally a singular space is hard to spot which makes reading difficult in some situations, for example this one:

class GreatWidget {
  final int x;
  void f(int num, Function g){/**/}

  // A lot of code

  void doSomething(){
     // Is x a class variable passed to x or an argument to g? The space isn't immediately obvious. 
     f(x) {
         print(x);
     };
}

So my input is this:

• Perhaps there is a better candidate than a whitespace?
• If this proposal goes through (which is by no means bad), a quick fix/intention to add a space in a case in which an absence of one causes a syntax error is definitely mandatory.

[eernstg]

Perhaps there is a better candidate than a whitespace?

Thinking about the exact same thing! This kind of syntax is a source of ambiguity for the developer as well as the parser, and it's obvious that we need to very careful about comprehensibility.

[ds84182]

Perhaps a placeholder _ type can be added to the language, essentially meaning "no type specified, use inference". Then you can solve the ambiguity by requiring parameter types:

f0 { xyz; }
f1 (_ foo) { print(foo); }

Other than that, I don't see a different way to disambiguate it (as of right now, without using completely different syntax).

As an aside, an _ inferred type can be useful in other situations where you need to provide only some type arguments, like for example when using dart:ffi:

void Function() lookupFoo(DynamicLibrary lib) {
  return lib.lookup<Void Function()>("foo").asFunction(); // [What we use today]
  return lib.lookupFunction<Void Function(), _>("foo"); // [lookupFunction is available today, but we can't omit the second parameter type]
}

[eernstg]

a placeholder _ type can be added

Interesting idea! We'd then need a complete set of grammar rules deriving a <parameterPart> that has no overlap with <argumentPart>, but that's probably not so hard to do.

But we still have to be careful. @lrhn mentioned the following fact:

main() {
  foo(b) {
    print(b);
  };
  ..
}

If we omit the semicolon after } that's a declaration of a local function taking an argument b of type dynamic. With the semicolon it will call some global foo with two arguments. If one of these is the intention and we get the other one by accident then we will probably detect the problem, but it may well yield some very confusing error messages.

Case 1: We intended to declare a local function, but added the semicolon by accident: Then maybe there is no foo in scope, or there is a foo but it doesn't accept the argument list (b, { print(b); }). This is probably OK, because it's quite unlikely that it will pass the type checker.

Case 2: We intended to perform the invocation, but forgot the semicolon: Then we just get a local function foo that (quite likely) nobody ever calls, and the body might not yield any errors because it's working on a formal parameter b of type dynamic. Of course, we do emit a hint if foo is unused.

On top of these issues, note that foo(_ b) {...} is also a local function declaration, so we still have the problem that a missing semicolon will (all too easily and silently) turn a function call into a function declaration, also with the "please infer this type" feature.

Checking: The keyword fun ensures that such a term won't be a function declaration in Kotlin, so they don't have that problem.

[eernstg]

Having explored this idea a little bit, I've reached the conclusion that function literal selectors will not fit very well into Dart.

Syntax like f(b) { ... } may be tempting because it allows for custom control structures to be syntactically similar to built-in control structures (like if (b) { ... }), but the fact that it is also extremely similar to existing Dart constructs (in particular, local function declarations) makes it unlikely to work well in actual software.

Based on that conclusion I'll close this issue.

[natanfudge]

That is extremely disappointing. I was hoping Dart was heading towards a future where

SharedPreferences.getInstance().then((preferences){
    setState(() {
        _x = preferences.getBool("x");
      });
 });

becomes

SharedPreferences.getInstance().then (preferences){
      setState {
        _x = preferences.getBool("x")
      }
}

Which is so much more pleasant to read.
I hope you (and the dart team and general) will rethink this and be more supple with the language so this could be fit in.

[eernstg]

@natanfudge wrote:

That is extremely disappointing

Right, I want a concise syntax as well. But any syntax that makes the expression myIf(b) { ... } desugar into myIf(b, { ... }) is simply too much of a clash with local function declarations.

However, that's not the only option available. For instance, the proposal for abbreviated function literals (#265) allows you to express the example more concisely. It relies on the context types to determine that the outer function literal takes one argument (whose name is then by definition it, and whose type is found by inference), and that the inner function literal takes zero arguments:

SharedPreferences.getInstance().then(=> setState(=> _x = it.getBool("x")));

If the function literals contain a larger amount of computations then it might work well to use anonymous methods (#260). An anonymous method is applied to a receiver (which is the term before the period(s)), so we'd want to compute that instance first, and then set the state (I'm assuming that then is Future.then, that is, that getInstance returns a future):

await SharedPreferences.getInstance()..{
  setState(=> _x = getBool("x"));
  // Could have more statements here, this is just a function body.
};

The invocation of getBool("x") is resolved to be an invocation on the receiver of the anonymous method, that is, the object obtained from await SharedPreferences.getInstance(). I suspect that anonymous methods will actually be able to handle a lot of the cases that you have in mind.