Forwarding function expressions and declarations

[eernstg]

The ability to concisely specify forwarding functions has been requested several times (e.g., #58, #59, #157, #418; and #493 requests a similar feature for constructors).

The main point is that it is verbose and error-prone to write a forwarding function, because it includes a full declaration of a parameter part (value parameters, with types, names, optionality, plus possibly type parameters, possibly with bounds) which is often very similar to the parameter part of that other function which is the forwardee. For example:

class A {
  void f(int i, int j, [int k = 0]) {}
}

class C {
  final A a;
  C(this.a);
  void g(int i, int j, int k, {bool isFoo = false}) => a.f(i, j, k); // Forward to `a.f`.
}

When the function types are identical it seems clearly suboptimal to have to write the signature again, and pass a suitable list of parameters as actual arguments. However, even with somewhat different function types, even in some cases where those function types are not subtypes of each other (in any direction, like in the example above), it is still possible to specify how the forwarding invocation should be performed.

Hence, this proposal contains rules for how to match up two function types that differ in several ways. It is of course trivial to eliminate this part of the proposal and simply require that we are dealing with two identical function types; in that case the expression form is probably useless and can be omitted as well.

Syntax

This proposal introduces a new function body and a new formal parameter list:

<functionBody> ::= // Add one new alternative at the end.
    '=>' <expression> ';' |
    <block> |
    'async' '=>' <expression> ';' |
    ('async' | 'async' '*' | 'sync' '*') <block> |
    'forward' '=>' 'await'? expression ';'

<formalParameterPart> ::= // Add one new alternative at the end.
    <typeParameters>? <formalParameterList> |
    '(' '-' ')';

Static analysis and semantics

We propose that this mechanism should be made available for declarations and for function literals, such that we can specify a forwarding function concisely both in a declaration context and in an expression context.

We use Function.forwardTo<G>(<expression>) as the concrete syntax for the expression case, and a function body of the form forward => <expression> for the declaration case.

We use (-) as a parameter part in a function declaration to indicate that the signature is obtained from some other source (e.g., in an instance method declaration that signature could be obtained from a superinterface). If the return type is provided then only the formal parameters are obtained from that other source. For example:

class A {
  void f(int i, int j, [int k = 0]) {}
}

abstract class B {
  void g(int i, int j, int k, {bool isFoo = false});
}

class C implements B {
  final A a;
  B(this.a);
  g(-) forward => a.f;
}

void topF(int i, int j, [int k = 0]) {}
typedef G = void Function(int, int, int, {bool? isFoo});

void main() {
  h(G g) {}
  h(Function.forwardTo(topF));
}

The meaning of the declaration of C.g is defined by desugaring. First, the function signature (return type and formal parameters, with all properties) are obtained from the interface of B, which yields the following declaration (that we could also have written in the first place, if preferred):

class C implements B {
  ...
  void g(int a0, int a1, int a2, {bool isFoo = false}) forward => a.f;
}

A default value for an optional parameter may be needed; it is obtained from the declaration that provides the signature, if that yields a unique result; otherwise, it is obtained from the corresponding parameter in the forwardee, if present; otherwise a compile-time error occurs.

Next, we determine two function types F and G: G is the type of the forwarding function, that is, the function type of the instance method g; so that's void Function(int, int, int, {bool isFoo}). F is the type of the value argument, that is, the static type of the instance method tearoff a.f, so this is void Function(int, int, [int]).

The semantics of a function body of the form forward => e; is defined as a desugaring step which replaces this body by an arrow body of the form => e<X1..Xm>(args) where args is a list of actual arguments which is computed as specified below, based on G and F, and X1..Xm is a list of type arguments which is the type parameters of G, or a prefix thereof (such that h receives the number of type parameters that it declares).

A compile-time error occurs unless e has a static type which is a function type.

It is a compile-time error if the declaration is static and e denotes an instance member; it is a compile-time error if the declaration declares an instance method or a static method, and e<X1..Xm>(args) is a compile-time error. Finally, it is a compile-time error unless we have the relation G <:: F, which is specified in the following.

The semantics of the expression form Function.forwardTo<G>(e) is that it is desugared to a function literal that declares formal parameters and type parameters based on the type argument G, which specifies the type of the resulting function. A compile-time error occurs if any non-null default values are required in the resulting function literal. The body of the function literal which is the result of the desugaring invokes e as e<X1..Xm>(args), where X1..Xm is the list of type parameters of G or a prefix thereof, such that the number of type arguments passed to e is equal to the number of type parameters declared by the static type of e. A compile-time error occurs unless G <:: F where F is the static type of e.

We need to ensure that args is uniquely determined, which at first implies that it is possible to compute args at all. We introduce the binary type relationship <:: to do that; this relationship is also known as the wraps relationship.

For example, we do have the property G <:: F if G is the type of g and F is the type of f, as defined in the very first example.

Let G and F be function types such that G has n_G positional parameters and k_G required positional parameters, and F has n_F positional parameters and k_F required positional parameters, and n is the minimum of n_G and n_F. The relationship G <:: F holds if each of the following is satisfied:

• G declares at least as many type parameters as F, and if T1..Tm is a list of actual type arguments that satisfy the bounds declared by G, then T1..Tl, where l <= m, is a list of actual type arguments that satisfy the bounds of F. So we can drop some type arguments, but we will not invent any. In the remaining bullets we assume that we are passing the l first type parameters of G as the actual type arguments to F.
• The return type of F is assignable to the return type of G.
• n_G >= k_F. That is, "G has enough positional parameters to be able to call an F".
• For each positional parameter p_j of G, where 1 <= j <= n, the type of p_j is assignable to the _j_th positional parameter of F.
• For each required named parameter q of F, there exists a named parameter p of G, and the type of the latter is assignable to the type of the former.
• For each named parameter p of G where there exists a named parameter q of F with the same name, the type of the former is assignable to the type of the latter.

We can now specify the actual arguments args that are passed in the implicitly induced forwarding invocation: It contains the positional parameters p_j of G, where 1 <= j <= n, in that order; moreover, it contains an actual argument of the form n: n for each named parameter with the name n which is declared by both G and F.

The names of positional parameters in G are fresh identifiers. The point is that the first positional parameter of the wrapper function is passed as the first positional parameter of the wrappee, and similarly for the second, etc, and there is no user written code which is able to refer to the positional parameters.

Discussion

Use type inference?

We could consider specifying the desugaring of the forwarding function literal and the forwarding function body such that it simply invokes h(args) where h is the value argument of forwardTo and the arguments args are as specified above. In this case we would leave it to the type inference process to come up with suitable actual type arguments in the invocation of the forwardee, if any.

This may be somewhat hard to handle from the outside, though, because we would have to make the success or failure of this type inference step part of the wraps relation (<::), and this could make expressions using forwardTo unstable in ways that are hard to understand, as part of software development and evolution. For instance, we might run dart pub upgrade, and then a tiny difference in typing cause one of these type inference steps to fail, and then an expression like Function.forwardTo(f) is suddenly an error, and it takes some work to track down how that happened.

Usage as an IDE feature

It seems likely that we would want to allow for variation in the implicitly induced invocation of the wrappee in the body of the new function declaration or the wrapper function literal.

However, it may be sufficient to provide a quick-fix that expands any given use of forwardTo in a declaration or Function.forwardTo in an expression, just like the compiler would do it anyway during compilation.

This would then allow developers to get the function declaration or function expression right when it is introduced, and henceforth it can be manually desugared and edited as needed, which means that the body could be changed to a block, and it could contain arbitrary code.

Higher-order constructs?

In order to express higher-order constructs, it might be useful to be able to abstract over the wraps relation.

typedef G1 = ... Function<...>(...); // Something.
typedef G2 = ... Function<...>(...); // Something else.
typedef F = List<X> Function<X, Y extends X>(...); // Assume that `G1` and `G2` both wrap this type.

List<X> foo<X, Y extends X>(...) {} // Has type `F`.

void foo<G1 <:: F, G2 <:: F, F>(F f, h1(G1 g1), h2(G2 g2)) {
  // The `wraps` relationship ensures that this is allowed:
  h1(Function.forwardTo(f));
  h2(Function.forwardTo(f));
}

However, this kind of generalization can hardly be supported, because the construct Function.forwardTo... and the declarations with body = forwardTo... are only capable of covering the situation where the given function types are fully known at compile time.

Versions

• v0.1 Sep 30 2021: Initial proposal.
• v0.2 Sep 30 2021: Introduced optional parameters in the forwardee as a source of default values for non-nullable optional parameters.
• v0.3 Oct 1 2021: Adjusted the declaration form to use idea from Levi-Lesches (letting forward be similar to async/sync*/async*).

[Levi-Lesches]

I was surprised to find out even this doesn't work:

class A {
  void f(int i, int j, [int k = 0]) => print("A.f");
}

abstract class B {
  void g(int i, int j, [int k = 0]);
}

class C implements B {
  final A a;
  C(this.a) : g = a.f;

  // Error: Can't declare a member that conflicts with an inherited one
  // Info: Annotate overriden members
  Function g;
}

Especially since these two are equivalent:

class D {
  void a() => print("Hello");
  final b = () => print("Hello");
}

void main() {
  final d = D();
  d.a();  // "Hello"
  d.b();  // "Hello"
}

On the one hand, Dart can invoke a field that has a Function type, but on the other hand, it doesn't recognize such a field as a method, even when the context would suggest it is.

Even if this did work, we'd still lose info about parameters and return types on C.g, but this is where my mind goes intuitively when discussing method forwarding, so I thought I should mention it. For context, here's what I'd do in Python (which obviously doesn't have the same type safety as Dart but does allow method forwarding quite nicely):

class A:
  def f(self, i, j, k=0): print(i, j, k)

class C:
  def __init__(self, a):
    self.a = a
    self.g = a.f

c = C(A())
c.g(0, 1)  # 0 1 0
c.g(0, 1, 2)  # 0 1 2

Maybe instead of - or forwardTo, we take setting a field to a function as an indication that it should be a full-fledged method?

[eernstg]

Dart has always had the rule that it is a compile-time error to override a method with a getter or vice versa, even though the types can be matched up as you show.

I think one of the main reasons for this basic rule was that it lowers the performance of the language as a whole if every method invocation may turn out to be a getter invocation, yielding a function object, followed by an invocation of that function object. We generally don't know that a method hasn't been overridden in the dynamic type of any given receiver, so it doesn't help that the denoted member statically resolves to a method.

[Levi-Lesches]

Makes sense. I'm just trying to think of a more intuitive way besides

  g(-) = forwardTo(a.f);

Would this work?

g => a.f;

It's currently invalid syntax (Error: Methods must have an explicit list of parameters), and it would have the same meaning, being desugared into

void g(int i, int j, int k, {bool isFoo = false}) => a.f(i, j, k);

[eernstg]

I think it would be difficult to specify the signature with the g => a.f syntax: We might just want to specify the signature as a matter of style, or maybe the declaration isn't an instance member, or maybe g is an instance member, but the superinterfaces of the enclosing class do not declare a member named g, or we have several and they don't give rise to a combined member interface (that is, they differ too much). In all those cases we'd need to write the return type and formal parameter part explicitly. And then we can't see that => a.f is a forwarding body, it looks just like a normal arrow body.

[Levi-Lesches]

You're right, my example was focusing too much on when g is known to be an override. How about :, like we use in constructors? The connotation is "forward parameters from this constructor to the super constructor", so "forward parameters from this method to another method" isn't that much of a stretch:

class A {
  void b(int i, int j, {int k=0}) => print("$i, $j, $k");
}

class B {
  final A a;
  const B(this.a);
  
  void c(int i, int j, {required int k, bool debug = false}) : a.f
  // translates to:
  void c(int i, int j, {required int k, bool debug = false}) => a.f(i, j, k: k);
}

---

EDIT: I'm actually more for = instead, like forwarding constructors. So:

class A {
  void b(int i, int j, {int k=0}) => print("$i, $j, $k");
}

class B {
  final A a;
  const B(this.a);
  
  void c(int i, int j, {required int k, bool debug = false}) = a.f
  // translates to:
  void c(int i, int j, {required int k, bool debug = false}) => a.f(i, j, k: k);
}

[Levi-Lesches]

@tatumizer, my impression of this proposal is that it's only for delegates, meaning the actual result of the forwarded call is not used further in the body Here's the example in issue #418:

class House {
  num setDimensions(num north, num south, num west, num east) => 
    north + south + west + east;
}

class HouseManager {
  final House house;
  const HouseManager(this.house);

  num setDimensions(num north, num south, num west, num east) => 
    house.setDimensions(north, south, west, east);
}

In this case, the above can be rewritten to something like:

class HouseManager {
  final House house;
  const HouseManager(this.house);

  num setDimensions(num north, num south, num west, num east) = house.setDimensions;
}

[eernstg]

@Levi-Lesches - I think the syntax <functionSignature> = <expression> could work to declare a function or method which is a forwarding method. Here's the example again:

class B {
  final A a;
  const B(this.a);
  
  void c(int i, int j, {required int k, bool debug = false}) = a.f;
}

It may or may not be useful to use a syntax that includes a very explicit indication of the nature of this declaration (by having the word forward somewhere). I guess the trade-off is that the word forward may help a reader who is not extremely familiar with this mechanism, but a syntax like c(-) = forwardTo(a.f) is also quite unusual, which might be seen as a source of irritation, and, secondly, it's also a bit more verbose.

If we assume that we don't need the word forward as a reminder, the biggest difficulty I can see is that there is no obvious place to put the type arguments. It could turn out to be useful to specify the signature of a forwarding function using a type argument (and it makes sense that we can specify the type arguments in any case, such that we can preempt type inference if we want to make a different choice):

typedef G = void Function(int, int, {required int k, bool? debug});

void f(int i, int j, {int k=0}) => print("$i, $j, $k");
g(-) = forwardTo<G>(f);

[Levi-Lesches]

Sorry, I'm a bit lost, are you trying to find a way to get rid of the argument list?

If so, what about the ... that keeps getting proposed? If it's just desugaring, and the argument list is extracted from the forwarded-to function, then it should work?

class A {
  void b(int i, int j, {int k=0}) => print("$i, $j, $k");
}

class B {
  final A a;
  const B(this.a);
  
  void c(...) = a.f;
  // translates to
  void c(int i, int j, {int k=0}) => a.f(i, j, k: k);
}

If you're talking about using typedef (or other sources for a Function type), then maybe as? Since the function is kind of being reshaped (extra parameters dropped when not used), which is most similar to dynamic casting.

typedef G = void Function(int, int, {required int k, bool? debug});

class A {
  void b(int i, int j, {required int k}) => print("$i, $j, $k");
}

class B {
  final A a;
  const B(this.a);
  
  void c(...) = a.f as G
  // translates to
  void c(int i, int j, {required int k, bool? debug}) => a.f(i, j, k: k);  // debug is dropped
}

[eernstg]

@Levi-Lesches wrote:

are you trying to find a way to get rid of the argument list?

I'm looking for a concise and automatic (hence less error-prone) mechanism to obtain a function of a given type based on forwarding to a function of a similar type.

The current approach would be to write a function/method declaration or a function literal, spelling out the entire parameter part, with a body which is just a return of an invocation of the forwardee, passing those parameters. So we have to write a lot of stuff, and it is easy to get it wrong. In that sense it is true that I'm getting rid of the parameter list and the argument list.

When it comes to the expression form, the mechanism offers support for signature adaptation. For instance, if you have a function f of type dynamic Function(int) that you want to use in a context where an int Function(int, int) is required (and you know it's OK to just ignore the second argument, and you know that the result will be an int), then you can just pass Function.forwardTo(f) rather than f.

You could of course also pass (int i, int _) => f(i) as int, but the whole point is that Function.forwardTo(f) is easier to get right, and probably easier to read and understand. Of course, this is one of the cases where it may be helpful if there is IDE support for hovering on forwardTo in order to show the inferred type arguments.

void c(...) = a.f as G

We could certainly use (...) to indicate that a function does not declare its signature. I chose (-) rather than (...) because we have had several different proposals about ... in connection with parameter lists, composite literals (where we already use it as a 'spread' operator), and also because I consider it unlikely that we'll have Haskell style sections.

My main worry is that I think forwarding declarations will be a rather small feature: It's going to pop up here and there, probably there will be some "delegation" classes where almost all methods are just forwarding to some other object, and this probably means that the syntax needs to be somewhat self-explanatory. The presence of the word forward seems to give us that property.

[Levi-Lesches]

So would the two above examples work? It seems to be able to handle both: 1) getting rid of long paramter lists (using ...) when they can be inferred by the forwarded-to function, and 2) using the "wrong" function type so long as it can be massaged into the expected type (using as G)

[eernstg]

So would the two above examples work?

I think they would work. I just tend to prefer to have the word forward shown somewhere.

[Levi-Lesches]

I just tend to prefer to have the word forward shown somewhere.

How about forward being used like async? Here are a few of the earlier examples again:

typedef G = void Function(int, int, {required int k, bool? debug});

class A {
  void b(int i, int j, {int k=0}) => print("$i, $j, $k");
}

class B {
  final A a;
  const B(this.a);
  
  c forward => a.f  // a "normal" forward
  // translates to
  void c(int i, int j, {int k=0}) => a.f(i, j, k: k);

  d forward => a.f as G  // forward with a cast
  // translates to
  void d(int i, int j, {required int k, bool? debug}) => a.f(i, j, k: k);

  void e(int i, int j, {required int k, bool debug = false) forward => a.f;  // explicit signature
  // translates to
  void e(int i, int j, {required int k, bool debug = false) => a.f(i, j, k: k);
}

One benefit to doing it this way is that you probably don't need the parenthesis for the grammar to be unambiguous, so you don't need ... or any special marker either. Personally, I think that even without a keyword, users will be able to understand the = syntax because it's so similar to constructor forwarding and method forwarding in other langauges like Python. Adding a keyword gives the impression that it's a completely different feature than what users are used to, which may cause confusion over a relatively simple feature.

[eernstg]

I like that!

[eernstg]

I adjusted the proposal to let forward play the same role as async in a function body. This means that we can have the following:

typedef G = void Function(int, int, {required int k, bool? debug});

class A {
  void b(int i, int j, {int k=0}) => print("$i, $j, $k");
}

class B {
  final A a;
  const B(this.a);

  c(-) forward => a.f;
  d(-) forward => a.f as G;
  void e(int i, int j, {required int k, bool debug = false}) forward => a.f;

  // translates to

  void c(int i, int j, {int k=0}) => a.f(i, j, k: k);
  void d(int i, int j, {required int k, bool? debug}) => a.f(i, j, k: k);
  void e(int i, int j, {required int k, bool debug = false}) => a.f(i, j, k: k);
}

I tried to use <identifier> as a new kind of <functionSignature>, but that causes too many ambiguities, so I kept (-).

[Edit: Going over the proposal and updating various details, I decided to drop the support for using an explicit result type, as in forward => a.f as G;. If we want that effect then we must write the desired result type as a normal function signature in the enclosing declaration.]

[Levi-Lesches]

Looks great! Just one detail: is it possible to switch out the - for something else, like ...? (-) seems to imply there are no parameters, or that whatever parameters there were are cancelled out for some reason. (...) or (*) more traditionally imply that there are an unspecified amount of parameters, which is more accurate.

[Levi-Lesches]

Also, now that we're using forward like async, maybe we can add the equivalent to await as well. It complicates things a little but adds much more flexibility and usefulness.

As @tatumizer and others in the linked issues pointed out, a common use of forwarding is to either do something with the result or simply wrap the function. After all, if you can do c() forward =>, you should be able to do c() forward {}. Borrowing from my earlier example:

 class House {
  num getPerimeter(num north, num south, num west, num east) => 
    north + south + west + east;
}

class Contractor {
  Future<void> orderMaterials(num amount) => 
    Future.delayed(Duration(seconds: amount.floor()), () {});
}

class HouseManager {
  final House house;
  final Contractor contractor;
  const HouseManager(this.house, this.contractor);

  /// Computes the perimeter and orders the needed materials
  Future<num> setDimensions(num north, num south, num west, num east) async {
    print("Getting dimensions");
    num perimeter = house.getPerimeter(north, south, west, east);
    await contractor.orderMaterials(perimeter);
    return perimeter;
  }
}

HouseManager.setDimensions essentially wraps House.getPerimeter, and just does something with the result. I also threw in a Future to spice things up. Now, the future does throw things off because now it's a different return type, and in your edit you say that if the signature changes we should just type the whole thing out, so I'll do one example with the future and one without. How about using forwards in the signature and forward as a keyword to actually call the function. So something like the following:

setDimensions(...) forwards {  // forwards indicates this signature needs to be expanded
  print("Getting dimensions");
  num perimeter = forward house.getPerimeter;  // similar to awaiting a Future
  print("Got perimeter: $result");
  return perimeter;
}

And with a different signature:

Future<void> setDimensions(num north, num south, num west, num east, {bool shouldOrder = true}) forwards async {
  print("Getting dimensions");
  num perimeter = forward house.getPerimeter;  // similar to awaiting a Future
  if (shouldOrder) await contractor.orderMaterials(perimeter);
}

In summary:

• Forwards is to forward as async is to await (maybe separated with a comma?)
• If the signature is explicitly set, forwards may not be needed since it's purpose is to desugar the signature
• Since forwards can be used with a different return type, it may be used together with async
• forwads is valid if and only if there is exactly one forward call

Again, this does add more complexity for the sake of saving a line or two, but just throwing it out there.

[eernstg]

@Levi-Lesches wrote:

switch out the - for something else, like ...?

I think that's a subjective decision (I don't expect any serious parsing problems with any of these), so we could certainly do that.

HouseManager.setDimensions essentially wraps House.getPerimeter,

That does make sense, but it is a very delicate relationship.

In particular I'm afraid that the code won't be sufficiently stable, maintainable, or readable, if we allow a forward e in the middle of a large function body to introduce names that are in scope for the whole function body. Consider the situation where changing the name of a positional parameter of getPerimeter would suddenly cause some expressions in the body of setDimensions to refer to something different (e.g., x used to be a parameter of getPerimeter, hence also a parameter of setDimensions, but now that parameter is called y, and x resolves to an instance variable declared by a superclass of HouseManager).

The simple proposal where a forwarding body can't do anything other than the forwarding call itself doesn't have that issue, because there is no user code that refers to the parameters.

[Levi-Lesches]

Agreed, it's a lot more complexity than it's worth. Besides, when showing a new coder around a codebase, one can easily explain why they need one function to wrap another, but if signatures start changing based on the code inside said function, that starts to cause confusion. I think it's one of those things that makes sense when you expect it but can otherwise feels magical and arbitrary.