Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Control Flow Collections #78

Closed
munificent opened this issue Nov 2, 2018 · 37 comments
Closed

Control Flow Collections #78

munificent opened this issue Nov 2, 2018 · 37 comments
Assignees
Labels
feature Proposed language feature that solves one or more problems
Milestone

Comments

@munificent
Copy link
Member

munificent commented Nov 2, 2018

In order to handle conditionally omitting elements from list literals (#62, #70), I propose that we allow if and for inside list and map literals.

Feature specification draft.

This issue is for discussing that proposal.

Edit: The proposal has been accepted as is being implemented. The implementation work is tracked in #165.

@munificent munificent added the feature Proposed language feature that solves one or more problems label Nov 2, 2018
@mraleph
Copy link
Member

mraleph commented Nov 6, 2018

Two questions:

  • Is there a reason for not make it more uniform and allow switch too?
  • Should we also allow if and switch outside of list literals in the expression context?

Seems a bit arbitrary and confusing to newcomers that the first like would be allowed and the second one would require conditional operator

var x = [if cond expr else expr].first;
var y = if cond expr else expr;

@lrhn
Copy link
Member

lrhn commented Nov 6, 2018

My answers to those two questions:

  • The reason to not allow while/do-while is that this is not imperative code, it's iterative and conditional code. That doesn't rule out switch, but it would change each case expression to be a single "element expression" with no break. That's far from the current switch syntax, IMO farther than the for/if cases. Nothing prevents you from doing () { switch ( ... ) case .... return ...; }() if you want an arbitrary statement.

  • I don't see if (cond) expr else expr as the common case for a literal element expression, rather the real use-case is conditional omission, if (cond) expr. You can already use ?: for choice. If we weren't planning spreads I would have preferred to not even allow the else for element expressions. With spreads, it does make sense to do if (test) ...[e1, e2] else ...[e3, e4, e5] since you a have a different amount of elements in the branches (you can do that with if (test) for .... else for ... as well, it's just not as readable, but ... iterable is equivalent to for (var x in iterable) x, so that's hardly surprising).

We could use if for expressions conditionals too, but we already have ?:, and I'd loathe to have two ways to do the same thing, where neither is inherently superior.
If we were designing the language from scratch, I'd be much more receptive to using if for expressions and never add ?:. We are not.

@munificent
Copy link
Member Author

@lrhn said it well.

The other practical reason for not doing switch was that I didn't see any places in the corpus I examined where it seemed like it would be valuable.

Allowing if as an expression in general is interesting, but that's a bigger feature with more subtle implications. It immediately raises questions of if everything should be an expression and that opens a whole can of complexity given Dart's existing syntax and semantics. It would have been nice to do before 1.0, but I think would be very challenging now.

@mraleph
Copy link
Member

mraleph commented Nov 7, 2018

While we are not designing our language from scratch we are trying to attract more new users to the language - and they are learning the language from scratch.

Put yourself in their shoes and ask yourself if this makes sense to you:

var x = [if cond expr else expr].first;  // ok
var y = if cond expr else expr;  // not ok
var z = cond ? expr : expr; // ok
var a = [cond ? expr : expr] // ok, but we suggest this can be converted to if

Cause it does not make sense to me. It makes this feature non-orthogonal - where it could easily be made orthogonal without any significant implications AFAIK.

If we make it so non-orthogonal - then we can at least make our tools produce meaningful errors, because currently it is not handled very well

dartpad

(Note that DartPad's analyzer happily parses this code - while CFE reports compilation errors)

Regarding the switch: this feature is requested quite often and modern language actually do have it, including Java (see http://openjdk.java.net/jeps/325 - which provides very good rationale to have it).

Again - adding switch makes the feature feel more complete, more orthogonal.

@munificent
Copy link
Member Author

munificent commented Nov 8, 2018

where it could easily be made orthogonal without any significant implications AFAIK.

I think the main implication of making if an expression is that users will likely expect to be able to use blocks as the then and else statements. We'd either have to make that a surprising error (surprising since the style guide requires you to use blocks in almost all cases for if statements) or we'd have to allow blocks as expressions too, which I think does open the whole "everything is an expression" can of worms.

@mraleph
Copy link
Member

mraleph commented Nov 8, 2018

I think the main implication of making if an expression is that users will likely expect to be able to use blocks as the then and else statements.

Would not users similarly expect to use blocks when if starts appearing in the lists?

When I see [if (x) x else y ] I actually think that if is allowed here because it is an expression, not because it is some weird list-literal specific thing.

The error that you would need to implement for [if (x) { x; y; z; } else { a; b; c; }] is exactly the same kind of error you would implement for var o = if (x) { x; y; z; } else { a; b; c; }.

I think the only unpleasant and hard to explain corner case here is the interpretation of [if (x) a] which is not an error while var v = if (x) a is an error. I think we could interpret as [if (x) a else _|_] and say that bottom value is not allowed in other contexts.

@lrhn
Copy link
Member

lrhn commented Nov 8, 2018

I actually believe that users will be able to distinguish collection comprehension for/if syntax from statement for/if syntax. If I didn't think that, I would not be in favor of reusing the syntax.

I had the same worries initially, but I actually got to the point where it felt natural.

It's arguable that we now have "similar syntax for different meanings", but what we have is actually
similar syntax for similar meanings, it's just that conditions/iterations are parameterized by the syntactic category that they apply to.

When used as a a statement, for (...) and if (...) allow a statement as their body.
When used as a list element, for (...) and if (...) allow a list element as their body.
When used as a map entry, for (...) and if (...) allow a map entry as their body.

Apart from that, they actually work exactly the same, performing iteration or choice at run-time. Since all the contexts have the notion of repetition and of doing/adding nothing, or doing/adding more than one thing, we can use both iteration and the if-with-no-else.

For expressions, that does not apply. An expression has one result, and it always has a result if it terminates normally. That's why neither repetition nor one-or-none conditions apply to expressions, and why we should not try to extend if to expressions. Not because we couldn't, but because that would be similar syntax for a different behavior (no else-less if in expressions, no for loops at all).

So, I do think the proposed syntax is consistent with the existing language, and allowing if in expressions is not.

(That does not explain why we can't have try/catch in literals. The switch/case makes sense, but only if we remove the requirement for break, and do?/while won't work well because they generally rely on the body to do the iteration.)

@munificent
Copy link
Member Author

Would not users similarly expect to use blocks when if starts appearing in the lists?

I am somewhat worried about that (the proposal mentions this as a concern), but I think it's less of a worry because users do understand they are inside the context of a list and that's a pretty explicitly "not statement" place. Something like this feels more ambiguous to me:

var foo = if (condition) {
   ...
} else {
  ...
}

Yes, technically the initializer is strictly an expression so it's definitely not a statement context, but it does look a lot like one.

Again - adding switch makes the feature feel more complete, more orthogonal.

The syntax for switch is already bad: weird label-looking things, mandatory break, fall-through, etc. I strongly believe we shouldn't make switch more complex. It's primary value in Dart is that it works identical to switch in other languages which eases porting and learning. Anything we add to switch to make it "better" worsens those attributes

If we want to make a "better switch" (and I do), I think we should add a separate new pattern-matching syntax that removes all of the warts of switch, allows destructuring, works on user-defined types, etc. If we do that, then I think it's reasonable to consider making it an expression and/or allowing it as a control flow element in a collection.

More pragmatically: switch just isn't used that often and I couldn't find any examples in the corpus I looked at where allowing switch inside a collection literal would be useful.

@gamebox
Copy link

gamebox commented Nov 23, 2018

It seems that outside of the context of constructing a collection literal, the die has been cast for if. A match expression as described by @munificent would cause the least confusion and surprises, and allow for cleaning up ugly initializing code(namely values that are only assigned once not being able to be const due to initialization happening in the if statement.

Now as far as far as the specification under question is concerned, is function application allowed within the context of an if or for, i.e. a function that returns a valid collection member? This would be interesting as it would create a bit of a discrepancy between using this feature with lists vs maps (or is there some value that could be returned as a valid member for a map from a function?). Obviously some languages with tuples allow lists of 2-tuples to be spread into a map. Sidenote: Are tuples on anyone's radar?

@lrhn
Copy link
Member

lrhn commented Nov 23, 2018

A function application is allowed for list literals, because a function application is an expression, and a list element is an expression.

In a map literal, the entry is a key:value pair, which is not a normal expression, nor is it an expressible value. You literally has to write "expression-colon-expression" to be a map entry. (You can perhaps write ... functionCallReturningMap() instead, because maps are expressible values).

@munificent
Copy link
Member Author

Now as far as far as the specification under question is concerned, is function application allowed within the context of an if or for, i.e. a function that returns a valid collection member?

Yes, any expression is allowed there, including a function call.

(You can perhaps write ... functionCallReturningMap() instead, because maps are expressible values).

That's right. You could spread a map into it:

var a = {1: 2, 3: 4};
var b = {
  5: 6,
  if (condition) ...a
};

@kmillikin
Copy link
Contributor

Hi @munificent, this looks very nice.

I don't think we should allow general if and for expressions. It's so non-obvious what they mean that it's better not to introduce them at all and there are pretty simple workarounds for where you'd use them. (What is the value of an if with no else and a false condition? What if it's a non-nullable type? Runtime error? What is the value of a for loop? The last value? The rest are evaluated for their side effects?)

The runtime semantics needs one more pass through it. Here are some things I spotted to clarify:

  1. What is the scoping of the variable bound by for elements? To avoid surprises it should probably be the same as for statements.

  2. Elements are evaluated but not added to the collection in all cases.

  3. A semantics in terms of adding to a collection does not work for a const collection.

I'll sketch up what this would look like as a Dart-to-Dart (actually Kernel) transformation. I'd like to see if we could come up with an operational and static semantics that says it behaves as if ... to explicitly bless such an implementation technique.

Notice that for elements gives you let for local variables through an encoding (without closures) which is kind of new.

@munificent
Copy link
Member Author

Thanks for the feedback!

What is the scoping of the variable bound by for elements?

The proposal says:

If a for element declares a variable, then a new namespace is created where that variable is defined. The body of the for element is resolved and evaluated in that namespace. The variable goes out of scope at the end of the element's body.

Is that clear enough? By "the element's body", it means "the for element's body". I'll tweak the text to clarify that.

Elements are evaluated but not added to the collection in all cases.

They may not be evaluated. If, for example, the condition in an if element evaluates to false, the "then" element is not evaluated at all. I.e.:

var list = [
  if (false) throw "!"
];
print(list); // []

Is that what you mean, or am I missing something?

A semantics in terms of adding to a collection does not work for a const collection.

Ah, you're right. For the spread proposal, I added a separate section to cover the const semantics. I need to do something similar here.

I'll try to get that in pretty soon.

I'll sketch up what this would look like as a Dart-to-Dart (actually Kernel) transformation.

That sounds great. It is my intent that this feature is effectively just syntax sugar, so if it's not possible to do that, that would be useful data.

Notice that for elements gives you let for local variables through an encoding (without closures) which is kind of new.

Yes. Importantly, it means it's transparent to async/sync*/async*, unlike a lambda. I don't think this is a particularly useful thing to use the feature for, but it's there. I look forward to some ML programmer writing a bunch of code like:

print([for (var i in [
  [for (var j in [
    [for (var k in [
      3
    ])]
  ])]
])]);

@kmillikin
Copy link
Contributor

For loop scoping is either clear enough but not correct, or it is correct but it's not clear enough.

I think Dart programmers will expect

[for (int i = 0; i < 5; ++i) () => i].map((f) => f()).toList()

to be [0, 1, 2, 3, 4] but it doesn't sound like that.

When I wrote that elements are not always added to the collection, I was just wrong. I had scrolled past the base cases.

@munificent
Copy link
Member Author

For loop scoping is either clear enough but not correct, or it is correct but it's not clear enough.

The proposal does state:

Each iteration of the loop binds a new fresh variable:

var closures = [for (var i = 1; i < 4; i++) () => i];
for (var closure in closures) print(closure());
// Prints "1", "2", "3".

I'll tweak the text above that to be a little clearer.

@kasperpeulen
Copy link

kasperpeulen commented Dec 11, 2018

Users may be unhappy that the syntax doesn't go far enough. This feature may lead them to expect, say, while to work inside a collection. They may expect to be able to put an entire block of statements as the body of an if. They may want to use if outside of a collection but in other expression contexts.

In other words, this may be a "garden path" feature that encourages a whole set of expectations, some of which are met and the rest of which are confounded.

This resonates with me. I don't understand the restrictions of the proposal.

I think the main implication of making if an expression is that users will likely expect to be able to use blocks as the then and else statements. We'd either have to make that a surprising error (surprising since the style guide requires you to use blocks in almost all cases for if statements) or we'd have to allow blocks as expressions too, which I think does open the whole "everything is an expression" can of worms.

I think users will expect this as well when if and else appear in collections. It seems like the rule for collections is now:

When you use if as an expression braces should be omitted and only one expression per branch is allowed.

This rule seems a bit odd, in the sense that I can't think of any other languages with such a rule that allows control structs as expressions.

But it may make sense in the Dart language. If so, this rule could generalise to if expressions:

Widget build(BuildContext context) {
  return if (loggedIn)
    LogoutButton()
  else
    LoginButton()
  };
}

What is the problem with "everything is an expression"? Kotlin allows if...else and try...catch in an expression context, and I've never seen anyone complain:
https://kotlinlang.org/docs/reference/idioms.html#if-expression

@gamebox
Copy link

gamebox commented Dec 11, 2018 via email

@kasperpeulen
Copy link

@gamebox I think what you said is what I meant to say.

@kasperpeulen
Copy link

kasperpeulen commented Dec 12, 2018

If we add spread syntax, then that covers some of these use cases. Spreading is fine, but when you want to do more than just insert a sequence in place, it forces you to chain a series of higher-order methods together to express what you want. That can get cumbersome, especially if you're mixing both repetition and conditional logic. You always can solve that using some combination of map(), where(), and expand(), but the result isn't always readable.

I doubt if the examples you gave actually shows that it is more readable than the spread syntax. I do think that using for as an expression could be useful, but I'm not sure if you gave the right examples.

var command = [
  engineDartPath,
  frontendServer,
  for (var root in fileSystemRoots) '--filesystem-root=$root',
  for (var entryPointsJson in entryPointsJsonFiles)
    if (fileExists("$entryPointsJson.json")) entryPointsJson,
  mainPath
];

With spread syntax:

var command = [
  engineDartPath,
  frontendServer,
  ...fileSystemRoots.map((root) => '--filesystem-root=$root'),
  ...entryPointsJsonFiles.where((entryPointsJson) => fileExists("$entryPointsJson.json")) 
  mainPath
];

Also the other example becomes with the spread syntax:

Widget build(BuildContext context) {
  final themeData = Theme.of(context);

  return MergeSemantics(
    child: Padding(
      padding: const EdgeInsets.symmetric(vertical: 16.0),
      child: Row(
        mainAxisAlignment: MainAxisAlignment.spaceBetween,
        children: [
          Expanded(
            child: Column(
              crossAxisAlignment: CrossAxisAlignment.start,
              children: [
                ...lines.sublist(0, lines.length - 1)).map((line) => Text(line)),
                Text(lines.last, style: themeData.textTheme.caption)
              ]
            )
          ),
          if (icon != null) SizedBox(
            width: 72.0,
            child: IconButton(
              icon: Icon(icon),
              color: themeData.primaryColor,
              onPressed: onPressed
            )
          )
        ]
      )
    ),
  );
}

It feels to me that for expressions as specified now are too restricted to be more useful or readable than the spread syntax combined with map and where.

Let's look at this problem from a different angle. We can now do the following in Dart:

var integers = () sync* {
  for (var i = 1; i < 5; i++) yield i;
}().toList();

And with the spread syntax you can do:

var integers = [...() sync* {
  for (var i = 1; i < 5; i++) yield i;
}()];

This of course looks very clunky, in Javscript there is a proposal to have do generator expressions, that would look like:

var integers = [...do* {
  for (var i = 1; i < 5; i++) yield i;
}];

I have earlier proposed for ecmascript to make it even shorter:

var integers = [...for (var i = 1; i < 5; i++) yield i];

In general, if you use for in an expression context, it desugars to an immediately invoked generator function with as body the for loop.

The for expression part of this proposal adds even more sugar then what I propose above:

var integers = [for (var i = 1; i < 5; i++) i];

This is short and readable, but it feels less in line with the rest of the language (and style guide). Also because it is so restricted you can do the things you would like to do with a for loop (because you can't do it with map/where without making it less readable). For example something like:

var solutionBatch = for (var solution in solutions) {
  var numberData = numberDatas.firstWhere(
    (numberData) => numberData.oldNumber == solution.number,
    orElse: () => null);
  if (numberData != null) {
    yield Solution(id: solution.id, number: numberData.newNumber);
  }
}

Last note about Map, the syntax here seems very specific. If you allow a "map entry" in this context, I would expect it that I can also use it in other places. For example in a generator function:

Map<String, String> generator() sync* {
  for (var demo in kAllGalleryDemos) {
    if (demo.documentationUrl != null) {
      yield demo.routeName: demo.documentationUrl;
    }
  }
}

tldr, my changes to the proposal would be:

  • Allow a block as the body of the "for expression"
  • Require yield
  • Make a for expression evaluate to an iterable by default, and use spread operator to cast it to a List/Set/Map etc.

@munificent
Copy link
Member Author

munificent commented Dec 13, 2018

I don't understand the restrictions of the proposal.

I think your mental model of the proposal is that if and for become expressions, but that's not how the proposal actually works. An expression is a piece of code that always evaluates to a single value. Consider:

[if (false) 123];

What is the value of the if "expression" inside this collection? It's not null, because that would result in the collection [null]. What you actually get is []. Likewise:

[for (var i = 1; i <= 3; i++) i];

What is the value of this for expression? It's not [1, 2, 3], because would result in the collection [[1, 2, 3]]. (Note the double nesting.) What you actually get is [1, 2, 3].

The if and for syntaxes are most analogous to the spread ... syntax that JS also has. In JavaScript, ... isn't an expression. It isn't even meaningful to consider code like:

var value = ...[1, 2, 3];

What spread and the if and for proposals really do is define a new syntactic category: a piece of code that can produce zero or more values. That category is only allowed in contexts where it's natural to consume zero or more of something — collection literals. (And, in particular, maps introduce another category where a piece of code can produce zero or more entries and an entry is not an expression.)

You are correct that if you have spread, then generators give you everything you need. Instead of:

var list = [
  if (condition) a else b,
  for (var i in stuff) i * 2
];

You can always do:

var list = [
  ...() sync* {
    if (condition) yield a; else yield b;
  }(),
  ...() sync* {
    for (var i in stuff) yield i * 2;
  }
];

The three main problems with that are:

  1. It's really ugly. Given that this whole feature is about making code easier on the eyes, that's a real problem. We could shave off some of the punctuation here and there, but it's still always going to be verbose.

  2. In particular, requiring an explicit yield to emit a value is verbose and the wrong default. This proposal is about making code more declarative and less imperative. If we use a syntax that defaults to assuming you want statements and then forces you to add a keyword (yield) to opt out of that and into declarative expression space, then we've picked the exact wrong default.

    If you want to use imperative statements to build up a collection, we've already got a nice way to do that:

    var list = [];
    if (condition) list.add(a); else list.add(b);
    for (var i in stuff) list.add(i * 2);

    This proposal is about letting you stay out of statement space and do more useful work inside a declarative expression.

  3. Using some kind of generator lambda doesn't work with things like await. A do expression syntax would be transparent to that but, again, the goal is not to allow statements in your expressions, it's to make your expressions not need statements in the first place.

I did consider something like implicit generator blocks. They do have the nice property that any statement is allowed, so the "garden path" problem is solved. But they also force you to explicitly yield. To me, that's the wrong trade-off. If you want statements and imperative code, you can already do that now outside of the collection literal just fine using add(), addAll(), etc. My goal was to let you do more as a pure expression.


Orthogonal to this is supporting if (and/or for) as expressions. I like languages that do that, and wish Dart always had. (I suggested it a number of times to the language team way back before 1.0, but alas.)

Doing that now would be very hard. One key reason is map literals. Consider:

var what = if (condition) {} else {}

Are those {} empty blocks or empty map literals? Kotlin and Swift don't have this problem because they (probably not coincidentally) don't use the same curly brace syntax for maps that JavaScript and Dart use.

Unfortunately, in Dart, we have a couple of places in the grammar where the exact same text means two different things in an expression and statement context. If we make everything an expression, then those contexts merge and we end up with an ambiguous grammar. Teasing that apart after we've reached 2.0 would be very difficult.

@kasperpeulen
Copy link

kasperpeulen commented Dec 14, 2018

What spread and the if and for proposals really do is define a new syntactic category: a piece of code that can produce zero or more values. That category is only allowed in contexts where it's natural to consume zero or more of something — collection literals. (And, in particular, maps introduce another category where a piece of code can produce zero or more entries and an entry is not an expression.)

I see.

I did consider something like implicit generator blocks. They do have the nice property that any statement is allowed, so the "garden path" problem is solved. But they also force you to explicitly yield. To me, that's the wrong trade-off. If you want statements and imperative code, you can already do that now outside of the collection literal just fine using add(), addAll(), etc. My goal was to let you do more as a pure expression.

I definitely think that allowing if in collections allows you to do more as a pure expression.

I doubt that the proposed for in collections (possibly combined with if) let's you do more as a pure expression. You could write that code always with a combination of spread, map and where. The question then becomes, why do we need another syntax for that?

The for expression (that works like an implicit generator blocks) allows you to write some imperative logic in an expression context, if it is hard read or hard to write with map, where fold etc.

Orthogonal to this is supporting if (and/or for) as expressions. I like languages that do that, and wish Dart always had. (I suggested it a number of times to the language team way back before 1.0, but alas.)

I really like it as well in other languages, also try expressions. I think they are allmost neccesary if you try to make immutable, non-nullable and type inferrable local variables. My main concern with this proposal I guess is that it will make if/for expressions impossible to ever implement in Dart. Because as you said, with if expressions: [if (false) 123] would evaluate to [null].

(Except if Dart would adopt something like void as a runtime type, that would do "nothing" when added to a collection, and also do nothing when passed to a named argument with a default value.)

Doing that now would be very hard. One key reason is map literals. Consider:

var what = if (condition) {} else {}

Ah I see. JS has something similar with arrow functions and map literals. For example what does the followiong code mean?

const a = () => {};

Here a() returns undefined if you want to return an empty object you have to write:

const c = () => ({});

@munificent
Copy link
Member Author

You could write that code always with a combination of spread, map and where. The question then becomes, why do we need another syntax for that?

Yeah, you always can, but it can be particularly cumbersome in some places. The typical bad cases where when you have some conditional logic inside the looping. To translate that to higher-order methods requires transform() or some combination of emitting nulls and then filtering them out later.

The for expression (that works like an implicit generator blocks) allows you to write some imperative logic in an expression context, if it is hard read or hard to write with map, where fold etc.

If you want to stuff some imperative code in a context where an expression is expected, you can always use an immediately-invoked lambda. In practice, it's usually better (more readable, maintainable) to hoist that out to a separate named function.

I think they are allmost neccesary if you try to make immutable, non-nullable and type inferrable local variables.

I have some concerns around that as well. I think definite assignment analysis is another viable approach. Even when you have if as expression, it doesn't gracefully handle cases where you are declaring and initializing multiple variables:

var a;
var b;
if (condition) {
  a = ...
  b = ...
} else {
  a = ...
  b = ...
}

Definite assignment analysis can handle that. You can do it with if as an expression along with some destructuring too:

var [a, b] = if (condition) {
  [..., ...]
} else {
  [..., ...]
}

But I'm not sure if that would feel natural in a language like Dart.

JS has something similar with arrow functions and map literals.

Ah, that's right. Yes, we could do something similar where we require you to parenthesize to indicate that you want a map literal.

@kasperpeulen
Copy link

Yeah, you always can, but it can be particularly cumbersome in some places. The typical bad cases where when you have some conditional logic inside the looping. To translate that to higher-order methods requires transform() or some combination of emitting nulls and then filtering them out later.

Can you show me examples of those two cases? For the examples in the proposal I could not help thinking that the spread operator combined with map and where would be an as readable option (and faster writable with method completion).

@munificent
Copy link
Member Author

Sure, here's a contrived one:

var list = [
  for (var i in items)
    for (var j = 0; j < i; j++)
      if (j.isEven) j
];

@munificent
Copy link
Member Author

one caveat is that there's no empty value available for "else".

You could do ...[] or ...?null. But, of course, the simpler answer is to omit the else clause entirely if you don't want to insert any elements.

@mdakin
Copy link

mdakin commented Jan 12, 2019

@tatumizer did your read the whole proposal? Most of the points you mentioned are discussed there and there are other aspects.

@munificent
Copy link
Member Author

Please correct me if I'm wrong, but I think that 1) the expression is valid - no warnings or errors from compiler 2) in case someCondition evaluates to true, it inserts 1-element set into the list, not an integer like before.

That's correct. This is one of my concerns with the syntax. Using if may lead users to think they can write a block, but they can't. Worse, they can use the same syntax as a block, but it means something else.

Though, in your example, the code wouldn't be syntactically valid as a block without a semicolon after count++. Obviously, optional semicolons would lead to this actually being ambiguous.

Overall, the users I've shown the syntax too so far don't seem too bothered by it, so this isn't keeping me up at night. In many cases, if you try to write a block and get a set instead, the resulting code will have a type error because the set doesn't match the list's expected type. We could potentially lint or warn if you try to use a set literal after an if element. Then, in cases where you do want that, you can silence the warning by wrapping it in parentheses.

But the current plan is to just try the current proposal and see how much of a problem this actually is in practice before we try to "fix" it.

@lrhn
Copy link
Member

lrhn commented Jan 22, 2019

The "void means nothing" approach is interesting, but it can't stand by itself. You need a type system that understands void-ness. The type of isAndroid ? IconButton(icon: Icon(Icons.search)) : void is IconButton-or-void. The fact that a voidable type is allowed here is non-trivial. You wouldn't/shouldn't be allowed to do int x = test ? 42 : void; because voidable int should not be assignable to int. You'd need a non-void cast.

This "voidable" type starts to look suspiciously like a nullable type. It means int-or-nothing.
The difference between this and a nullable type is that null is itself an expressible value.
(And that's also one of the big problems with nullable types: A null can occur both as a value and as a marker).

If Dart had a proper empty void type, then we could introduce int | void as a type that is allowed in positions where an optional int is allowed. We would need a way to recognize the void case, and act around it, but it would never become a value itself, so it differs from null.

So, I guess I'm saying that what is being proposed here could have been an alternative to nullable types, but I very much doubt we will have both.

@eernstg
Copy link
Member

eernstg commented Jan 22, 2019

@tatumizer, I agree that void is not just a type. Like, dynamic, it is an additional name for the top type, and (like dynamic) it has some associated rules that apply during static analysis, and those extra rules are unrelated to the subtype relation.

The special discipline applied to expressions whose static type is void is that the value of such an expression must be discarded (except for a short whitelist of exceptions). We do not maintain soundness for the higher-order case (e.g., List<Object> xs = <void>[]; is allowed), but separate tools like the linter can add restrictions in this area as needed. (Check the section 'Void Soundness' in the language specification for more on this.)

So void in Dart basically means "discard that value", but other than that it is just another name for the top type.

it's not assignable to any type!

This is not quite precise enough (e.g., if that had been true then it would then have been an error to have List<Object> xs = <void>[];). Given that the special restrictions on expressions with static type void are applicable in the first-order case only, it's misleading to explain what's going on in terms of subtyping (in particular, the word 'assignable' should not be used).

You can't write var x=foo()

Actually, that is allowed. The main reason is that we wanted to support the situation where an argument of a function is being ignored, and the call site is allowed to know about it. For example:

X seq<X>(void _, X x) => x;

seq allows for invocations where two expressions are evaluated, and the first value thus obtained is ignored (or seq(seq(...), ...) can be used to make it a longer sequence).

In order to allow things like seq(print("...Some Debug Info..."), expr) where we'd otherwise just have expr (and preserve the static type), seq must be able to indicate to the static analysis that the first argument is ignored (such that it includes the case where it must be ignored, because the actual argument has static type void). Because of this, we also allow void x = someVoidExpression;, and var x = foo(); where foo returns void is the same thing with inference. But then you can't use x!

With respect to the notion of 'lateinit' variables, I really think Dart needs a mechanism that is supported at run time, that is, we should be able to allow for cases where the static analysis does not establish a guarantee that the variable has been initialized, and then we'll have a dynamic error if it is not initialized when it is first evaluated.

With non-null types, this works perfectly for cases like int x; where we can use null as the initial value and any evaluation of x where there is no static guarantee that it has been initialized can check for null. This must mean "uninitialized" because there is no way we can assign null to x, because its type is non-null.

Of course, this property must be predictable for developers, so we could require something like lateinit as a modifier on a variable whose initialization is enforced dynamically rather than statically.

However, that doesn't have much to do with the type void, because that's a top type, and this means that every object can be given type void (any object can be discarded), and there is no meaningful way to associate the type void with a dynamic check.

I think the problem is less severe for variables with no initializer whose type is nullable, because the variable would be initialized to null and all usages would be subject to null checks just because of the type.

optional parameters

(I'll skip that topic today ;-)

express conditional elements in lists and map literals (by filtering out all void values)

Every object has type void, as I mentioned, so that couldn't be supported dynamically.

However, the upcoming null-aware spread operator actually does this using null:

List<int> xs = null;
if (something) xs = [3];
var ys = [1, 2, ...?xs, 4, 5];

With this, ys will be [1, 2, 3, 4, 5] if xs is [3] and otherwise [1, 2, 4, 5].

In summary, I can see the temptation to make the type void do all these things, but I don't think it will work for Dart, and it's certainly not very compatible with the existing interpretation of that type which says that it can be any object, but it must be discarded.

@leafpetersen
Copy link
Member

Just a quick drive by comment: it looks to me like what folks are reaching for here is a generalization from a single valued expression to a 0/1/many valued expression. What you're looking for is not really void, but rather the notion of an expression which may evaluate to 1 int, or 0 ints. You then, of course, need to decide what happens when you get 0 ints. This proposal basically introduces a context in which 0/1/many valued terms makes sense: you spread out the values into the enclosing collection. You can potentially extend the language to allow this in other places as well (e.g. imagine something like if (int x = if (b) then 3) { print (x); } else {print("Didn't get anything")}). There's lots you can do with this, but there's rather a lot of work to do to come up with a coherent design based on it.

@eernstg
Copy link
Member

eernstg commented Jan 23, 2019

@tatumizer wrote:

Still, maybe there's a chance to revisit the issue?

I don't think that will be easy.

We started off from Dart 1 where void was only a return type, wanting to support cases like Visitor<void> (which would have the meaning "the value returned by visit should be ignored", which is actually a useful concept).

But void was effectively a supertype of all other types: T Function() <: void Function() was true for all T != void, and it was a proper subtype relationship (because void Function() <: T Function() only holds for T == void), so we couldn't treat void as anything other than a top type, and we couldn't hope to support "being void" by a dynamic check.

(Of course, we could have made void a new "super-top-type", strictly above all other types, and we could have introduced a new value, "the bomb", whose type is only void. So having that value at run time would be definite evidence of having a value of type void, all other values would just mean "could be void", and then we would be able to perform a number of dynamic checks to maintain the "void discipline" a bit more tightly. But we decided that it was too much machinery, developers actually only wanted that ability to have stuff like Visitor<void> and have compile-time feedback if they tried to use the returned value.)

So: void is a static-only property of a type T, that type T must be the top type, and the main purpose is to say "you told me you didn't want to use this value, now don't use it!", and it would probably be hugely breaking to try to give void a new conceptual foundation (and then change it technically to fit the new "meaning"). I'd love to have a strictly enforced dynamic discipline on it, but it's not easy to get that.

We still have an option to introduce the value of "nothing",

Interestingly, that could match up with the notion of void as a super-top-type, with one object having the type void and no other types. For the example:

var map = {
   "foo": x ?? _,
   "bar": x?.y?.z ?? _ // TRY DOING IT WITH IF :)
}

... the rule would be that "the bomb" is written as _ and when it's used as the value of a key/value pair in a map, or when it is an element expression in a list or set literal, insertion is omitted. It does match up with the idea that "this value should be discarded", and the exception is that we may have such a value in those literals because they are inherently able to omit an element, whereas we still get an error for foo(_).

I think the typing would match up better, though, if we were to generalize void to take a type argument: void<T> would then be a supertype of T (just an epsilon above T), with rules and semantics corresponding to T | {"the bomb"}, and it could be used in contexts where it is acceptable for a value to be missing:

foo([int i = 42]) => i;

main() {
  print(foo()); // Prints '42', argument omitted syntactically.
  print(foo("the bomb")); // Prints '42', argument omitted semantically.
}

I don't know whether it will work out. ;-)

@eernstg
Copy link
Member

eernstg commented Jan 23, 2019

@tatumizer wrote:

If this doesn't work out

I'm afraid it is very unlikely that we will change void to be a proper supertype of the other current top-types and introduce a new value to support the distinction at run time. So it might be possible, but it seems unlikely that it could be done without breaking a massive amount of code....

@munificent
Copy link
Member Author

munificent commented Jan 23, 2019

@munificent: could you please fix the link in the opening post of this thread ("Feature specification draft") - it currently leads to 404 page.

Done, thanks for reminding me. :)

Leaf's comment is spot on. People are looking at this proposal as having to do with making if an expression but it's much closer to making if a generator in the Icon sense.

This proposal and spread narrowly define a couple of "generator elements" (..., if, and for) and allow them only in places where their semantics are obvious — inside collection literals where 0, 1, or more items have an obvious interpretation.

Extending that to allow these elements in arbitrary expressions is interesting but much less clearer semantically. I don't think we could expect users to infer what, say, this means:

var k = ["a", "b", "c"];
var v = [1, 2, 3, 4];
var map = {...k: ...v};

I could see us expanding both the set of generator elements over time and the set of places where they are allowed somewhat. But I don't think it will make sense to collapse elements and expressions into a single category. Dart isn't Icon.

@eernstg
Copy link
Member

eernstg commented Feb 13, 2019

@tatumizer wrote:

can you see any problem with introducing new type (Bomb),
with a single value (e.g. underscore), which can be used only
in some contexts - in the same contexts where dart today silently
"converts" (in a sense) void to null?

(Oops, didn't see this, I was traveling at the time.)

I think this extra "I'm not here" object Z wouldn't need to have many properties. Various parts of the code need to recognize that the given object is Z, and then do something else than usual, like not adding an element to a list (that is: certain language elements need to be specified to do so, but arbitrary user code could just be written to follow the same conventions, as long as Z appears in some context where it has no special semantics, presumably including a built-in isZ(...) function).

It (presumably) just needs one special property: It should be allowed to occur everywhere, such that we can abstract over "I'm not here"-ness.

But isn't that a perfect match for the null object, using nullable types to indicate that some value "may not be there"?

This would mean that we are statically tracking whether absence is permitted (nullable/non-null types are built for that already), and we have the ability to write code when a value is tested (using if for branching and a e! operator to assert that absence is not expected right here after all, and check dynamically), etc.

The missing bit, as far as I can see, would be that we'd want language support for a few extra cases: We already have null-aware method invocation etc., so there are lots of ways to explicitly allow absence and do nothing when it occurs.

But we could add the ability for a composite literal whose element (or key or value) type is non-null to accept a nullable typed expression and interpret the null object as "skip", and similarly for named parameters:

void foo({int x = 42}) => print(x);

main() {
  foo(); // Prints '42', as usual.
  foo(x?: null); // Ditto. Proposed by Lasse at some point, I think.
  foo(x: someNullableExpression); // Error.
  <int>[1, 2, ?null]; // Yields <int>[1, 2].
  // etc ...
}

So apart from the fact that you'd have to announce that there is a potential for receiving the null object in a situation where the context type is a non-null type (thus flagging the possible "I'm not here" semantics explicitly), I think this is very nearly an approach that we have already had on the table several times.

With that, I'm tempted to say that we don't need a Z which is different from the null object. It does confine the "I'm not here"-ness to nullable typed expressions in a context where a non-null instance is expected (except that we added some ?s exactly to bridge that gap), but that also seems to make sense. Or WDYT?

@eernstg
Copy link
Member

eernstg commented Feb 13, 2019

Cool, thanks!

@aartbik
Copy link

aartbik commented Feb 21, 2019

@aartbik

dart-bot pushed a commit to dart-lang/sdk that referenced this issue Mar 8, 2019
Rationale:
Implementation of control-flow collections is
done through the notion of block expressions.
This is a first implementation to get this
working. It takes a few shortcuts (like disabling
OSR inside block expressions for now) that may be
refined later.

dart-lang/language#78
dart-lang/language#47

Change-Id: I966bf10942075052fcfd9bac00298a179efc551b
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/94441
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
@mit-mit
Copy link
Member

mit-mit commented May 1, 2019

Closing; this is launching in Dart 2.3

@mit-mit mit-mit closed this as completed May 1, 2019
@mit-mit mit-mit added this to the Dart 2.3 milestone May 1, 2019
@mit-mit mit-mit moved this to Done in Language funnel May 27, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature Proposed language feature that solves one or more problems
Projects
Status: Done
Development

No branches or pull requests