Non-growable list literal syntax

[srawlins]

Non-growable lists are implemented more efficiently than growable lists, but creating one is rather verbose and inconvenient.

// Growable:
var items = foo.map(toItem).toList();
// Non-growable:
var items = foo.map(toItem).toList(growable: false);

// Growable:
var items = [
  ...firstThings.map((e) => e.foo),
  ...secondThings.map((e) => e.foo),
  if (condition) lastThing,
];

// Non-growable:
var intermediate1 = firstThings.map((e) => e.foo);
var intermediate2 = secondThings.map((e) => e.foo);
// Oops, we cannot use List() any more.
var items = List(intermediate1.length + intermediate2.length + 1);

// Concatenate the Iterables?
var items = List.of(
    intermediate1.followedBy(intermediate2).followedBy([lastThing]),
    growable: false);

Oh! to give up our precious and novel UI-as-code, our spreads and for-elements and if-elements... ☹️ And we're forced to create an intermediate 1-element list with followedBy([lastThing]), and I didn't even implement the if (condition) part.

Solution

What a beautiful, space-efficient, fun and quirky world we'd live in, if we had a non-growable list literal syntax. I'm thinking:

// growable: true
var items = foo.map(toItem).toList();
// growable: false
var items = [...foo.map(toItem)]gf;

// Growable:
var items = [
  ...firstThings.map((e) => e.foo),
  ...secondThings.map((e) => e.foo),
  if (condition) lastThing,
];

// Non-growable:
var items = [
  ...firstThings.map((e) => e.foo),
  ...secondThings.map((e) => e.foo),
  if (condition) lastThing,
]gf;

Or if we want to reserve gf for a gluten-free option, we can use []ng ("non-growable"), []f ("fixed-length"), whatever. A preceding f would be great (f[]), but I can't see how that wouldn't blatantly conflict with index-expressions.

[mateusfccp]

I love the idea. I am not sold on this syntax, tho.

[ykmnkmi]

var items = @fixedLength <int>[1, 2, 3];

[srawlins]

My investigation into non-growable lists stemmed from @mkustermann 's analysis at dart-lang/sdk#49858. I have mailed out dart-lang/dartdoc#3151 for dartdoc which indicates a ~7% memory use reduction in documenting one package. And there is this change which saved a ton of memory for dart2js. Other than these examples, I don't have super rigorous data on the benefits of the non-growable list.

[mateusfccp]

I use non-growable lists by default. Although for many cases the difference is negligible, the cost to have this little optimization is also negligible, i.e. just put a growable: false parameter.

However, I know of some people that would think it's verbose and even refuse to use const constructors in Flutter just because they find a hassle to put const before the widgets. Thus, having a literal for non-growable lists may motivate those people to use them.

---

Another problem with non-growable lists, although not in the scope of this issue, is that we can't statically avoid that it grows. If we had a NonGrowableList class that didn't provide a growing API (like add method), we could make it more safe. Maybe this is something solvable through extension views...

[Levi-Lesches]

This looks really helpful. It seems like an unusual restriction on an otherwise mutable concept, but I'd regard it similarly to const: something that you could change (a variable vs a collection) is now non-changeable (deeply immutable vs non-growable) for performance reasons.

[lrhn]

I'd be all for a way to select the kind of list you create.

The default is already available, but I'd want both unmodifiable and fixed-length.
Then I'd want Uint8List, Int8List, etc.
Plus unmodifiable versions of those.

Maybe we can use final [...] to define an unmodifiable list. I think that should be fine. (Works for sets and maps too).

Using the suffix idea, we can do [1, 2]u8 for an Uint8List and final [1, 2]u8 for an unmodifiable Uint8List.
Only thing missing is the modifiable fixed-length list. We could do [1, 2]f. Or [1, 2]a for "array".

Using a prefix would be nice because then we can also apply it to strings.
Consider final u8"Abracadabra\n" as a way to define an unmodifiable Uint8List containing the code points for "Abracadabra\n". Having the "u8" after the list, but before the string, is weird. But we already have r before the string.
(We can keep that, and put u8 after the string. In case you want to have r"a \$ sign"u8 as a list.)

Syntax is hard, let's do math!

[kevmoo]

Beat you by 3+ years, @srawlins - #117 😀

[kevmoo]

See also https://github.com/tc39/proposal-record-tuple. Their syntax is #[...] etc.

I'd take final [] too – just...gimme!!!

[mmcdon20]

Beat you by 3+ years, @srawlins - #117 😀

A non-growable list can still be mutable, so it is not quite the same request.

I would definitely like to see collection literal syntax (with collection-for, collection-if, spreads ...) work with a greater variety of collection types.

Iterable literals #1633 are another example where I think collection literal syntax would be nice.

[lrhn]

Generalizing collection literal syntax seems like a useful thing.
If we want generality, which is always a good thing in language feature design, we can't just use postfix "keywords" to hit a set of specific known types.

We'd need to open it up to any user collection class.

Say something like MyCollectionType{ elements }, which calls the the MyCollectionType constructor with ... something. And then maybe does something more. Maybe depending on the constructor's signature.
If (), call add on the result for each element, if (Iterable<X> elms), pass an iterable with the elements to the constructor, etc).

Then we can special-case the platform constructors to do things normal classes can't (or at least do it more efficiently, without having to go through an iterator). Which means List.unmodifiable{1, 2, 3} would work!

[srawlins]

I really like List.unmodifiable{1, 2, 3} which sort of begs the question of whether List.unmodifiable([1, 2, 3]) is already optimized (in the VM, say) to not allocate (and then GC) the list literal separate from the List.unmodifiable...

Another syntax idea: who says we can't have named arguments in [] 😁 ??? [1, 2, 3, unmodifiable: true] or [1, 2, 3, growable: false]. I'm not really advocating for those, but they sort of avoid new new syntax, they just add sort of new syntax.

[lrhn]

About "named parameters" in literals, that won't fly for unmodifiable set literals like {unmodifiable: true} ... which is a map literal.

[Jetz72]

I like the CollectionType{elements} approach, because it forms a nice conceptual parallel with the tagged strings proposal - "let this identifier handle the interpretation of this literal", and the current proposed syntax for that is an identifier directly followed by a string literal.

[gintominto5329]

Hello @lrhn,

Im unable to understand, that in the comment, does u8"str" mean utf8.encode("str"), or Uint8List.fromList("str".codeUnits), or some else,

Thanks

[gintominto5329]

Hello everyone,

I think:

1. A new fixed length list type, named Array, like in Java, should be added
2. List's growable: false parameter should be replaced, with Array([0, 1, 2]), and Array(3, 0)(like List.filled)
3. Uint8List(3), should be replaced withUint8List.filled(3, 0)(like List.filled)
4. Uint8List.fromList([0, 1, 2]), should be replaced withUint8List([0, 1, 2]), like the new Array([0, 1, 2])

Also, I think there is no real benefit in adding [0, 1, 2]u8 as a synonym for Uint8List.fromList([0, 1, 2]), in fact this would complicate, pollute, and add bloat, to the clean(till now, at least) syntax of dart,

The last point of syntax pollution, is actually very important now, as we see more and more request issues, to add useless features, with ugly syntax, to dart,

Just my opinion,
Thanks

[gintominto5329]

CollectionType{elements}

Please explain, CollectionType{elements} vs CollectionType(elements), other than adding, newer and newer, complications to dart's syntax,

I seek Peace,
Thanks,

[gintominto5329]

Regarding replacing

• Uint8List(3),withUint8List.filled(3, 0), and
• Uint8List.fromList([0, 1, 2]),withUint8List([0, 1, 2]),

I understand the cost/benefit ratio argument, for a critical breaking change, but:

• Uint8List(3) does not follow the List<int>.filled(3, 0) style, despite both being similar,
• Shouldn't syntax be Usability-first, as fromList constructor is far more useful, than filled(len, 0),

Thanks

[lrhn]

@gintominto5329

My idea with u8"string content" is that it would be a list literal for a Uint8List which contains the bytes of the content of that string.
I left it undefined what that means. There are two reasonable options, and I can't decide which one I like better:

1. It's the UTF-8 encoding of the string content, which can contain any Unicode code point other than unpaired surrogates. You specify characters and convert them to bytes. It's primarily a way to specify UTF-8 data directly as bytes.
2. It's the code points of the string which can only contain code points up to 255. You specify the bytes using acceptable characters. It's a primarily a way to specify bytes, which makes it easy to include the bytes of ASCII text.

In the latter case, u8"\x00\x01\x02\xff" contains the bytes 0, 1, 2 and 255.
In the former case, that \xff would be UTF-encoded as 0xC3 0xBF, and the list will have five elements.

Ok, I like 2. better. I don't need a way to specify UTF-8 bytes directly, but I do want a way to specify Uint8Lists.

(In either case, the string must be a constant, so it can only contain constant string interpolations.)

Also, about Uint8List vs. List.filled, maybe it's Uint8List which is doing the right thing here.
We're removing the List constructor in Dart 3.0 (has been unusable in new code inside 2.12).
I'd be happy to repurpose it at a later point, working like List.filled, so you do List<String>(3, ""). Then we can also add a fill value to Uint8List and do Uint8List(3, 42). That'd beconsistent and useful.

[gintominto5329]

From this issue(regarding Uint8Lists and List<int>s),

I think the implementation of these classes should be changed so that this documentation is true

This could be a good opportunity, to add Uint8List.filled(3, 255), without depreciating Uint8List(3), and also adding final <int>[0, 1, 2]( or maybe, also a new Array class),

Thanks

[Abion47]

I like the idea of having an Array class to logically separate non-growable "arrays" and growable "lists", since it would also bring Dart inline with many other typed languages that distinguish between the two. I also like that it would render the optional growable parameter redundant on many List constructors and methods since their use feels a bit cumbersome at times.

I'm not sure I like the idea that the only way to define a byte list literal would be with a string using ASCII code. It would end up being not very intuitive to use and only really sensible for situations where you're specifically using bytes in a string context. Any other situation would make it really undesirable in context, for example, if I had a byte array I wanted to use as a default header for generated WAV files:

final wavHeader = u8'RIFF\x14`(\x00WAVEfmt\x20\x10\x00\x00\x00\x01\x00\x01\x00"V\x00\x00D\xac\x00\x00\x02\x00\x10\x00data\xf0_(\x00';

If I ever wanted to change a couple of the bytes in the literal, it would be nontrivial to decipher that string to find the byte I wanted to modify and then know the exact ASCII character that I needed to change it to. On the other hand, a list in plain list literal syntax with either base10 or base16 integers is simple to understand and modify, even if it's not great on conciseness.

Having the u8[...] syntax for byte array literals would be interesting, but it also feels like the ambiguity with indexers would be irreconcilable:

final u8 = ['a', 'b' ,'c']; 
final foo = u8[1]; // array literal or indexer? 
// is `foo` inferred to be a String or a Uint8List?

I don't see getting around the issue that pushing this feature would require a breaking change that "u8" would become a reserved keyword in order to resolve this.

One possible solution that comes to mind is to use the type parameter to specify literals of various numerical type literals:

// possible solution?
final foo = <u8>[1]; 

It's unambiguous, but the problem now is that this would require all of the numeric "primitives" to have type names in the global space, increasing pollution and possibly breaking some codebases. It would also be confusing that you could use u8 in a type literal but only for this specific purpose, and you couldn't use it to define a byte variable like u8 b = 255;, though I suppose you could if u8 was simply a type alias for int, but then it could hold values outside of 0-255 which would be unexpected behavior.

Maybe just add all the other fully-featured numeric primitives themselves to Dart? Sounds simple enough. /s

[bernaferrari]

My biggest issue is that you do const list = []; list.add(1); and it crashes at runtime because it can't verify that an immutable list is mutable at compile.

I wish something like

abstract class BaseList {}
class List implements BaseList {}
class ImmutableList implements BaseList {}
class FixedList implements BaseList {} // or extends List

So ImuutableList doesn't have .add(), so const [] becomes type ImmutableList and doesn't fail at runtime.

[Reprevise]

My biggest issue is that you do const list = []; list.add(1); and it crashes at runtime because it can't verify that an immutable list is mutable at compile.

I wish something like

abstract class BaseList {}
class List implements BaseList {}
class ImmutableList implements BaseList {}
class FixedList implements BaseList {} // or extends List

So ImuutableList doesn't have .add(), so const [] becomes type ImmutableList and doesn't fail at runtime.

Perhaps inline classes could be used so we'd have:

inline class ImmutableList<T> {
    ImmutableList(this.list);

    final List<T> list;
}

This way, it'd be zero-cost and you wouldn't be able to call mutating methods on this ImmutableList object.

[bernaferrari]

I had a ton of bugs from JsonSerializable due to this, I never know if a parameter is mutable or immutable. Took me a lot of tries and a many bugs :(.

It is worth considering if this/other solutions would 'fix' JsonSerializable.

[Abion47]

My biggest issue is that you do const list = []; list.add(1); and it crashes at runtime because it can't verify that an immutable list is mutable at compile.

I wish something like

abstract class BaseList {}
class List implements BaseList {}
class ImmutableList implements BaseList {}
class FixedList implements BaseList {} // or extends List

So ImuutableList doesn't have .add(), so const [] becomes type ImmutableList and doesn't fail at runtime.

Interestingly, as far as I know, this is already how lists are represented internally, at least in the Dart VM. (Not sure about the web/JS side.) That said, exposing it might not be so trivial - I'd imagine if it was, they would've already done so a while ago.

That said, I can't imagine it would be that difficult to add a bool get modifiable to the base class that the other classes/interfaces/mixins can set to true/false.

I'm curious how a list's (im)mutability would pose a problem with JsonSerializable though. Typically a list wouldn't be modified during the serialization process and I can't think of a reason why it would be.

[bernaferrari]

I think they simplified because they wanted, but now it is time to unsimplify again.

The issue is that all classes need to have const default parameters (ideally), so if it is null it becomes const [], if it has a value, it gets the value. So sometimes you can call add, sometimes you can't. This frustrates me a lot.

[Wdestroier]

@bernaferrari Ohhh yes. It's annoying to have a runtime error when trying to add an element to an immutable List, it happens all the time in Java. I guess it's a better idea to have an Array class that is an immutable List, but a completely unrelated type; instead of trying to have the same name for classes that behave differently in a very special way.

[bernaferrari]

If const [] made ImmutableList instead of List I would be really happy already.

[Abion47]

In the meantime, this can be used as a workaround:

extension ImmutableListExtension<T> on List<T> {
  bool get isMutable {
    try {
      addAll(const []);
      return true;
    } catch (e) {
      return false;
    }
  }
}

It's not ideal since it relies on a try-catch, but AFAIK there is no way to know if an add operation will succeed except to just try it and see if it works.

---

EDIT: Wow, okay. I already said the workaround wasn't ideal, but if a better workaround exists, by all means, someone please post it. Otherwise, I don't understand the down thumbs seeing as people are literally explaining how this behavior is an impedance and frustration, and until an official method is introduced (which doesn't rely on mirrors since Flutter devs can't use that), there isn't really a better option than a stupid little try-catch wrapper.

I mean, sure, we can say it's better to refactor the code so it doesn't happen in the first place, but that's not something that can always be guaranteed when packages and third-party code come into play. So when code can completely break in an unintuitive and undetectable way simply due to a tiny change to how it's used...

// Imagine this class was buried in a package somewhere
class Foo {
  final List<int> list;

  Foo([this.list = const [0]]);
}

void main() {
  final a = Foo([0]);
  print(a.list);             // Prints "[0]"
  print(a.list.runtimeType); // Prints "List<int>"
  a.list.add(1);             // Completely fine

  final b = Foo();
  print(b.list);             // Prints "[0]"
  print(b.list.runtimeType); // Prints "List<int>"
  b.list.add(1);             // Throws a runtime error
}

...then there is a fundamental issue. Two lists in two instances of the same class that otherwise appear identical will behave differently at runtime, creating a time bomb that cannot possibly be checked for before it goes off.

There is no list.isGrowable. There is no list is ImmutableList. There is no list.tryAdd(..). Ultimately the only way to guard against this scenario is to wrap the whole thing in a try-catch and let it fail at runtime.

That isn't just problematic, it's borderline unacceptable.

[gintominto5329]

Hello everyone,

The new syntax, being proposed here, is:

• <int>[0, 1, 2] for growable list, same as current/present
• [NEW] final <int>[0, 1, 2] for non-growable/fixed-length list/array
• [NEW] const <int>[0, 1, 2] for non-modifiable/immutable list

Problem with the proposed syntax

The reserved words, final, and const act on the underlying Object(List<int> here), and have NO relation with the grow-ability or im-mutability of the Object, the proposed syntax seem like patchy work to me, and would definitely degrade the quality/logic in the syntax, and the trust of public, in dart, and its team

Solution

Once #1014 gets implemented, then we would be able to add the following:

• List<int>(0, 1, 2) for growable list,
• Array<int>(0, 1, 2) for non-growable/fixed-length list/array
• FirmList<int>(0, 1, 2) for non-modifiable/immutable list

ArrayList over Array, is possible, but more chars, and same for ImmutableList/UnmodifiableList over FirmList, maybe ImmuList/UnmodList could be used,

thanks

[gintominto5329]

I think #1014, is difficult to approve for dart team, as useless, in front of [], then i think the only option we have is to change the default of [], from growable, to non-growable,

thanks

[Abion47]

@gintominto5329

<int>[0, 1, 2] for growable list, same as current/present
[NEW] final <int>[0, 1, 2] for non-growable/fixed-length list/array
[NEW] const <int>[0, 1, 2] for non-modifiable/immutable list

The idea for final [...] is succinct and familiar, though I agree that it might be less clear that final means non-growable when some people might assume it means immutable. It's the most promising proposal mentioned so far, though, particularly because I like the idea of reserving list prefixes/suffixes for specifying literals of specific kinds of lists. (e.g. [...]u8 for a Uint8List.)

The note on const [...] isn't a proposal for a syntactic change since it's something you can already do today. Rather, it's a request to change the runtime type of a list declared with const to something that can be verified as immutable, i.e. an ImmutableList.

List(0, 1, 2) for growable list,
Array(0, 1, 2) for non-growable/fixed-length list/array
FirmList(0, 1, 2) for non-modifiable/immutable list

This would be a fundamental syntactic change in how lists are constructed in Dart, and personally, I'm against such a drastic change unless the Dart team had no other choice. Furthermore, it has already been said that a likely re-use of the soon-to-be-removed default List constructor is to remake it to behave like List.filled to bring it in line with the behavior of the typed lists.

I think #1014, is difficult to approve for dart team, as useless, in front of [], then i think the only option we have is to change the default of [], from growable, to non-growable,

I would hazard a guess that this change would break 90%+ of the code in packages or in production everywhere. It's a non-starter.

[gintominto5329]

I think,
<u8>[ ] or <byte>[ ] would be more logical than [ ]u8, and adding a u8 primitive type

Also I think, inference is the root cause of all the problems, here

If it weren't there, then we could do both, <String, Object>{}(map) and <int>{ }(non-growable Array), without conflict, and <int>[ ] could be kept reserved for grow-able lists,

thanks

[Abion47]

Personally, I like the idea of there being a byte/u8 primitive type, but adding an entirely new primitive type is probably a non-starter as well unless there was a very compelling reason (or reasons) to do so that goes well beyond the scope of just this ticket.

Also, <int>{...} is already in use as a Set literal. A non-growable array syntax that relied on bracket shape alone would need to use an unconventional character, e.g. <int> | ... | or <int>\ ... /, and I'm not a fan of that approach if only because it would add a lot of complexity for relatively little gain and may end up just being confusing to use.

[Reprevise]

We could do something like a function call syntax for it:

final fixedLengthList = [
  // ...
](growable: false);

[tatumizer]

Inventing a special ad-hoc syntax for the literals won't get you very far. There should be a systematic way of adding properties to the language constructs of any kind, be it literals, keywords or anything. The @annotaion syntax looks quite appropriate for that purpose. For some reason, dart continues to believe that the annotation cannot be allowed to change the semantics of a program, at the same time introducing annotation-based macros designed specifically with the goal of changing the semantics.
Without artificial self-imposed restrictions, we could write final list = @ng [1, 2, 3] or final x= @sync await myFuture, etc.

[Abion47]

@Reprevise

This syntax is problematic for many reasons, most notably because it would run afoul of Dart’s convention to make any type callable by adding a call method to the class. If this is the direction they are going, it would be better to just use the List.unmodifiable constructor in the first place. (And frankly, a compiler optimization on this syntax is probably the most likely solution, if somewhat of a letdown. If the compiler sees a List.unmodifiable call where the argument is a list literal, it could replace the entire call with a pre-calculated unmodifiable list value.)

@tatumizer

Implementing meta-programming isn’t a trivial undertaking, and the biggest reason the Dart team hasn’t worked on it thus far is because they had other higher priorities. They are working on it now to a degree, but it will likely be a very long time if ever before it can support what you’re suggesting.

(And as an irrelevant aside, I have no idea how @sync await myFuture would even work. Ignoring the fact that sync over async is nearly always a bad idea, that annotation would probably need to modify the entire function it’s in rather than just the statement it’s attached to. Otherwise, it would either be redundant [why use @sync in an async function?], unnecessary [just call myFuture.then], or it would risk deadlocking the thread.)

[tatumizer]

@Abion47:
for the context, see #3625, #726, #2033

My comment was about the general syntax of "annotations" (using the term loosely) modifying the meaning of keywords, literals, etc.
In this specific case, it could be @sync or @allowSync or some other word.