Optional Entries in Map Patterns

[ds84182]

Currently the specification for patterns states that a map that does not contain a key will not match. Unfortunately this means that patterns aren't able to be used to destructure JSON with optional fields.

Say we have the following schema: {"type": "location", "latlon": [number, number], "name": string?, "visits": int (defaults to 0 if unspecified)}

Matching this is not possible due to the optional fields:

json = {"type": "location", "latlon": [123.4, -56.78]};
return switch (json) {
  case {"type": "location", "latlon": [lat as num, lon as num], "name": name as String?, "visits": visits as int?} => ...
};

Perhaps a ? suffix could be used on the map key to indicate that it should be optional?

[ykmnkmi]

maybe put ? after or before keys:

return switch (json) {
  case {
    "type": "location",
    "latlon": [lat as num, lon as num],
    "name"?: name as String?,
    "visits"?: visits as int?,
    } {
      ...
    }
};

[lrhn]

I can see the use-case in destructuring, but it's not working very well with pattern matching.

A pattern match will either succeed, and bind variables, or reject, and not.
The "name"?: var name as String? map-entry pattern, as described, is irrefutable, but it does not guarantee to bind the name variable. That means it can't actually work, because the var name pattern must be part of the accepting match in order for the variable to be accessible afterwards.

What we have here is more like an optional declaration, which ... I guess could make sense, since it can be initialized to null.
(Could variable patterns have default values, which are applied if the pattern matches without them, like:

   case int i = 0 | String s = "": print("$i: $s");

With the current rules, the variables on each side of an | must be the same. With this, a non-participating variable declaration can still have a value. Syntax might be a little confusing.

Since the pattern it's irrefutable, you don't need it as part of the match, and you can just do:

  var name = json["name"] as String?;

in the body of the switch.
But that takes away some of the convenience of destructuring.

For a plain destructuring assignment, it would make a kind of sense to allow:

var {"type": var type as String,
       "latlon": [var lat as num, var lon as num],
       "name"?: var name = null as String?,
       "visits"? : var visits = 0 as int?
 } = json;

(Not sure if this brings us closer to parameter lists with optional parameters, or if it diverges in a different direction.)

[munificent]

I'm definitely not sold on default values for patterns. That starts to get really hard for me to follow.

Optional entries in map patterns is a pretty interesting idea, though. I think what we'd say is that the matched value type of the optional entry's subpattern is V?, and runtime semantics are:

1. Call containsKey(k).
2. If it returns false, then match the subpattern against null.
3. Else, call v[k] and match the subpattern against the result.

In other words, the entry itself always matches, but the value sent to the subpattern may be null or an actual value. That in turn suggests that you could use these in irrefutable contexts to avoid throwing a runtime exception if the key is absent:

var map = <String, String>{};

var {'x': a} = map; // Throws.
var {'x'?: b} = map; // Fine. `b` has type `String?` and contains `null`.

It's an interesting idea. I suspect we shouldn't put it in the initial release because I want to see how much user demand there is for it first. We already use ? for so many things that I'm very hesitant to add another use without knowing for certain that it's value, but it's a very nice suggestion.

[ds84182]

Coming back to this with a snippet from Discord's documentation:

Specifically, this uses a similar syntax to the one proposed. There are optional keys AND optional values.

Current option is to do something like:

typedef JsonMap = Map<String, Object?>;

extension on JsonMap {
  Object? get snowflake => this['snowflake'];
  Object? get name => this['name'];
  Object? get animated => this['animated'];
}

// ...

Emoji? parseEmoji(Object? json) =>
  switch (json) {
    case JsonMap(:int id, :String? name, :bool? animated): CustomEmoji(id: id, name: name, animated: animated ?? false),
    case JsonMap(id: null, :String name): UnicodeEmoji(name: name),
    default: null,
  };

when ideally I'd love to be able to skip the extension getters:

Emoji? parseEmoji(Object? json) =>
  switch (json) {
    case {"id": int id, "name": String? name, "animated"?: bool animated, ...}: CustomEmoji(id: id, name: name, animated: animated ?? false),
    case {"id": null, "name": String name, ...}: UnicodeEmoji(name: name),
    default: null,
  };

or if you could call operator[] with an extractor pattern:

Emoji? parseEmoji(Object? json) =>
  switch (json) {
    case JsonMap(['id']: int id, ['name']: String? name, ['animated']: bool? animated): CustomEmoji(id: id, name: name, animated: animated ?? false),
    case JsonMap(['id']: null, ['name']: String name): UnicodeEmoji(name: name),
    default: null,
  };

or, another alternative using records. still not ideal:

Emoji? parseEmoji(Object? json) =>
  json is! JsonMap ? null : switch ((json["id"], json["name"], json["animated"])) {
    case (int id, String? name, bool? animated): CustomEmoji(id: id, name: name, animated: animated ?? false),
    case (id: null, :String name, ...): UnicodeEmoji(name: name),
    default: null,
  };

[lrhn]

At this point, I'd probably just go with;

switch (json) {
  case {"id": int id, "name": String? name}: 
    var (bool animated, bool managed, bool available, ...) = 
      (json["animated"] ?? false, json["managed"] ?? false, json["available"] ?? false, ...);
   ...
}

We don't allow [...] as an extractor pattern selector (yet!), but if we do at some point, we can change that to:

switch (json) {
  case {"id": int id, "name": String? name} &&
    Map(["animated"]: bool? animated, ["managed"]: bool? managed, ["available"]: bool? available, ...):
       ...
   ...
}

Default values do make a kind of sense. We have ? which only matches if non-null, we could also allow ?? constantValue which replaces null with a value and then matches. Probably don't want a default value which fails a later test, so it should only be applicable to binding patterns.

Nullability, the gift that keeps giving. :)

[supposedlysam-bb]

I started converting models over to pattern matching destructuring for JSON results and I personally expected for Maps with a missing key to be set to null when using nullable syntax.

After getting the "Bad state: Pattern matching error" back, the first thing I tried was adding a question mark after the string literal but before the colon like @munificent suggested in his previous post.

Attempt 1

"myKeyValue": String? myVariable

Attempt 2

"myKeyValue"?: String? myVariable

[natebosch]

For JSON models with optional keys we have to fall back on jsonMap['maybeKey'] today. The new syntax is nicer to use for getting Dart structures out of JSON, but it's a large edit I need to make when I find out the keys I expected were always present can be omitted.

[mmcdon20]

Consider the following example:

Future<Map<String, String>> nameService(int id) async => switch (id) {
      1 => {'first': 'Mike', 'last': 'Jones'},
      2 => {'first': 'Kevin', 'last': 'Smith', 'middle': 'Jacob'},
      3 => {'first': 'Mary', 'last': 'Davis', 'title': 'Dr'},
      _ => throw Exception('unknown id'),
    };

void main() async {
  final name = await nameService(1);

  final {
    'first': String first,
    'last': String last,
    'middle': String? middle,
    'title': String? title,
  } = name;

  print('$title $first $middle $last');
}

Since we can not specify that 'middle' and 'title' keys are optional we get the following error:

Unhandled exception:
Bad state: Pattern matching error

In order to get it working you have to rewrite to not use map patterns:

Future<Map<String, String>> nameService(int id) async => switch (id) {
      1 => {'first': 'Mike', 'last': 'Jones'},
      2 => {'first': 'Kevin', 'last': 'Smith', 'middle': 'Jacob'},
      3 => {'first': 'Mary', 'last': 'Davis', 'title': 'Dr'},
      _ => throw Exception('unknown id'),
    };

void main() async {
  final name = await nameService(1);

  final String first = name['first']!;
  final String last = name['last']!;
  final String? middle = name['middle'];
  final String? title = name['title'];

  print('$title $first $middle $last');
}

Having a syntax for optional keys would allow us to express the following:

final {
  'first': String first,
  'last': String last,
  'middle'?: String? middle,
  'title'?: String? title,
} = name;

[TekExplorer]

I'd prefer the question mark at the front, so it's immediately clear without needing to find the end of the key, which may be long. also nicer to have it lined up.

[munificent]

This is still a feature we could consider doing, but note that the upcoming null-aware elements feature does sit on some of the potential syntactic space we might want to use.

The ? operator is getting really heavily overloaded.

[mmcdon20]

IMO without this feature, I would recommend to people to simply never use map patterns in an irrefutable context like a variable destructuring assignment. You inevitably run into cases where you cannot express it properly, and then you have to rewrite it. You are better off not using the feature to begin with to avoid rewriting it later.