[Proposal]: Dictionary expressions

[CyrusNajmabadi]

Dictionary Expressions

• Proposed
• Prototype: Not Started
• Implementation: Not Started
• Specification: Not Started

Summary

Collection Expressions were added in C# 12. They enabled a lightweight syntax [e1, e2, e3, .. c1] to create many types of linearly sequenced collections, with similar construction semantics to the existing [p1, p2, .. p3] pattern matching construct.

The original plan for collection expressions was for them to support "dictionary" types and values as well. However, that was pulled out of C# 12 for timing reasons. For C# 13 we would to bring back that support, with an initial suggested syntax of [k1: v1, k2: v2, .. d1]. As a simple example:

private readonly Dictionary<string, int> _nameToAge = [
    "Cyrus": 21,
    "Dustin": 22,
    "Mads": 23,
];

The expectation here would be that this support would closely track the design of collection expressions, with just additions to that specification to support this k:v element syntax, and to support dictionary-like types as the targets of these expressions.

Motivation

• Collection-like values are hugely present in programming, algorithms, and especially in the C#/.NET ecosystem. Nearly all programs will utilize these values to store data and send or receive data from other components. Currently, almost all C# programs must use many different and unfortunately verbose approaches to create instances of such values.

• An examination of the ecosystem (public repos, nuget packages, and private codebases we have access to), indicate that dictionaries are used as a collection type in APIs around 15% of the time. While not nearly as ever-present as the sequenced collections (like arrays, spans, lists, and so on), this is still substantial, and warrants the equivalently pleasant construction provided by collection-expressions.

• Like with linear collections, there are numerous different types of dictionary-like types in the .net ecosystem. This includes, but is not limited to, simply constructed dictionaries like Dictionary<TKey, TValue> and ConcurrentDictionary<TKey, TValue>, but also interfaces like IDictionary<TKey, TValue> and IReadOnlyDictionary<TKey, TValue>, as well as immutable versions like `ImmutableDictionary<TKey, TValue>. Supporting all these common forms is a goal for these expressions.

Detailed design

Main specification here: https://github.com/dotnet/csharplang/blob/main/proposals/collection-expressions-next.md

The only grammar change to support these dictionary expressions is:
 

collection_literal_element
  : expression_element
+ | dictionary_element
  | spread_element
  ;

+ dictionary_element
  : expression ':' expression
  ;

Spec clarifications

• dictionary_element instances will commonly be referred to as k1: v1, k_n: v_n, etc.

Conversions

The following implicit collection literal conversions exist from a collection literal expression:

• ...

• To a type that implements System.Collections.IDictionary where:

  • The type contains an applicable instance constructor that can be invoked with no arguments or invoked with a single argument for the 0-th parameter where the parameter has type System.Int32 and name capacity.
  • For each expression element Ei:
    • the type of Ei is dynamic and there is an applicable indexer setter that can be invoked with two dynamic arguments, or
    • the type of Ei is a type System.Collections.Generic.KeyValuePair<Ki, Vi> and there is an applicable indexer setter that can be invoked with two arguments of types Ki and Vi.
  • For each dictionary element Ki:Vi, there is an applicable indexer setter that can be invoked with two arguments of types Ki and Vi.
  • For each spread element Si:
    • the iteration type of Si is dynamic and there is an applicable indexer setter that can be invoked with two dynamic arguments, or
    • the iteration type is System.Collections.Generic.KeyValuePair<Ki, Vi> and there is an applicable indexer setter that can be invoked with two arguments of types Ki and Vi.

• To an interface type I<K, V> where System.Collections.Generic.Dictionary<TKey, TValue> implements I<TKey, TValue> and where:

  • For each expression element Ei, the type of Ei is dynamic, or the type of Ei is a type System.Collections.Generic.KeyValuePair<Ki, Vi> and there is an implicit conversion from Ki to K and from Vi to V.
  • For each dictionary element Ki:Vi there is an implicit conversion from Ki to K and from Vi to V.
  • For each spread element Si, the iteration type of Si is dynamic, or the iteration type is System.Collections.Generic.KeyValuePair<Ki, Vi> and there is an implicit conversion from Ki to K and from Vi to V.

Syntax ambiguities

• dictionary_element can be ambiguous with a conditional_expression. For example:

  var v = [ a ? [b] : c ];

  This could be interpreted as expression_element where the expression is a conditional_expression (e.g. [ (a ? [b] : c) ]). Or it could be interpreted as a dictionary_element "k: v" where a?[b] is k, and c is v.

Alternative designs

Several discussions on this topic have indicated a lot of interest and enthusiasm around exploring how close this feature is syntactically (not semantically) to JSON. Specifically, while we are choosing [k: v] for dictionary literals, JSON chooses { "key": value }. As "key" is already a legal C# expression, this means that [ "key": value ] would be nearly identical to JSON (except for the use of brackets instead of braces). While it would make it so we would have two syntaxes for collection versus dictionary expressions, we should examine this space to determine if the benefits we could get here would make up for the drawbacks.

Specifically, instead of reusing the collection_expression grammar, we would introduce:

primary_no_array_creation_expression
  ...
+ | dictionary_expression
  ;

+ dictionary_expression
  : '{' '}'
  | '{' dictionary_element ( ',' dictionary_element )* '}'
  ;

+ dictionary_element
  : expression_element
  | dictionary_element
  | spread_element
  ;

+ dictionary_element
  : expression ':' expression
  ;

You could then write code like:

private readonly Dictionary<string, int> _nameToAge = {
    "Cyrus": 21,
    "Dustin": 22,
    "Mads": 23,
};

Or:

Dictionary<int, string> combined = { .. ageToName1, .. ageToName2 };

Or

Dictionary<int, string> combined = { kvp1, key2: value2, .. ageToName1, .. ageToName2 };

The downside here is:

1. The potential ambiguity with existing constructs.
2. The potential conflicts with future constructs.
3. The challenge in creating a corresponding pattern form.

In order:

First, the above syntax already conflicts with int[] a = { 1, 2, 3, 4 } (an array initializer). However, we could trivially sidestep this by saying that if the initializer contained a spread_element or dictionary_element it was definitely a dictionary. If it did not (it only contains normal expressions, or is empty), then it will be interpreted depending on what the target type is.

Second, this could definitely impact future work wanted in the language. For example, "block expressions" has long been something we've considered. Where one could have an expression-valued "block" that could allow statements and other complex constructs to be used where an expression is needed. That said, such future work is both speculative, and itself could take on some new syntax. For example, we could mandate that an expression block have syntax like @{ ... }.

Third, the pattern form here presents probably the hardest challenges. We would very much like patterns and construction to have a syntactic symmetry (with patterns in the places of expressions). Such symmetry would motivate having a pattern syntax of { p1: p2 }. However, this is already completely the completely legal pattern syntax for a property pattern. In other words, one can already write if (d is { Length: 0 }). Indeed, it was all the ambiguities with { ... } patterns in the first place that motivated us to use [ ... ] for list patterns (and then collection-expressions). We will end up having to resolve these all again if we were to want this to work. It is potentially possible, but will likely be quite subtle with various pitfalls. Alternatively, we can come up with some new syntax for dictionary-patterns, but we would then break our symmetry goal.

Regardless, even with the potential challenges above, there is a great attractiveness on aligning syntactically with JSON. This would make copying JSON into C# (particularly with usages in API like Json.net and System.Text.Json) potentially no work at all. However, we may decide the drawbacks are too high, and that it is better to use the original syntactic form provided in this specification.

That said, even the original syntactic form is not without its own drawbacks. Specifically, if we use [ ... ] (just with a new element type, then we will likely have to address ambiguities here when we get to the "natural type" question.

For example:

var v1 = []; // What if the user wants a dictionary?
var v2 = [.. dict1, .. dict2]; // should combining dictionaries produce a dictionary?

and so on.

Open Questions

1. Should dictionary expressions have "Add" semantics or "Overwrite semantics". The working group is leaning toward the latter, specifically so you can write code like:

Dictionary<TKey, TValue> updated = [.. initialValues, k: v]; //or
Dictionary<TKey, TValue> merged = [.. d1, .. d2]; //or
Dictionary<TKey, TValue> overwritten = [k1: v1, k2: v2, .. augments]; // etc.

The space of allowing merging, with "last one wins" seems pretty reasonable to us. However, we want a read if people would prefer that throw, or if there should be different semantics for dictionaries without spreads for example.

2. Unlike linear-collections, augmenting a dictionary is a much more common practice. For example, providing a suitable IEqualityComparer. Should that be something supported somehow in this syntax. Or would/should that require falling back out to a normal instantiation?

Design Meetings

• https://github.com/dotnet/csharplang/blob/main/meetings/2024/LDM-2024-03-11.md#dictionary-expressions
• https://github.com/dotnet/csharplang/blob/main/meetings/2024/LDM-2024-05-15.md#dictionary-expressions
• https://github.com/dotnet/csharplang/blob/main/meetings/2024/LDM-2024-09-06.md#dictionary-expressions

[KennethHoff]

As noted by open question 2, I think losing support for IEqualityComparer - especially for string dictionaries - would be very unfortunate. For me personally equality comparisons is one of the primary reasons I use dictionaries over other data structures.

[HaloFour]

Would an extension method not suffice here?

var dictionary = [ "foo": "bar", "fizz": "buzz" ].WithComparer(StringComparer.OrdinalIgnoreCase);

// or

var dictionary  = StringComparer.OrdinalIgnoreCase.CreateDictionary([ "foo": "bar", "fizz": "buzz" ]);

[GabeSchaffer]

Would an extension method approach to specifying comparers be able work without creating a second dictionary?

[CyrusNajmabadi]

@GabeSchaffer Likely yes (though we would still need to get extensions working with collection exprs). Specifically, the keys/values in the expr woudl likely be passed to the extension method as a ReadOnlySpan of KeyValuePairs. These would normally just be on the stack, and would be given as a contiguous chunk of data for the dictionary to create its internal storage from once.

[GabeSchaffer]

Apologies for my lack of understanding, but it seems like a builder approach to augmentation could be made to work, like with String.Create.

That seems like it could make a syntax like this possible:

var dictionary = Dictionary.Create(StringComparer.OrdinalIgnoreCase, [ "foo": "bar", "fizz": "buzz" ]);

Another options that seems feasible is to allow constructor arguments to be specified in parens after the collection expression:

[ "foo": "bar", "fizz": "buzz" ](StringComparer.OrdinalIgnoreCase) // function call syntax for passing ctor args

An uglier possibility is to use a semicolon:

[ "foo": "bar", "fizz": "buzz"; StringComparer.OrdinalIgnoreCase ] // ; delimits comparer

I think if we want to have hope of being able to specify a comparer in a pattern, the latter two are better.

[Perksey]

Wow I had no idea the C# team were so young ;)

private readonly Dictionary<string, int> _nameToAge = [
    "Cyrus": 21,
    "Dustin": 22,
    "Mads": 23,
];

[hez2010]

How about reusing the existing dictionary syntax?

Given that we already have

var dict = new Dictionary<K, V> { [key] = value }

We can somehow use a similar syntax:

var dict = { [key]: value }

This is also matching how TypeScript defines an interface with indexed field:

interface Foo {
    [key: string]: number;
}

And we can even support dictionary patterns along with existing patterns as well:

var obj = new C();

if (obj is {
    [string key]: 42,
    ["foo"]: int value,
    SomeProp: float x
}) ... // matching an object which has a indexed value 42 while its key is string, and a key "foo" whose value is int

class C
{
    public float SomeProp { get; }
    public int this[string arg] => ...
}

As well as the case where the indexer has multiple parameters:

if (obj is {
    ["foo", "bar"]: 42
})

[TahirAhmadov]

Specifically, the keys/values in the expr woudl likely be passed to the extension method as a ReadOnlySpan of KeyValuePairs.

What about spreads, those will have to be iterated and placed on the stack before calling the extension method? Or is there a way to pass them in somehow?

[GabeSchaffer]

Specifically, the keys/values in the expr woudl likely be passed to the extension method as a ReadOnlySpan of KeyValuePairs.

What about spreads, those will have to be iterated and placed on the stack before calling the extension method? Or is there a way to pass them in somehow?

I don't see how that can be implemented in any way other than to copy every element into a contiguous array in order to create a ReadOnlySpan. Because if the elements aren't contiguous, it's not a span, right?

[mpawelski]

@hez2010

How about reusing the existing dictionary syntax?

Given that we already have

var dict = new Dictionary<K, V> { [key] = value }

We can somehow use a similar syntax:

var dict = { [key]: value }

I think dictionary-like objects that have only one indexer are so common that it's worth to consider special, more succinct, syntax that doesn't require unnecessary [ and ]. Such as the one proposed in this thread.

Actually, your proposed syntax is not far from what we can do now in C#:

Dictionary<string, int> dict = new() { 
    ["aaa"] = 1, 
    ["bbb"] = 1
};

And we can even support dictionary patterns along with existing patterns as well:

var obj = new C();

if (obj is {
    [string key]: 42,
    ["foo"]: int value,
    SomeProp: float x
}) ... // matching an object which has a indexed value 42 while its key is string, and a key "foo" whose value is int

class C
{
    public float SomeProp { get; }
    public int this[string arg] => ...
}

As well as the case where the indexer has multiple parameters:

if (obj is {
    ["foo", "bar"]: 42
})

Actually I was kinda surprised that C# doesn't have pattern matching for indexers. This code doesn't work:

Dictionary<string, int> dict = new() { 
    ["aaa"] = 1, 
    ["bbb"] = 2
};

if (dict is { ["aaa"] : 1 })
{
}

But this feels looks like another "pattern matching improvements" feature, not strictly related to dictionary expressions we discuss in this thread.

[scarletquasar]

I suggest using object notation, like TypeScript.

Like:

{
    Type = "Person",
    Name = "John"
}

[TahirAhmadov]

Can we make the "add/overwrite" be switchable? Say, we default to the safe "add" approach, and if overwrite is desired:

var d = [(k1: v1)!, (..d1)!];

Here, ! means "overwrite".
Frankly, I'm not crazy about this syntax, but I couldn't come up with anything better at the moment (admittedly, I thought about it for a few minutes only).
However, syntax aside, I'd still like to raise this as a suggestion.

PS. In addition to the above, if it should be "overwrite" for the whole list:

var d = ([k1: v1, ..d1])!;

[colejohnson66]

That syntax looks a bit weird. If an "or overwrite" operation was added, I'd prefer to augment the with syntax:

IDictionary<string, string> d = GetDictionary();
(string k, string v) = GetNewKVP();
IDictionary<string, string> d2 = d with { [k1]: v1 };

But maybe that would be confusing, as one would have to remember that spreading is adding and with is overwrite (or add if missing).

[TahirAhmadov]

@colejohnson66 with your approach, you would always have to create a temp dictionary variable to then "overwrite" things onto it to produce the final desired result, which effectively changes it from a "collection expression" to a "procedural code block".

[alrz]

You can do this today

Dictionary<string, int> _nameToAge = new(OptionalComparer) {
    ["Cyrus"] = 21,
    ["Dustin"] = 22,
    ["Mads"] = 23,
};

or just [new(key ,value)] given an extension Add(this Dictionary, KeyValuePair).

I'd rather see dictionary/indexer patterns to complete the matching side of the existing [e]=v initialization syntax. Meanwhile, I think stronger type inference could help a lot within the existing syntax for more concise maps.

var x = new Dictionary() { .. };

There's endless optimization possibilities for collection expressions, not so much with dictionaries, and spreads with maps are just confusing or so rare at best.

Having said that, I think immutable dictionaries could use some less awkward initialization API, if possible.

[CyrusNajmabadi]

I don't quite follow how a major pillar in the design is to actively avoid using orthogonal syntax

A core virtue here is brevity. We already have verbose options available to us today:-)

I fail to see the appeal of picking the latter.

Only if they are truly equal on every other dimension. That remains to be seen. A better syntax, for collections only, would likely still be preferred.

is my suggestion that less intuitive or harder to use

To me, it is. Especially given that we already have a new(...) form. Now we'd have new:(...) form as well. This is not a positive for me.

[julealgon]

I fail to see the appeal of picking the latter.

Only if they are truly equal on every other dimension. That remains to be seen. A better syntax, for collections only, would likely still be preferred.

Fair.

is my suggestion that less intuitive or harder to use

To me, it is. Especially given that we already have a new(...) form. Now we'd have new:(...) form as well. This is not a positive for me.

Oh, just to be clear, I was asking that question when comparing it to @TahirAhmadov's original proposed syntax, not in general. I also dislike the new: syntax. The point was that "if you are doing it that way, at least move the elements outside of the array literal so it could be reused for other use cases later".

[tzographos]

Currently when we want to create e.g. a Dictionary<string, double> we do something like this:

Dictionary<string, double> airComponents = new(capacity, comparer) {
        {  "CO", 10.0 },
        {  "CO2", 200.0 },
}

How will we be able to utilize capacity and comparer parameters in the proposed syntax?

Something like this maybe in order to be as close to the existing collection initialization?

Dictionary<string, double> airComponents = [
    new("CO",  10.0),
    new("CO2", 200.0)
].WithCapacity(...).WithComparer(....)

[TahirAhmadov]

@tzographos see these:
#7822 (comment)
#7822 (comment)
It's based on a proposed syntax I noticed in LDM notes but I can't find that link now.

[slang25]

Given that there is no branch in Roslyn for this (that I can see), is it safe to say this won't be a C# 13 feature?

[333fred]

Correct.

[makeman-p81]

Not suitable for practical usage, of course, but there is an another unsafe syntactically correct way of dictionary creation via anonymous type and reflection

var dictionary = new
{
    Cyrus = 21,
    Dastin = 22,
    Mads = 23,
}
.ToDictionary<int>();

Console.WriteLine(dictionary["Cyrus"]); // 21

public static class Ext
{
    public static Dictionary<string, TValue> ToDictionary<TValue>(this object item) => item
        .GetType()
        .GetProperties()
        .ToDictionary(p => p.Name, p => (TValue)p.GetValue(item))
        ;
}

So, there are possible relaxations of syntax

var dictionary = new Dictionary<string, int>
{
    Cyrus = 21, // instead: ["Cyrus"] = 21,
    Dastin = 22,  // instead: "Dastin" = 21,
    "Mads" = 23, // allowed
    "Key With Spaces" = 24,
};

Not sure that it is a value proposal, but may be considered for fun.

[Atulin]

Not sure about yet another notation for dictionaries, to be honest. We already have two:

var dict = new Dictionary<string, int>() {
    { "one", 1 },
    { "two", 2 },
    { "three", 3 },
};

and

var dict = new Dictionary<string, int>() {
    ["one"] = 1,
    ["two"] = 2,
    ["three"] = 3,
};

and the proposal would add a third notation?

As much as I am a fan of adding some collection literal syntax to dictionaries, I'd rather reuse one of the existing notations, ideally the latter:

new Dictionary<string, int> dict = [
    ["one"] = 1,
    ["two"] = 2,
    ["three"] = 3,
];

[CyrusNajmabadi]

We already have two:

This is not two syntaxes. Those are the existing syntaxes for Add vs Indexing. They serve different purposes.

[CyrusNajmabadi]

and the proposal would add a ... notation?

Yes. Like with collection expressions, the point would be to be terse. If it's not terse, you can use the existing syntax.

[ ["one"] = 1

This is syntactically ambiguous, and it needs two more tokens than the simpler k:v syntax.

Brevity was a primary goal of collection expressions. We don't want to roll that back extending to the simple case of dictionaries.

[Atulin]

This is not two syntaxes. Those are the existing syntaxes for Add vs Indexing. They serve different purposes.

They both serve the same purpose of initializing a dictionary literal. Sure, technically one of them compiles down to .Add("x", 0) and another to ["x"] = 0;, but their purpose is the exact same.

You're right that k:v is more terse that [k]=v or {k,v}, there's no doubt about it. I'd simply rather not have a notation that's inconsistent with either of the two existing ones. It's like, I don't know, having a collection expression use | for item separator instead of ,.

It's ultimately no big deal, and I'll use whatever the syntax ends up being, but I still wanted to throw in my two cents.

[hez2010]

I would still like to resolve this in a more general way: instead of dictionary expressions which only treat Dictionary specially, we introduce "indexable expressions", and particularly, Dictionary is an indexable object which only takes one parameter as key.

Then we use the following syntax:

T obj = {
    [key, ...]: value
}

For dictionary or any type which has an indexer with only 1 param set_Item(arg1), we can write the expression like:

Foo1 obj = {
    ["foo"]: "bar"
}

class Foo1
{
    public string this[string key] { set => ... }
}

And for types have an indexer with 2 params set_Item(arg1, arg2), we can write the expression like:

Foo2 obj = {
    ["foo", 42]: "bar"
}

class Foo2
{
    public string this[string key1, int key2] { set => ... }
}

This will allow us to use arbitrary number of keys.

[AArnott]

Unlike linear-collections, augmenting a dictionary is a much more common practice. For example, providing a suitable IEqualityComparer. Should that be something supported somehow in this syntax. Or would/should that require falling back out to a normal instantiation?

Some linear collections do take comparers. e.g. SortedSet<T> which has no keys, but has a Comparer<T> anyway, with a default constructor and one that takes the Comparer<T>.

So whatever syntax you may come up with for maps, ideally it would work for vectors too.

[CyrusNajmabadi]

Unlike linear-collections, augmenting a dictionary is a much more common practice. For example, providing a suitable IEqualityComparer. Should that be something supported somehow in this syntax. Or would/should that require falling back out to a normal instantiation?

Some linear collections do take comparers. e.g. SortedSet<T> which has no keys, but has a Comparer<T> anyway, with a default constructor and one that takes the Comparer<T>.

So whatever syntax you may come up with for maps, ideally it would work for vectors too.

The latest proposal spec makes this point as well. It's something we will discuss.

[alrz]

From: https://github.com/dotnet/csharplang/blob/main/proposals/collection-expression-arguments.md

HashSet<string> s = [...] with ...;

I remember there was a request to specify the comparer on switch as well (I couldn't find the issue/discussion on that unfortunately),

switch (e) with (StringComparer.OrdinalIgnoreCase) {..}

map ??= new Dictionary<string /*constant pattern*/, int /* case index */>(comparer) {..};
if (map.TryGetValue(e, out var index)) /* jump to section */ else /* jump to default */;

Could that possibly extended to switch?