Proposed changes for Pattern Matching in C# 9.0 - Draft Specification

[gafter]

We are considering a small handful of enhancements to pattern-matching for C# 9.0 that have natural synergy and work well to address a number of common programming problems:

• Champion: Type Patterns (VS 16.8, .NET 5) #2925 Type patterns
• Champion "and, or, and not patterns" (VS 16.8, .NET 5) #1350 Parenthesized patterns to enforce or emphasize precedence of the new combinators
• Champion "and, or, and not patterns" (VS 16.8, .NET 5) #1350 Conjunctive and patterns that require both of two different patterns to match;
• Champion "and, or, and not patterns" (VS 16.8, .NET 5) #1350 Disjunctive or patterns that require either of two different patterns to match;
• Champion "and, or, and not patterns" (VS 16.8, .NET 5) #1350 Negated not patterns that require a given pattern not to match; and
• Proposal: allow comparison operators in switch case labels C# (VS 16.8, .NET 5) #812 Relational patterns that require the input value to be less than, less than or equal to, etc a given constant.

Parenthesized Patterns

Parenthesized patterns permit the programmer to put parentheses around any pattern. This is not so useful with the existing patterns in C# 8.0, however the new pattern combinators introduce a precedence that the programmer may want to override.

primary_pattern
    : parenthesized_pattern
    ;
parenthesized_pattern
    : '(' pattern ')'
    ;

Relational Patterns

Relational patterns permit the programmer to express that an input value must satisfy a relational constraint when compared to a constant value:

    public static LifeStage LifeStageAtAge(int age) => age switch
    {
        < 0 =>  LiftStage.Prenatal,
        < 2 =>  LifeStage.Infant,
        < 4 =>  LifeStage.Toddler,
        < 6 =>  LifeStage.EarlyChild,
        < 12 => LifeStage.MiddleChild,
        < 20 => LifeStage.Adolescent,
        < 40 => LifeStage.EarlyAdult,
        < 65 => LifeStage.MiddleAdult,
        _ =>    LifeStage.LateAdult,
    };

We imagine supporting <, <=, >, and >= patterns on all of the built-in types that support such binary relational operators with two operands of the same type in an expression. Specifically, we support all of these relational patterns for sbyte, byte, short, ushort, int, uint, long, ulong, char, float, double, and decimal.

primary_pattern
    : relational_pattern
    ;
relational_pattern
    : '<' expression
    | '<=' expression
    | '>' expression
    | '>=' expression
    ;

The expression is required to evaluate to a constant value. It is an error if that constant value is double.NaN or float.NaN. It is an error if the expression is a null constant and the relational operator is <, <=, >, or >=.

When the input is a type for which a suitable built-in binary relational operator is defined that is applicable with the input as its left operand and the given constant as its right operand, the evaluation of that operator is taken as the meaning of the relational pattern. Otherwise we convert the input to the type of the expression using an explicit nullable or unboxing conversion. It is a compile-time error if no such conversion exists. The pattern is considered not to match if the conversion fails. If the conversion succeeds then the result of the pattern-matching operation is the result of evaluating the expression e OP v where e is the converted input, OP is the relational operator, and v is the constant expression.

Open Issue: Should the expression be shift_expression to have precedence corresponding to a relational expression?

Pattern Combinators

Pattern combinators permit matching both of two different patterns using and (this can be extended to any number of patterns by the repeated use of and), either of two different patterns using or (ditto), or the negation of a pattern using not.

I expect the most common use of a combinator will be the idiom

if (e is not null) ...

More readable than the current idiom e is object, this pattern clearly expresses that one is checking for a non-null value.

The and and or combinators will be useful for testing ranges of values

bool IsLetter(char c) => c is >= 'a' and <= 'z' or >= 'A' and <= 'Z';

This example illustrates our expectation that and will have a higher parsing priority (i.e. will bind more closely) than or. The programmer can use the parenthesized pattern to make the precedence explicit:

bool IsLetter(char c) => c is (>= 'a' and <= 'z') or (>= 'A' and <= 'Z');

Like all patterns, these combinators can be used in any context in which a pattern is expected, including nested patterns, the is-pattern-expression, the switch-expression, and the pattern of a switch statement's case label.

pattern
    : disjunctive_pattern
    ;
disjunctive_pattern
    : disjunctive_pattern 'or' conjunctive_pattern
    | conjunctive_pattern
    ;
conjunctive_pattern
    : conjunctive_pattern 'and' negated_pattern
    | negated_pattern
    ;
negated_pattern
    : 'not' negated_pattern
    | primary_pattern
    ;
primary_pattern
    : // all of the patterns forms previously defined
    ;

Open Issues with Proposed Changes

Syntax for relational operators

Are and, or, and not some kind of contextual keyword? If so, is there a breaking change (e.g. compared to their use as a designator in a declaration-pattern).

Should we support some combination of declaration pattern along with a relational pattern? For example,

if (o is int x <= 100) // x is an int with value < 100 here.

Or will the and combinator be sufficient?

if (o is int x and <= 100) // x is an int with value < 100 here.

Semantics (e.g. type) for relational operators

We expect to support all of the primitive types that can be compared in an expression using a relational operator. The meaning in simple cases is clear

bool IsValidPercentage(int x) => x is >= 0 and <= 100;

But when the input is not such a primitive type, what type do we attempt to convert it to?

bool IsValidPercentage(object x) => x is >= 0 and <= 100;

We have proposed that when the input type is already a comparable primitive, that is the type of the comparison. However, when the input is not a comparable primitive, we treat the relational as including an implicit type test to the type of the constant on the right-hand-side of the relational. If the programmer intends to support more than one input type, that must be done explicitly:

bool IsValidPercentage(object x) => x is
    >= 0 and <= 100 or    // integer tests
    >= 0F and <= 100F or  // float tests
    >= 0D and <= 100D;    // double tests

Flowing type information from the left to the right of and

It has been suggested that when you write an and combinator, type information learned on the left about the top-level type could flow to the right. For example

bool isSmallByte(object o) => o is byte and < 100;

Here, the input type to the second pattern is narrowed by the type narrowing requirements of left of the and. We would define type narrowing semantics for all patterns as follows. The narrowed type of a pattern P is defined as follows:

1. If P is a type pattern, the narrowed type is the type of the type pattern's type.
2. If P is a declaration pattern, the narrowed type is the type of the declaration pattern's type.
3. If P is a recursive pattern that gives an explicit type, the narrowed type is that type.
4. If P is a constant pattern where the constant is not the null constant and where the expression has no constant expression conversion to the input type, the narrowed type is the type of the constant.
5. If P is a relational pattern where the constant expression has no constant expression conversion to the input type, the narrowed type is the type of the constant.
6. If P is an or pattern, the narrowed type is the common type of the narrowed type of the subpatterns if such a common type exists. For this purpose, the common type algorithm considers only identity and implicit reference conversions, and it considers all subpatterns of a sequence of or patterns (ignoring parenthesized patterns).
7. If P is an and pattern, the narrowed type is the narrowed type of the right pattern. Moreover, the narrowed type of the left pattern is the input type of the right pattern.
8. Otherwise the narrowed type of P is P's input type.

Variable definitions and definite assignment

The addition of or and not patterns creates some interesting new problems around pattern variables and definite assignment. Since variables can normally be declared at most once, it would seem any pattern variable declared on one side of an or pattern would not be definitely assigned when the pattern matches. Similarly, a variable declared inside a not pattern would not be expected to be definitely assigned when the pattern matches. The simplest way to address this is to forbid declaring pattern variables in these contexts. However, this may be too restrictive. There are other approaches to consider.

One scenario that is worth considering is this

if (e is not int i) return;
M(i); // is i definitely assigned here?

This does not work today because, for an is-pattern-expression, the pattern variables are considered definitely assigned only where the is-pattern-expression is true ("definitely assigned when true").

Supporting this would be simpler (from the programmer's perspective) than also adding support for a negated-condition if statement. Even if we add such support, programmers would wonder why the above snippet does not work. On the other hand, the same scenario in a switch makes less sense, as there is no corresponding point in the program where definitely assigned when false would be meaningful. Would we permit this in an is-pattern-expression but not in other contexts where patterns are permitted? That seems irregular.

Related to this is the problem of definite assignment in a disjunctive-pattern.

if (e is 0 or int i)
{
    M(i); // is i definitely assigned here?
}

We would only expect i to be definitely assigned when the input is not zero. But since we don't know whether the input is zero or not inside the block, i is not definitely assigned. However, what if we permit i to be declared in different mutually exclusive patterns?

if ((e1, e2) is (0, int i) or (int i, 0))
{
    M(i);
}

Here, the variable i is definitely assigned inside the block, and takes it value from the other element of the tuple when a zero element is found.

It has also been suggested to permit variables to be (multiply) defined in every case of a case block:

    case (0, int x):
    case (int x, 0):
        Console.WriteLine(x);

To make any of this work, we would have to carefully define where such multiple definitions are permitted and under what conditions such a variable is considered definitely assigned.

Should we elect to defer such work until later (which I advise), we could say in C# 9

• beneath a not or or, pattern variables may not be declared.

Then, we would have time to develop some experience that would provide insight into the possible value of relaxing that later.

Diagnostics, subsumption, and exhaustiveness

These new pattern forms introduce many new opportunities for diagnosable programmer error. We will need to decide what kinds of errors we will diagnose, and how to do so. Here are some examples:

case >= 0 and <= 100D:

This case can never match (because the input cannot be both an int and a double). We already have an error when we detect a case that can never match, but its wording ("The switch case has already been handled by a previous case" and "The pattern has already been handled by a previous arm of the switch expression") may be misleading in new scenarios. We may have to modify the wording to just say that the pattern will never match the input.

case 1 and 2:

Similarly, this would be an error because a value cannot be both 1 and 2.

case 1 or 2 or 3 or 1:

This case is possible to match, but the or 1 at the end adds no meaning to the pattern. I suggest we should aim to produce an error whenever some conjunct or disjunct of a compound pattern does not either define a pattern variable or affect the set of matched values.

case < 2: break;
case 0 or 1 or 2 or 3 or 4 or 5: break;

Here, 0 or 1 or adds nothing to the second case, as those values would have been handled by the first case. This too deserves an error.

byte b = ...;
int x = b switch { <100 => 0, 100 => 1, 101 => 2, >101 => 3 };

A switch expression such as this should be considered exhaustive (it handles all possible input values).

In C# 8.0, a switch expression with an input of type byte is only considered exhaustive if it contains a final arm whose pattern matches everything (a discard-pattern or var-pattern). Even a switch expression that has an arm for every distinct byte value is not considered exhaustive in C# 8. In order to properly handle exhaustiveness of relational patterns, we will have to handle this case too. This will technically be a breaking change, but one no user is likely to notice.

[HaloFour]

GitHub should let me add 🎉and ❤️more than once. I'm especially excited by the prospect of variable patterns being declared in multiple mutually exclusive patterns. F# allows it and it's a powerful feature.

[yaakov-h]

Specifically, we support all of these relational patterns for sbyte, byte, short, ushort, int, uint, long, ulong, char, float, double, and decimal. Additionally, we support == and != for string and bool.

Would we support these patterns for user-defined types that are implicitly convertible to any of those types?

[gafter]

@HaloFour If you have a use case that really makes sense in C# that would be helpful.

@yaakov-h No, we do not use user-defined conversions in pattern-matching, except to convert the expression for the switch statement once (a necessary nod to compatibility).

[alrz]

A few points:

• The or pattern semantics could be applied to multi-label switch sections as well:

  case (var x, 1): case (1, var x):

• Also, it would be nice to bind identifiers to types (as a fallback when a constant is not available) so we can say x is Int32 or Int64 instead of x is Int32 _ or Int64 _ .
• Edit: I think having both postfix and prefix relational operators could be very helpful wrt readability.

  bool IsLetter(char c) => c is 'a' <= and <= 'z' or 'A' <= and <= 'Z';`

And there's already some code that could benefit from mutually exclusive pattern variables:

https://github.com/dotnet/roslyn/blob/c1cd335f1306919e68a414fc18f51a3e874ff0dd/src/Features/Core/Portable/ConvertIfToSwitch/AbstractConvertIfToSwitchCodeRefactoringProvider.Analyzer.cs#L310-L317

avoiding code duplication.

[Thaina]

This example illustrates our expectation that and will have a higher parsing priority (i.e. will bind more closely) than or

I would like to voice my disagreement on this. and and or and other binary keyword in the same category should take the same precedence and prioritize by left to right in the same way as normal operator. We should always explicitly use parentheses when needed

[Thaina]

if ((e1, e2) is (0, int i) or (int i, 0))
{
    M(i);
}

If we would allow this case. Would it be possible for switch to declare same variable in fall through case?

Such as

switch((e1, e2))
{
    case (0, int i):
    case (int i, 0):
        M(i);
        break;
}

or maybe #2703

[Thaina]

case >= 0 and <= 100D:

This case can never match

This is unexpected. I expect that we don't need to care about the type of object we put into the comparison operator

byte b = 99;
if(b >= 0 && b <= 100D)
{
    // should be here
}

if(b is >= and <= 100D)
{
    // not come here ??
}

[canton7]

public static LifeStage LifeStageAtAge(int age) => age select

Is this a typo, or a sneak peek at a new keyword? 😄

Should we support some combination of declaration pattern along with a relational pattern?

IMO, no. if (o is int x <= 100) was slightly hard to parse for me. if (o is int x and <= 100) is only a few characters longer, and IMO quite a bit clearer.

[ronnygunawan]

public static LifeStage LifeStageAtAge(int age) => age select

Is this a typo, or a sneak peek at a new keyword? 😄

Must be a typo. The last code snippet is written using a switch.

[DavidArno]

Whilst likely beyond the scope of C# 9, could I please ask that the ideas in the user-defined positional/active patterns threads (#1047 and #277) and support of typeof as a value that can be used in patterns be born in mind when designing the features discussed here? It would be a great shame to make changes now that would make it harder to add active patterns etc later.

[ronnygunawan]

if (o is int x and >= 0 and < 10) ...
// or
if (o is int x >= 0 and < 10) ...

I think the intent of this expression is unclear. Will < 10 be evaluated against x or against o?

[canton7]

I think the intent of this expression is unclear. Will < 10 be evaluated against x or against o?

True. I skimmed over that without thinking about it too much.

if (o is int x && x >= 0 and < 10)

Clearer, but puts && and and right next to each other, which looks a little jarring.

[DavidArno]

Will < 10 be evaluated against x or against o?

Possibly playing devil's advocate here, but does it matter?

[canton7]

Possibly playing devil's advocate here, but does it matter?

Given:

case >= 0 and <= 100D:

This case can never match (because the input cannot be both an int and a double).

The answer appears to be yes: if the datatype needs to be the same between the thing being matched and the type of the pattern, then there is a difference between < 10 being applied to an int and an object: it will match for the int but not for the object.

[Thaina]

if (o is int x and >= 0 and < 10) ...

I think the intent of this expression is unclear. Will < 10 be evaluated against x or against o?

In my understanding, above is evaluated against o

This

if (o is int x && x is >= 0 and < 10)

is evaluated against x

I think and and or would take the left side of the most nearest left is keyword to be evaluated against it

[YairHalberstadt]

if the datatype needs to be the same between the thing being matched and the type of the pattern, then there is a difference between < 10 being applied to an int and an object: it will match for the int but not for the object.

Not if we view < 10 as implicitly defining an int check. Then I believe there's no difference and it should be an implementation detail to allow performance optimizations.

[Thaina]

@YairHalberstadt How do you think about

float x = 9.9f;
if(x is > 0 and < 10)
{
   // is not matched ?
}

[YairHalberstadt]

@Thaina

That seems to be the intent of @gafter, that a float would not match > 0.

Whilst conceptually that makes sense. I think it may not be ideal, since it leads to a pit of failure. By default I would probably write something like that, and it won't actually work.

[canton7]

I do think that:

float x = 9.9f;
if (x is > 0 and < 10)
{
}

Needs to have exactly the same behaviour (both in terms of any compiler warnings/errors, and runtime behaviour) as:

float x = 9.9f;
if (x > 0 && x < 10)
{
}

[CyrusNajmabadi]

Note: there is precedence (no pun intended) for the language shipping features that don't behave the same as one would intuit. For example, without using operator-overloading, it's possible to have the following a >= b evaluates to false, but a > b || a == b returns true.

Yes, that's surprising. But it's also NBD in practice. Users rarely (if ever) hit the corners that make this stuff appear. It tends to worry people who think about hte whole language (including myself) and how it all fits together. But it's often overblown as an actual problem for the language and the near total majority of users of it.

[gafter]

For example, without using operator-overloading, it's possible to have the following a >= b evaluates to false, but a > b || a == b returns true

Please explain.

[gafter]

Did everyone miss this sentence in the draft spec?

When the input is a type for which a suitable built-in binary relational operator is defined that is applicable with the input as its left operand and the given constant as its right operand, the evaluation of that operator is taken as the meaning of the relational pattern.

[CyrusNajmabadi]

Please explain.

Sure!

        int? a = null;
        int? b = null;
        Console.WriteLine(a >= b);
        Console.WriteLine(a > b || a == b);

As defined in the language itself, a is not >= to b. But it is == to b. Leading the above resulting in the false then true being printed above.

This is a core, built-in, inconsistency with the language and how people might generally intuit things to behave.

[LeftofZen]

When and why would I prefer x is >= 0 over x >= 0?

[HaloFour]

@LeftofZen

When and why would I prefer x is >= 0 over x >= 0?

By itself you probably wouldn't prefer to use the pattern, but combined with recursive patterns it can be quite powerful:

if (person is Student { Gpa: >= 3.0 } student) { ... }

[Thaina]

Did everyone miss this sentence in the draft spec?

When the input is a type for which a suitable built-in binary relational operator is defined that is applicable with the input as its left operand and the given constant as its right operand, the evaluation of that operator is taken as the meaning of the relational pattern.

@gafter Admittedly this sentence was too hard to understand and picture all the scenario relate to it (at least for me) until we see the example we then know that this sentence was breaking our common sense's expectation

[gafter]

Added

• Champion: Type Patterns (VS 16.8, .NET 5) #2925 Type patterns

To the list of new pattern forms in scope for C# 9.0.

[thargol1]

Will patterns be a 'type' so you can reuse them in different switch statement etc?

Like this (contrived example):

pattern IsLetter(char c) => c is >= 'a' and <= 'z' or >= 'A' and <= 'Z';
pattern IsPersonId(int i) => i is >=1 and <=100;

var result = obj switch
{
    IsLetter x and x=='x' => "X marks the spot" ,
    IsLetter => "Not an X",
    IsPersonId => "A person id",
    _ => "Argle blargle glob glif"
};

[HaloFour]

@thargol1

That would fall under "active patterns" (#1047 and #277).

That said, syntax like that could be nice shorthand for writing user-defined positional patterns if it would expand out into a static class with the static Deconstruct method.

[alrz]

beneath a not or or, pattern variables may not be declared.

That would have been a lifesaver, I think this is the only workaround for now:

case (0, int x):
case (int x0, 0) when (x = x0) is var _:
    Console.WriteLine(x)

Ugly but codegen is as efficient as built-in support.

[gafter]

6. If P is an or pattern, the narrowed type is the common type of the narrowed type of the left pattern and the narrowed type of the right pattern if such a common type exists.

This doesn't work. In a pattern such as int or long, there is no common type for pattern-matching purposes, so the narrowed type should be the input type.

[gafter]

Changed

6. If P is an or pattern, the narrowed type is the common type of the narrowed type of the left pattern and the narrowed type of the right pattern if such a common type exists.

to

6. 6. If P is an or pattern, the narrowed type is the common type of the narrowed type of the subpatterns if such a common type exists. For this purpose, the common type algorithm considers only identity and implicit reference conversions, and it considers all subpatterns of a sequence of or patterns (ignoring parenthesized patterns).

[gafter]

Moving to a checked-in document via #3361

[mpawelski]

This looks great.
However, I wonder if there is anything planned for matching against value of variable?
I guess there will be many people that when they learn that you can write this:

if(foo.Bar is > 10 and <10) 

they will see this just as nicer syntax for making conditional check without repeating variable name and will try to write this:

var min = 10;
var max = 100
if(foo.Bar is > min and < max) 

And they will get error.

I see it was briefly discussed (in @HaloFour response to removed user) but I didn't saw it mentioned anywhere else.

[CyrusNajmabadi]

However, I wonder if there is anything planned for matching against value of variable?

There is nothing planned for that. Sorry! :(

[mpawelski]

@CyrusNajmabadi
:( Thanks for answer. Now at least we know ;-)

Can I ask if there is some fundamental "problem" with it? I only had bigger experience with pattern matching in F# and there it is not possible (you need to use when clause), similar in Rust and Scala also seems to not allow it.

My guess the reason is that pattern matching in this languages is used mainly with dedicated match/switch/case syntax and there you can use when/is guards to compare against variable. You can also do it in c#:

var foo = 11m;
var bar = 22m;

switch (foo)
{
    case var _ when foo == bar:
        Console.WriteLine("foo is bar");
        break;
}
var r = foo switch
{
    _ when foo == bar => "foo is bar",
    _ => ""
};
Console.WriteLine(r);

So not real benefit of explicit "match value of variable" pattern there. But in C# you can use patterns in if clause with is operator (which is awesome!) and proposed enchantments will make it far more useful and common than now.

C# luckily requires var keyword or type name for variable pattern. In F#, Rust and Scala you just provide name of new variable. I wonder if this leaves door open for possible "match value of variable" that will basically have the same "syntax" as current constant pattern (just use the variable name)?

[CyrusNajmabadi]

Can I ask if there is some fundamental "problem" with it?

I know of no fundamental problem with it. We've simply never designed it to do the above. One benefit of the current approach is that it's strictly declarative and isolated from things like order of operations or side-effects. At the point that arbitrary values are allowed, it would be necessary to design what that means.

[gafter]

Can I ask if there is some fundamental "problem" with it?

It doesn't do the same thing that pattern-matching was designed to do. Specifically, it was designed to work with constants specifically so that the compiler can analyze, diagnose, and optimize the totality of the set of pattern-matching operations. If the compiler doesn't know the value it is matching against, it cannot do any of that. That's not to say it is necessarily a bad idea, but it is a pretty different animal. Do ordinary expressions not satisfy that need?

[mpawelski]

thanks for answers guys

That's not to say it is necessarily a bad idea, but it is a pretty different animal. Do ordinary expressions not satisfy that need?

I don't see having "match against value of variable" pattern to be a must-have killer feature. It's use will probably be limited to just small patterns in if statements so no big loss.

I just think there will be many people who will see using is and or keywords as just a different way of writing boolean expressions. Just another small C# feature that makes code more succinct. Similar to tons of other we got in recent years ( expression-bodied members, out variables, tuples, null-conditional operators operator, "default" literal, etc). They won't know it's part of greater feature like "pattern-matching" which "was designed to work with constants". They will just be confused when instead of constant they will use variable name and get "CS0150: A constant value is expected" error.

[gafter]

Closing this issue as it has been moved to a checked-in document via #3361

[alrz]

However, what if we permit i to be declared in different mutually exclusive patterns?

Do we have a tracking issue for that? Possibly a candidate for 10.0 working set? @gafter