Lifting support for nullable types (and other LINQy entities).

[orthoxerox]

Right now C# supports implicit lifting for operators on nullable value types like int?:

int? a = null;
var b = a * 2;

However, this doesn't work on other methods:

struct Foo { public int A }
Foo? foo = null;
var a = foo.A; //Won't compile

With explicitly nullable reference types coming to C# in #5032, this gets a bit problematic. Lifting all functions implicitly would be a huge problem, since it would silently return null where you would get a NRE before:

Foo? foo = GetFoo();
Bar? bar = GetBar();

Func<Foo, Bar, string> func = (f, b) => f.ToString() + b.ToString();

var result = func(foo, bar); //result is inferred to be a string, 
//but if we do implicit lifting, it will be a `string?`

I want to propose explicit lifting using ? as a postfix operator. For example, var result = func(foo?, bar?); from my previous example would be rewritten as:

var result = foo==null?null:bar==null?null:func((Foo)foo, (Bar)bar);
//of course, if foo and bar weren't locals, they would be replaced with temp values first.

I also propose to generalize the lifting operator to support all types that support LINQ functions. For example, the same var result = func(foo?, bar?); could be rewritten as

var result =
    from foo_ in foo
    from bar_ in bar
    select func(foo_, bar_);

Or:

IEnumerable<int> as = GetAs();
IEnumerable<int> bs = GetBs();
var cs = as? + bs?;
//cs is IEnumerable<int>
//NB: cs is a Cartesian product of two Enumerables!

Or the perverse examples:

IEnumerable?<BigInteger?> as = GetAs();
IEnumerable?<BigInteger?> bs = GetBs();
var cs = as???.CompareTo(bs???);
//yes, three ?'s!
//1: to IEnumerable<BigInteger?>
//2: to BigInteger?
//3: to BigInteger
//cs is IEnumerable?<int?>
//NB: cs is a Cartesian product of two Enumerables!

IEnumerable?<PointF?> as = GetAs();
IEnumerable?<PointF?> bs = GetBs();
var cs = as?? + bs??;
//just two ?'s are needed, because + lifts its operands from PointF? to PointF
//cs is IEnumerable?<PointF?>
//NB: cs is a Cartesian product of two Enumerables!

[orthoxerox]

I've just realized that ? doesn't really play well with non-nullable lifting, e.g.

IEnumerable<Foo> foos = GetFoos();
foos?.Frob();

Did I want to say foos.Select(f => f.Frob()) or foos==null?null:foos.Select(f => f.Frob())?

I see two possible solutions:

1. C#7 with nullability tracking switched on no longer allows you to use ?. on values of non-nullable reference types.
2. We keep using ? for nullability lifting only, but all other functions are lifted using a different postfix on their operands.

[alrz]

What is the problem with this?

int? a = null;
var b = a * 2; // b will be null if a is null

struct Foo { public int A; }
Foo? foo = null;
var a = foo?.A; // will compile, a will be null if foo is null

object? a = null;
var b = a.GetType(); // error/warning depending on #5032

object? a = null;
var b = a?.GetType(); // OK

[orthoxerox]

@alrz the nullable object is the dispatcher in all your examples.

class Foo { int X; bool Foop(Foo other) => this.X == other.X;}
Foo? a = new Foo() {X = 4};
Foo? b = null;
var eq = a?.Foop(b);

a.Foop(b) will fail with a NRE. Existing libraries' methods check their arguments for nulls, but methods that will be written using nullability tracking won't. To make them accept nullable arguments we either have to write a wrapper if we need some specific behavior or we can lift the function automatically. For example, a.Foop(b?) will return a bool? that will be set to null if b is null.

In addition to that it will simplify LINQ queries (but see my comment about reusing ?). It won't work with Where (I think), but all Selects and SelectManys will become easier to use.

IEnumerable<Foo> foos = GetFoos();
//var xs = foos.Select(foo => foo.X);
var xs = foos?.X;

[alrz]

You are clearly passing a nullable Foo to the method.

Foo? b = null;
var eq = a?.Foop(b);

And what it has to do with LINQ?

[orthoxerox]

@alrz Am I allowed to pass a nullable Foo to a method that accepts a non-nullable Foo?

Enumerables, Nullables, Tasks are all monads. Right now they all have different ways to simplify working with them. Nullables have lifted operators (even though == is not technically correctly lifted), Enumerables have from-notation, Tasks have await, which works a lot like my ? if you squint hard enough.

From-notation is the only extensible way to handle monadic entities, similar to do-notation in Haskell. (Awaiting is technically extensible as well, but results in weird-looking code for other monads). F# has computation expressions that perform this role, by the way.

However, from-notation forces you to introduce additional variables to handle them. For example, to zip two enumerables you either have to use a Zip higher order function (or a SelectMany) or to write something like

from x in xs
from y in ys
select MyFunc(x, y)

With a lifted MyFunc you could write MyFunc(x?, y?) instead. Or MyFunc(x^, y^), if you are weirded out by ? for non-nullable use cases.

[alrz]

Am I allowed to pass a nullable Foo to a method that accepts a non-nullable Foo?

I think not, that involves an implicit unwrapping which totally misses the point of nullable reference, however, as far as I know the following is OK,

Foo? b = null;
if(b != null ) a.Foop(b); // b is definitely not null

which doesn't throw in the method.

The syntax you are using is similar to what I've been proposed for #5961 which as discussed wouldn't be really practical (see this comment).

As for,

IEnumerable<Foo> foos = GetFoos();
//var xs = foos.Select(foo => foo.X);
var xs = foos?.X;

I don't think for C# that would be really good style of coding, I'd rather use #5444 to make it more concise like var xs = foos.Select(::X); (for f(x?,y?) even in Haskell you need to bind every variable separately) and also, for each type that you have mentioned there is a separate syntax,

var result = from item in enumerable select item.X;
var result = (await task).X;
var result = nullable?.X;

Just as you said, Enumerables, Nullables and Tasks are monads, but all of them have their own syntax, and we don't have any abstraction over these types. With traits and higher kinded generics you might be able to declare an abstraction over them, but there is no general syntactic sugar in C# like do in Haskell or computation expressions in F#.

[orthoxerox]

@alrz Hah, I actually forgot what your proposal was about. Perhaps it's too late for C# to unify its approach to monads beyond from-notation. Or perhaps not.

[orthoxerox]

Looks like someone wrote a Scala macro with a very similar syntax: https://github.com/pelotom/effectful

[gafter]

See also #5961

[orthoxerox]

I must say, writing a functional LINQ expression to zip and sum three IEnumerable<Optional<T>> by hand was a great exercise. Automating this transformation will be even harder.

[orthoxerox]

I've been jotting down an expression-rewriting algorithm to implement this and decided to share with you the madness that mixing different monads creates.

var x = new [] {1, 2, 3};
var y = new Option<int>(5);
var w = (Func<int>)(() => 100);
var z = x! + y! * w!;
//is ultimately transformed into:
var z = y.Select(y_ => w.Select(w_ => y_ + w_)).Select(yw_ => x.Select(x_ => yw_.Select(yw__ => x_ + yw__)));
//I have no idea how to even format this monstrosity!

Unfortunately, the type of z is Option<IEnumerable<Func<int>>>, while you would expect it to be IEnumerable<Option<Func<int>>>, so the transformation rules have to be rewritten to handle both this and double-wrapped values case in the least unexpected way possible:

var x = new Option<int>(5); //Option<int>
var y = new [] {Some(1), Some(2), Some(3)}; //IEnumerable<Option<int>>
var z = x! + y!!; //should be //IEnumerable<Option<int>>
//is correctly transformed into:
var z = y.Select(y_ => x.Zip(y_, (x_, y__) => x_ + y__));

[gafter]

We are now taking language feature discussion on https://github.com/dotnet/csharplang for C# specific issues, https://github.com/dotnet/vblang for VB-specific features, and https://github.com/dotnet/csharplang for features that affect both languages.