Add WhereNotNull to help with nullability tracking

[YairHalberstadt]

See dotnet/csharplang#8383

Currently the C# compiler can't track how the lambda in a Where method effects nullability.

For example

using System;
using System.Collections.Generic;
using System.Linq;

#nullable enable
public class C {
    public IEnumerable<string> WhereNotNull(IEnumerable<string?> strings)
    {
        return strings.Where(x => x != null);
    }
}

Warns with:

warning CS8619: Nullability of reference types in value of type 'IEnumerable<string?>' doesn't match target type 'IEnumerable'.

This could definitely be solved by improved compiler tooling, but doing so would be non-trivial.

A workaround would be to introduce a WhereNotNull method into CoreFX, so that consumers won't need to use the suppression operator themselves.

A further advantage would be that using WhereNotNull instead of Where would avoid allocating a lambda for an extremely common Linq method. This could be done either by caching the lambda as a Func<object, bool>, which avoids the need to update the rest of Linq to be aware of a new Linq method, or by creating a custom enumerator for WhereNotNull which avoids the overhead of a virtual function call, but requires updating the rest of Linq.

API Proposal

namespace System.Linq
{
    public static partial class Enumerable
    {
        public static IEnumerable<T> WhereNotNull<T>(this IEnumerable<T?> source);
        public static IEnumerable<T> WhereNotNull<T>(this IEnumerable<T?> source) where T: struct;
    }
}

API Usage

I've deliberately not constrained the first API (i.e. omitted where T: class) so that unconstrained generic code can use this API as well. Of course, this won't work well with nullable value types, but that's a general problem of unconstrained generic code and nullability in general, so this isn't a new problem.

Risks

None.

[stephentoub]

A further advantage would be that using WhereNotNull instead of Where would avoid allocating a lambda for an extremely common Linq method. This could be done either by caching the lambda as a Func<object, bool>

If you write:

Where(x => x != null)

the compiler is already going to cache the delegate automatically.

[stephentoub]

This could definitely be solved by improved compiler tooling, but doing so would be non-trivial.

It might be non-trivial, but there are a myriad number of places this same kind of issue can show up, and I don't think we should address each as a one-off solved by adding additional methods for each such case. My preference is to wait and see what the actual broad impact is (we don't even have LINQ annotated yet), and then ideally come up with a holistic solution around such cases. If it turns out that this specific case is literally the only one that matters, then it could be relevant, but we should not rush it.

[YairHalberstadt]

the compiler is already going to cache the delegate automatically.

The compiler will cache one delegate per type parameter, per use site. There are likely to be hundreds of identical such delegates across a large code base - a quick search just for "x => x != null" turned up 97 in ours. It's a small benefit, but perhaps not irrelevant.

My preference is to wait and see

Agreed. I was just bringing this up here as a point of discussion. Hopefully the Roslyn and CoreFx teams can work something out on this together. Thanks for your input!

[stephentoub]

The compiler will cache one delegate per type parameter, per use site.

This is something Roslyn could have an optimization pass to consolidate if deemed impactful. Developers shouldn't be required to manually do such folding by hand.

[kronic]

Maybe WhereNotDefault

[terrajobst]

Linq is now annotated and the issue remains.

dotnet/roslyn#41363 has some good comments:

• Add Enumerable.WhereNotNull() #41363 (comment)
• Add Enumerable.WhereNotNull() #41363 (comment)

[john-h-k]

why isn't OfType<object> good enough? Avoids the lambda, isn't particularly cryptic (imo, given the fact null is T is never true), and works great with nullability