Proposal: introduce left and right join clauses to C# query expression syntax

[roji]

Summary

Introduce left join and right join clauses into LINQ query expression syntax, e.g.:

from student in students
left join department in departments on student.DepartmentID equals department.ID
select new { student.Name, department?.Name }

Motivation

LINQ has a Join operator, which, like its SQL INNER JOIN counterpart, correlates elements of two sequences based on matching keys; the C# language has a corresponding join clause which translates to this operator (section 12.20.3.5 of the C# specs).

In addition to INNER JOIN, SQL also has LEFT JOIN, which returns outer elements even if there's no corresponding inner ones; LINQ and C#, in contrast, lack this operator. The LINQ conceptual documentation shows how to combine existing operators to achieve a left join:

var query =
    from student in students
    join department in departments on student.DepartmentID equals department.ID into gj
    from subgroup in gj.DefaultIfEmpty()
    select new
    {
        student.FirstName,
        student.LastName,
        Department = subgroup?.Name ?? string.Empty
    };

Or using method syntax:

var query = students
    .GroupJoin(departments, student => student.DepartmentID, department => department.ID, (student, departmentList) => new { student, subgroup = departmentList })
    .SelectMany(
        joinedSet => joinedSet.subgroup.DefaultIfEmpty(),
        (student, department) => new
        {
            student.student.FirstName,
            student.student.LastName,
            Department = department.Name
        });

Although functionality sufficient for expressing a left join operation, this combining approach has the following drawbacks:

• It's complicated, requiring combining multiple different LINQ operators in a specific way to form a complex construct, and is easy to accidentally get wrong. Many EF users have complained about the complexity of this construct for expressing a simple SQL LEFT JOIN.
• It is inefficient - the combination of operators adds significant overhead compared to a single operator using an internal lookup table (or "hash join", as Join is implemented).

dotnet/runtime#110292 is an approved API proposal for introducing LeftJoin and RightJoin operators into the System.Linq for .NET 10; see that proposal for additional performance information and benchmarks. Aside from allowing users to more easily express SQL LEFT/RIGHT JOIN when using a LINQ provider (such as EF Core), it would also allow both easier and more efficient in-memory left/right joins, exactly in the way that inner joins are already supported.

Along with the introduction of the operators into System.Linq, the C# query expression syntax can be extended with new left join and right join clauses which would translate to these new operators, allowing for simpler and more efficient code where C# query syntax is used.

Design

The syntax for the proposed left join and right join expression clauses would be identical to the existing join clause. For example:

from student in students
left join department in departments on student.DepartmentID equals department.ID
select new { student.Name, department?.Name }

Note that after the left join clause, department is nullable, as left join also returns students with no corresponding departments (unlike the existing join clause, which does not).

Similar to how the join clause internally translates to the LINQ Join operator, left join would translate to the new LeftJoin operator being added in dotnet/runtime#110292:

students.LeftJoin(
   departments,
   student => student.DepartmentID,
   department => department.CustomerID,
   (student, department) => new { student.Name, department?.Name })

right join represents the opposite operation, returning inner items when no corresponding outer ones are found:

from department in departments
right join student in students on department.ID equals student.DepartmentID
select new { department?.Name, student.Name }

This would translate to:

departments.RightJoin(
   students,
   department => department.CustomerID,
   student => student.DepartmentID,
   (department, student) => new { department?.Name, student.Name })

Notes

• See [API Proposal]: Introduce LeftJoin LINQ operator runtime#110292 for discussion on the naming for the new operators, and motivations for the adopted LeftJoin and RightJoin naming. For the proposed C# query expression, the clauses could be named left join/right join, or possibly leftjoin/rightjoin (similar to the orderby clause).

/cc @eiriktsarpalis @terrajobst @CyrusNajmabadi @dotnet/efteam

Previous related issues

• Please consider to extend Linq to add a new keyword leftjoin or rightjoin #361
• Add new keyword "leftjoin" to make LINQ more readable #508
• Add leftjoin/rightjoin syntax to linq to end the horrific syntax currently required runtime#53236
• Support the new .NET 10 LeftJoin/RightJoin operators efcore#12793