With clause in query expressions (support for zip)

[orthoxerox]

I propose to add a with clause to the query expression syntax. This will allow us to zip sequences together using either LINQ syntax.

A query expression with a with clause followed by a select clause

from x1 in e1
with x2 in e2 
select v

is translated into

( e1 ) . Zip( e2, ( x1 , x2 ) => v )

A query expression with a with clause followed by something other than a select clause

from x1 in e1
with x2 in e2 
…

is translated into

from * in ( e1 ) . Zip(
        e2 , ( x1 , x2 ) => new { x1 , x2 })
…

This transformation would happen in section 7.16.2.4 of the spec:

from x1 in e1
from x2 in e2
with x3 in e3
…

is first translated into

from * in ( e1 ) . SelectMany( x1 => e2 , ( x1 , x2 ) => new { x1 , x2 } )
with x3 in e3
…

and then translated into (if I understand how transparent identifiers work)

from ** in ( ( e1 ) . SelectMany( x1 => e2 , ( x1 , x2 ) => new { x1 , x2 } ) ) . Zip(
        e2 , ( * , x3 ) => new { * , x3 })
…

And

from x1 in e1
with x2 in e2
from x3 in e3
…

is first translated into

from * in ( e1 ) . Zip( e2 , ( x1 , x2 ) => new { x1 , x2 })
from x3 in e3
…

and then translated into

from ** in ( ( e1 ) . Zip( e2 , ( x1 , x2 ) => new { x1 , x2 }) ).SelectMany( 
        * => e3 , ( * , x3 ) => new { * , x3 } )
…

[orthoxerox]

I tried implementing this, and it looks like making with a new contextual keyword would break existing query expressions. I'll try to come up with an alternative syntax. Perhaps a comma instead of with or a join x2 in e2 zip clause would work better with existing code.

[alrz]

@orthoxerox Utilizing extension implementations (#8127):

// assuming covariant tuples
extension<T, U> (IEnumerable<T>, IEnumerable<U>) : IEnumerable<(T, U)> { 
  IEnumerator<(T, U)> GetEnumerator() {
    return this.Item1.Zip(this.Item2, (x, y) => (x, y)).GetEnumerator();
  }
}

// tada
var result =
  from tuple in (list1, list2)
  select tuple case (var i, var j) : i + j;

// if we could use patterns in linq
var result =
  from (var i, var j) in (list1, list2)
  select i + j;

Today you can do this the uninteresting way, via a regular extension method,

static IEnumerable<Tuple<T, U>> Zip<T, U>(this Tuple<IEnumerable<T>, IEnumerable<U>> source) {
  return source.Item1.Zip(source.Item2, Tuple.Create);
}

var result =
  // explicit type arguments are required because tuples are invariant
  from tuple in Tuple.Create<IEnumerable<int>, IEnumerable<int>>(list1, list2).Zip()
  select tuple.Item1 + tuple.Item2;

[AdamSpeight2008]

What's wrong with using zip as the keyword? AlsoZip doesn't necessarily return a tuple.
I believe it is the only standard LINQ operator that pulls from two sources simultaneously though.

[orthoxerox]

@AdamSpeight2008 someone has zip as an identifier in their query expression already, plus the word makes sense only for enumerables, not other monads.

[AdamSpeight2008]

@orthoxerox with is already a keyword in vb.net.

Having zip return a IEnumerable<Tuple<T1,T2>> doesn't make sense either.
As in the case where you don't want one. You'd have to de-construct the tuple.

Imports System.Runtime.CompilerServices

Module Module1

    Sub Main()
        Dim e1 = {1, 2, 3, 4}
        Dim e2 = {"A"c, "B"c, "C"c, "D"c}
        Dim ie = e1.ZipT(e2).Select(Function(t)
                                        Dim x = t.Item1
                                        Dim y = t.Item2
                                        Return New String(y, x)
                                    End Function)
    End Sub

    <Extension>
    Function ZipT(Of T1, T2)(e1 As IEnumerable(Of T1), e2 As IEnumerable(Of T2)) As IEnumerable(Of Tuple(Of T1, T2))
        Return e1.Zip(e2, Function(x, y) Tuple.Create(x, y))
    End Function
End Module

Using Zip Query syntax. Similar in style and context to an Aggregate Query operation.

Dim  ie = Zip x In e1, y In e2 Into New String(y,x)

[AdamSpeight2008]

Old discussion #100

[AdamSpeight2008]

@orthoxerox

I've devised the following grammar definition for Zip ( x In xs) , (y In ys ) Into ... that is based on the grammar from the Visual Basic Language Specification.

ZipQueryOperator ::= 
    [ LineTerminator ] Zip [LineTerminator]
     CollectionRangeVariableDeclaration [LineTerminator] , [LineTerminator]
     CollectionRangeVariableDeclaration [LineTerminator] Into [LineTerminator]
     ExpressionRangeVariableDecarationList


QueryExpression ::=
    FromOrAggregateOrZipQueryOperator | 
    QueryExpression QueryOperator   


FromOrAggregateQueryOperator ::= FromQueryOperator | AggregateQueryOperator

FromOrAggregateOrZipQueryOperator ::= FromOrAggregateQueryOperator | ZipQueryOperator

QueryOperator ::= 
    FromQueryOperator       |
    AggregateQueryOperator  |
    ZipQueryOperator        |
    SelectQueryOperator     |
    DistinctQueryOperator   |
    WhereQueryOperator      |
    OrderByQueryOperator    |
    LetQueryOperator        |
    GroupByQueryOperator    |
    JoinOrGroupJoinQueryOperator

Suggests that adding a Query Syntax for Zip would relatively easy (at least for VB).

[alrz]

@orthoxerox I have to ask, why there is a need for a new contextual keyword,

from tuple in (list1, list2) select ...

Isn't good enough?

[orthoxerox]

@alrz This would require tuples to support LINQ functions that turn them inside out, and we can't do that without HKT/traits:

M<TResult> Select<T1, T2, TResult>(
    this VTuple(M<T1>, M<T2>) source, 
    Func<VTuple<T1, T2>, TResult> projector) 
    where M : TMonad //if Zip is added to the baseline LINQ functions
{
    return source.Item1.Zip(source.Item2).Select(projector);
}

[orthoxerox]

Moved to dotnet/csharplang#117