[Proposal] extend language integrated query clauses with "apply" clause

[gafter]

@paulomorgado commented on Wed Jun 17 2015

Every now and then there's a new request for a new query clauses (C#, Visual Basic).

Sometimes it's to add to C# some query clause that Visual Basic already has (Aggregate, Distinct, Skip, Skip While, Take, Take While), or materialization of the query (ToArray, ToList, ToDictionary or ToLookup) or soe other selection or aggregation (First, FirstOrDefault, Last, LastOrDefault, Single, SingleOrDefault, Count, etc.).

There are just too many and chances are that more will come.

How about extending the query language with an apply clause?

This apply clause would be composed of an apply keyword followed by the method name (instance or extension like already happens to Select or Where) and any other parameters.

Then it would be possible to write something like this:

from c in customers
apply Distinct c.CountryID
select c.Country.Name

Or this:

from o in customer.Orders
select o
apply FirstOrDefault

Or this:

from c in customers
select c
apply ToDictionary c.ID

I don't know if I like this or not, but looks to me a lot more consistent and all-encompassing than individual proposals.

---

@svick commented on Thu Jun 18 2015

For the record,the previous proposals on GitHub are #3486 and #100 (and I'm sure there were some discussions on CodePlex too).

---

@svick commented on Thu Jun 18 2015

I would really like something like this, since it's completely general. But I think the syntax needs to be fleshed out much more. Some questions:

• How do you apply methods with multiple parameters (like the two-selector overload of ToDictionary())?

  The variants consistent with this proposal are apply ToDictionary c.ID c.Name and apply ToDictionary c.ID, c.Name, but personally I don't like either. Instead, I would prefer call-like syntax for all methods: apply ToDictionary(c.ID, c.Name) and e.g. apply First().

• When no range variable is mentioned, how do you differentiate between normal parameters (e.g. apply Take(10)) and lambda parameters (e.g. the second parameter of apply ToDictionary(c.ID, 0))?

  F# LINQ (which already has similar kind of extensibility) uses attributes for this. Another option would be to use the variant that compiles, while preferring one of them when both compile (though that might easily lead to confusing error messages).

• How do you differentiate between methods that maintain range variables and so can be used in the middle of a query (like Distinct) and those that don't and so have to be used after the final select (like First)?

  F# LINQ again uses attributes for this. And another option would be to just assume the user is right, but I think that would require lots of care to make the error messages understandable (instead of something like "Could not find an implementation of the query pattern for source type 'int'. 'Select' not found." for the query from i in list apply First select 2*i).

---

@gordanr commented on Thu Jun 18 2015

'Apply' could be very good starting point for discussion.

Call-like syntax is maybe more suitable for C# language.

from c in customers
select c
apply ToDictionary c.ID

from c in customers
select c
apply ToDictionary(c.ID)

When there are not parameters, is it good idea to allow both forms?

from o in customer.Orders
select o
apply FirstOrDefault

from o in customer.Orders
select o
apply FirstOrDefault()

Some methods are natural after the final select.

... select expression apply ToList/ToArray/ToDictionary/...

Some methods can be after the final select.

... select expression apply FirstOrDefault/First/Single/...

But also can be proposed in infix notation.

...select FirstOrDefault/First/Single/... expression

Some methods are maybe more natural in infix notation (as in sql).

...select Distinct expression

But also can be written on the end

...select expression apply Distinct

---

@paulomorgado commented on Thu Jun 18 2015

Not all methods of Enumerable are available through LINQ clauses. So, for the lack of better syntax, I intentionally left out those not translatable to this syntax.

I intentionally did not use a call-like syntax because that's how all existing clauses were defined.

All the rules that apply now and to method invocation style would still apply. The fact that you don't have a TResult Select<TSource, TResult>(TSource source, Func<TSource, TResult> selector) where source and result are not enumerables doesn't mean you can't have. In fact, in the last few years, I've seen Bart de Smet twisting LINQ in very interesting ways.

---

@gordanr commented on Sat Jun 20 2015

So, for the lack of better syntax, I intentionally left out those not translatable to this syntax.

Can we make a minimum subset of methods that are suitable for implementation?

... select expression apply ToList
... select expression apply ToArray
... select expression apply ???

Is it good idea for the first time to avoid methods with parameters?

---

@gordanr commented on Sat Jun 20 2015

@paulomorgado

Can you explain this example.

from c in customers
apply Distinct c.CountryID
select c.Country.Name

I don't understand how this query can be translated into pure methods chain. Does it mean

(from c in customers select c.Country.Name).Distinct()

Maybe you thought

from c in customers
select c.Country.Name 
apply Distinct

When we talk about Distinct we should have in mind how works default comparator.

---

@gordanr commented on Sat Jun 20 2015

More precisely, I don't understand how to use 'apply' before 'select expression'.

Now we can write 'where' more times between first 'from' and the last 'select'.
For example, I usually write 'let' between two 'where'.

from ...
...
where condition
...
where condition
...
where condition
...
select expression

Do we think about this form of 'apply'?

from ...
...
apply Method
...
apply Method
...
apply Method
...
select expression
apply Method

Can we make a list of suitable methods in this case?
Is it good approach?

---

@paulomorgado commented on Sat Jun 20 2015

@gordanr,

So, for the lack of better syntax, I intentionally left out those not translatable to this syntax.

Can we make a minimum subset of methods that are suitable for implementation?

... select expression apply ToList
... select expression apply ToArray
... select expression apply ???

Is it good idea for the first time to avoid methods with parameters?

The point of this proposal is to not be limited to any set of methods. And to allow methods with none or one parameter.

Can you explain this example.
    from c in customers
    apply Distinct c.CountryID
    select c.Country.Name

I don't understand how this query can be translated into pure methods chain. Does it mean
    (from c in customers select c.Country.Name).Distinct()

Maybe you thought
    from c in customers
    select c.Country.Name
    apply Distinct

When we talk about Distinct we should have in mind how works default comparator.

This:

from c in customers
apply Distinct c.CountryID
select c.Country.Name

will translate into:

customers.Distinct(c => c.CountryID).Select(c => c.Country.Name)

On the other hand, this:

from c in customers
select c.Country.Name 
apply Distinct

will translate into:

customers.Select(c => c.Country.Name).Distinct()

More precisely, I don't understand how to use 'apply' before 'select expression'.

Now we can write 'where' more times between first 'from' and the last 'select'.
For example, I usually write 'let' between two 'where'.
    from ...
    ...
    where condition
    ...
    where condition
    ...
    where condition
    ...
    select expression

Do we think about this form of 'apply'?

    from ...
    ...
    apply Method
    ...
    apply Method
    ...
    apply Method
    ...
    select expression
    apply Method

Can we make a list of suitable methods in this case?
Is it good approach?

apply is used as any other clause with the only difference that the first operand is a method.

You can write this:

from ...
...
where condition
...
where condition
...
where condition
...
select expression

as this:

from ...
...
apply Where condition
...
apply Where condition
...
apply Where condition
...
apply Select expression

if you want to.

---

@gordanr commented on Sat Jun 20 2015

Thank you. Very, very interesting. I will think more about that.

---

@gordanr commented on Sun Jun 21 2015

That was a key.

apply Where condition

Now, I understand better your proposal.

I intentionally did not use a call-like syntax because that's how all existing clauses were defined.

Yes. Now when I understand I agree with you regarding parameters.

---

@gordanr commented on Sun Jun 21 2015

This is a piece of code from one of my old WinForms application.

List<int> awardsIds = panelAwards.Controls.OfType<CheckBox>()
                     .Where(c => c.Checked)
                     .Select(c => (int)c.Tag).ToList();

Now we can write

var awardIds = from c in panelAwards
               apply OfType<CheckBox>
               where c.Checked  // CheckBox has property Checked.
               select (int)c.Tag
               apply ToList;

Very nice.

---

@gordanr commented on Sun Jun 21 2015

Element operations have Immediate Execution. These methods are always on the end of query or subquery.

apply ElementAt n
apply ElementAtOrDefault n
apply First
apply FirstOrDefault 
apply Last
apply LastOrDefault
apply Single
apply SingleOrDefault

var x = from ... 
        where ...
        select expression 
        apply First;

Or in some form of possible infix notation. This notation can coexist with apply.

var x = from ... 
        where ...
        select First expression 
     // select first expression 

---

@gordanr commented on Sun Jun 21 2015

This group of my favourite methods have Immediate Execution. These methods are always on the end of query or subquery.

ToArray, ToDictionary, ToList, ToLookup

var r = from n in numbers
        where n % 2 == 0
        select n
        apply ToList;

var r = from n in numbers
        where n % 2 == 0
        apply ToList;

var r = from n in numbers
        where n % 2 == 0
        select n 
        as List;

Dictionary<int,Order> orders =  
    customers.SelectMany(c => c.Orders)
    .Where(o => o.OrderDate.Year == 2005).ToDictionary(o => o.OrderId);

var orders = from c in customers
             from o in c.Orders
             where o.OrderDate.Year == 2005
             apply ToDictionary o.OrderId

---

@gordanr commented on Sun Jun 21 2015

Methods All and Any have Immediate Execution. These methods are always on the end of query or subquery.

Here is one VB example.

Dim query = From pers In people 
            Where (Aggregate pt In pers.Pets Into All(pt.Age > 2)) 
            Select pers.Name

Dim query = From pers In people 
            Where (Aggregate pt In pers.Pets Into Any(pt.Age > 7)) 
            Select pers.Name

And some F#.

query {
    for student in db.Student do
    where (query { for courseSelection in db.CourseSelection do
                   exists (courseSelection.StudentID = student.StudentID) })
    select student
}

I am not convinced that it is good to introduce special form of syntax for this purpose which exists in VB.

var query = from pers in people 
            where (from pt In pers.Pets select pet apply Any pt.Age > 7) 
         // where (from pt In pers.Pets apply Any pt.Age > 7) 
            select pers.Name

---

@gordanr commented on Sun Jun 21 2015

These methods have Immediate Execution. These methods are always on the end of query or subquery.

Here is VB syntax.

Dim avg = Aggregate temp In temperatures Into Average()

Dim highTemps As Integer = Aggregate temp In temperatures Into Count(temp >= 80)

Dim numTemps As Long = Aggregate temp In temperatures Into LongCount()

Dim maxTemp = Aggregate temp In temperatures Into Max()

Dim minTemp = Aggregate temp In temperatures Into Min()

Dim orderTotal = Aggregate order In orders Into Sum(order.Amount)

And some F#

query {
    for student in db.Student do
    averageByNullable (Nullable.float student.Age)
    }

query {
    for student in db.Student do
    averageBy (float student.StudentID)
}

query {
   for student in db.Student do
   sumBy student.StudentID
   }

let student =
    query {
        for student in db.Student do
        maxBy student.StudentID
    }

We can avoid special form of Aggregate syntax that exists in VB.

Dim customerMax = From cust In customers
                  Aggregate order In cust.Orders Into MaxOrder = Max(order.Amount)
                  Select cust.CompanyName, MaxOrder

var customerMax = from cust in customers
                  let maxOrder = from o cust.Orders select o.Amount apply Max
               // let maxOrder = from o cust.Orders apply Max o.Amount
                  select cust.CompanyName, MaxOrder

Some possible infix improvements that can coexist with apply.

var customerMax = from cust in customers
               // let maxOrder = from o cust.Orders select Max o.Amount // infix form
               // let maxOrder = from o cust.Orders select max o.Amount // infix form, deep integration
                  select cust.CompanyName, MaxOrder

---

@gordanr commented on Sun Jun 21 2015

Here is VB syntax.

Dim query = From word In words Skip 4

Dim query = From word In words Skip While word.Substring(0, 1) = "a" 

Dim query = From word In words Take 2

Dim query = From word In words Take While word.Length < 6

And some F#

 query {
    for number in data do
    skipWhile (number < 3)
    select student
    }

C# Examples

var numbersSubset = numbers.Take(5).Skip(4);

from n in numbers
select n
apply Take 5
apply Skip 4

from n in numbers
select n
take 5 // possible syntax in the future?
skip 4

var remaining = numbers.SkipWhile(n => n < 9);

var remaining = from n in numbers
                apply SkipWhile n < 9
                select n

var remaining = from n in numbers
                skipWhile n < 9 // possible syntax in the future? deep integration?
                select n

var result = products.OrderByDescending(p => p.UnitPrice).Take(10);

var result = from p in products
             orderby p.UnitPrice descending
             apply Take 10;

var result = from p in products
             orderby p.UnitPrice descending
             take 10; // possible syntax in the future?

---

@gordanr commented on Sun Jun 21 2015

VB example

Dim distinctQuery = From grade In classGrades Select grade Distinct

C# example

IEnumerable<string> productCategories = products.Select(p => p.Category).Distinct();

var productCategories = from p in products
                        select p.Category
                        apply Distinct;

Or some possible alternatives.

var productCategories = from p in products
                        select p.Category Distinct;
                     // select p.Category distinct;
                     // select distinct p.Category;
                     // select Distinct p.Category;

@paulomorgado I am not sure that method Distinct has correct syntax.

customers.Distinct(c => c.CountryID).Select(c => c.Country.Name)

---

@gordanr commented on Mon Jun 22 2015

Some examples in C# and F#

char[] apple = { 'a', 'p', 'p', 'l', 'e' };
char[] reversed = apple.Reverse().ToArray();

from c in apple
select c
apply Reverse
apply ToArray;

from c in apple
select c
apply Reverse
as Array; // That's my proposal for materialization with 'as'.

let b =
    query {
        for student in db.Student do
        select student.Age
        contains 11
    }

bool b = from student in db.Student
         select student.Age
         apply Contains 11

---

@paulomorgado commented on Sun Jun 21 2015

@gordanr,

@paulomorgado I am not sure that method Distinct has correct syntax.
customers.Distinct(c => c.CountryID).Select(c => c.Country.Name)

I was thinking of some of my extensions - LINQ: Enhancing Distinct With The SelectorEqualityComparer

---

@weitzhandler commented on Tue Dec 15 2015

dotnet/roslyn#100 (comment)

I strongly vote for this one.

I never use LINQ language queries and instead my coding guidelines is avoid using them and always use the extension methods directly, and that's because the ugly parentheses each query has to be wrapped in order to materialize it (ToArray, ToList, Sum, SingleOrDefault etc.).

Until this issue is addressed, language-build-in LINQ is merely useless.
I really hope to see this implemented soon. Maybe to a more limited extent (avoiding the introduction of a gazillion new language keywords.

I'd say the syntax should provide an operator that expects a shortcutted extension method available for IEnumerable<TElement>, for instance:

//parents is Parent[]
var parents = from student in students
                     where student.Age < 18
                     select student.Parent
                     call ToArray()

//student is Student
var student = from st in students
                      call SingleOrDefault(st => st.Id == id);

Asynchronous methods should also be supported:

   var student = from st in students
                         call await SingleOrDefaultAsync(st => st.Id == id);

Maybe there should be a verbose LINQ fashioned way to pass the arguments and completely avoid the use of parenthesis, but I personally don't see it as necessary.

Anyway this feature is crucial for the completeness of the LINQ syntax.

Some suggestions above proposed the ToList at the beginning of the query, but that's a no go, since we want to be able to process it after the selection, and we don't want to be limited to parameterless ex. methods only. What if we wanna call ToLookup with a key selector, bottom line we can't tie it up to the language, we just need to find a way to call any Enumerable ex. methods on the current query state and treat its result as the final type of the query.

[john-cd]

Can we skip apply and just use convention - the query contextual keyword is the Enumerable extension method name, lowercased ?

.Select(...) --> select
.Take(...) --> take

On a side note, ToLookup needs some love - meaning corresponding classes in` System.Collections.Generic.

[paulomorgado]

Why limit it to a single class? It should be any available method, instance or extension.

[orthoxerox]

This could be emulated with |> (#74), but the pipe would terminate the query, so while something like this would work:

var string6789 =
    from x in Enumerable.Range(1, 10)
    |> @.Skip(5)
    |> @.Take(4)
    |> @.Aggregate("", (y, z) => y + z);

this wouldn't:

var sixseveneightnine =
    from x in Enumerable.Range(1, 10)
    |> @.Skip(5)
    |> @.Take(4)
    select SpellOut(x, CultureInfo.CurrentCulture); //error
    |> @.Aggregate("", (y, z) => y + z);

Our hypothetical apply keyword would have to be very smart and disambiguate between continuing the query expression and terminating it with Aggregate or ToList. Since LINQ is 100% syntax sugar with no semantic influence, this is impossible. Therefore I see multiple options:

1. Introduce a keyword that continues the query and a keyword that terminates it:

var sixseveneightnine =
    from x in Enumerable.Range(1, 10)
    foo Skip(5)
    foo Take(4)
    select SpellOut(x, CultureInfo.CurrentCulture);
    bar Aggregate("", (y, z) => y + z);

2. Introduce a keyword that continues the query and use |> to terminate it. It forces you to use @., so it's a bit less aesthetically pleasing.

var sixseveneightnine =
    from x in Enumerable.Range(1, 10)
    foo Skip(5)
    foo Take(4)
    select SpellOut(x, CultureInfo.CurrentCulture);
    |> @.Aggregate("", (y, z) => y + z);

3. Introduce multiple keywords for existing standard LINQ methods and a keyword that terminates a LINQ query expression.

var sixseveneightnine =
    from x in Enumerable.Range(1, 10)
    skip 5
    take 4
    select SpellOut(x, CultureInfo.CurrentCulture);
    bar Aggregate("", (y, z) => y + z);

4. Introduce multiple keywords for existing standard LINQ methods and use |> to terminate the expression. It forces you to use @., so it's a bit less aesthetically pleasing.

var sixseveneightnine =
    from x in Enumerable.Range(1, 10)
    skip 5
    take 4
    select SpellOut(x, CultureInfo.CurrentCulture);
    |> @.Aggregate("", (y, z) => y + z);

5. Introduce a keyword that continues the query and multiple keywords for existing standard LINQ terminators:

var sixseveneightnine =
    from x in Enumerable.Range(1, 10)
    foo Skip(5)
    foo Take(4)
    select SpellOut(x, CultureInfo.CurrentCulture);
    aggregate from "" using string.Concat;

6. Introduce multiple keywords for existing standard LINQ methods and multiple keywords for existing standard LINQ terminators:

var sixseveneightnine =
    from x in Enumerable.Range(1, 10)
    skip 5
    take 4
    select SpellOut(x, CultureInfo.CurrentCulture);
    aggregate from "" using string.Concat;

7. Introduce an attribute that makes a method a LINQ query syntax keyword (a la F#), this would be visually similar to 2, 3 and 6.

[weitzhandler]

Personally I don't like the idea of polluting the language with so many new keywords. I vote for C# being minimalistic when not-necessary otherwise, and I specially dislike the idea of |> F#/Haskell-flavored operators in C#.

I'd rather seeing what was proposed above, just one additional keyword (do? instead foo in @orthoxerox' example? or whatever you see fit), that will use as a materialization bridge and would accept a method of the previous query, i.e. (A):

var userNames = from user in _DbContext.Users
                where user.Rating > 3.0
                select user.UserName      
                do Skip(5)
                do Take(4)                   
                do ToArray;

We could expand it with some rules, i.e. if the materialization method takes a single reference/value parameter, or the same delegate/predicate with a parameter as the main type (from user), or parameterless, the method's parameters can be omitted and used directly, as also suggested above) (B):

var userNames = from user in _DbContext.Users
                //some gazillion other search operations, sorting etc.
                do SingleOrDefault user.UserName == "hello";

var users = from user in _DbContext.Users
            do Take 3
            do ToDictionary user.UserId, user

If the pattern isn't a type of user, not parameterless, or doesn't take a single non-LINQ-expression type, full method with parameters is used:

var roles = from user in _DbContext.Users
            select user.Roles
            do Distinct
            do ToDictionary(role => role.Id, role => role.Name);

I'm fine with only part (A), I'm sure part (B) could be achieved in a much more efficient a way if necessary at all.
Some things like the plain apply etc. suggested above would be sufficient, IMHO.

[mattwar]

When working on the original LINQ design we did consider having a 'do' clause, but eventually decided against it because it just ended up being the same thing as dot. Maybe time to reconsider?

[paulomorgado]

@mattwar, if you look at this one, it's not the same thing:

var roles = from user in _DbContext.Users
            do Skip 5
            where user.Rating > 3.0
            select user;

[orthoxerox]

@paulomorgado you're mixing up two dos. One continues the expression, the other terminates it.

[paulomorgado]

@orthoxerox, why do they need to be different? Apart from select that might be a noop, aren't they all method calls?

[weitzhandler]

@orthoxerox @paulomorgado they shouldn't. If the called method contains an iteration (i.e. ToArray), it will lead to the termination of the query, so what's now? Will the do be restricted to LINQ context only? Imagine do Console.WriteLine "Hello" which introduces a whole new syntax mess.
The idea of query materialization is awesome and required, still warning flags should be raised.

[paulomorgado]

Any instance or extension method of the expression type with none or one expression (Func<T1,T2> or Expression<Func<T1,T2>>, depending on if it's for enumerables or queryables) should work.

...
do ToArray
do ToString

Could work. But should it work?

[CyrusNajmabadi]

I personally want this. However, i think one of the cleanest ways we could do this would be to make the syntax either:

do    binding-expression

or

apply    binding-expression

'binding expressions' are what you can get after a ?. i.e. the .X().Y in z?.X().Y. They innately represent "the thing that comes on the right side of some instance".

Substantively, what this would mean is that your query would actually look like:

var result = from c in customers
             where c.Age > 21
             do .ToList();

The reason i like this is that it feels inherently weird to me to just have a standalone expression ToList() that then needs to be resolved as if it was being called off of something. We already have all the language support and all the plumbing to support 'Binding Expressions' in the compiler. So i actually think this would also be extremely cheap to provide.

Finally, i think it would be nice for the syntax to clearly call out that you're not calling some instance member of your own type (i.e. if you had your own ToList method in scope). By being .ToList() it would be clear that it is being called off of something. And that thing is the result of the query so far.

[weitzhandler]

@CyrusNajmabadi yeah, so how about params? ToDictionary? Did you read the whole thread?

[paulomorgado]

@CyrusNajmabadi, at this point, looks like a punctuation character would be better than do. I keep seeing suggestions of using @ and it doesn't look so bad here:

var result = from c in customers
             where c.Age > 21
             @.ToList();

Not wanting to discard the already existing infrastructure you mentioned, I would prefer something where I can do this:

var result = from o in orders
             from c in customers
             do Distinct c.Id
             where c.Age > 21
             do ToList;

over this:

var result = from o in orders
             from c in customers
             do .Distinct(c => c.Id)
             where c.Age > 21
             do .ToList;

because, in the end, I've gained nothing over:

var result = (from o in orders
              from c in customers.Distinct(c => c.Id)
              where c.Age > 21
             ).ToList;

[CyrusNajmabadi]

yeah, so how about params?

Sorry, not sure what you mean by that. Do you mean 'parameters'? Or actual 'params'?

ToDictionary

In my mind, 'do/apply' is just a way to bridge to all things outside of queries. As such, what you put after 'do/apply' is simply a 'BindingExpression' with all the rules inherent therein. So, if you want to use ToDictionary, you write:

previous-clauses
do .ToDictionary(x => x.Whatever)
following-clauses

Did you read the whole thread?

Yes. I've been following since the beginning. But i thought i would weigh in on an approach that would stand the best chance of being implemented, while also solving the largest pain points for me, namely:

1. Needing to parenthesize queries, just to be able to 'dot' them makes for a very irksome coding experience.
2. once i 'dot' something, i'm no longer in a query and it's a pain to continue on.

I'd like to be able to just add a do .SkipWhile(e => NotThere(e, etc)) in the middle of my query and not be pulled out so unpleasantly like i am today. Right now when i hit that sort of issue, effectively the entire query will be rewritten to be in method-call form. I'd like a gentler path that can avoid that from happening.

[CyrusNajmabadi]

because, in the end, I've gained nothing over:

I disagree. For me, i get pulled out of the 'flow' of queries when i now have to go back, wrap it in parens, and then dot off it. Furthermore, if i want to continue the query onwards, that's another piece of awkwardness.

I'd like a gentler way to bridge the two worlds. But i'm not really looking for a way to take a huge swath of C# constructs and have them auto-translated to some sort of 'new query syntax' that maps to/from them.

--

Seriously, in your example, i find:

var result = from o in orders
             from c in customers
             do .Distinct(c => c.Id)
             where c.Age > 21
             do .ToList();

To be far more readable than your lower piece of code.

[paulomorgado]

@HaloFour, why would ToDictionary have to be terminal?

[mattwar]

@HaloFour probably a function that changes type illustrates the problem better.

var query = from x in Enumerable.Range(0, 10)
    do .ConvertToString()
    select x;  // what the Heck is x supposed to be here?

[paulomorgado]

@mattwar, what's the signature of .ConvertToString?

Can you write, on today's LINQ, this?

from i in ints
select i.ToString()
select i

[mattwar]

@paulomorgado No, you'd have to write it this way.

from i in ints
select i.ToString()

or if you really want to select again, you'd do it like this

from i in ints
select i.ToString() into s
select s

[HaloFour]

@paulomorgado

what's the signature of MultiplyBy2?

public static class MyExtensions {
    public static IEnumerable<int> MultiplyBy2(this IEnumerable<int> numbers) {
        foreach (var number in numbers) {
            yield return number * 2;
        }
    }
}

why would ToDictionary have to be terminal?

It wouldn't have to be, but as a non-terminal operation is it certainly projecting a value different than the input values. There's no reasonable way to correlate any particular range variables leading up to that transform with whatever might be enumerated from that operation. It's not like the Dictionary<TKey, TValue> returned by ToDictionary will enumerate the values in the same order in which they were input.

[paulomorgado]

@mattwar, so, why would do ConvertToString be any different?

[paulomorgado]

@HaloFour, so, MultiplyBy2 takes an IEnumerable<T> and produces an IEnumerable<T>. Is that any different from Where?

[orthoxerox]

@paulomorgado

from x in xs
where x > 2 //there
do .ToString()
where x.EndsWith("0") //there
select x

In two lines marked //there the range variable x has a different type. Currently supported methods generally ensure this never happens (unless you do something like this). By opening the doors to all possible methods we abandon this assumption. I generally do not think it will be too dangerous, but I must agree that it doesn't support the whole "pit of success" concept. It does give you more options to shoot yourself in the foot.

[paulomorgado]

@orthoxerox,

That would be equivalent to:

from x in xs
where x > 2 //there
select x.ToString()
where x.EndsWith("0") //there
select x

Can you do that now?

[CyrusNajmabadi]

@paulomorgado You can not write that today. 'select' ends a query.

[CyrusNajmabadi]

IMO, a do binding clause could only be followed by more do binding clauses. However, if the last do binding clause was a do binding into v clause, then you'd be able to continue adding more clauses, which would only be able to use the projected variable you just created.

[orthoxerox]

@CyrusNajmabadi isn't into v its own clause, "query continuation"? So basically we say "between select/group by" and an optional "into" you can have as many "do" clauses as you want?

I think the only tricky part will be IntelliSense for the consecutive "do" clauses, but you definitely know more about that than anyone.

[CyrusNajmabadi]

Right now this is the query grammar:

query-body:
    query-body-clause*   select-or-group-clause   query-continuation?

I'm proposing changing that to:

query-body:
    query-body-clause*   select-or-group-clause  do-clause*   query-continuation?

I think the only tricky part will be IntelliSense for the consecutive "do" clauses, but you definitely know more about that than anyone.

I think it should be fine. A ... do .<Binding> just gets translated (...).<Binding> (syntactically i might add).

If you have a continuation, i.e. ... do .<Binding> into x ... then that gets translated into first into:

(...).<Binding> into x which then gets translated exactly as we already state in the spec to:
from x in (...).<Binding>.

All of this should fall out naturally from both how we handle BindingExpressions today, and how 'into' continuations already work.

[phi1010]

@HaloFour probably a function that changes type illustrates the problem better.

var query = from x in Enumerable.Range(0, 10)
do .ConvertToString()
select x;  // what the Heck is x supposed to be here?

@mattwar: What about throwing all variables of the context into named tuples? Some custom linq statement implementations accepting an IEnumerable<T> and something like a Func<T,TSomethingElse> (e.g. Where(tuple=>Test(tuple.x)) with bools) or the proper specific tuple signature would still be able to handle them -- and the return type would determine whether there is an x defined after executing the function -- which e.g. would not be the case after using an aggregate function.

(See #117 (comment) for an example with a Zip()-Linq-Statement -- still using another syntax)

[orthoxerox]

@CyrusNajmabadi I've played around a bit with this change, and "All of this should fall out naturally from both how we handle BindingExpressions today" wasn't 100% true. Both types of binding access expressions were really attached to conditional access expressions and I had to slap them around a bit to make them accept a do clause as their new daddy. I'll see if this change can be made more polished after I'm done with take while.