[Proposal] Syntactic ranges

[GeirGrusom]

I suggest a feature in C# to easily define ranges. It's a replacement for Enumerage.Range except it defines start and stop points rather than start and count.

Sometime you wish to create a range to perhaps .Select from. I propose the following syntax:

'[' expression '..' expression ']'

The requirements of the type from expression is that both have to be of the same type. You cannot create a range between float and integer for example. The result set is [a → b].

edit: changed to inclusive. F# is inclusive, so why should C# be different?

The type of expression also must support an appropriate incremental operator, prefix ++, postfix++ or += (int), or += T if expression is implicitly convertible from int, in that order.

If the order is reversed (high to low) the result of the expression is an empty set.

Example:

foreach(var item in [1 .. 100])
{
  Console.WriteLine(item);
}

An extension could be that it allowed a third argument:

'[' expression '..' expression (',' expression)? ']'

This allows the developer to specify an increment step. The type of the argument must be implicitly convertible to T.

foreach(var radian in [0.0 .. Math.PI, Math.PI / 180])
{
  DrawPoint(Math.Cos(radian) * radius, Math.Sin(radian) * radius);
}

edit: removed left hand type inference.

[orthoxerox]

Without commenting yet on any other parts of your proposal, I object to the use of .. to indicate a [a, b) interval. I think having .. to indicate [a, b] and ..< to indicate [a, b) would be a better option.

[alrz]

The form with lambda expression is a no. "Increment operation" seems silly. it should use a value for custom steps, just like F# or any other language.

Also it would be nice if it would capable of representing infinite ranges.

[GeirGrusom]

@alrz I agree.

[GeirGrusom]

@orthoxerox I thought a lot about this, and I do agree with you, but I used this simply because in my opinion it's the most useful behavior without complicating syntax.

[alrz]

I believe #6949 should use target type to return desired type of collection. Same is applicable here.

int[] arr = [ 0 .. 10 ];
List<int> list = [ 0 .. 10 ];
IEnumerable<int> infinite = [ 0 .. ];

[GeirGrusom]

Why not .ToArray() and .ToList()?

[alrz]

@GeirGrusom I don't know what translations you have in mind for ranges. (1) Having all of them translated to Enumerable.Range is not a good idea and (2) with steps, it's not even possible. (3) For my first example I think ToArray has a performance hit, since compiler knows what type should be generated at first place.

F# fills arrays for both [ ... ] and [| ... |], and uses ToList in case of the former. For infinite ranges it implements IEnumerable I guess.

[orthoxerox]

@GeirGrusom I think having .. combined with square brackets doesn't adequately communicate to the reader that the right-hand value is excluded from the range.

On the other hand,

    public static IEnumerable<int> r(int from, int to) {
        for (var i = from, i <= to, i++)
            yield return i;
    }

allows you to write r(0,100) and is just as brief. Yes, this cannot be generalized to all addable values, but writing another helper is quite easy.

[alrz]

@orthoxerox You love this one don't you? 😉

[GeirGrusom]

@alrz

(1) It's not, and that wasn't the point. It's to make a better substitution for Enumerable.Range, because Enumerable.Range is both a long expression to write, and there are tons of things it can't express.

(3) You are making an assumption on what the compiler will do. However you are treading into something I think is outside of this proposal's scope.

@orthoxerox yes I agree that the square brackets may come off as a little bit misleading.

However the generator has the issue of creating a state machine in every use-case, and you cannot generalize it as you mentioned. I think ranges is a common enough use case that it warrants syntax.

The C# compiler can generate a normal for-loop for common ranges, but using your generators it will never do so.

[alrz]

you are treading into something I think is outside of this proposal's scope.

oohhhkay.

[GeirGrusom]

@alrz

Fine! Jeez!

edit: but any performance benefits I would claim is imaginary.

[GeirGrusom]

What do you think about the range issue? Should it simply be inclusive, or should there be a way to show that the range obviously is exclusive?

[GeirGrusom]

@alrz What is really the benefit of having an implicit conversion from range to arrays and collections?
I struggle to see it as anything but a complication of the proposal.

[orthoxerox]

@GeirGrusom You will be surprised by the implementation of Enumerable.Range(), then. It is a generator with a few helper functions for collecting the values.

How do you propose these ranges work on doubles, btw?

[GeirGrusom]

You will be surprised by the implementation of Enumerable.Range(), then. It is a generator with a few helper functions for collecting the values.

Yes I know. It's up to the compiler to actually use it as a generator though. It probably does, but it doesn't have to. Same with ranges. The compiler can absolutely generate generators, but it doesn't have to. It depends on context. for(var i in [0 .. 100]) for example can turn into for(int i = 0; i < 100; i++) which is why the tail is exclusive instead of inclusive.

For doubles (or any type really) the default behavior should be incremented by 1 implicitly (should probably allow explicit as well) cast to the expression type. This is the rule for any type, not just integers. Any issue with precision is in the hands of the developer like in every other case.

[alrz]

@GeirGrusom You cannot assume anything about how this feature will be used. In F# you can explicitly specify the desired type:

let arr = [| 0 .. 10 |]
let list = [ 0 .. 10 ]
let seq = seq { 0 .. 10 }

As well for clusivity. You shouldn't limit a feature to just one possibility, otherwise it woudn't be usable that much. These are not "out of scope of this proposal". I should ask, what is the benefit of having ranges as a language feature in the first place? Yes, clarity. So if ranges are meant to be used instead of any other mechanism they should provide best performance and not be bound to a specific type. See #6949 for more on this.

for(var i in [0 .. 100]) for example can turn into for(int i = 0; i < 100; i++)

It's not up to the compiler to do these optimizations. Just like #7580.

[GeirGrusom]

You cannot assume anything about how this feature will be used. In F# you can explicitly specify the desired type:

You should check the IL generated by those expressions, because the F# creates .ToArray() and .ToList() on the iterators. So [| |] is syntactic sugar for .ToArray and [ ] is the same for .ToList.

edit: which is really obvious. How else should it be represented in IL?

As well for clusivity. You shouldn't limit a feature to just one possibility, otherwise it woudn't be usable that much.

How did I limit anything? I just proposed a range syntax.

These are not "out of scope of this proposal". I should ask, what is the benefit of having ranges as a language feature in the first place? Yes, clarity. So if ranges are meant to be used instead of any other mechanism they should provide best performance and not be bound to a specific type.

There is really nothing in .ToList() or .ToArray() that indicates poor performance. It's a hint for the compiler (the JIT or AOT compiler).

It's not up to the compiler to do these optimizations

For Roslyn? Perhaps not, but for JIT and AOT compilers it certainly is. It may be smart to define a concrete type that is returned by ranges though, but I would rather let compiler developers think about performance impact than me as a customer. They are much more competent in the matter.

[alrz]

@GeirGrusom Yes, you're right F# doesn't do that. I'm thinking that it can fill an array, by performance hit I meant iterating an IEnumerable which is indeed slower than iterating an array.

[alrz]

I think brackets are not necessary and also it can be a pattern, e.g.

foreach(var i in 1..10) {}

var integer = 5;
switch(integer) {
    case 1..10: break;
}

If we don't use a generic expression (which I believe is ambiguous), inclusive ranges would be just a minus away,

var n = 10;
var i = 1..n; // 1 to 10
var j = 1..n - 1; // 1 to 9

Besides of stepped ranges 0..2..10, 0.. or 0..2.. would denote an infinite lazy range which also can be used as a pattern.

[orthoxerox]

@alrz shouldn't .. have higher precedence than -? 1..n - 1 should be 0 to 9 if n is 10.

[alrz]

@orthoxerox Actually it should have lower precedence (then - get evaluated earlier), anyway, that was my point. In the original post it uses a general expression which makes a lot of (ambiguous) unnecessary things possible.

[paulomorgado]

I'm failing to see the value of this over a library/libraries.

[alrz]

@paulomorgado If you ask me, anything more than earlier versions of C# could be done with a library or a bunch of boilerplate code, I think the point is to avoid just that, otherwise, none of these are really necessary. (don't take this personal but the whole value of #3571 is to avoid a pair of parentheses, right?)

[orthoxerox]

@alrz you're right, I always confuse the two meanings.

[paulomorgado]

@alrz

I don't see Enumerable.Range(1, 10) as "bunch of boilerplate code".

The minimal value of #3571 is a pair of parenthesis and a do,.but can be more. It was just my take on the many proposals for something like that. But, in the end, it's not a way to replace libraries but a way to leverage libraries.

[alrz]

@paulomorgado To support your point, I can tell that range patterns also can be implemented via is

class Between { public static bool operator is(this int self, int from, int to) => self <= from && self >= to; }
if(integer is Between(1, 10)) {}
integer switch(case Between(1, 10): ...)
// etc

But given that this is really a basic pattern to follow I think language should make it easier to do, nevertheless, I can live without it. 😄

[GeirGrusom]

@paulomorgado

I'm failing to see the value of this over a library/libraries.

More expressive. To the point.

I don't see Enumerable.Range(1, 10) as "bunch of boilerplate code".

If you need anything other than Int32 you have to reimplement it. That's boilerplate.

[paulomorgado]

@GeirGrusom,

I don't see Enumerable.Range(1, 10) as "bunch of boilerplate code".

If you need anything other than Int32 you have to reimplement it. That's boilerplate.

Give that the libraries would have to exist, what would that boilerplate code would be?

You can't justify new language features that will require libraries with the fact that those libraries don't exist now.

[GeirGrusom]

@paulomorgado

Give that the libraries would have to exist, what would that boilerplate code would be?

Imagine if Nullable wasn't a language feature. Imagine that you couldn't create a generic nullable container (because unlike nullable, ranges cannot be generic). Would you advocate that they should keep nullable out of the language and rather that everyone implemented NullableInt, NullableBool, NullableVector4 etc. everywhere? It's not boilerplate if there's a library for it you claim.

You can't justify new language features that will require libraries with the fact that those libraries don't exist now.

No one is going to write a library to create ranges for every imaginable type. System.Numerics.Vectors.Vector4 for example doesn't have a Range implementation. You would have to either find someone who has implemented it, which is a waste of time because 1) it doesn't take much code to implement Range for it and 2) No one is going to be running around implementing a range library simply for Vector ranges. However having a range syntax spares you from doing this work at all, and the result very clearly states the intention of the code in a format that everyone understands.

[GeirGrusom]

@alrz brackets may not be necessary. Also perhaps increment could be specified using += -= *= /=?

var range = 0 .. 10 += 2;

[alrz]

@GeirGrusom It will be ambiguous, it can be parsed like 0..(10 += 2).

[GeirGrusom]

[0 .. 10] += 2

[alrz]

you mean like built-in operators for ranges? See, IEnumerable should be generated and returned by range so it doesn't make sense to apply += to actually change the whole sequence.

[GeirGrusom]

Delegate's += swaps out the delegate. How is this any different?

[GeirGrusom]

The idea is that += operator applied on a range type results in a new range type with a different step. It doesn't change it, it creates a new type.

IEnumerable shouldn't be returned by a range. It should return some type that implements IEnumerable.

[alrz]

So actually, when I need a stepped range I have to allocate two and abandon the first one somehow? Then what is the point of ranges at all!

[GeirGrusom]

So actually, when I need a stepped range I have to allocate two and abandon the first one somehow? Then what is the point of ranges at all!

What? Where do you get that from? It's not like 1 + 2 + 3 + 4 * 3 actually does all those calculations in runtime, so why on earth does the ranges have to?

[paulomorgado]

@GeirGrusom, Nullable<T> is not a good example because it needed CLR changes and could e done, as it was, as library.

You are using Nullable<T> to make a point that you haven't proved yet. What in your proposal cannot be done as a library, regardless of needing or not boilerplate code?

[GeirGrusom]

@paulomorgado

What in your proposal cannot be done as a library, regardless of needing or not boilerplate code?

It would support NodaTime or Vector4, or any type that supports increment without someone writing a library or boilerplate for it. It also clearly indicates what the expression does. It also means that the expression makes sense to the compiler such that a foreach can be implemented as a normal loop instead of using the iterator.

[Thaina]

I was making my own extension method

public static IEnumerable<int> To(this int start,int end)
{
    while(start < end)
        yield return start++;
}

foreach(int x in 1.To(10))
    x;

Should be just this

Any other transform could be made from math on Linq

1.To(10).Select((x) => x * 2)

[gafter]

See dotnet/csharplang#185 for a feature planned for C# 8. Future language discussion should be directed to https://github.com/dotnet/csharplang .