Proposal: Compile time expressions

[jcouv]

@nvmkpk commented on Mon Jun 27 2016

It would be useful to specify something like "Hello World".Length in some special syntax that gets evaluated at compile time and replaced with 11 (becoming a constant) in the compiled binaries.

---

@choikwa commented on Mon Jun 27 2016

You mean it's not already converted to constant?

---

@nvmkpk commented on Mon Jun 27 2016

JIT compiler might optimize it but I want the language compiler to do it. In the generated IL code the string is constant but the value of its length is not.

---

@nvmkpk commented on Tue Jun 28 2016

I gave string as example. I want to be able to use more expressions to evaluated at compile time. Something like eval(simple expression).

---

@svick commented on Tue Jun 28 2016

What is the reason? Is it about performance? Or something else?

---

@alrz commented on Tue Jun 28 2016

This is also related to #10972 and #9627 (there are similar examples in the latter issue).

---

@nvmkpk commented on Tue Jun 28 2016

The reason is simple. If we know that the value of an expression at run time is exactly same as at compile time, there is no point in deferring evaluation to run time.

---

@Unknown6656 commented on Wed Jun 29 2016

@nvmkpk : I do quite like the idea, I do however see the following problem:

Imagine, that one is writing a (memory) profiler. The profiler would not register the function call int System::String::get_Length(), as the function would be "precompiled" into its result. Thus the application profiling result could be incorrect compared to expected data/values.

My second question/concern: To which degree do you want this "optimization"? Should it apply to each completely deterministic function(-chain) with only constant input?
e.g. should the expression "Hello,World!".ToUpper().Remove(0,1) be precompiled into "ELLO,WORLD!" ? Where do we draw the line?

EDIT: or should one use a pseudo-attribute like [compile], which one could put in front of fields/expressions?

---

@svick commented on Wed Jun 29 2016

@Unknown6656 That already happens, in the JIT compiled machine code, there is no call to get_Length(), since it's inlined.

And you can't put attributes on expressions, at least not without #6671.

---

@Unknown6656 commented on Wed Jun 29 2016

@svick : Interesting, I did not know the fact about get_Length()-inlining.

Concerning the usage of attributes on expressions: During the composition of my previous comment I assumed, that issue #6671 will be implemented in the future C# version, as it has been marked as
[2 - Ready]

---

@choikwa commented on Wed Jun 29 2016

My second question/concern: To which degree do you want this "optimization"?

Hopefully in Release build, all of the optimizations.

Where do we draw the line?

Where-ever compiler can statically determine static or read-only or const-ness and exploit it 👯

---

@nvmkpk commented on Wed Jun 29 2016

That is fine for optimization part but I would like the developer to also call for compile time evaluation by using something like eval(expression). This would be better way than using attributes.

---

@Unknown6656 commented on Wed Jun 29 2016

@choikwa (, @nvmkpk ): So, would the following expression be optimized (assuming, that a array<string> shall be interpreted as constant, and that all functions do not use non-constant internal parameters and are fully deterministic)?

char a = (new string[] { "test", "string" })[1].Reverse().ToArray()[3]) - 'B';
// a now stores the character `0`

My point is: I really do love your proposal, but does too much optimization not take too much time on a big scale? Are arrays defined at compile time also a constant, if they remain unchanged?
Imagine the calculation time for large for-loops, nested conditions etc. which will be precompiled. Does not take this too much compilation time?

---

If anyone is interested, how the character is calculated in the example above:

string[] arr = new string[] { "test", "string" };
char[] elem = arr[1].Reverse().ToArray(); // [g,n,i,r,s,t]
char r = elem[3]; // 'r' (0x72)
char a = r - 'B'; // 0x72 - 0x42 == 0x30 = '0'
// a now stores the character `0`

---

@choikwa commented on Wed Jun 29 2016

would the following expression be optimized?

A compiler worth its salt would optimize. I am hoping CoreCLR is to be such one. Maybe you can help make that happen 😄

does too much optimization not take too much time on a big scale?

C++ already does this with constexpr and template meta. A saving grace is that you only do this once during compilation. In C# context, maybe this makes more sense in AOT compilation or more expensive ReJIT (don't we all want tiered JIT?).

PS. a lot of compilers already neuter optimization if method size is too large, to the detriment of performance just because compilation time would be excessive. Would not surprise me if CoreCLR had to do something similar since it's a VM.

---

@Unknown6656 commented on Wed Jun 29 2016

@choikwa I think that you have me convinced on that point! 😄
I knew that C++ has precompiled templates instead of runtime-generated generics, but I did not yet know about constexpr. I think, that this is an excellent definition of constant expressions/methods (which needs some adaptation for C#)

Maybe you can help make that happen

I will try as much as I can 😉

---

@choikwa commented on Wed Jun 29 2016

FYI this very issue is open in https://github.com/dotnet/coreclr/issues/3633

---

@nvmkpk commented on Wed Jun 29 2016

That is for optimzation. This proposal is about letting developer write general compile time expressions not just for string length.

---

@choikwa commented on Wed Jun 29 2016

Sorry for hijacking the issue. You are correct, and I agree that this should have some dependence on success of https://github.com/dotnet/roslyn/issues/6671

---

@ashmind commented on Tue Aug 23 2016

On the syntax -- I suggested const(...) in #11259 which has a benefit of already being a keyword.

---

@alrz commented on Thu Apr 20 2017

@nvmkpk May you move this to csharplang repo?

[tuespetre]

👍 for @ashmind's suggestion

[alrz]

I don't think there is a need for a new syntax, the compiler could evaluate some properties/methods at compile-time, the resulting expression could be used in a const context as well, e.g.

switch(obj) {
  case string_constant.Length:
  case string_constant.Substring(...): // string_constant could be a `const`, `nameof`, literal, etc.
  case typeof(T).Namespace:
  case typeof(T).Name:
    break;
}

and so on.

[MgSam]

@alrz People have asked for all kinds of stuff to be compile time constants- including mathematical expressions, string manipulation, date manipulation, regex manipulation, etc. A more general feature allowing anything known statically to be evaluated at compile time would solve all of these types of requests rather than waiting for the dev team to add support for each little use case someone might think of.

[alrz]

a new syntax doesn't mean that it would magically enable all expressions to be evaluated at compile-time. there will be a set rules for the said expressions so that does not lessen the amount of work to make this happen. I don't think making it an opt-in feature would be useful either. some expressions are inherently a constant, evaluating them at compile-time has an advantage of lighting up without any source changes.

[alrz]

If you're talking about "const functions" I'll agree with you, being able to write a function that is able to be evaluated at compile-time will be very useful, but even then the compiler needs to validate the requirements of such functions and at the end, there is no need to use a new syntax at the call-site.

[bbarry]

no need to use a new syntax at the call-site.

There may be a small need for some new syntax at the call site in places where the existing syntax would be allowed runtime syntax. I've suggested before to have a (const) "cast" be available for these expressions.

[tannergooding]

Don't we already have 3 or 4 proposals that cover some form of constexpr for C#?

[TonyValenti]

I would suggest that any method (even non-determinstic ones) should be constexprable.

var BuildDate = constexpr(DateTime.Now.ToString());

[tannergooding]

I would suggest that any method (even non-determinstic ones) should be constexprable.

If that were supported, I would fully expect it to fail if you compiled with /deterministic

[alrz]

@bbarry

Can you give an example where it's absolutely needed to explicitly mark an expression as const? If you're suggesting that we allow evaluating arbitrary expressions at compile-time, I don't think that's something constexpr is about. We make guarantees at declaration-site and pass it over to metadata for client code. There is a reason why we don't allow non-constants in case labels: because we don't want undefined behavior depending on implementation detail. For example, if compiler turns a switch statement to a binary tree, you might lose some intended side effects, or worse, they might evaluate in an unexpected order.

[alrz]

Wouldn't new pattern syntax (dotnet/roslyn#22718) block this feature? imagine you have a compile-time function like this:

static const int Add3(reaodnly int a) => a + 3;

then if you want to use it in a switch case:

case Add3(4):

it would be parsed as a pattern after this change, and not an invocation expression.

[Pzixel]

@TonyValenti you're already being answered here: dotnet/roslyn#15079 (comment)

There's a rule in the compiler world: if a compiler evaluates an expression at compile time then it better ensure that the result is identical with the result obtained if the expression is evaluated at runtime. Anything less than that is usually considered a bug, plain and simple.

I totally agree with this.

However, I'd like to have pure methods even if they aren't currently optimized in any way. All public string methods are pure so it would be really nice if we could use them at compile time.

[HaloFour]

@Pzixel

Even with String there could be subtle differences. For example, if the result of two expressions are the same string value they would still be different string instances. But if those expressions were evaluated at compile-time into string literals then they would be the same string instance. Probably a trivial concern, but a change nonetheless.

[Pzixel]

@HaloFour I think we're ok with that. I can't imagine a real case when it would break anything. You want compare strings, but only by reference? Whoa. That doesn't seems to be performed often. And it works with constants only, so it wouldn't affect most of code anyway. It's just a better indicator for developers hey, you are calling a pure method but do not use return value, and source of some optimizations.

It seems much smaller and safer change than change in loop closures from 3 3 3 to 1 2 3.

[CyrusNajmabadi]

All public string methods are pure so it would be really nice if we could use them at compile time.

I'm not sure this is true. For example, string.CompareTo says this:

This method performs a word (case-sensitive and culture-sensitive) comparison using the current culture.

Therefore:

1. If the compiler runs in a different culture than the actual program, this will result in different behavior.
2. If the user were to change the culture at runtime, this would now not be reflected properly.

String, IMO, would be one of the hardest APIs to get right for compile time evaluation unless you were extremely careful to only whitelist the ordinal methods. Even then, i wouldn't be certain that something wouldn't be going wrong :-/

[TonyValenti]

I really wish I had some compiler experience because I really don’t see this as very difficult at all. 1. Compile the code. 2. Everywhere there is a const expr, reflection.invoke the member and replace the source node with the evaluated value. 3. Recompile.

On Fri, Nov 17, 2017 at 4:53 PM Joseph Musser ***@***.***> wrote: Nonetheless, it's something that easily the majority of us are mentally waiting for, especially in light of all the threads where the reasonable response is, "wait and do this with source generation." So please don't just close your eyes, with that or with this since they are related. =) — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <https://github.com/dotnet/csharplang/issues/1028#issuecomment-345387659>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AM-qVs465b6qMr9y_fQNgD8sRAjkeDVAks5s3g6GgaJpZM4QB0Sx> .

-- Tony Valenti

[jnm2]

I'm not saying it's not difficult.

[jrmoreno1]

For anyone questioning whether constant expressions could be useful, I’ suggest watching this video:
https://youtu.be/zBkNBP00wJE. It’s in c++ and not directly applicable, but it shows that this can have a huge impact.

[TonyValenti]

@jrmoreno1 -
I've come up with a solution that kind of gets me to where I want. It is a nasty, nasty hack but it is better than nothing.

1. I have a solution that has my standard, non-constant methods in it.
2. I create a second solution that has T4 files that reference my primary solution. My T4 files invoke my methods and then write the output to constant fields and such.
3. I add a reference to the output.g.cs file that is generated wherever I want to use the constant values.

That type of workflow should be natively part of C# and shouldn't require such a glorious hack.

Below is an easy to understand sample showing what I'm doing (in this case, capturing version and build date info):

<#@ template debug="true" hostspecific="false" language="C#" #>
<#@ output extension=".g.cs" #>
<#@ assembly name="System.Core" #>
<#@ import namespace="System.Linq" #>
<#@ import namespace="System.Text" #>
<#@ import namespace="System.Collections.Generic" #>

<#

	var Date = DateTimeOffset.Now.ToString(System.Globalization.DateTimeFormatInfo.InvariantInfo);

    var Version = String.Format("{0:00}.{1:00}.{2:00}", DateTime.Now.Year % 100, DateTime.Now.Month,DateTime.Now.Day);

#>


using System;

    static partial class InternalAssemblyInfo {

		public const string AssemblyBuildDateString = "<#= Date #>" ;
		public static DateTimeOffset AssemblyBuildDate => DateTimeOffset.Parse(AssemblyBuildDateString, System.Globalization.DateTimeFormatInfo.InvariantInfo);

        public const string Version = "<#= Version #>";

	}

[TonyValenti]

I should be able to just write the following code:

public static partial class InternalAssemblyInfo {
  public const string AssemblyBuildDate = compile(DateTimeOffset.Now.ToString(System.Globalization.DateTimeFormatInfo.InvariantInfo));
//...etc.
}

[CyrusNajmabadi]

The above worries me. It means that each time i build the code i get a different set of IL. That seems to me to be the antithesis of what i would ever want in a programming language.

[jnm2]

@CyrusNajmabadi Imagine code generators! 😱

[CyrusNajmabadi]

@jnm2 Indeed. i would think it was an anti-pattern even for code-generators to not be deterministic.

[TonyValenti]

@CyrusNajmabadi -
In those cases, the reason I make it const is so that the value can be used in Attributes.

Also, while this case does involve generating different IL each time, that is only because it is particularly useful for this case.

[jnm2]

@CyrusNajmabadi I'm not sure why I'd particularly be bothered by it. If there's a particular point where it becomes annoying, you can always just stop using DateTime.Now.

[theunrepentantgeek]

Here's an example of why non-determinism is bad ... admittedly a little contrived, but only a little ...

It's regrettably common for people to screw up date/time calculations - and there are a lot of ways to do it. Here are three examples linked to leap day:

• I've seen systems that failed on 29 February of a leap year because they calculate a date 1 year in the future by doing new DateTime(year+1, month, day);
• Others that failed on 31 December of a leap year because it was day 366 of the year, and therefore out of range;
• And I've seen systems fail on 1 March in the year prior to a leap year because they added 365 days to the date and it wasn't in March.

Combine that with compile time constants like this one (slightly modified from an example above):

public static partial class InternalAssemblyInfo {
    public const DateTime AssemblyBuildDate 
        = compile(DateTimeOffset.Now);
}

What do you now have?

You can build the application, run all the automated tests, do some manual exploratory testing, then push it into production, and everything works fine.

Later, you can build exactly the same source code with exactly the same compiler and exactly the same libraries and it now behaves in a different way.

The very fact that this would be a rare edge case makes it even worse, because practically no developer would think of this as a possible cause for odd behavior when they were trying to work out why the latest update in production is crashing.

(And, yes, I know we can get exactly the same problem right now by generating source code in our build scripts ...)

[Pzixel]

@TonyValenti

I should be able to just write the following code:

public static partial class InternalAssemblyInfo {
  public const string AssemblyBuildDate = compile(DateTimeOffset.Now.ToString(System.Globalization.DateTimeFormatInfo.InvariantInfo));
//...etc.
}

Being said, compiler that produces different output for every call is being considered a buggy one.

---

I already have said it before, but you probably don't listen: #1028 (comment)

[jrmoreno1]

@theunrepentantgeek: that's not bad, that's an external dependency. One that in rare circumstances you'd have to be aware of. But you'd have the same problem if you examined the assembly to get the build date:

https://stackoverflow.com/questions/1600962/displaying-the-build-date

You may not like this, but it happens, and it is doing what people want it to.

[arekbal]

@theunrepentantgeek:

Allowing only [Pure] and enforcing it for compile-time code should root out that kind of code(file access, network, devices, timers, all of these would go away). Taking current Date is non-pure(as in non-determinisitic) in my book.

[julealgon]

Would this allow something like:

[MyCaching(expireIn: TimeSpan.FromMinutes(10))]

?

Working with attributes currently is fairly limited because you can only pass constant values to it.

At the same time, dealing with "durations of time" (in the example above) is usually best handled by leveraging TimeSpan instead of raw int, since it provides better semantic, readability (no need to suffix variables with "seconds" and things like that) and uniformity in usage (no need for conversions to take place).

Similarly, Uri is another common type where it is a good practice to use it to represent arbitrary urls instead of using raw string, but then you cannot actually pass values to it in an attribute since they are not constant.

Would this proposal allow things like TimeSpan.FromX and Uri(string) to be constexpr methods and be used in contexts where const values are required, like attribute declarations?

[TahirAhmadov]

I've been thinking about this a little. While it's easy to understand how to allow certain static methods to be marked as "const" (or "deterministic" - word doesn't matter as much), it gets a little more complicated with struct instance methods. (Obviously classes instance methods cannot be made deterministic by definition - because a heap object by definition is not constant.) If we take something like TimeSpan, we have an immutable struct, and calling it's Add method is not much different from a hypothetical TimeSpanHelper.Add(ts1, ts2) helper method.
We can devise a rule that if a struct is immutable (as defined by readonly keyword usage), then its methods can be optionally marked "deterministic", as well.
Most importantly, this hinges on making it possible in the runtime to treat immutable structs as constants - which would also give you the attribute parameter values scenario.