Weird system.linq.expressions behavior with regard to interned vs. non-interned strings

[elgonzo]

TL;DR: A (relatively) simple BinaryExpression equality-comparing two string instances (as ConstantExpressions) will evaluate to different results solely depending on whether the involved strings are interned or not interned. Is this a bug, or is it intended behavior? If it is intended behavior, where is it documented?

Lets create a simple Equality-comparison expression which compares two string values:

using System;
using System.Linq.Expressions;

var strLeft = "foobar";
var strRight = new string(new char[] { 'f', 'o', 'o', 'b', 'a', 'r' });

var left = Expression.Constant(strLeft, typeof(object));
var right = Expression.Constant(strRight, typeof(string));

var eq = Expression.Equal(left, right);

var lam = Expression.Lambda(eq);
var result = lam.Compile().DynamicInvoke();
Console.WriteLine(result);

strLeft (a string literal) is an interned string. strRight is not an interned string.

Crucially important is that the left ConstantExpression is typed as object, not as string.

Executing this will yield a true result, as expected. (Dotnetfiddle: https://dotnetfiddle.net/waFmqT)

However, now lets change strLeft also to a non-interned string (reminder: strLeft is the one used in the object-typed ConstantExpression):

var strLeft = new string(new char[] { 'f', 'o', 'o', 'b', 'a', 'r' });
var strRight = new string(new char[] { 'f', 'o', 'o', 'b', 'a', 'r' });

...

Both strLeft and strRight are now non-interned string instances. But they are still equal. Yet, now the equality expression with the left-hand side typed as object will unexpectedly and surprisingly evaluate to false (Dotnetfiddle: https://dotnetfiddle.net/TxGjPQ). This doesn't look right to me.

If i "manually" intern strLeft like so:

var strLeft = new string(new char[] { 'f', 'o', 'o', 'b', 'a', 'r' });
var strRight = new string(new char[] { 'f', 'o', 'o', 'b', 'a', 'r' });

strLeft = string.Intern(strLeft);

...

the equality expression will now again correctly evaluate to true (Dotnetfiddle: https://dotnetfiddle.net/iEQU96).

What is going on here? Why does the result of the expression evaluation depend on whether strLeft is an interned string or not?

Do i lack a fundamental understanding of how System.Linq.Expressions is supposed to work? Or is this a bug?

(My apologies if this has been reported elsewhere already. I wasn't able to find any other report about this here in the discussion or issues section; not sure what other terms to search for than something along the lines of "expression string intern" or alike.)

[EgorBo]

@elgonzo your expression compares objects by reference, Expression.Equal is basically ceq in IL. If you want to compare two string objects you have to emit a call to string.Equals

[EgorBo]

Result of that for strings is purely an implementation detail

[elgonzo]

@EgorBo

your expression compares objects by reference

If that were the case, the three cases i outlined in my report should all evaluate to false (because strLeft and strRight are separate string instances in every of these three cases), no?

How exactly does it happen that if the strLeft string is interned the expression evaluates to true, otherwise to false? Reference comparison does not explain this behavior.

[EgorBo]

If that were the case

It is the case, here is the IL for your expression after compilation:

IL_0000  02                ldarg.0     
IL_0001  7b 02 00 00 04    ldfld        0x4000002
IL_0006  16                ldc.i4.0    
IL_0007  9a                ldelem.ref  <--- gets 
IL_0008  72 03 00 00 70    ldstr        0x70000003 <--- "foobar" literal
IL_000d  fe 01             ceq         <---
IL_000f  2a                ret  

Your strRight is converted into a string literal because you asked it to be so: Expression.Constant(strRight, typeof(string)); dynamically. strLeft is already the same literal despite the fact it's emitted as object (an access to some internal array by index).

[elgonzo]

Ah, thanks! Now i understand.

I wish the (some) documentation for Expression.Constant would mention the fact that it is interning strings under the hood...

[svick]

It doesn't always do that. Try modifying your code with .Compile(preferInterpretation: true).

[elgonzo]

Oh, that's good to know, too. Thanks!

That makes me wonder: What would be the authoritative documentation for things like how expression trees are being compiled or interpreted precisely? Or is this knowledge passed on more like "word of mouth" / "bag of StackOverflow questions and answers"? The documentation at docs.microsoft.com about the types in System.Linq.Expressions (and any further docs.microsoft.com documentation linked from there) seem to keep mum about these details, unfortunately.

[EgorBo]

Comparing strings by references is always a bad idea and not guaranteed, e.g. at some point GC might start deduping strings etc.