Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support of lambda expressions #140

Merged
merged 20 commits into from
Oct 8, 2021

Conversation

metoule
Copy link
Contributor

@metoule metoule commented May 18, 2021

This is a proposition to add support of lambda expressions (to fix #25). This is still a work in progress, but it's a promising start.
In particular, there are a few regressions, and some use cases are not working yet.

I'll keep working on it, but I still wanted to open a PR to have some feedback :)

@metoule metoule changed the title Add support of lambda expressions Add support of lambda expressions (WIP) May 18, 2021
@metoule metoule marked this pull request as draft May 18, 2021 08:44
@davideicardi
Copy link
Member

davideicardi commented May 18, 2021

It looks very promising! Very well!

…nt, so that it can access the same variables and types.

Improve resolution of methods where the parameters are generic types.
…g. It's disabled by default because it has a slight performance cost.
Allow a single lambda expression to be converted to a Func type.
Edited documentation.
@metoule metoule marked this pull request as ready for review May 18, 2021 21:27
@metoule
Copy link
Contributor Author

metoule commented May 18, 2021

I think the PR is ready for review now. Parsing the lambda expressions has a slight performance penalty, because every statement can be one: we try to parse each statement as a lambda expression, and we backtrack if we don't detect the lambda arrow token.

Because of this, I added a new parser settings to enable / disable the lambda expressions parsing.

To make all this work, I had to modify a bunch of code related to the generic types; hopefully it won't impact any existing code.

@metoule metoule changed the title Add support of lambda expressions (WIP) Add support of lambda expressions May 18, 2021
@davideicardi
Copy link
Member

I just released v2.5.0 with all prev PRs... next days I will review in depth this one and we can then have a new release. Thank you again for your recent work!

@Pentiva
Copy link
Contributor

Pentiva commented Jun 3, 2021

There appears to be a problem when using nested lambdas with net6.0. Adding net5.0 and net6.0 as unit test targets results in LambdaExpressionTest.Nested_lambda failing only on the net6.0 tests.

Here is the test output/exception message:

System.ArgumentException : Type T contains generic parameters (Parameter 'type')
   at System.Dynamic.Utils.TypeUtils.ValidateType(Type type, String paramName, Int32 index) in System.Linq.Expressions.dll:token 0x600125c+0x2e
   at System.Dynamic.Utils.TypeUtils.ValidateType(Type type, String paramName, Boolean allowByRef, Boolean allowPointer) in System.Linq.Expressions.dll:token 0x600125b+0x0
   at System.Linq.Expressions.Expression.Validate(Type type, Boolean allowByRef) in System.Linq.Expressions.dll:token 0x6000304+0xb
   at System.Linq.Expressions.Expression.Parameter(Type type, String name) in System.Linq.Expressions.dll:token 0x6000302+0x0
   at DynamicExpresso.Parameter..ctor(String name, Type type, Object value) in .\src\DynamicExpresso.Core\Parameter.cs:line 29
   at DynamicExpresso.Reflection.ReflectionExtensions.GetDelegateInfo(Type delegateType, String[] parametersNames) in .\src\DynamicExpresso.Core\Reflection\ReflectionExtensions.cs:line 29
   at DynamicExpresso.Interpreter.ParseAs(Type delegateType, String expressionText, String[] parametersNames) in .\src\DynamicExpresso.Core\Interpreter.cs:line 407
   at DynamicExpresso.Interpreter.ParseAsExpression(Type delegateType, String expressionText, String[] parametersNames) in .\src\DynamicExpresso.Core\Interpreter.cs:line 396
   at DynamicExpresso.Parsing.Parser.InterpreterExpression.EvalAs(Type delegateType) in .\src\DynamicExpresso.Core\Parsing\Parser.cs:line 2694
   at DynamicExpresso.Parsing.Parser.CheckIfMethodIsApplicableAndPrepareIt(MethodData method, Expression[] args) in .\src\DynamicExpresso.Core\Parsing\Parser.cs:line 1735
   at DynamicExpresso.Parsing.Parser.<>c__DisplayClass90_0.<FindBestMethod>b__0(MethodData m) in .\src\DynamicExpresso.Core\Parsing\Parser.cs:line 1619
   at System.Linq.Enumerable.WhereEnumerableIterator`1.ToArray() in System.Linq.dll:token 0x6000227+0x1d
   at System.Linq.Enumerable.ToArray[TSource](IEnumerable`1 source) in System.Linq.dll:token 0x600011a+0x1b
   at DynamicExpresso.Parsing.Parser.FindBestMethod(IEnumerable`1 methods, Expression[] args) in .\src\DynamicExpresso.Core\Parsing\Parser.cs:line 1618
   at DynamicExpresso.Parsing.Parser.FindBestMethod(IEnumerable`1 methods, Expression[] args) in .\src\DynamicExpresso.Core\Parsing\Parser.cs:line 1613
   at DynamicExpresso.Parsing.Parser.FindExtensionMethods(String methodName, Expression[] args) in .\src\DynamicExpresso.Core\Parsing\Parser.cs:line 1553
   at DynamicExpresso.Parsing.Parser.ParseExtensionMethodInvocation(Type type, Expression instance, Int32 errorPos, String id, Expression[] args) in .\src\DynamicExpresso.Core\Parsing\Parser.cs:line 1233
   at DynamicExpresso.Parsing.Parser.ParseMethodInvocation(Type type, Expression instance, Int32 errorPos, String methodName) in .\src\DynamicExpresso.Core\Parsing\Parser.cs:line 1215
   at DynamicExpresso.Parsing.Parser.ParseMemberAccess(Type type, Expression instance) in .\src\DynamicExpresso.Core\Parsing\Parser.cs:line 1187
   at DynamicExpresso.Parsing.Parser.ParsePrimary() in .\src\DynamicExpresso.Core\Parsing\Parser.cs:line 599
   at DynamicExpresso.Parsing.Parser.ParseUnary() in .\src\DynamicExpresso.Core\Parsing\Parser.cs:line 542
   at DynamicExpresso.Parsing.Parser.ParseMultiplicative() in .\src\DynamicExpresso.Core\Parsing\Parser.cs:line 475
   at DynamicExpresso.Parsing.Parser.ParseAdditive() in .\src\DynamicExpresso.Core\Parsing\Parser.cs:line 444
   at DynamicExpresso.Parsing.Parser.ParseTypeTesting() in .\src\DynamicExpresso.Core\Parsing\Parser.cs:line 415
   at DynamicExpresso.Parsing.Parser.ParseComparison() in .\src\DynamicExpresso.Core\Parsing\Parser.cs:line 321
   at DynamicExpresso.Parsing.Parser.ParseLogicalAnd() in .\src\DynamicExpresso.Core\Parsing\Parser.cs:line 307
   at DynamicExpresso.Parsing.Parser.ParseLogicalXor() in .\src\DynamicExpresso.Core\Parsing\Parser.cs:line 293
   at DynamicExpresso.Parsing.Parser.ParseLogicalOr() in .\src\DynamicExpresso.Core\Parsing\Parser.cs:line 279
   at DynamicExpresso.Parsing.Parser.ParseConditionalAnd() in .\src\DynamicExpresso.Core\Parsing\Parser.cs:line 265
   at DynamicExpresso.Parsing.Parser.ParseConditionalOr() in .\src\DynamicExpresso.Core\Parsing\Parser.cs:line 251
   at DynamicExpresso.Parsing.Parser.ParseConditional() in .\src\DynamicExpresso.Core\Parsing\Parser.cs:line 229
   at DynamicExpresso.Parsing.Parser.ParseAssignment() in .\src\DynamicExpresso.Core\Parsing\Parser.cs:line 203
   at DynamicExpresso.Parsing.Parser.ParseExpressionSegment() in .\src\DynamicExpresso.Core\Parsing\Parser.cs:line 96
   at DynamicExpresso.Parsing.Parser.ParseExpressionSegment(Type returnType) in .\src\DynamicExpresso.Core\Parsing\Parser.cs:line 71
   at DynamicExpresso.Parsing.Parser.Parse() in .\src\DynamicExpresso.Core\Parsing\Parser.cs:line 62
   at DynamicExpresso.Parsing.Parser.Parse(ParserArguments arguments) in .\src\DynamicExpresso.Core\Parsing\Parser.cs:line 24
   at DynamicExpresso.Interpreter.ParseAsLambda(String expressionText, Type expressionType, Parameter[] parameters) in .\src\DynamicExpresso.Core\Interpreter.cs:line 468
   at DynamicExpresso.Interpreter.ParseAs(Type delegateType, String expressionText, String[] parametersNames) in .\src\DynamicExpresso.Core\Interpreter.cs:line 409
   at DynamicExpresso.Interpreter.ParseAsExpression(Type delegateType, String expressionText, String[] parametersNames) in .\src\DynamicExpresso.Core\Interpreter.cs:line 396
   at DynamicExpresso.Parsing.Parser.InterpreterExpression.EvalAs(Type delegateType) in .\src\DynamicExpresso.Core\Parsing\Parser.cs:line 2694
   at DynamicExpresso.Parsing.Parser.CheckIfMethodIsApplicableAndPrepareIt(MethodData method, Expression[] args) in .\src\DynamicExpresso.Core\Parsing\Parser.cs:line 1735
   at DynamicExpresso.Parsing.Parser.<>c__DisplayClass90_0.<FindBestMethod>b__0(MethodData m) in .\src\DynamicExpresso.Core\Parsing\Parser.cs:line 1619
   at System.Linq.Enumerable.WhereEnumerableIterator`1.ToArray() in System.Linq.dll:token 0x6000227+0x1d
   at System.Linq.Enumerable.ToArray[TSource](IEnumerable`1 source) in System.Linq.dll:token 0x600011a+0x1b
   at DynamicExpresso.Parsing.Parser.FindBestMethod(IEnumerable`1 methods, Expression[] args) in .\src\DynamicExpresso.Core\Parsing\Parser.cs:line 1618
   at DynamicExpresso.Parsing.Parser.FindBestMethod(IEnumerable`1 methods, Expression[] args) in .\src\DynamicExpresso.Core\Parsing\Parser.cs:line 1613
   at DynamicExpresso.Parsing.Parser.FindExtensionMethods(String methodName, Expression[] args) in .\src\DynamicExpresso.Core\Parsing\Parser.cs:line 1553
   at DynamicExpresso.Parsing.Parser.ParseExtensionMethodInvocation(Type type, Expression instance, Int32 errorPos, String id, Expression[] args) in .\src\DynamicExpresso.Core\Parsing\Parser.cs:line 1233
   at DynamicExpresso.Parsing.Parser.ParseMethodInvocation(Type type, Expression instance, Int32 errorPos, String methodName) in .\src\DynamicExpresso.Core\Parsing\Parser.cs:line 1215
   at DynamicExpresso.Parsing.Parser.ParseMemberAccess(Type type, Expression instance) in .\src\DynamicExpresso.Core\Parsing\Parser.cs:line 1187
   at DynamicExpresso.Parsing.Parser.ParsePrimary() in .\src\DynamicExpresso.Core\Parsing\Parser.cs:line 599
   at DynamicExpresso.Parsing.Parser.ParseUnary() in .\src\DynamicExpresso.Core\Parsing\Parser.cs:line 542
   at DynamicExpresso.Parsing.Parser.ParseMultiplicative() in .\src\DynamicExpresso.Core\Parsing\Parser.cs:line 475
   at DynamicExpresso.Parsing.Parser.ParseAdditive() in .\src\DynamicExpresso.Core\Parsing\Parser.cs:line 444
   at DynamicExpresso.Parsing.Parser.ParseTypeTesting() in .\src\DynamicExpresso.Core\Parsing\Parser.cs:line 415
   at DynamicExpresso.Parsing.Parser.ParseComparison() in .\src\DynamicExpresso.Core\Parsing\Parser.cs:line 321
   at DynamicExpresso.Parsing.Parser.ParseLogicalAnd() in .\src\DynamicExpresso.Core\Parsing\Parser.cs:line 307
   at DynamicExpresso.Parsing.Parser.ParseLogicalXor() in .\src\DynamicExpresso.Core\Parsing\Parser.cs:line 293
   at DynamicExpresso.Parsing.Parser.ParseLogicalOr() in .\src\DynamicExpresso.Core\Parsing\Parser.cs:line 279
   at DynamicExpresso.Parsing.Parser.ParseConditionalAnd() in .\src\DynamicExpresso.Core\Parsing\Parser.cs:line 265
   at DynamicExpresso.Parsing.Parser.ParseConditionalOr() in .\src\DynamicExpresso.Core\Parsing\Parser.cs:line 251
   at DynamicExpresso.Parsing.Parser.ParseConditional() in .\src\DynamicExpresso.Core\Parsing\Parser.cs:line 229
   at DynamicExpresso.Parsing.Parser.ParseAssignment() in .\src\DynamicExpresso.Core\Parsing\Parser.cs:line 203
   at DynamicExpresso.Parsing.Parser.ParseExpressionSegment() in .\src\DynamicExpresso.Core\Parsing\Parser.cs:line 96
   at DynamicExpresso.Parsing.Parser.ParseExpressionSegment(Type returnType) in .\src\DynamicExpresso.Core\Parsing\Parser.cs:line 71
   at DynamicExpresso.Parsing.Parser.Parse() in .\src\DynamicExpresso.Core\Parsing\Parser.cs:line 62
   at DynamicExpresso.Parsing.Parser.Parse(ParserArguments arguments) in .\src\DynamicExpresso.Core\Parsing\Parser.cs:line 24
   at DynamicExpresso.Interpreter.ParseAsLambda(String expressionText, Type expressionType, Parameter[] parameters) in .\src\DynamicExpresso.Core\Interpreter.cs:line 468
   at DynamicExpresso.Interpreter.Parse(String expressionText, Type expressionType, Parameter[] parameters) in .\src\DynamicExpresso.Core\Interpreter.cs:line 357
   at DynamicExpresso.Interpreter.Eval(String expressionText, Type expressionType, Parameter[] parameters) in .\src\DynamicExpresso.Core\Interpreter.cs:line 445
   at DynamicExpresso.Interpreter.Eval[T](String expressionText, Parameter[] parameters) in .\src\DynamicExpresso.Core\Interpreter.cs:line 433
   at DynamicExpresso.UnitTest.LambdaExpressionTest.Nested_lambda() in .\test\DynamicExpresso.UnitTest\LambdaExpressionTest.cs:line 119

@metoule
Copy link
Contributor Author

metoule commented Jun 3, 2021

I think it's linked to the new SingleOrDefault overload taking default parameters (see dotnet/core#6098, FirstOrDefault/LastOrDefault/SingleOrDefault overloads taking default parameters).

The issue is not related to nested lambdas: I can reproduce the same issue with this simpler test:

[Test]
public void String_single_or_default_lambda()
{
	var target = new Interpreter(_options);
	var str = "cd";
	target.SetVariable("str", "cd");

	var result = target.Eval<char>("str.SingleOrDefault(c => c == 'd')");
	Assert.AreEqual(str.SingleOrDefault(c => c == 'd'), result);
}

It feels like the resolution doesn't find the proper method to use, and therefore can't find the proper type parameters.
I'll see if I can find a solution.

Only promote a lambda expression if the method's parameter is a delegate with the matching number of generic arguments.
@metoule
Copy link
Contributor Author

metoule commented Jun 7, 2021

I managed to find and fix the issue. It occurred when a lambda expression could be matched to a generic argument, for example:

public static TSource MySingleOrDefault<TSource>(this IEnumerable<TSource> source, TSource defaultValue)
{
	return source.SingleOrDefault();
}

str.MySingleOrDefault(c => c == 'd')

In that case, this method should be ignored during resolution, because a lambda expression can't be a generic argument.
The resolution now ignores candidates if a lambda expression is matched to a generic argument, or to a delegate with the incorrect number of generic arguments.

@davideicardi
Copy link
Member

If you are able to rebase from master we should have new PR validation using github actions...

Copy link
Member

@davideicardi davideicardi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@metoule When you have some time can you rebase with latest master version?

One small thing: maybe it is better to remove the EnableLambdaExpression function and just leave the possibility to set the feature using InterpreterOptions? What do you think? Just to have a single way to change these kind of options.

Thank you!

@metoule
Copy link
Contributor Author

metoule commented Jul 6, 2021

@davideicardi I've merged the master branch.

For EnableLambdaExpression, I had the feeling that each parser setting had a corresponding Enable method (e.g. SetDefaultNumberType, EnableAssignment, etc), but I can certainly remove it if you prefer!

@bortolin
Copy link
Contributor

bortolin commented Jul 6, 2021

If I can express my opinion, as the options and features increase it is a bit limiting to use an enum. For the future it might be interesting to use a class object that centralize all the options, even the most complex ones.

@davideicardi
Copy link
Member

davideicardi commented Jul 6, 2021

@metoule

For EnableLambdaExpression, I had the feeling that each parser setting had a corresponding Enable method (e.g. SetDefaultNumberType, EnableAssignment, etc), but I can certainly remove it if you prefer!

The idea for now is to use for all flags (where a simple true/false is enough) just the InterpreterOptions. Where I need more data I used a dedicated function (like SetDefaultNumberType). EnableAssignment is probably an exception of this rule but because I expected to have different kind of assignments, but probably it is not the case.
So yes, if you don't have any strong opinion maybe it is better to just set it via InterpreterOptions.

@davideicardi
Copy link
Member

davideicardi commented Jul 6, 2021

@bortolin
Yes this is a good point, I agree. Maybe we can think about a refactoring of the settings for a next major release.
For now I think it is simpler to just maintain the same interface, we still have some numbers available ;-)

@metoule
Copy link
Contributor Author

metoule commented Jul 6, 2021

@davideicardi

The idea for now is to use for all flags (where a simple true/false is enough) just the InterpreterOptions.

that makes sense! I've removed the EnableLambdaExpressions method.

Copy link
Member

@davideicardi davideicardi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added some small comments, feel free to discuss it.
But overall I think it is very good! Very good idea and implementation!

Are you planning to do something else? Or do you think that we can then merge it? (in the PR description is still marked as "work in progress" ...)

src/DynamicExpresso.Core/Parsing/Parser.cs Outdated Show resolved Hide resolved
src/DynamicExpresso.Core/Parsing/Parser.cs Outdated Show resolved Hide resolved
README.md Outdated Show resolved Hide resolved
@Pentiva
Copy link
Contributor

Pentiva commented Jul 9, 2021

I don't know if it needs to be supported, as I cannot find/think of a usage for it in LINQ, but this implementation does not support parameter-less lambdas e.g. () => 3 - 1, () => new Object().
I think the main usage in the runtime is specifying a factory func, such as for the Lazy type.

@metoule
Copy link
Contributor Author

metoule commented Jul 12, 2021

@Pentiva that was an oversight, thanks for reporting! You can now write a parameterless lambda:

[Test]
public void Lambda_expression_no_arguments()
{
	var target = new Interpreter(InterpreterOptions.Default | InterpreterOptions.LambdaExpressions);
	var lambda = target.Eval<Func<int>>("() => 5 + 6");
	Assert.AreEqual(11, lambda.Invoke());
}

Copy link
Member

@davideicardi davideicardi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you!

@davideicardi davideicardi merged commit ea77214 into dynamicexpresso:master Oct 8, 2021
@metoule metoule deleted the fix_25 branch July 23, 2022 08:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Support lambda syntax
4 participants