How can I replace constans expression?

[iRumba]

Hi. I need to add several functions to EF Core that must be replaced to maintained expression.
Also I don't want to break switched provider's relations (same provider: Npgsql, sqlite, SQL Server etc).

I found several points of entry - IMethodCallTranslator, IMemberTranslator. But these translators produce SqlExpression, but I need transform source expression. Also, there is IEvaluatableExpressionFilterPlugin. this service's method returns bool.

Also, I found IQueryTranslationPreprocessorFactory, that I can replace, But at first, I think, that some providers can replace this service too, an also i have other problem. Expression, that a want transform looks as this i => i.Id.Is(spec)

Is - extension method of object, that takes other object of concrete type (for example MyClass) and returns bool.
spec - is instance of MyClass, that contains information for predicate creating.

So, I ecpect take ConstantExpression with MyClass instance in MyTranslationPreprocessor.Process, but find ParameterExpression. I looked to source code and understood, that constants became parameters before MyTranslationPreprocessor.Process calling. And I tried find way to win this, but my time is over. Please heeeeelp!

Target framework: NET 5.0

[roji]

Can you provide a bit more information on what you're trying to achieve? What SQL are you trying to translate the Is function into?

Regardless, have you looked at user-defined function mappings? That is the standard way for users to add a function translation. Regarding the parameterization, you may need to place [NotParameterized] on the function parameter.

But more info would allow us to help you better.

[iRumba]

Can you provide a bit more information on what you're trying to achieve? What SQL are you trying to translate the Is function into?

Regardless, have you looked at user-defined function mappings? That is the standard way for users to add a function translation. Regarding the parameterization, you may need to place [NotParameterized] on the function parameter.

But more info would allow us to help you better.

Hi. I wrote

I need transform source expression

So, I don't want to translate this function into SQL. I want to transform source expression (System.Expression) to other form (other expression)

i => i.Id.Is(spec) - LambdaExpression of predicate. I need transform it to other predicate, but I can't, because I need "spec" as ConstantExpression (with value) but constant transforms to Parameter before MyTranslationPreprocessor.Process calling.

[roji]

So, I don't want to translate this function into SQL. I want to transform source expression (System.Expression) to other form (other expression)

Which other expression do you want to translate it to? It may be simpler to simply translate i.Id.Is(spec) directly into the final SQL you're looking for, rather than doing a source translation from i.Id.Is(spec) to some other LINQ expression, and have that translated to your final SQL by EF Core. Please provide a full description of what you're trying to achieve - what you're trying to transform your function into, since there may be a simpler way to do what you want.

Re the constant/parameter question, see my suggestion above to use [NotParameterized].

[iRumba]

Which other expression do you want to translate it to? It may be simpler to simply translate i.Id.Is(spec) directly into the final SQL you're looking for, rather than doing a source translation from i.Id.Is(spec) to some other LINQ expression, and have that translated to your final SQL by EF Core. Please provide a full description of what you're trying to achieve - what you're trying to transform your function into, since there may be a simpler way to do what you want.

spec contains property with other expression (predicate) from i.Id type.
For example i.Id is integer. spec.Expression has value x =>x == 5. Destinational expression must be i => i.Id == 5.

It is impossible, to translate i.Id.Is(spec) into SQL.

[roji]

It sounds to me that you're trying to do dynamic queries, where a WHERE predicate needs to change from query to query. To do that, it's usually much simpler to simply pass an expression directly to e.g. the Where operator:

Expression<Func<Blog, bool>> predicate = b => b.Id == 5;

var blogs = ctx.Blogs.Where(predicate).ToList();

The predicate parameter (just like spec in your question) can be an entirely different expression every time. This doesn't require any special transformations or translations. Is there a special reason this wouldn't work for your scenario?

Note that if you're doing this (or any other kind of dynamic query construction), be very careful with query caching, or else performance may suffer; see this doc article.

[iRumba]

@roji no, it is not correct example.

Expression<Func<Blog, bool>> predicate = b => b.Id.Is(spec);

var blogs = ctx.Blogs.Where(predicate).ToList();

try this ;)

In other your comment you tried to simplify my source expression. But please just help me with my problem. My problem is real. My solution is not overenginering. It is usefull in different cases. I tired to make crutches instead of clear approach.

[roji]

In other your comment you tried to simplify my source expression. But please just help me with my problem.

That's true, but we get many, many cases of users going down a very complicated route where a simpler one is sufficient. I still haven't seen any info above on why a dynamic Where predicate isn't sufficient; if you could provide that context, it would help us understand better and possibly make changes on the EF Core side to support it better.

Regardless, if you insist on your own direction, you could indeed replace the IQueryTranslationPreprocessorFactory to perform the necessary transformation. Providers don't typically have their own version of IQueryTranslationPreprocessorFactory (though there's a relational one for all relational providers), but you could still have a per-provider thing to replace the IQueryTranslationPreprocessorFactory for that provider.

[iRumba]

Regardless, if you insist on your own direction, you could indeed replace the IQueryTranslationPreprocessorFactory to perform the necessary transformation. Providers don't typically have their own version of IQueryTranslationPreprocessorFactory (though there's a relational one for all relational providers), but you could still have a per-provider thing to replace the IQueryTranslationPreprocessorFactory for that provider.

I got it. It is useless without spec as ConstantExpression. But spec transformed to ParameterExpression before MyTranslationPreprocessor.Process calling.

That's true, but we get many, many cases of users going down a very complicated route where a simpler one is sufficient. I still haven't seen any info above on why a dynamic Where predicate isn't sufficient; if you could provide that context, it would help us understand better and possibly make changes on the EF Core side to support it better.

My english is bad. I have dificulty with explaining this. But I will try.

Look, You have a Blog entity. Blog has Posts, Post has Comments. Also Blog and Comment have Author.

You should output posts to line. Posts should be filtered by some rules. Customer said that banned authors shouldn't be showed. Ok, you write request ctx.Posts.Where(x => !x.Blog.Author.Banned). Great!

Also in Post page you should hide comments of banned authors. ctx.Comments.Where(x => x.Post.Id == postId && !x.Author.Banned).

Also you should output banned list in admin panel. ctx.Authors.Where(x => x.Banned)

Look at this expressions. You have repeating code. I don't know about you, but I want to create stored predicate. Because tomorrow this filter maybe will change. For example property Banned will changed with new table with banned list.

Expression<Func<Author, bool>> isBanned = x => x.Banned

But how can I use it?

In admin panel. ctx.Authors.Where(isBanned)

In posts line. ctx.Authors.Where(isBanned.Not()).Select(x => x.Posts) It will work, but Select() in this case switches off tracking. In this case I don't need tracking, but in other case I will need it. So I can implement extension method for object.

public static bool Is<T>(this object obj, Func<T, bool> predicate)
{
   // impl
}

After it I can write request ctx.Posts.Where(x => !x.Blog.Author.Is(isBanned.Compile()))

Can you translate this expression to SQL or other form? I can't. Change method Is().

public static bool Is<T>(this object obj, Expression<Func<T, bool>> predicate)
{
   // impl
}

Now we can write ctx.Posts.Where(x => !x.Blog.Author.Is(isBanned))

Can you translate this? No. Can you transform this? NO! Because in MyTranslationPreprocessor.Process Expression isBanned will sended as ParameterExpression with name __isBanned and type LambdaExpression. But for transforming we need ConstantExpression with Value.

It is that what I want. But I send to Is() other type instead of Expression (it is also long story), but solution of this problem will same.

Additional dificulty - I can send Expression in different forms. As variable, as new instancing, as initializator, as method call. EF already has evaluator, that evaluates values for parameters. So, in best way I want have service, that contains parameter's values in the Preprocessor.

[roji]

First, for the case above, it really sounds like you're looking global query filters - the "IsDeleted" example shown there seems very similar to your "Banned" case. Query filters allow you to define the filter once in the model - on the Author entity type - and it would get automatically applied in all queries which touch Authors. In the solution you're trying to implement with Is, you'd have to pass the isBanned variable everywhere and explicitly apply it to all queries in your program - this is problematic and risky (it's easy for someone to accidentally write a query forgetting the filter).

I got it. It is useless without spec as ConstantExpression. But spec transformed to ParameterExpression before MyTranslationPreprocessor.Process calling.

I've pointed to using [NotParameterized] twice above - have you tried that?

Finally, what you're asking for - a hook to intercept and transform expressions early in the query pipeline - is tracked by #19748.

[iRumba]

No, globally query filters doesn't suit me.

In the solution you're trying to implement with Is, you'd have to pass the isBanned variable everywhere and explicitly apply it to all queries in your program

No, I have a choose. Also it is clear.

What is [NotParameterized]. A found description, but I need example. https://docs.microsoft.com/en-us/dotnet/api/microsoft.entityframeworkcore.query.notparameterizedattribute?view=efcore-5.0

Finally, what you're asking for - a hook to intercept and transform expressions early in the query pipeline - is tracked by #19748.

I didn't find solution here. Please show.

[roji]

See what I wrote above:

Regarding the parameterization, you may need to place [NotParameterized] on the function parameter.

So the thing that's getting parameterized - the parameter to your Is function, needs to be annotated with [NotParameterized].

[iRumba]

So the thing that's getting parameterized - the parameter to your Is function, needs to be annotated with [NotParameterized].

Is there other way? Assembly with function doesn't have reference to EF. Maybe is there fluent api for that setup?

[roji]

Note that NotParameterizedAttribute is in Microsoft.EntityFrameworkCore.Abstractions, which is a very thin package designed precisely only to contain such attributes.

There isn't any other mechanism or fluent API as far as I know.

[iRumba]

Thanks, I will try.

But I want solution in other assembly.

AssemblyForSetupFunctionsForEf --> AssemblyWithFunctions

[roji]

There's nothing like that as far as I know.

The logic here is that the function (Is in your case) is there precisely (and only) so that EF can translate it. That makes it reasonable for the function to live in an assembly which references the EF Core abstractions package. I can't really say more about your specific case because I don't know about your dependency structure.

[iRumba]

No, function Is can execute directly. Same as IEnumerable.Select or Any

[iRumba]

Why issue closed? I have not received an answer to my question.

[ajcvickers]

@iRumba It seems to me that the issue has been discussed extensively and that what you are specifically asking for is not something that EF supports. If that's not the case, then can state your question again, as specifically as possible?

[iRumba]

Not supported? Who and where did say that not supported?

[ajcvickers]

@iRumba I'm saying that, based on my interpretation of the discussion above.

[iRumba]

Maybe somebody knows what to change in EF for that? :)

[ajcvickers]

@iRumba The EF team discussed this issue in the last triage meeting. The conclusion was that there was nothing more to discuss here. If someone does know, then they are free to contribute to the discussion, and at that point we can even re-open the issue if needed. However, at this time, there is nothing more actionable here.

[smitpatel]

Use NotParameterized attribute on the function parameter you want to leave as a constant expression. If you don't want to reference EF Core assembly in your project then that is your choice. We don't have plan to support doing it without referencing EF Core in any way.

P.S. trying to change expression tree based on arbitrary constant expression during query compilation will have drastic impact. If your constant expression doesn't implement equality properly to identify tree has changed when different constant came in then, you will be used previously cached query which doesn't do compilation again means the replacement will be skipped altogether and if replacement was supposed to generate different tree than earlier query, it will generate incorrect results.

[iRumba]

P.S. trying to change expression tree based on arbitrary constant expression during query compilation will have drastic impact. If your constant expression doesn't implement equality properly to identify tree has changed when different constant came in then, you will be used previously cached query which doesn't do compilation again means the replacement will be skipped altogether and if replacement was supposed to generate different tree than earlier query, it will generate incorrect results.

I found solution but it uses Internal EF Core API. I replaced IQueryCompiler and ambed my replace visitor into Execute and ExecuteAsync before base functional. Visitor works before caching.

I have some class that has Expression. Some instance (theInstance) for example has expression (string) x => x == "string"

My query looks as ctx.Authors.Where(x => x.Name.Is(theInstance))

Visitor replaces x.Name.Is(theInstance) to x.Name == "string". It is safe for caching and perfomance.

But it Internal EF Core API. I want legal solution.