Dictionary (hstore, json, and jsonb) query support

[yinzara]

This PR adds support for querying on hstore type columns which can map to either Dictionary<string, string> or ImmutableDictionary<string, string> in C# and for querying any other Dictionary<,> or ImmutableDictionary<,> for json and jsonb types.

See npgsql/doc#369 for full documentation.

Fixes #212

[roji]

Thanks @yinzara, looks pretty good overall! See comments below.

[roji]

I attempted Keys and Values and could never get it to work correctly.

If you translate Keys via akeys, that simply returns an array, so shouldn't be complicated (note that the provider will automatically call unnest on that array if you start composing arbitrary operators on it). If you translate via skeys which returns a set, things are definitely more complicated.

[yinzara]

Apologies for the mountains of force pushes :-) I just kept realizing there was more I could do or make better :-)

I think I'm done now lol all tests pass and everything I could possibly translate I have translated.
Every operator in:
https://www.postgresql.org/docs/current/hstore.html
that has some kind of C# equivalent is now covered.

I would have loved to have supported:

string[] keys = ["key1", "key2"];

_ => _.Store.Keys.Any(k => keys.Contains(k)) 
which would become
"Store" ?|  __keys

and the equivalent .All => ?&

however that simply isn't possible using the IMethodCallTranslator or a IMemberTranslator

FINALLY
ready for review @roji

[yinzara]

I started to write up the changes to the npgsql/doc site for hstore and realized that I could use EF.Functions to get the rest of the supported operators for hstore so that we can have the implementation completely done.

I've still got a few more test cases to write, but the new and improved hstore functionality is now complete. See the documentation PR at:
npgsql/doc#369

For more details.

[roji]

@yinzara thanks for making these changes and for the enthusiasm.

First, in any case where there's a straightforward LINQ construct that can be used to express something, that's preferred over introducing a method on EF.Functions. For example, something like ContainsAnyKeys can be expressed in LINQ via dict.Keys.Contains(foo), so I wouldn't introduce EF.Functions.ContainsAnyKeys(). It's indeed a bit more complicated to do translations like the former (not possible in a method translator), but I think it's fine to leave them out for the time being.

Second, I appreciate the work, but I'd rather we didn't eagerly introduced translations for each and every possible thing that PostgreSQL supports, but rather base the work on what users actually express a need for. The hstore type generally is rarely used, and crucially, has been made somewhat obsolete by PG's json support - JSON can do pretty much everything hstore can, and more. Remember that at the end of the day, I have to maintain all this stuff going forward...!

So I'd be happier if at least as a first step, we added the basic, simple stuff which users would be more likely to use, and then wait for people to actually ask for more features before implementing them.

Hope that makes sense.

[yinzara]

So it turns out this PR and its related issue is highly related to

dotnet/efcore#29825
And
dotnet/efcore#26903

I think that I can actually change this PR so that it is focused on that instead and just happens to work with 'hstore' types as well that way this ends up being something people want as a whole. That would mean I'd refocus the PR on translating any Dictionary or ImmutableDictionary query no matter if it's hstore, json or jsonb

[yinzara]

@roji
I think I've got another incarnation that's worth your eyes to at least give some guidance if you think I'm headed in the right direction. I still have a few more tests to write if you agree.

This PR is now focused 100% on all forms of Dictionary<string,TValue> and ImmutableDictionary<string,TValue> based querying no matter if it's hstore, json or jsonb no matter their TValue types.

I removed all of the DBFunction extensions that could ever have a translation that would be supported without the DBFunctions. A future PR (or maybe if I get ansy) can address doing the expression visitor changes to support the other queries.

I now have a test case with a Dictionary<string, Dictionary<string, string>> NestedDictionary {get;set;} that shows how the functionality even works recursively :-) and another test case with Dictionary<string, int> to show it works with numbers as well.

Finally I made the new DictionaryTranslator not look up any TypeMappings inside the constructor so it's very efficient to create new instances of it with no overhead.

This does not fix dotnet/efcore#26903 as I was thinking it might, but it still seems quite useful overall and eventually it could support it.

[yinzara]

I know it's been a while since you've had time to look at this. Any chance you could take a peak and at least tell me if you are ok with my direction now? If so I'll update all the test cases and make sure it's ready for merge.

[yinzara]

I realize this isn't necessary now and I'll make sure it's removed before the PR merges.

[roji]

@yinzara I'll do my best to look soon, but it's a very busy time in the post 9.0 phase; so it may take a little while. Don't hesitate to ping me again in a month or two if you see no progress here.