*:{!property} No longer working

[dargolith]

With the latest patch (3.0.14) my queries stopped working. I assume those changes are unwanted since the release is only a patch.

OrientDB Version:

3.0.14

Java Version / OS:

The official docker container from orienttechnologies on dockerhub. 3.0.14 (or latest) tag.

Expected behavior

SELECT *:{!@version, !@class, !out_*, !in_*} FROM V to return:

{
    "result": [
        {
            "testProp": "test1",
            "@rid": "#10:0"
        },
        {
            "testProp": "test2",
            "@rid": "#11:0"
        }
    ],
    ...
}

Actual behavior

SELECT *:{!@version, !@class, !out_*, !in_*} FROM V returns:

{
    "result": [
        {
            "out_": [
                {
                    "in": "#11:0",
                    "out": "#10:0"
                }
            ],
            "testProp": "test1",
            "@rid": "#10:0",
            "@version": 2,
            "@class": "V"
        },
        {
            "testProp": "test2",
            "in_": [
                {
                    "in": "#11:0",
                    "out": "#10:0"
                }
            ],
            "@rid": "#11:0",
            "@version": 2,
            "@class": "V"
        }
    ],
    ...
}

Steps to reproduce

CREATE VERTEX V SET testProp = 'test1'
CREATE VERTEX V SET testProp = 'test2'
CREATE EDGE E FROM (SELECT FROM V WHERE testProp = 'test1') TO (SELECT FROM V WHERE testProp = 'test2')
SELECT *:{!@version, !@class, !out_*, !in_*} FROM V

[luigidellaquila]

Hi @dargolith

thank you for reporting. I don't know if it can be considered a regression or if the old behaviour was incorrect.
Let me analyse it a bit, I'll let you know ASAP

Thanks

Luigi

[dargolith]

Thanks @luigidellaquila !

I didn't know it was un-intended to use it that way. If so, how can I make a projection that excludes fields (if I know what I don't want but not what is actually in the record)?

Thanks for the quick response!

[luigidellaquila]

Hi @dargolith

I thought a bit on this and I came to the conclusion that the correct behaviour is current one.
the nested projection refers to the projected field, eg.

SELECT parent:{name, surname} FROM ...

the name and surname refer to the linked object (the parent), that is expanded. Star * means all the attributes so the following:

SELECT *:{name, surname} FROM ...

means get all the attributes, the nested projection refers to each of them, not to the current record.

There is a way to refer to current record though, that is @this, so you can write the query as follows:

SELECT @this:{!@version, !@class, !out_*, !in_*} FROM V 

This will return a slightly different result than a SELECT *, ie. it will return a record with a single attribute, called @this, containing the current record and without the excluded attributes.

To return what you expect, probably you need to do the following:

select expand(p) from (
  SELECT @this:{!@version, !@class, !out_*, !in_*} as p FROM V 
)

I hope it helps

Thanks

Luigi

[andreyvk]

Hi @luigidellaquila,

Couple of quetions:

1. Why have a nested expand when we could write select expand(@this:{ ... })?
2. Also @this:{!attr1} does not return @rid, @class or @version. So when you expand in the outer query, record could not be formed. I think that's a bug.
3. I think *:{..} and @this:{...} should be used interchangeably. Consider an example:

create class Temp

create property Temp.id string
create property Temp.name string
create property Temp.parent link

# insert a few records and link them up

# QUERY 1: this should return Temp records including "parent" attribute as a LINK. So in this case the nested projection refers only to current record
select *:{id, parent} from Temp

# QUERY 1 VARIATION: nested projections should allow nested projections, so that `*` has nothing to do with referring to linkbags and attempting to apply projections on them. If this approch is not possible at the moment, then leave it at returning parent as a LINK to the user. At least we can issue a separate query to get additional information from parent attribute. 
select *:{id, parent:{id as pid, name as pname}} from Temp

# QUERY 2: this refers to getting parent's id and name only through a link of the current record
select parent:{id, name} from Temp

# QUERY 3: should work exacly as QUERY 1
select expand(@this:{id, parent}) from Temp

So in my mind * is pretty much the same as @this.

Also a note: drastically changing behavior of the query screws everyone's work up quite bad. In the future please consider that there are large projects using your database. I've spend quite a bit of effort developing a REST driver and moving off of from PhpOrient. And I was using 3.0.13 as my main testbed. Not mentioning a fact that an immense amount of queries was restructured to make things work again. A change from 3.0.13 or 3.0.14 should signify a bug fix and not an user facing functionality change that changes dynamics of queries and renders someone's work completely unusable.

[luigidellaquila]

Hi @andreyvk

1. Why have a nested expand when we could write select expand(@this:{ ... })?

just because you can use nested projections only as projections, not as general expressions (so not inside another expression)

2. Also @this:{!attr1} does not return @rid, @class or @version. So when you expand in the outer query, record could not be formed. I think that's a bug.

That's true, if you want @rid @class @version you have to explicitly declare them, eg. @this:{!attr1, @class}. That's because a nested projection is just intended to return actual data by default, not internal info

3. I think *:{..} and @this:{...} should be used interchangeably.

I understand your point, but if it's true, how can you write a query like this #8724?

I agree with you, this was a significant fix and can have a significant impact on existing (my fault, definitely), but it's still a fix, so I cannot just ignore an actual problem... I hope you understand

Thanks

Luigi

[andreyvk]

Hi @luigidellaquila,

I understand your point, but if it's true, how can you write a query like this #8724?

Do you mean select *:{*} ? If I looked at a query then from a first glance it actually looks and feels like a plain select * to me - i.e. get all attributes for the current record. Please correct me if I am wrong here, because I am really trying to grasp this concept.

In any case what's happening now in 3.0.14 is inconsistency between including and excluding attributes in projections. While *:{a, b) produces some result, it does not make sense why *:{!a, !b) shouldnt produce an inverted result of *:{a, b}. Am I right? -- Forget this plz. I've tested more on 3.0.14 and I am starting to understand the difference now.

[andreyvk]

Hi @luigidellaquila ,

That's true, if you want @rid @class @version you have to explicitly declare them, eg. @this:{!attr1, @class}. That's because a nested projection is just intended to return actual data by default, not internal info

That doesnt work:

create class Temp
# insert a few records with this
insert into Temp content {"a": 1, "b": 2, "c": 3}
select @this:{!a, !b, @rid} from Temp

This should return attribute c together with @rid but it doesnt. Only @rid is returned

[andreyvk]

just because you can use nested projections only as projections, not as general expressions (so not inside another expression)

That's a bit strange. While @this.include and @this.exclude used to work well in expand and also in and out projections are working fine, it doesnt make sense that {} projection doesnt. In the end, you are allowing expanding of {} in a sub-query approach (which complicates things quite a bit really), but not in the same query. Why? Current limitation?

[luigidellaquila]

Hi @andreyvk

Sorry, here's the right query:

select @this:{*, !a, !b, @rid} from Temp

The point is that * IS different from *:{*}, the same way as parent is different from parent:{*}.
If * and *:{*} were the same, then also *:{*:{*}} would be the same, so I just wouldn't be able to expand all the attributes at an arbitrary depth.

About the nested projections in the expand(), well, it's just a technical detail, the parser allows nested projections only at projection level, not on general expressions. It's nothing impossible to do, but it requires some work and I don't know if I can do it now.

Again, I understand your concern and the fact that it someway breaks backward compatibility, but the previous behaviour was a bug, I cannot leave it there...

Thanks

Luigi

[andreyvk]

Hi @luigidellaquila,

Thanks for the explanation, it's clear now. I guess as long as query dymamics stay the same from now on, I can continue my development using the latest database, because until now I wasnt sure what to expect.