Query optimization summary based on cache experiment

[wenhaocs]

With experiments on empty cache, we summarize the queries that will generate non-existing keys as follows. Some are expected but some are not.

1. GoNStep without specifying tags in return

GO 1 STEP FROM {} OVER KNOWS YIELD properties($$) --- large number of non-existing keys
vs.
GO 1 STEP FROM {} OVER KNOWS YIELD $$.Person.birthday

2. Match

2.1 queries returning vertices without a tag in matching condition

MATCH (v) RETURN v LIMIT 3 --- expected

2.2 queries returning vertices with a tag in matching condition

MATCH (v:player) RETURN v LIMIT 3 ---debatable

2.3 Any path matching query

MATCH ()<-[e]-() RETURN e LIMIT 3 --- not expected
This query is expected to return edges, but will call appendVertices -> getProp -> TagNode to get properties of nodes. It is not optimized.

2.4 Implicitly specifying edge type

MATCH (v:player{name:"Tim Duncan"})--(v2:player) RETURN v2.player.name AS Name --- not expected
This query implicitly indicates that the edge type to query is "follow'. But when it assembles the edge key prefix, it still tries all possible edge types, which result in the destination ids which belong to the "Team" tag are also retrieved. Then the key combined by that kind of destination ids and tag of player will cause keys not found for sure.

[wenhaocs]

2.2 is by design in cypher. 2.3 and 2.4 are not expected and should be optimized at query layer.

[Shylock-Hg]

I think query of these nonexistent keys is caused by there is only RPC getProps which only scan by tags instead of RPC like getVertices which scan by vertex id. If we have primitive to scan vertices, graph layer could use it to implement query to scan vertices instead of mocking it by scanning all tags in space as now.

[wenhaocs]

I think query of these nonexistent keys is caused by there is only RPC getProps which only scan by tags instead of RPC like getVertices which scan by vertex id. If we have primitive to scan vertices, graph layer could use it to implement query to scan vertices instead of mocking it by scanning all tags in space as now.

@Shylock-Hg In storage, the tag to search is passed by query layer. If tag is specified, the Go queries will not generate non-existing keys. Match query will however return keys with all possible tags. That is by design in Cypher. The issue is 2.3 and 2.4 above.

GetProps is not doing scanning. It just returns the properties given a vertex id and tags, by appending tag to vertex id so a key is generated to retrieve from RocksDB. I don't understand GetVertices. Given a vertex id, what does getVertices return?

[Shylock-Hg]

I think query of these nonexistent keys is caused by there is only RPC getProps which only scan by tags instead of RPC like getVertices which scan by vertex id. If we have primitive to scan vertices, graph layer could use it to implement query to scan vertices instead of mocking it by scanning all tags in space as now.

@Shylock-Hg In storage, the tag to search is passed by query layer. If tag is specified, the Go queries will not generate non-existing keys. Match query will however return keys with all possible tags. That is by design in Cypher. The issue is 2.3 and 2.4 above.

GetProps is not doing scanning. It just returns the properties given a vertex id and tags, by appending tag to vertex id so a key is generated to retrieve from RocksDB. I don't understand GetVertices. Given a vertex id, what does getVertices return?

getVertices return all properties of given vertices.