Bug: Assertion failure in PathPropertyProbe when running Fintech Benchmark

[royi-luo]

Kùzu version

master

What operating system are you using?

Ubuntu 22.04.5 LTS

What happened?

Occasionally in the Fintech benchmark when running queries ComplexRead1 or ComplexRead3, in PathPropertyProbe::probe the hash table probe will fail for some path rel ID value.

for (auto i = 0u; i < sizeToProbe; ++i) {
        localState->probedTuples[i] = hashTable->getTupleForHash(localState->hashes[i]); // this will be nullptr
}

This will result in the assertion failure (which will probably manifest as a segfault if we have runtime checks disabled)

Error: Assertion failed in file "/home/kuzu/kuzu/src/processor/operator/recursive_extend/path_property_probe.cpp" on line 96: localState->matchedTuples[i] != nullptr

Another potentially important observation is if you checkpoint and run the query again the assertion failure will not occur.

More specifically, in HashJoinBuild::finalize when we build the hash slots the number of tuples in the fTable will be different between the two runs.

Are there known steps to reproduce?

1. Build kuzu with runtime checks enabled
2. Open the attached DB
3. Run the following query:

MATCH
  p = (account:Account {id: 4615360861165330226})-[edge1:AccountTransferAccount*1..3 (r, n | WHERE timestamp("2021-07-23T06:10:16.747Z") < r.timestamp AND r.timestamp < timestamp("2022-11-29T02:52:22.640Z"))]->(other:Account),
  (other)<-[edge2:MediumSignInAccount]-(medium:Medium {isBlocked: true})
WITH
  edge2.timestamp AS signInTime,
  properties(rels(p), 'timestamp') AS ts,
  length(p) AS pathLength,
  other, medium
WHERE
  timestamp("2021-07-23T06:10:16.747Z") < signInTime AND signInTime < timestamp("2022-11-29T02:52:22.640Z")
  AND ts = list_sort(ts, 'ASC')
WITH
  other.id AS otherId, pathLength AS accountDistance, medium.id AS mediumId, medium.type AS mediumType
RETURN otherId, CAST(accountDistance, "INT32"), mediumId, mediumType
ORDER BY accountDistance, otherId, mediumId;

Alternatively, you can run the fintech benchmarks from the start using the driver and you will probably run into a similar issue. I'll try to get something else that is easier to reproduce.

[royi-luo]

Alternatively, the following queries can be run on top of the base sf1 database to reproduce this issue:

CALL threads=1;
MATCH (account:Account {id: 4900480993799974672})
WITH account
OPTIONAL MATCH (account)-[:AccountRepayLoan]->(loan1:Loan)
WITH account, loan1
OPTIONAL MATCH (loan2:Loan)-[:LoanDepositAccount]->(account)
DETACH DELETE account, loan1, loan2;
MATCH
  p = (account:Account {id: 2251799813685615})-[edge1:AccountTransferAccount*1..3 (r, n | WHERE timestamp("2021-07-23T06:10:16.747Z") < r.timestamp AND r.timestamp < timestamp("2022-11-29T02:52:22.640Z"))]->(other:Account),
  (other)<-[edge2:MediumSignInAccount]-(medium:Medium {isBlocked: true})
WITH
  edge2.timestamp AS signInTime,
  properties(rels(p), 'timestamp') AS ts,
  length(p) AS pathLength,
  other, medium
WHERE
  timestamp("2021-07-23T06:10:16.747Z") < signInTime AND signInTime < timestamp("2022-11-29T02:52:22.640Z")
  AND ts = list_sort(ts, 'ASC')
WITH
  other.id AS otherId, pathLength AS accountDistance, medium.id AS mediumId, medium.type AS mediumType
RETURN otherId, CAST(accountDistance, "INT32"), mediumId, mediumType
ORDER BY accountDistance, otherId, mediumId;