Implement OpenCypher TCK

[lvca]

OpenCypher provides Apache 2 licensed TCK to test if an implementation is compliant:

https://github.com/opencypher/openCypher/tree/main/tck

I'm looking to the best way to integrate it into our test suite.

Originally posted by @lvca in #3259

[lvca]

I was able to implement the TCK into the ArcadeDB pipeline, it's now on a separate branch (opencypher-tck). Current results are not great in many areas, but hopefully I can focus on this only for the next days.

Current TCK Results

  ┌───────────────────────────┬────────┬───────┬───────┐
  │         Category          │ Passed │ Total │ Rate  │
  ├───────────────────────────┼────────┼───────┼───────┤
  │ expressions/conditional   │ 13     │ 13    │ 100%  │
  ├───────────────────────────┼────────┼───────┼───────┤
  │ expressions/null          │ 39     │ 44    │ 88%   │
  ├───────────────────────────┼────────┼───────┼───────┤
  │ clauses/remove            │ 25     │ 33    │ 75%   │
  ├───────────────────────────┼────────┼───────┼───────┤
  │ expressions/quantifier    │ 449    │ 604   │ 74%   │
  ├───────────────────────────┼────────┼───────┼───────┤
  │ clauses/unwind            │ 9      │ 14    │ 64%   │
  ├───────────────────────────┼────────┼───────┼───────┤
  │ clauses/return-skip-limit │ 19     │ 31    │ 61%   │
  ├───────────────────────────┼────────┼───────┼───────┤
  │ expressions/aggregation   │ 21     │ 35    │ 60%   │
  ├───────────────────────────┼────────┼───────┼───────┤
  │ expressions/literals      │ 75     │ 131   │ 57%   │
  ├───────────────────────────┼────────┼───────┼───────┤
  │ clauses/return            │ 33     │ 63    │ 52%   │
  ├───────────────────────────┼────────┼───────┼───────┤
  │ clauses/return-orderby    │ 18     │ 35    │ 51%   │
  ├───────────────────────────┼────────┼───────┼───────┤
  │ ...                       │        │       │       │
  ├───────────────────────────┼────────┼───────┼───────┤
  │ Overall                   │ 1153   │ 3897  │ 29.6% │
  └───────────────────────────┴────────┴───────┴───────┘

To Run TCK suite (from project root on the specific branch until is merged):

cd engine && mvn test -Dtest=OpenCypherTCKRunner -DfailIfNoTests=false -Dsurefire.excludeJUnit5Engines=""

[lvca]

I kept TCK build separated, once all tests pass @robfrank will run it on our CI. Now it takes 47s on my laptop to execute the whole suite. This is the latest results:

=======================================================
         OpenCypher TCK Compliance Report
=======================================================
Total scenarios:  3897
Passed:           1558 (39%)
Failed:           2289 (58%)
Skipped:          50 (1%)
-------------------------------------------------------
By category:
  clauses/call                               2/ 52 passed (3%)
  clauses/create                            62/ 78 passed (79%)
  clauses/delete                            24/ 41 passed (58%)
  clauses/match                            278/381 passed (72%)
  clauses/match-where                       14/ 34 passed (41%)
  clauses/merge                             46/ 75 passed (61%)
  clauses/remove                            28/ 33 passed (84%)
  clauses/return                            32/ 63 passed (50%)
  clauses/return-orderby                    20/ 35 passed (57%)
  clauses/return-skip-limit                 19/ 31 passed (61%)
  clauses/set                               26/ 53 passed (49%)
  clauses/union                              6/ 12 passed (50%)
  clauses/unwind                            10/ 14 passed (71%)
  clauses/with                              12/ 29 passed (41%)
  clauses/with-orderBy                      61/292 passed (20%)
  clauses/with-skip-limit                    3/  9 passed (33%)
  clauses/with-where                         4/ 19 passed (21%)
  expressions/aggregation                   21/ 35 passed (60%)
  expressions/boolean                       70/150 passed (46%)
  expressions/comparison                    29/ 72 passed (40%)
  expressions/conditional                   13/ 13 passed (100%)
  expressions/existentialSubqueries          2/ 10 passed (20%)
  expressions/graph                         25/ 61 passed (40%)
  expressions/list                          79/185 passed (42%)
  expressions/literals                      75/131 passed (57%)
  expressions/map                           18/ 44 passed (40%)
  expressions/mathematical                   3/  6 passed (50%)
  expressions/null                          44/ 44 passed (100%)
  expressions/path                           0/  7 passed (0%)
  expressions/pattern                        7/ 50 passed (14%)
  expressions/precedence                    18/121 passed (14%)
  expressions/quantifier                   470/604 passed (77%)
  expressions/string                        11/ 32 passed (34%)
  expressions/temporal                       0/1004 passed (0%)
  expressions/typeConversion                19/ 47 passed (40%)
  useCases/countingSubgraphMatches           6/ 11 passed (54%)
  useCases/triadicSelection                  1/ 19 passed (5%)
=======================================================

[ExtReMLapin]

"okay claude code, fix all the issues"

[lvca]

Still many tests fail because of missing functions. Temporal functions alone have 1004 tests.

[lvca]

Last update:

=======================================================
         OpenCypher TCK Compliance Report
=======================================================
Total scenarios:  3897
Passed:           3812 (97%)
Failed:           0 (0%)
Skipped:          85 (2%)
-------------------------------------------------------
By category:
  clauses/call                               2/ 52 passed (3%)
  clauses/create                            78/ 78 passed (100%)
  clauses/delete                            41/ 41 passed (100%)
  clauses/match                            380/381 passed (99%)
  clauses/match-where                       34/ 34 passed (100%)
  clauses/merge                             75/ 75 passed (100%)
  clauses/remove                            33/ 33 passed (100%)
  clauses/return                            63/ 63 passed (100%)
  clauses/return-orderby                    35/ 35 passed (100%)
  clauses/return-skip-limit                 31/ 31 passed (100%)
  clauses/set                               53/ 53 passed (100%)
  clauses/union                             12/ 12 passed (100%)
  clauses/unwind                            14/ 14 passed (100%)
  clauses/with                              29/ 29 passed (100%)
  clauses/with-orderBy                     268/292 passed (91%)
  clauses/with-skip-limit                    9/  9 passed (100%)
  clauses/with-where                        19/ 19 passed (100%)
  expressions/aggregation                   35/ 35 passed (100%)
  expressions/boolean                      150/150 passed (100%)
  expressions/comparison                    68/ 72 passed (94%)
  expressions/conditional                   13/ 13 passed (100%)
  expressions/existentialSubqueries         10/ 10 passed (100%)
  expressions/graph                         60/ 61 passed (98%)
  expressions/list                         185/185 passed (100%)
  expressions/literals                     131/131 passed (100%)
  expressions/map                           44/ 44 passed (100%)
  expressions/mathematical                   6/  6 passed (100%)
  expressions/null                          44/ 44 passed (100%)
  expressions/path                           7/  7 passed (100%)
  expressions/pattern                       50/ 50 passed (100%)
  expressions/precedence                   121/121 passed (100%)
  expressions/quantifier                   599/604 passed (99%)
  expressions/string                        32/ 32 passed (100%)
  expressions/temporal                     1004/1004 passed (100%)
  expressions/typeConversion                47/ 47 passed (100%)
  useCases/countingSubgraphMatches          11/ 11 passed (100%)
  useCases/triadicSelection                 19/ 19 passed (100%)
=======================================================