tests: Add SQL file-based test framework

[andygrove]

Summary

• Adds a sqllogictest-inspired framework for writing Comet SQL tests as plain .sql files
• Unlike sqllogictest, there is no need to store actual results in the files since we compare to Spark and we test against multiple Spark versions with different configs, so results can vary between runs
• Converts 11 tests from CometExpressionSuite into 8 .sql files under spark/src/test/resources/sql-tests/expressions/
• Adds CometSqlFileTestSuite (test runner) and SqlFileTestParser (parser)
• Adds the new suite to Linux and macOS CI workflows

CometSqlFileTestSuite:
26/01/29 07:56:07 INFO core/src/lib.rs: Comet native library version 0.14.0 initialized
- sql-file: expressions/parquet_default_values.sql (5 seconds, 578 milliseconds)
- sql-file: expressions/in_set.sql [spark.sql.optimizer.inSetConversionThreshold=100, parquet.enable.dictionary=false] (1 second, 97 milliseconds)
- sql-file: expressions/in_set.sql [spark.sql.optimizer.inSetConversionThreshold=100, parquet.enable.dictionary=true] (610 milliseconds)
- sql-file: expressions/in_set.sql [spark.sql.optimizer.inSetConversionThreshold=0, parquet.enable.dictionary=false] (565 milliseconds)
- sql-file: expressions/in_set.sql [spark.sql.optimizer.inSetConversionThreshold=0, parquet.enable.dictionary=true] (512 milliseconds)
- sql-file: expressions/arithmetic.sql [parquet.enable.dictionary=false] (886 milliseconds)
- sql-file: expressions/arithmetic.sql [parquet.enable.dictionary=true] (487 milliseconds)
- sql-file: expressions/datetime.sql [parquet.enable.dictionary=false] (352 milliseconds)
- sql-file: expressions/datetime.sql [parquet.enable.dictionary=true] (230 milliseconds)
- sql-file: expressions/boolean.sql [parquet.enable.dictionary=false] (566 milliseconds)
- sql-file: expressions/boolean.sql [parquet.enable.dictionary=true] (471 milliseconds)
- sql-file: expressions/bitwise.sql [parquet.enable.dictionary=false] (823 milliseconds)
- sql-file: expressions/bitwise.sql [parquet.enable.dictionary=true] (719 milliseconds)
- sql-file: expressions/string.sql (634 milliseconds)
- sql-file: expressions/hash.sql [parquet.enable.dictionary=false] (308 milliseconds)
- sql-file: expressions/hash.sql [parquet.enable.dictionary=true] (238 milliseconds)
Run completed in 15 seconds, 660 milliseconds.
Total number of tests run: 16
Suites: completed 1, aborted 0
Tests: succeeded 16, failed 0, canceled 0, ignored 0, pending 0
All tests passed.

Supported directives

Directive	Purpose
-- Config: key=value	Static Spark/Comet SQL config
-- ConfigMatrix: key=val1,val2	Run file once per value (cross-product)
statement	Execute DDL/DML
query	Run checkSparkAnswerAndOperator
query spark_answer_only	Run checkSparkAnswer (no operator check)
query tolerance=0.001	Floating-point tolerance comparison

Why

• No recompilation needed to add or modify SQL tests — just edit a .sql file and run
• Aligns with upstream DataFusion, which uses sqllogictest .slt files extensively for SQL testing
• Lower barrier for contributors to add test coverage — no Scala knowledge required
• Declarative configs via Config/ConfigMatrix headers replace nested Seq.foreach/withSQLConf boilerplate

Test plan

• All 16 SQL file tests pass (including ConfigMatrix expansion)
• Remaining CometExpressionSuite tests still pass (118 tests)
• Total: 134 tests succeeded, 0 failed

🤖 Generated with Claude Code

[codecov-commenter]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 54.98%. Comparing base (f09f8af) to head (4ab3f19).
⚠️ Report is 905 commits behind head on main.

Additional details and impacted files

@@             Coverage Diff              @@
##               main    #3325      +/-   ##
============================================
- Coverage     56.12%   54.98%   -1.15%     
- Complexity      976     1437     +461     
============================================
  Files           119      175      +56     
  Lines         11743    16167    +4424     
  Branches       2251     2682     +431     
============================================
+ Hits           6591     8889    +2298     
- Misses         4012     6011    +1999     
- Partials       1140     1267     +127     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[andygrove]

Actually, I may as well replace this with #3328