feat: Support Spark expression years

[YutaLin]

Which issue does this PR close?

Closes #3131

Rationale for this change

Comet does not currently support the Spark years function, causing queries using this function to fall back to Spark's JVM execution instead of running natively on DataFusion.

This patch adds serialization support for Years, allowing Comet to recognize the expression

What changes are included in this PR?

• Map Spark Years expression to DataFusion `datepart('year, child)' and cast to integer
• Implement getSupportLevel to restrict support type DateType, TimestampType and TimestampNTZType
• Register the new handler in QueryPlanSerde

How are these changes tested?

• Add unit test in CometExpressionSerde to verify Years expression is identified correctly and serialized into Protobuf message.
• Add unit test to verify that getSupportLevel returns Compatible for supported temporal types.

[mbutrovich]

Thank you for the contribution @YutaLin! Could you add/migrate the expression tests to the new SQL test framework? https://datafusion.apache.org/comet/contributor-guide/sql-file-tests.html

[YutaLin]

Hi @mbutrovich, thanks for the review!
I investigated adding a SQL test, but found out Years is a PartitionTransformExpression that extends Unevaluable in Spark, which means spark itself cannot execute Years at run time, it would throws [INTERNAL_ERROR] cannot generate code for expression, I cannot write SELECT years(col) or WHERE years(col) = 2024, it's only used as partition metadata, could you guide me on the best way to add a solid test in SQL test framework to verify the partitioning?
I've added tests to verify serialization and support level.

[hsiang-c]

@YutaLin

Thank you for your contribution!

I cannot write SELECT years(col) or WHERE years(col) = 2024, it's only used as partition metadata, could you guide me on the best way to add a solid test in SQL test framework to verify the partitioning?

If I understand it correctly, you might need to create a table with years partition first, insert a few records, then query/check the result.

Not sure if SQL statements can help you with table creation: https://datafusion.apache.org/comet/contributor-guide/sql-file-tests.html#statements

[YutaLin]

Hi @mbutrovich @hsiang-c, i've add the test, let me know if i miss anything, appreciate!

[hsiang-c]

Since Years is a PartitionTransformExpression, maybe we can move it to a new Map(e.g. partitionTransformExpression instead of having it in temporalExpressions?

[hsiang-c]

years is supported by Iceberg only.

Switching to Parquet throws exception b/c CatalogV2Implicits::convertTransform of https://github.com/apache/spark/blob/branch-4.1/sql/catalyst/src/main/scala/org/apache/spark/sql/connector/catalog/CatalogV2Implicits.scala#L62-L101

I am not sure how to verify partition pruning is effective on partitioned Iceberg tables.