Can we build GraphAr Spark library using Spark 3.3.1 ?

[DancyHuang]

GraphAr Spark uses Maven Profiles to support multiple Spark Versions. By default it is built with Spark 3.2.x or profile datasources-32. To built with Spark 3.3.4 use -P datasources-33 (mvn clean install -DskipTests -P datasources-33).

The library built with 3.3.4 is incompatible with spark 3.3.1.

Can we build GraphAr Spark library using Spark 3.3.1 ?

[SemyonSinchenko]

Apache Spark follows semantic versioning and all the versions from 3.3.x are back compatible. In my understanding there should not be any problem of running app on spark cluster with spark 3.3.1 with GraphAr built for 3.3.4. If you face an error, can you please share the stacktrace?

[DancyHuang]

graphar-commons.jar and graphar-datasource.jar are built with spark 3.3.4, and my test code is write graphar and read it . With spark 3.3.4 , I can write graphAr and read it successfully, but when I set the spark jars 3.3.1 , write graphar is ok , but when I read vertex files report the above error.

This is my code of reading graphar parquet files.

[SemyonSinchenko]

I see. It seems to we that is a rare case and we can workaround it.

[SemyonSinchenko]

@DancyHuang FYI: #639