Build failure with JDK 11

[zhztheplayer]

Apache Iceberg version

None

Query engine

None

Please describe the bug 🐞

With latest Iceberg code (6e432fc):

On Ubuntu 24.04:

update-java-alternatives -s /usr/lib/jvm/java-1.11.0-openjdk-arm64 # Switch to JDK 11
./gradlew clean build -x test -x integrationTest # Build Iceberg

Error:

> Task :iceberg-spark:iceberg-spark-4.0_2.13:compileScala
[Error] /opt/code/iceberg/spark/v4.0/spark/src/main/java/org/apache/iceberg/SparkDistributedDataScan.java:41:33:  error: cannot access JavaRDD
[Error] /opt/code/iceberg/spark/v4.0/spark/src/main/java/org/apache/iceberg/SparkDistributedDataScan.java:42:33:  error: cannot access JavaSparkContext
[Error] /opt/code/iceberg/spark/v4.0/spark/src/main/java/org/apache/iceberg/SparkDistributedDataScan.java:43:42:  error: cannot access FlatMapFunction
[Error] /opt/code/iceberg/spark/v4.0/spark/src/main/java/org/apache/iceberg/SparkDistributedDataScan.java:44:34:  error: cannot access Broadcast
[Error] /opt/code/iceberg/spark/v4.0/spark/src/main/java/org/apache/iceberg/SparkDistributedDataScan.java:45:28:  error: cannot access SparkSession
...

After switching to JDK 17, the failure disappeared.

I guess it's because Spark 4.0 has moved to JDK 17, while Iceberg still supports JDK 11?

Willingness to contribute

• I can contribute a fix for this bug independently
• I would be willing to contribute a fix for this bug with guidance from the Iceberg community
• I cannot contribute a fix for this bug at this time

[zhztheplayer]

@huaxingao Have you or anyone faced something similar? Thanks!

[manuzhang]

@zhztheplayer iceberg-spark-4.0 needs to be built with JDK 17+, as per spark-ci

[zhztheplayer]

@zhztheplayer iceberg-spark-4.0 needs to be built with JDK 17+, as per spark-ci

@manuzhang Thank you for the explanation. As Spark 4.0 is the default version in gradle.properties, this could mean Iceberg doesn't compile with JDK 17- with the default settings?

Is it a good idea to enforce the Java version in Gradle when Spark 4.0 is enabled? So one doesn't have to troubleshoot these compilation errors.

[manuzhang]

Is it a good idea to enforce the Java version in Gradle when Spark 4.0 is enabled? So one doesn't have to troubleshoot these compilation errors.

Yeah, that will be ideal. Would you mind submitting a PR?

[zhztheplayer]

Is it a good idea to enforce the Java version in Gradle when Spark 4.0 is enabled? So one doesn't have to troubleshoot these compilation errors.

Yeah, that will be ideal. Would you mind submitting a PR?

Yes I will have a try. Thanks!

[nastra]

fixed by #13381

[huaxingao]

Thanks @zhztheplayer for resolving the issue!