Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Revert [SPARK-43836][SPARK-43845][SPARK-43858][SPARK-43858] to keep Scala 2.12 for Spark 3.x #41416

Closed
wants to merge 6 commits into from
Closed

Revert [SPARK-43836][SPARK-43845][SPARK-43858][SPARK-43858] to keep Scala 2.12 for Spark 3.x #41416

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
33b202c
Revert "[SPARK-43858][INFRA] Make benchmark Github Action task use Sc…
dongjoon-hyun Jun 1, 2023
110472f
Revert "[SPARK-43845][INFRA] Setup Scala 2.12 Daily GitHub Action Job"
dongjoon-hyun Jun 1, 2023
30d5a0a
Revert "[SPARK-43849][BUILD] Enable unused imports check for Scala 2.13"
dongjoon-hyun Jun 1, 2023
8ff1520
Revert "[SPARK-43836][BUILD] Make Scala 2.13 as default in Spark 3.5"
dongjoon-hyun Jun 1, 2023
f989e02
[SPARK-43849][BUILD] Enable unused imports check for Scala 2.13
LuciferYang May 29, 2023
828e0fc
Revert "[SPARK-43858][INFRA][FOLLOWUP] Restore the Scala version to 2…
dongjoon-hyun Jun 1, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .github/workflows/benchmark.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ on:
scala:
description: 'Scala version: 2.12 or 2.13'
required: true
default: '2.13'
default: '2.12'
failfast:
description: 'Failfast: true or false'
required: true
Expand Down Expand Up @@ -182,7 +182,7 @@ jobs:
"`find . -name 'spark-core*-SNAPSHOT-tests.jar'`" \
"${{ github.event.inputs.class }}"
# Revert to default Scala version to clean up unnecessary git diff
dev/change-scala-version.sh 2.13
dev/change-scala-version.sh 2.12
# To keep the directory structure and file permissions, tar them
# See also https://github.com/actions/upload-artifact#maintaining-file-permissions-and-case-sensitive-files
echo "Preparing the benchmark results:"
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/build_scala212.yml → .github/workflows/build_scala213.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@
# under the License.
#

name: "Build (master, Scala 2.12, Hadoop 3, JDK 8)"
name: "Build (master, Scala 2.13, Hadoop 3, JDK 8)"

on:
schedule:
Expand All @@ -36,7 +36,7 @@ jobs:
hadoop: hadoop3
envs: >-
{
"SCALA_PROFILE": "scala2.12"
"SCALA_PROFILE": "scala2.13"
}
jobs: >-
{
Expand Down
4 changes: 2 additions & 2 deletions assembly/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,12 +20,12 @@
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>org.apache.spark</groupId>
<artifactId>spark-parent_2.13</artifactId>
<artifactId>spark-parent_2.12</artifactId>
<version>3.5.0-SNAPSHOT</version>
<relativePath>../pom.xml</relativePath>
</parent>

<artifactId>spark-assembly_2.13</artifactId>
<artifactId>spark-assembly_2.12</artifactId>
<name>Spark Project Assembly</name>
<url>https://spark.apache.org/</url>
<packaging>pom</packaging>
Expand Down
4 changes: 2 additions & 2 deletions common/kvstore/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,12 +21,12 @@
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>org.apache.spark</groupId>
<artifactId>spark-parent_2.13</artifactId>
<artifactId>spark-parent_2.12</artifactId>
<version>3.5.0-SNAPSHOT</version>
<relativePath>../../pom.xml</relativePath>
</parent>

<artifactId>spark-kvstore_2.13</artifactId>
<artifactId>spark-kvstore_2.12</artifactId>
<packaging>jar</packaging>
<name>Spark Project Local DB</name>
<url>https://spark.apache.org/</url>
Expand Down
4 changes: 2 additions & 2 deletions common/network-common/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,12 +21,12 @@
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>org.apache.spark</groupId>
<artifactId>spark-parent_2.13</artifactId>
<artifactId>spark-parent_2.12</artifactId>
<version>3.5.0-SNAPSHOT</version>
<relativePath>../../pom.xml</relativePath>
</parent>

<artifactId>spark-network-common_2.13</artifactId>
<artifactId>spark-network-common_2.12</artifactId>
<packaging>jar</packaging>
<name>Spark Project Networking</name>
<url>https://spark.apache.org/</url>
Expand Down
4 changes: 2 additions & 2 deletions common/network-shuffle/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,12 +21,12 @@
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>org.apache.spark</groupId>
<artifactId>spark-parent_2.13</artifactId>
<artifactId>spark-parent_2.12</artifactId>
<version>3.5.0-SNAPSHOT</version>
<relativePath>../../pom.xml</relativePath>
</parent>

<artifactId>spark-network-shuffle_2.13</artifactId>
<artifactId>spark-network-shuffle_2.12</artifactId>
<packaging>jar</packaging>
<name>Spark Project Shuffle Streaming Service</name>
<url>https://spark.apache.org/</url>
Expand Down
4 changes: 2 additions & 2 deletions common/network-yarn/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,12 +21,12 @@
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>org.apache.spark</groupId>
<artifactId>spark-parent_2.13</artifactId>
<artifactId>spark-parent_2.12</artifactId>
<version>3.5.0-SNAPSHOT</version>
<relativePath>../../pom.xml</relativePath>
</parent>

<artifactId>spark-network-yarn_2.13</artifactId>
<artifactId>spark-network-yarn_2.12</artifactId>
<packaging>jar</packaging>
<name>Spark Project YARN Shuffle Service</name>
<url>https://spark.apache.org/</url>
Expand Down
4 changes: 2 additions & 2 deletions common/sketch/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,12 +21,12 @@
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>org.apache.spark</groupId>
<artifactId>spark-parent_2.13</artifactId>
<artifactId>spark-parent_2.12</artifactId>
<version>3.5.0-SNAPSHOT</version>
<relativePath>../../pom.xml</relativePath>
</parent>

<artifactId>spark-sketch_2.13</artifactId>
<artifactId>spark-sketch_2.12</artifactId>
<packaging>jar</packaging>
<name>Spark Project Sketch</name>
<url>https://spark.apache.org/</url>
Expand Down
4 changes: 2 additions & 2 deletions common/tags/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,12 +21,12 @@
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>org.apache.spark</groupId>
<artifactId>spark-parent_2.13</artifactId>
<artifactId>spark-parent_2.12</artifactId>
<version>3.5.0-SNAPSHOT</version>
<relativePath>../../pom.xml</relativePath>
</parent>

<artifactId>spark-tags_2.13</artifactId>
<artifactId>spark-tags_2.12</artifactId>
<packaging>jar</packaging>
<name>Spark Project Tags</name>
<url>https://spark.apache.org/</url>
Expand Down
4 changes: 2 additions & 2 deletions common/unsafe/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,12 +21,12 @@
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>org.apache.spark</groupId>
<artifactId>spark-parent_2.13</artifactId>
<artifactId>spark-parent_2.12</artifactId>
<version>3.5.0-SNAPSHOT</version>
<relativePath>../../pom.xml</relativePath>
</parent>

<artifactId>spark-unsafe_2.13</artifactId>
<artifactId>spark-unsafe_2.12</artifactId>
<packaging>jar</packaging>
<name>Spark Project Unsafe</name>
<url>https://spark.apache.org/</url>
Expand Down
4 changes: 2 additions & 2 deletions common/utils/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,12 +21,12 @@
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>org.apache.spark</groupId>
<artifactId>spark-parent_2.13</artifactId>
<artifactId>spark-parent_2.12</artifactId>
<version>3.5.0-SNAPSHOT</version>
<relativePath>../../pom.xml</relativePath>
</parent>

<artifactId>spark-common-utils_2.13</artifactId>
<artifactId>spark-common-utils_2.12</artifactId>
<packaging>jar</packaging>
<name>Spark Project Common Utils</name>
<url>https://spark.apache.org/</url>
Expand Down
8 changes: 4 additions & 4 deletions connector/avro/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,12 +20,12 @@
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>org.apache.spark</groupId>
<artifactId>spark-parent_2.13</artifactId>
<artifactId>spark-parent_2.12</artifactId>
<version>3.5.0-SNAPSHOT</version>
<relativePath>../../pom.xml</relativePath>
</parent>

<artifactId>spark-avro_2.13</artifactId>
<artifactId>spark-avro_2.12</artifactId>
<properties>
<sbt.project.name>avro</sbt.project.name>
</properties>
Expand Down Expand Up @@ -70,12 +70,12 @@
<groupId>org.apache.spark</groupId>
<artifactId>spark-tags_${scala.binary.version}</artifactId>
</dependency>
<!-- #if scala-2.13 -->
<!-- #if scala-2.13 --><!--
<dependency>
<groupId>org.scala-lang.modules</groupId>
<artifactId>scala-parallel-collections_${scala.binary.version}</artifactId>
</dependency>
<!-- #endif scala-2.13 -->
--><!-- #endif scala-2.13 -->
<dependency>
<groupId>org.tukaani</groupId>
<artifactId>xz</artifactId>
Expand Down
4 changes: 2 additions & 2 deletions connector/connect/client/jvm/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,12 +21,12 @@
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>org.apache.spark</groupId>
<artifactId>spark-parent_2.13</artifactId>
<artifactId>spark-parent_2.12</artifactId>
<version>3.5.0-SNAPSHOT</version>
<relativePath>../../../../pom.xml</relativePath>
</parent>

<artifactId>spark-connect-client-jvm_2.13</artifactId>
<artifactId>spark-connect-client-jvm_2.12</artifactId>
<packaging>jar</packaging>
<name>Spark Project Connect Client</name>
<url>https://spark.apache.org/</url>
Expand Down
4 changes: 2 additions & 2 deletions connector/connect/common/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,12 +21,12 @@
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>org.apache.spark</groupId>
<artifactId>spark-parent_2.13</artifactId>
<artifactId>spark-parent_2.12</artifactId>
<version>3.5.0-SNAPSHOT</version>
<relativePath>../../../pom.xml</relativePath>
</parent>

<artifactId>spark-connect-common_2.13</artifactId>
<artifactId>spark-connect-common_2.12</artifactId>
<packaging>jar</packaging>
<name>Spark Project Connect Common</name>
<url>https://spark.apache.org/</url>
Expand Down
8 changes: 4 additions & 4 deletions connector/connect/server/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,12 +21,12 @@
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>org.apache.spark</groupId>
<artifactId>spark-parent_2.13</artifactId>
<artifactId>spark-parent_2.12</artifactId>
<version>3.5.0-SNAPSHOT</version>
<relativePath>../../../pom.xml</relativePath>
</parent>

<artifactId>spark-connect_2.13</artifactId>
<artifactId>spark-connect_2.12</artifactId>
<packaging>jar</packaging>
<name>Spark Project Connect Server</name>
<url>https://spark.apache.org/</url>
Expand Down Expand Up @@ -156,12 +156,12 @@
</exclusion>
</exclusions>
</dependency>
<!-- #if scala-2.13 -->
<!-- #if scala-2.13 --><!--
<dependency>
<groupId>org.scala-lang.modules</groupId>
<artifactId>scala-parallel-collections_${scala.binary.version}</artifactId>
</dependency>
<!-- #endif scala-2.13 -->
--><!-- #endif scala-2.13 -->
<dependency>
<groupId>com.google.guava</groupId>
<artifactId>guava</artifactId>
Expand Down
4 changes: 2 additions & 2 deletions connector/docker-integration-tests/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,12 +21,12 @@
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>org.apache.spark</groupId>
<artifactId>spark-parent_2.13</artifactId>
<artifactId>spark-parent_2.12</artifactId>
<version>3.5.0-SNAPSHOT</version>
<relativePath>../../pom.xml</relativePath>
</parent>

<artifactId>spark-docker-integration-tests_2.13</artifactId>
<artifactId>spark-docker-integration-tests_2.12</artifactId>
<packaging>jar</packaging>
<name>Spark Project Docker Integration Tests</name>
<url>https://spark.apache.org/</url>
Expand Down
4 changes: 2 additions & 2 deletions connector/kafka-0-10-assembly/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,12 +20,12 @@
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>org.apache.spark</groupId>
<artifactId>spark-parent_2.13</artifactId>
<artifactId>spark-parent_2.12</artifactId>
<version>3.5.0-SNAPSHOT</version>
<relativePath>../../pom.xml</relativePath>
</parent>

<artifactId>spark-streaming-kafka-0-10-assembly_2.13</artifactId>
<artifactId>spark-streaming-kafka-0-10-assembly_2.12</artifactId>
<packaging>jar</packaging>
<name>Spark Integration for Kafka 0.10 Assembly</name>
<url>https://spark.apache.org/</url>
Expand Down
8 changes: 4 additions & 4 deletions connector/kafka-0-10-sql/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,13 +20,13 @@
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>org.apache.spark</groupId>
<artifactId>spark-parent_2.13</artifactId>
<artifactId>spark-parent_2.12</artifactId>
<version>3.5.0-SNAPSHOT</version>
<relativePath>../../pom.xml</relativePath>
</parent>

<groupId>org.apache.spark</groupId>
<artifactId>spark-sql-kafka-0-10_2.13</artifactId>
<artifactId>spark-sql-kafka-0-10_2.12</artifactId>
<properties>
<sbt.project.name>sql-kafka-0-10</sbt.project.name>
</properties>
Expand Down Expand Up @@ -74,12 +74,12 @@
<type>test-jar</type>
<scope>test</scope>
</dependency>
<!-- #if scala-2.13 -->
<!-- #if scala-2.13 --><!--
<dependency>
<groupId>org.scala-lang.modules</groupId>
<artifactId>scala-parallel-collections_${scala.binary.version}</artifactId>
</dependency>
<!-- #endif scala-2.13 -->
--><!-- #endif scala-2.13 -->
<dependency>
<groupId>org.apache.kafka</groupId>
<artifactId>kafka-clients</artifactId>
Expand Down
4 changes: 2 additions & 2 deletions connector/kafka-0-10-token-provider/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,13 +20,13 @@
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>org.apache.spark</groupId>
<artifactId>spark-parent_2.13</artifactId>
<artifactId>spark-parent_2.12</artifactId>
<version>3.5.0-SNAPSHOT</version>
<relativePath>../../pom.xml</relativePath>
</parent>

<groupId>org.apache.spark</groupId>
<artifactId>spark-token-provider-kafka-0-10_2.13</artifactId>
<artifactId>spark-token-provider-kafka-0-10_2.12</artifactId>
<properties>
<sbt.project.name>token-provider-kafka-0-10</sbt.project.name>
</properties>
Expand Down
8 changes: 4 additions & 4 deletions connector/kafka-0-10/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,12 +20,12 @@
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>org.apache.spark</groupId>
<artifactId>spark-parent_2.13</artifactId>
<artifactId>spark-parent_2.12</artifactId>
<version>3.5.0-SNAPSHOT</version>
<relativePath>../../pom.xml</relativePath>
</parent>

<artifactId>spark-streaming-kafka-0-10_2.13</artifactId>
<artifactId>spark-streaming-kafka-0-10_2.12</artifactId>
<properties>
<sbt.project.name>streaming-kafka-0-10</sbt.project.name>
</properties>
Expand Down Expand Up @@ -59,12 +59,12 @@
<type>test-jar</type>
<scope>test</scope>
</dependency>
<!-- #if scala-2.13 -->
<!-- #if scala-2.13 --><!--
<dependency>
<groupId>org.scala-lang.modules</groupId>
<artifactId>scala-parallel-collections_${scala.binary.version}</artifactId>
</dependency>
<!-- #endif scala-2.13 -->
--><!-- #endif scala-2.13 -->
<dependency>
<groupId>org.apache.kafka</groupId>
<artifactId>kafka-clients</artifactId>
Expand Down
4 changes: 2 additions & 2 deletions connector/kinesis-asl-assembly/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,12 +20,12 @@
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>org.apache.spark</groupId>
<artifactId>spark-parent_2.13</artifactId>
<artifactId>spark-parent_2.12</artifactId>
<version>3.5.0-SNAPSHOT</version>
<relativePath>../../pom.xml</relativePath>
</parent>

<artifactId>spark-streaming-kinesis-asl-assembly_2.13</artifactId>
<artifactId>spark-streaming-kinesis-asl-assembly_2.12</artifactId>
<packaging>jar</packaging>
<name>Spark Project Kinesis Assembly</name>
<url>https://spark.apache.org/</url>
Expand Down
4 changes: 2 additions & 2 deletions connector/kinesis-asl/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,13 +19,13 @@
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>org.apache.spark</groupId>
<artifactId>spark-parent_2.13</artifactId>
<artifactId>spark-parent_2.12</artifactId>
<version>3.5.0-SNAPSHOT</version>
<relativePath>../../pom.xml</relativePath>
</parent>

<!-- Kinesis integration is not included by default due to ASL-licensed code. -->
<artifactId>spark-streaming-kinesis-asl_2.13</artifactId>
<artifactId>spark-streaming-kinesis-asl_2.12</artifactId>
<packaging>jar</packaging>
<name>Spark Kinesis Integration</name>

Expand Down
Loading