[Task]: Planned removal of Spark 2 runner support

[mosche]

What needs to happen?

The runner for Spark 2 was deprecated quite a while back in August 2022 with the release of Beam 2.41.0. We’re planning to move ahead with this and finally remove support for Spark 2 (beam-runners-spark) to only maintain support for Spark 3 (beam-runners-spark-3) going forward.

Note, Spark 2.4.8 is the latest release of Spark 2. It was released in May 2021 and no further releases of 2.4.x should be expected, even for bug fixes.

Issue Priority

Priority: 2 (default / most normal work should be filed as P2)

Issue Components

• Component: Python SDK
• Component: Java SDK
• Component: Go SDK
• Component: Typescript SDK
• Component: IO connector
• Component: Beam examples
• Component: Beam playground
• Component: Beam katas
• Component: Website
• Component: Spark Runner
• Component: Flink Runner
• Component: Samza Runner
• Component: Twister2 Runner
• Component: Hazelcast Jet Runner
• Component: Google Cloud Dataflow Runner