[Bug]: Dataflow Example Java11/17 PreCommits actually running on Java 8 container

[Abacn]

What happened?

It has puzzled me when doing #28967, after compiling the example with Java11/17, Dataflow gives unsupported Java version error. Digging into the log one can see

Downloaded container image us-central1-artifactregistry.gcr.io/google.com/dataflow-containers/worker/v1beta3/beam-java-streaming:20231008-20-rc00 checksummed successfully.

which means a Java 8 container is used, even though the test is running on Java17 and the latest dev container version is 20231026

It has been working so far because currently the examples are still compiled on Java8, and the test (compile and submitted by Java17) is only a thin wrapper of the former

Issue Priority

Priority: 2 (default / most bugs should be filed as P2)

Issue Components

• Component: Python SDK
• Component: Java SDK
• Component: Go SDK
• Component: Typescript SDK
• Component: IO connector
• Component: Beam YAML
• Component: Beam examples
• Component: Beam playground
• Component: Beam katas
• Component: Website
• Component: Spark Runner
• Component: Flink Runner
• Component: Samza Runner
• Component: Twister2 Runner
• Component: Hazelcast Jet Runner
• Component: Google Cloud Dataflow Runner

[Abacn]

Found that the cause is here:

beam/runners/google-cloud-dataflow-java/examples/build.gradle

Line 102 in 7b38f74

"--workerHarnessContainerImage=${actualWorkerHarnessContainerImage}"

It assigns an empty string to workerHarnessContainerImage pipeline option. This has caused Dataflow to use an old container in us-central1-docker.pkg.dev/google.com/dataflow-containers/worker/v1beta3/beam-java-streaming

This might be an internal bug. Removing this line, the tag for container is determined by

beam/runners/google-cloud-dataflow-java/build.gradle

Line 54 in 7b38f74

ext.dataflowLegacyContainerVersion = 'beam-master-20231026'

and is using correct java version now.