Cannot build Bazel with local JDK21

[coeuvre]

Description of the bug:

When building Bazel itself with a local JDK21 (local_java_runtime), it fails with following error:

ERROR: /usr/local/google/home/chiwang/.cache/bazel/_bazel_chiwang/0d7b80a634c8f6dac7c916ddfc337473/external/rules_jvm_external~6.0/private/tools/java/com/github/bazelbuild/rules_jvm_external/BUILD:1:13: Compiling Java headers external/rules_jvm_external~6.0/private/tools/java/com/github/bazelbuild/rules_jvm_external/librules_jvm_external-hjar.jar (4 source files) [for tool] failed: (Exit 1): turbine_direct_graal failed: error executing Turbine command (from target @@rules_jvm_external~6.0//private/tools/java/com/github/bazelbuild/rules_jvm_external:rules_jvm_external) external/rules_java~7.4.0~toolchains~remote_java_tools_linux/java_tools/turbine_direct_graal --output ... (remaining 33 arguments skipped)

Use --sandbox_debug to see verbose messages from the sandbox and retain the sandbox build root for debugging
java.lang.NullPointerException: attempted to use --release, but JAVA_HOME is not set
        at java.base@20.0.2/java.util.Objects.requireNonNull(Objects.java:259)
        at com.google.turbine.binder.CtSymClassBinder.bind(CtSymClassBinder.java:55)
        at com.google.turbine.main.Main.bootclasspath(Main.java:309)
        at com.google.turbine.main.Main.compile(Main.java:142)
        at com.google.turbine.main.Main.compile(Main.java:133)
        at com.google.turbine.main.Main.main(Main.java:89)
Target //src:bazel failed to build

It seems like the turbine.ctSymPath property isn't correctly set, because java_runtime.lib_ct_sym is None. By following the code in _java_runtime_rule_impl, it's because local_java_runtime doesn't set srcs nor lib_ct_sym .

This prevent us using RBE with JDK21.

Which category does this issue belong to?

Java Rules

What's the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.

1. Clone https://github.com/bazelbuild/bazel
2. Add a local JDK runtime into BUILD:

load("@rules_java//toolchains:local_java_repository.bzl", "local_java_runtime")
local_java_runtime(
    name = "local",
    java_home = "<path to local JDK21>",
    version = "21",
)

3. bazel build --java_runtime_version=local --tool_java_runtime_version=local --extra_toolchains=//:all //src:bazel

Which operating system are you running Bazel on?

linux

What is the output of bazel info release?

release 7.0.2

If bazel info release returns development version or (@non-git), tell us how you built Bazel.

No response

What's the output of git remote get-url origin; git rev-parse HEAD ?

No response

Is this a regression? If yes, please try to identify the Bazel commit where the bug was introduced.

No response

Have you found anything relevant by searching the web?

No response

Any other information, logs, or outputs that you want to share?

No response

[coeuvre]

cc @hvadehra

[coeuvre]

it also fails with local JDK11.

[fmeum]

I have a fix I can submit shortly, I'm just trying to understand why the existing tests didn't cover this.

[coeuvre]

The rbe toolchain we generated provides the wrong version, e.g.:

local_java_runtime(
    name = "rbe_jdk",
    java_home = "/usr/lib/jvm/java-11-openjdk-amd64",
    version = "11.0.19",
)

so the java_runtime cannot parse it and defaults to 0. I guess in that case, the rule doesn't try to add --release flag?

[hvadehra]

I did some code archaeology, and from what I can tell, the local_java_runtime was never meant to be a general public API. It was only intended to be used by the BUILD files generated by the local_java_repository repo rule (which is why it had to be publicly exported from the bzl file). The two implementations are somewhat tightly coupled.

Unfortunately, the definitions evolved over a series of changes, and certain API design choices were made (likely driven by the need for backwards compatibility) which led us to this situation. The intended use of local_java_runtime seems to haven been to always declare one's own java_runtime (more specifically, it must use JDK_BUILD_TEMPLATE) and pass that as the runtime_name parameter to local_java_runtime.

I think it might be preferable if we updated https://github.com/bazelbuild/bazel-toolchains/blob/master/pkg/rbeconfigsgen/rbeconfigsgen.go to use the local_java_repository repo rule. @coeuvre Do you think that would be possible?

[hvadehra]

Had a discussion with Chi offline. The situation is trickier than I had hoped. The runtime in this case is naturally non-hermetic, but what's worse is it's not local and doesn't exist where Bazel is executing, so globing the sources is a no-go. #21610 will have to be our way forward for now.