distinct host configuration doesn't work for Java targets anymore

[ianoc-stripe]

Description of the problem / feature request:

Since the 0.16.0rc's it looks like the distinct_host_configuration=false flag is broken

Bugs: what's the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.

Enable the flag and no java targets will compile

What operating system are you running Bazel on?

Both os x and linux fail

What's the output of bazel info release?

0.16.0rc3

Any other information, logs, or outputs that you want to share?

ERROR: /Users/ianoc/github/rules_scala/src/java/io/bazel/rulesscala/jar/BUILD:1:1: Building src/java/io/bazel/rulesscala/jar/libjar.jar (2 source files) failed (Exit 1): java failed: error executing command
  (cd /private/var/tmp/_bazel_ianoc/1c83cb230054428e22566b8d0a2fa63a/execroot/io_bazel_rules_scala && \
  exec env - \
    LC_CTYPE=en_US.UTF-8 \
  external/local_jdk/bin/java '--add-exports=jdk.compiler/com.sun.tools.javac.api=ALL-UNNAMED' '--add-exports=jdk.compiler/com.sun.tools.javac.code=ALL-UNNAMED' '--add-exports=jdk.compiler/com.sun.tools.javac.comp=ALL-UNNAMED' '--add-exports=jdk.compiler/com.sun.tools.javac.file=ALL-UNNAMED' '--add-exports=jdk.compiler/com.sun.tools.javac.main=ALL-UNNAMED' '--add-exports=jdk.compiler/com.sun.tools.javac.tree=ALL-UNNAMED' '--add-exports=jdk.compiler/com.sun.tools.javac.util=ALL-UNNAMED' '--add-opens=jdk.compiler/com.sun.tools.javac.file=ALL-UNNAMED' '--patch-module=java.compiler=external/bazel_tools/third_party/java/jdk/langtools/java_compiler.jar' '--patch-module=jdk.compiler=external/bazel_tools/third_party/java/jdk/langtools/jdk_compiler.jar' '--add-opens=java.base/java.nio=ALL-UNNAMED' -jar external/bazel_tools/tools/jdk/JavaBuilder_deploy.jar @bazel-out/darwin-fastbuild/bin/src/java/io/bazel/rulesscala/jar/libjar.jar-2.params)
Unrecognized option: --add-exports=jdk.compiler/com.sun.tools.javac.api=ALL-UNNAMED
Error: Could not create the Java Virtual Machine.
Error: A fatal exception has occurred. Program will exit.

Per platform CI runs failing on travis via PR @ bazelbuild/rules_scala#556

[ianoc-stripe]

On this PR there are even errors in the task before i enabled this flag, so there seems to be wider issues here with the JDK change

[jin]

looks like a configurability issue, cc @gregestren any ideas?

[gregestren]

I can reproduce this. Will try to diagnose cause tomorrow.

Thanks for the report.

[gregestren]

When I repro with a trivial //java/com/domain/testapp:foo, my error is:

Exception in thread "main" java.lang.UnsupportedClassVersionError:
javax/lang/model/element/TypeElement has been compiled by a more recent version of
Java Runtime (class file version 53.0), this version of the Java Runtime only recognizes
the class file versions up to 52.0

TypeElement comes from:

$ jar tf bazel-out/k8-fastbuild/bin/java/com/domain/testapp/foo.runfiles/
  local_jdk/jre/lib/rt.jar | grep TypeElement
javax/lang/model/element/TypeElement.class

Curiously, there's no work being done in the host configuration for this build. However, when I run

$ bazel build //java/com/domain/testapp:foo -s
external/embedded_jdk/bin/java ...

vs.:

$ bazel build //java/com/domain/testapp:foo --nodistinct_host_configuration -s
external/local_jdk/bin/java ...

you see it's choosing a different jdk root.

So Bazel is trying to build with the system's local JDK for --nodistinct_host_configuration instead of the JDK it embeds.

This looks related to 849df36, which creates a distinction between the local vs. embedded JDK for different modes of building. One of --nodistinct_host_configuration's implications is that it flattens these distinctions.

Referring to @cushon, who should have more JDK-specific insight about the implications.

[gregestren]

As a side note, do you actually need --distinct_host_configuration=false? That's an obscure setting that often doesn't actually do much good.

[ulfjack]

The referenced change 849df36 is already in 0.15, so either there's already a regression in 0.15, or that's not the one.

[cushon]

This is the outcome of the default RHS --host_javabase being JDK 9, and the default --javabase being JDK 8.

Using a JDK 8 RHS --host_javabase should work around it, but that configuration won't be supported indefinitely.

Using a JDK 9 --javabase would also resolve this, but that isn't supported yet.

[ianoc-stripe]

distinct_host_configuration i've used a few times with mixed success, we mostly have pure jvm repos, and some usage of scripts or other oddities has caused us to sometimes to build for both host and target which is a waste. So this was good to just enforce we don't do it twice. That said i don't need this option -- but if its broken we probably need to flag it as a breaking change/going away?

The first failing commit on that PR branch happened before enabling the distinct host configuration too -- https://travis-ci.org/bazelbuild/rules_scala/jobs/402681221

[cushon]

build for both host and target which is a waste

I agree this is unfortunate. I think maybe Bazel team has some ideas about a more focused solution to this, but I'm not sure what the current status is.

[dslomov]

Please do not assign issues to more than one team.

[bhack]

Any news on this?

[gregestren]

I don't think --distinct_host_configuration is the most commonly used flag, hence this not being a higher priority. Do you need that functionality? What's your use case?

[bhack]

We are still using this in TF

[cushon]

Did this regress recently for you? I think the issue that was originally reported was a different between JDK 8 and 9 in the host/target config, are you seeing something similar but with different Java versions?

Also note that tensorflow/tensorflow#52598 is using --javabase, which is removed in the latest versions of Bazel: #12704

[bhack]

We had a commit from @comius at tensorflow/toolchains#23 for his comment at #13241 (comment)

But then we reverted with tensorflow/toolchains#25 as It was failing on MacOS.

So when It was renabled to speedup the MacOS build It failed again:
tensorflow/tensorflow@c58cda5

[cushon]

Closing this as --distinct_host_configuration is being removed: #17731