Bazel 0.21.0

[Profpatsch]

First commit is a patch created by @dtzWill to improve the substituteAll stuff.

Second commit is my progress of getting 0.21 to run so far. 0.21 removed the bundled openjdk-distribution. Instead, tries to fetch the “right” distribution on-the-fly when building. So we need to provide our own openjdk.

According to bazelbuild/bazel#6865 (comment) we should set --host_javabase if we want to do that.

We copy the buildJdk nix path to a temporary directory, make it look like a bazel repository (init BUILD and WORKSPACE) and point to it via local_repository.
As it turns out, exposing the files via filegroup is not enough, the bazel build needs it to expose a JavaRuntimeInfo provider:

ERROR: /tmp/.bazel-1000/bazel_HPQZBqno/out/external/bazel_tools/tools/jdk/BUILD:19:1:
in :alias attribute of java_host_runtime_alias rule
@bazel_tools//tools/jdk:current_host_java_runtime:
'@nix_jdk//:all-files' does not have mandatory providers:
'JavaRuntimeInfo'

I am unsure how to achieve that.

This commit also replaces the line-number-based sed invocations with
something more stable.

---

I actually tried to follow the upstream suggestion first and use --host_javabase='@local_jdk//:jdk', but that did not work at all:

ERROR: /build/src/java_tools/singlejar/java/com/google/devtools/build/singlejar/BUILD:82:1: error executing shell command: '/nix/store/l04m8mvir9ljd9dngnkxxgmw83wrxzs2-bash/bin/bash -c set -e;rm -rf bazel-out/host/bin/src/java_tools/singlejar/java/com/google/devtools/build/singlejar/bootstrap_deploy.jar.build_output;mkd...' failed (Exit 127): bash failed: error executing command
  (cd /tmp/.bazel-1000/bazel_18rkBpuk/out/execroot/io_bazel && \
  exec env - \
    PATH=/nix/store/can00lfiynqkbsdkkmgp6qg8p8w92cxa-bash-4.4-p23/bin:/nix/store/0q4i5ll9gxs6giq7kqkniww934j9j8dk-coreutils-8.30/bin:/nix/store/izhcvddpv3n052wp82zm5s8zcg514kr3-findutils-4.6.0/bin:/nix/store/zzzq8a9af192wfsi7lvf0mndpc8ykp4q-gnugrep-3.1/bin:/nix/store/4q0b6gz1yvb4bdzfcbyicz3vmlq05nxa-gnused-4.5/bin:/nix/store/c2czqxjf05amp81si5lvjxswx2bd7j6a-which-2.21/bin \
  /nix/store/l04m8mvir9ljd9dngnkxxgmw83wrxzs2-bash/bin/bash -c 'set -e;rm -rf bazel-out/host/bin/src/java_tools/singlejar/java/com/google/devtools/build/singlejar/bootstrap_deploy.jar.build_output;mkdir bazel-out/host/bin/src/java_tools/singlejar/java/com/google/devtools/build/singlejar/bootstrap_deploy.jar.build_output
unzip -qn bazel-out/host/bin/src/java_tools/singlejar/java/com/google/devtools/build/singlejar/libbootstrap.jar -d bazel-out/host/bin/src/java_tools/singlejar/java/com/google/devtools/build/singlejar/bootstrap_deploy.jar.build_output
unzip -qn bazel-out/host/bin/src/java_tools/singlejar/java/com/google/devtools/build/singlejar/libskylark-deps.jar -d bazel-out/host/bin/src/java_tools/singlejar/java/com/google/devtools/build/singlejar/bootstrap_deploy.jar.build_output
unzip -qn third_party/auto/auto-value-1.5.4.jar -d bazel-out/host/bin/src/java_tools/singlejar/java/com/google/devtools/build/singlejar/bootstrap_deploy.jar.build_output
unzip -qn third_party/error_prone/error_prone_annotations-2.2.0.jar -d bazel-out/host/bin/src/java_tools/singlejar/java/com/google/devtools/build/singlejar/bootstrap_deploy.jar.build_output
unzip -qn third_party/guava/guava-25.1-jre.jar -d bazel-out/host/bin/src/java_tools/singlejar/java/com/google/devtools/build/singlejar/bootstrap_deploy.jar.build_output
unzip -qn third_party/jcip_annotations/jcip-annotations-1.0-1.jar -d bazel-out/host/bin/src/java_tools/singlejar/java/com/google/devtools/build/singlejar/bootstrap_deploy.jar.build_output
unzip -qn third_party/jsr305/jsr-305.jar -d bazel-out/host/bin/src/java_tools/singlejar/java/com/google/devtools/build/singlejar/bootstrap_deploy.jar.build_output
unzip -qn bazel-out/host/bin/src/main/java/com/google/devtools/build/lib/shell/libshell-skylark.jar -d bazel-out/host/bin/src/java_tools/singlejar/java/com/google/devtools/build/singlejar/bootstrap_deploy.jar.build_output
external/local_jdk/bin/jar cmf bazel-out/host/bin/src/java_tools/singlejar/java/com/google/devtools/build/singlejar/bootstrap_MANIFEST.MF bazel-out/host/bin/src/java_tools/singlejar/java/com/google/devtools/build/singlejar/bootstrap_deploy.jar -C bazel-out/host/bin/src/java_tools/singlejar/java/com/google/devtools/build/singlejar/bootstrap_deploy.jar.build_output .
touch bazel-out/host/bin/src/java_tools/singlejar/java/com/google/devtools/build/singlejar/bootstrap_deploy.jar.build_output
')
Execution platform: @bazel_tools//platforms:host_platform

@local_jdk looks like a magic repository, from what I’ve seen it also has special support in bazel source code. Maybe it’s still the way to go.

@mboes @uri-canva @guibou, maybe you can find a way to get this to work.

[Profpatsch]

It was brought to my attention that there exists an undocumented built-in java_runtime rule that should provide the JavaRuntimeInfo provider and thus hopefully the last missing piece to get this working.

https://github.com/bazelbuild/bazel/blob/master/src/main/tools/jdk.BUILD#L166-L174

[Profpatsch]

Adding java_runtime does work, but has the same effect as just using @local_jdk//:jdk, leading back to the same build failure. The solution was to add unzip to defaultShellPath in addition to nativeBuildInputs.

@GrahamcOfBorg build bazel