Bazel build, using Docker, failing on macOS Big Sur

[manikandan-ravikumar]

Description of the problem / feature request:

I am trying to compile TensorFlow Java (r2.4) with AVX2 instructions from source using Bazel (3.1.0). I am using Docker for the compilation. The build produces the following error,

[3,700 / 8,277] Compiling external/boringssl/src/crypto/fipsmodule/bcm.c [for host]; 0s local ... (2 actions, 1 running)
917
[4,067 / 8,281] Compiling external/com_google_protobuf/src/google/protobuf/descriptor.cc [for host]; 1s local ... (2 actions, 1 running)
918
[5,022 / 8,898] Compiling external/com_github_grpc_grpc/src/compiler/csharp_generator.cc [for host]; 1s local ... (2 actions, 1 running)
919
[5,444 / 9,104] Compiling tensorflow/core/kernels/cwise_op_select.cc; 71s local ... (2 actions, 1 running)
920
[5,554 / 9,104] Compiling external/llvm-project/mlir/lib/Transforms/SCCP.cpp; 3s local ... (2 actions, 1 running)
921
[5,754 / 9,104] Compiling tensorflow/core/kernels/slice_op.cc; 13s local ... (2 actions, 1 running)
922
[5,983 / 9,104] Compiling external/llvm-project/llvm/lib/Object/ELFObjectFile.cpp; 3s local ... (2 actions, 1 running)
923
[6,382 / 9,104] Compiling tensorflow/core/kernels/conv_ops.cc; 70s local ... (2 actions, 1 running)
924
[6,789 / 9,105] Compiling external/llvm-project/mlir/lib/Dialect/Linalg/Transforms/Vectorization.cpp; 4s local ... (2 actions, 1 running)
925
[7,277 / 9,106] Compiling external/llvm-project/llvm/lib/Transforms/Scalar/InferAddressSpaces.cpp; 3s local ... (2 actions, 1 running)
926
[7,734 / 9,106] Compiling tensorflow/core/kernels/bias_op.cc; 34s local ... (2 actions, 1 running)
927
[8,380 / 9,106] Compiling external/llvm-project/llvm/lib/Target/TargetLoweringObjectFile.cpp; 0s local ... (2 actions, 1 running)
928
[8,917 / 9,106] Compiling tensorflow/compiler/tf2xla/kernels/mirror_pad_op.cc; 5s local ... (2 actions, 1 running)
929
ERROR: /tensorflow/tensorflow/java/BUILD:73:1: Building tensorflow/java/libprocessor_library-class.jar (1 source file) failed (Exit 1)
930
warning: -parameters is not supported for target value 1.7. Use 1.8 or later.
931
error: warnings found and -Werror specified
932
INFO: Elapsed time: 12050.681s, Critical Path: 113.11s
933
INFO: 8910 processes: 8910 local.
934
FAILED: Build did NOT complete successfully
935
FAILED: Build did NOT complete successfully

What operating system are you running Bazel on?

Docker on macOS Big Sur v11.4

What's the output of bazel info release?

ERROR: The 'info' command is only supported from within a workspace (below a directory having a WORKSPACE file).
See documentation at https://docs.bazel.build/versions/master/build-ref.html#workspace

Output of Bazel version

Build label: 3.1.0
Build target: bazel-out/k8-opt/bin/src/main/java/com/google/devtools/build/lib/bazel/BazelServer_deploy.jar

JDK Installation

RUN apt-get update &&
apt-get install -y openjdk-11-jdk-headless &&
apt-get clean;

# Fix certificate issues
RUN apt-get update && \
    apt-get install ca-certificates-java && \
    apt-get clean && \
    update-ca-certificates -f;

#setup java env variable
ENV JAVA_HOME /usr/lib/jvm/java-11-openjdk-amd64
ENV PATH $JAVA_HOME/bin:$PATH

Build command I used

git clone https://github.com/tensorflow/tensorflow.git \
  && cd tensorflow \
  && git checkout r$TENSORFLOW_VERSION \
  && bazel clean \
  && ./configure \
  && bazel build --config=opt \
      --define=no_tensorflow_py_deps=true \
      --define=ABSOLUTE_JAVABASE=/usr/lib/jvm/java-11-openjdk-amd64 \
      --javabase=@bazel_tools//tools/jdk:absolute_javabase \
      --host_javabase=@bazel_tools//tools/jdk:absolute_javabase \
      --java_toolchain=@bazel_tools//tools/jdk:toolchain_java11 \
      --host_java_toolchain=@bazel_tools//tools/jdk:toolchain_java11 \
      //tensorflow/java:tensorflow \
      //tensorflow/java:libtensorflow_jni

Have you found anything relevant by searching the web?

I found 19770 but I am using OpenJDK. I am not sure if Bazel is compatible with OpenJDK 11. If there are other suggestions, please share.

My project is using TensorFlow 2.4, so I can't try other versions.
Please help

[manikandan-ravikumar]

@meteorcloudy Any help on this?

[meteorcloudy]

There is a warning in your build and -Werror was specified, which tells the compiler to treat warning as error.
Perhaps you can try to figure out where the -Werror flag comes from and remove it?

I'll close this one since it's not a Bazel issue, it makes more sense to file this to TensorFlow.

[manikandan-ravikumar]

@meteorcloudy Thanks for the response. But the warning/error message is not specific or descriptive enough I can see the followings warnings in the GitHub action console,

1. WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
   I am not sure if this is the warning that causes the build to fail. This happened because of the following command
   pip3 install numpy wheel
2. WARNING: Download from https://mirror.bazel.build/github.com/aws/aws-sdk-cpp/archive/1.7.336.tar.gz failed: class
   The Bazel issue history suggests that this could be due to VPN or unstable internet. But I keep receiving this every time I trigger the tensor flow build. Moreover, when I open the link I get the following error
   <Error> <Code>NoSuchKey</Code> <Message>The specified key does not exist.</Message> </Error>
   So, am not sure if this is causing the build to fail.
3. warning: -parameters is not supported for target value 1.7. Use 1.8 or later
   I can see a long discussion about it here but I don't see a possible solution. also am not sure if this is causing my build to fail.
   Your thoughts on this?

[meteorcloudy]

warning: -parameters is not supported for target value 1.7. Use 1.8 or later

This is the one I meant, it comes from the javac compiler.

You can probably fix this by removing this line:
https://github.com/tensorflow/tensorflow/blob/5dcfc51118817f27fad5246812d83e5dccdc5f72/tensorflow/java/build_defs.bzl#L15

[manikandan-ravikumar]

Thank you @meteorcloudy