Most actions ignore --action_env

[ulfjack]

I mostly want to file this so I can add todos to the code. 6e940e5 added a new abstraction for tracking the environment, and we need to update all actions (and all callers) to use the new abstraction, and use it consistently, i.e., not randomly drop env variables.

[ulfjack]

@laszlocsomor updated some of the Android actions to do this properly.

[lisendong]

is there any progress?

[ulfjack]

I fixed the C++ actions recently, but there are still a bunch missing.

[ulfjack]

I also fixed the symlink tree action.

[robinp]

I noticed that when compiling protobuf, the env vars specified by --action_env are missing. In my case, I need special ENV for gcc to pick up locations of some libs (here zlib).

Bazel version: 0.15.0 (from release).

Failure message:

external/com_google_protobuf/BUILD.bazel:116:1: C++ compilation of rule '@com_google_protobuf//:protobuf' failed (Exit 1)                                                                           
In file included from external/com_google_protobuf/src/google/protobuf/io/gzip_stream.cc:38:0:                                                                                      
external/com_google_protobuf/src/google/protobuf/io/gzip_stream.h:48:10: fatal error: zlib.h: No such file or directory                                                             
 #include <zlib.h>                               

Dependency chain:

bazel query 'somepath(kythe/cxx/extractor:cxx_extractor, @com_google_protobuf//:protobuf)' 
//kythe/cxx/extractor:cxx_extractor
//kythe/cxx/extractor:cmdlib
@com_google_protobuf//:protobuf

Is there a way to circumvent this? +@creachadair

Update: I found protocolbuffers/protobuf#2508 which sounds related, but you are likely better suited to sort it out between bazel and protobuf than me. Thank you.

[ulfjack]

@robinp Which version of bazel are you on? You need a bazel binary that includes the changes I made in May.

[robinp]

@ulfjack

Bazel version:

Build label: 0.15.2
Build target: bazel-out/k8-opt/bin/src/main/java/com/google/devtools/build/lib/bazel/BazelServer_deploy.jar
Build time: Tue Jul 17 12:22:40 2018 (1531830160)
Build timestamp: 1531830160
Build timestamp as int: 1531830160

I execute bazel build -s kythe/cxx/extractor/... --action_env NIX_CFLAGS_COMPILE.

For example, a good subcommand:

SUBCOMMAND: # @boringssl//:crypto [action 'Compiling external/boringssl/src/crypto/x509/x509_r2x.c']
(cd /home/ron/.cache/bazel/_bazel_ron/625e58b234afe92344d95f69120484a0/execroot/io_kythe && \
  exec env - \
    NIX_CFLAGS_COMPILE=' -isystem /nix/store/y8cfvcvya61l260jil989lcmkia5b5gh-zlib-1.2.11-dev/include -isystem /nix/store/y8cfvcvya61l260jil989lcmkia5b5gh-zlib-1.2.11-dev/include' \
    PATH=......

A failing subcommand:

SUBCOMMAND: # @com_google_protobuf//:protobuf [action 'Compiling external/com_google_protobuf/src/google/protobuf/io/gzip_stream.cc [for host]']
(cd /home/ron/.cache/bazel/_bazel_ron/625e58b234afe92344d95f69120484a0/execroot/io_kythe && \
  exec env - \
    PATH=......

I can observe that the failing subcommand doesn't have the NIX_CFLAGS_COMPILE propagated as I requested. The failure (zlib not found) matches this observation.

Thanks!

[robinp]

Ok, Bazel 0.16 seems to have most of my problems fixed. For the remaining:

Could you clarify how should action_env work for Skylark rules? For example, there's a Kythe rule which has

cc_extract_kindex = rule(
    attrs = {
      ...
      "extractor": attr.label(
            default = Label("//kythe/cxx/extractor:cxx_extractor"),
            executable = True,
            cfg = "host",
      ),
      ...

and using this rule seems to not use the action_env, so the extractor dependency is attempted to be compiled without the right environment.

[SimonBoorer]

I'm still having this issue with protobuf when trying to bootstrap Bazel 0.21.0 using:

export EXTRA_BAZEL_ARGS="--action_env=PATH --action_env=LD_LIBRARY_PATH"

ERROR: /user_data/bazel/third_party/protobuf/3.6.1/BUILD:123:1: C++ compilation of rule '//third_party/protobuf/3.6.1:protobuf_lite' failed (Exit 127): gcc failed: error executing command
  (cd /user_data/.tmp/bazel_xT9qaTfS/out/execroot/io_bazel && \
  exec env - \
    PATH=/bin:/usr/bin \
    PWD=/proc/self/cwd \
  /builds/gcc/4.9.4/12a8bec7dd/bin/gcc -U_FORTIFY_SOURCE -fstack-protector -Wall -Wunused-but-set-parameter -Wno-free-nonheap-object -fno-omit-frame-pointer -g0 -O2 '-D_FORTIFY_SOURCE=1' -DNDEBUG -ffunction-sections -fdata-sections '-std=c++0x' -MD -MF bazel-out/host/bin/third_party/protobuf/3.6.1/_objs/protobuf_lite/bytestream.d '-frandom-seed=bazel-out/host/bin/third_party/protobuf/3.6.1/_objs/protobuf_lite/bytestream.o' -iquote . -iquote bazel-out/host/genfiles -iquote bazel-out/host/bin -isystem third_party/protobuf/3.6.1/src -isystem bazel-out/host/genfiles/third_party/protobuf/3.6.1/src -isystem bazel-out/host/bin/third_party/protobuf/3.6.1/src -g0 -g0 -DHAVE_PTHREAD -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -Wno-unused-function -Wno-writable-strings -fno-canonical-system-headers -Wno-builtin-macro-redefined '-D__DATE__="redacted"' '-D__TIMESTAMP__="redacted"' '-D__TIME__="redacted"' -c third_party/protobuf/3.6.1/src/google/protobuf/stubs/bytestream.cc -o bazel-out/host/bin/third_party/protobuf/3.6.1/_objs/protobuf_lite/bytestream.o)
Execution platform: @bazel_tools//platforms:host_platform
/builds/gcc/4.9.4/12a8bec7dd/bin/gcc: line 19: exec: gcc_actual: not found

The only way I can get it to work is by setting --noincompatible_strict_action_env so that Bazel does not use a static PATH environment variable.