-
Notifications
You must be signed in to change notification settings - Fork 15.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
add a key to ctx.action dict to prevent protoc losing the default env #2508
add a key to ctx.action dict to prevent protoc losing the default env #2508
Conversation
Thanks for your pull request. The automated tests will run as soon as one of the admins verifies this change is ok for us to run on our infrastructure. |
1 similar comment
Thanks for your pull request. The automated tests will run as soon as one of the admins verifies this change is ok for us to run on our infrastructure. |
Can one of the admins verify this patch? |
Thanks for your pull request. It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA). 📝 Please visit https://cla.developers.google.com/ to sign. Once you've signed, please reply here (e.g.
|
I signed it! |
CLAs look good, thanks! |
This really solves a problem compiling protobuf with tensorflow with environment variables. |
+1. Can we get a merge on this? This is very useful when building with non-default toolchains and setting LD_LIBRARY_PATH. |
@cgrushko Can you help review this PR? |
@cgrushko The change seems good to me, but given that default value of that flag is False, I'm wondering if there are concerns about making it True. Maybe bazel team has other recommended solutions for overriding CC/LD_LIBRARY_PATH? |
@xfxyjwf @cgrushko Thanks for LGTM and the comment. I guess Bazel team made that flag False as default just because Bazel usually will let the user define a customized compiler toolchain in a crosstool file, so it means Bazel could setup an internal version of customized environment and always pass them to the binary command right before it (it doesn't export it to create global side effects). When this is true, then it will certainly not use the default shell 's environment. However, the case here is: |
@yliu120 is correctly interpreting the use of this attribute - it's intended to allow the user to override. Bazel accepts @xfxyjwf Can we merge this? It's something we'd like for the Envoy Bazel build, to allow it to be used internally in Google where we point |
Update to master branch, which includes a fix for cc_proto_library() to make it work in custom build environment (CC/CXX/LD_LIBRARY_PATH), see: protocolbuffers/protobuf#2508 Signed-off-by: Piotr Sikora <piotrsikora@google.com>
* Use protobuf from HEAD to ensure we have the protocolbuffers/protobuf#3327 fix for CI and local builds. Users can opt to use a specific release tag with the ENVOY_PROTOBUF_COMMIT env var. * Remove the workaround for protocolbuffers/protobuf#3327 in do_ci.sh. * Remove the multiple protobuf versions that existed because protocolbuffers/protobuf#2508 wasn't merged. * Add some evil symlink stuff to get @protobuf_bzl inferred from wherever WORKSPACE points the envoy_dependencies path at. As a bonus, enabled more verbose build of external deps so we don't sit around for minutes on initial checkout with no activity indicator. This can be done safely now as everyone should be at Bazel 0.5.2.
* Remove the workaround for protocolbuffers/protobuf#3327 in do_ci.sh. * Remove the multiple protobuf versions that existed because protocolbuffers/protobuf#2508 wasn't merged. * Add some evil symlink stuff to get @protobuf_bzl inferred from wherever WORKSPACE points the envoy_dependencies path at. As a bonus, enabled more verbose build of external deps so we don't sit around for minutes on initial checkout with no activity indicator. This can be done safely now as everyone should be at Bazel 0.5.2.
* build: undo some protobuf hacks, put some new ones in. * Remove the workaround for protocolbuffers/protobuf#3327 in do_ci.sh. * Remove the multiple protobuf versions that existed because protocolbuffers/protobuf#2508 wasn't merged. * Add some evil symlink stuff to get @protobuf_bzl inferred from wherever WORKSPACE points the envoy_dependencies path at. As a bonus, enabled more verbose build of external deps so we don't sit around for minutes on initial checkout with no activity indicator. This can be done safely now as everyone should be at Bazel 0.5.2. * Don't put verbose Bazel spew into query targets for coverage. * Always start from a clean test/coverage/BUILD, don't include it in the query.
Similar to protocolbuffers/protobuf#2508, when using protoc with non-standard LD_LIBRARY_PATH, we need to be able to have it pick up from the environment the correct LD_LIBRARY_PATH.
This is a small change on the protobuf.bzl but very reasonable and useful.
In practice, a few protobuf client (like tensorflow, see below and etc) would call
_proto_gen_impl(ctx)
indirectly to generate proto implementations. However by the current definition of bazel's ctx.action rule (http://www.bazel.io/versions/master/docs/skylark/lib/ctx.html#action), the default user's custom shell env (PATH/LD_LIBRARY_PATH) is not passed to the compiled protoc binary because the user_default_shell_env is set to beFalse
as default. This will invoke link error cuz users may link protoc with a customized library. There are a lot of reports on this issue.In principle, when calling
_proto_gen_impl(ctx)
to generate proto impls, we should pass user's env to the protoc binary.Another small change is to delete an extra space. (a bonus created by my vim editor).
Please see some issues related to this small CL:
tensorflow/tensorflow#3261
This change will not affect any other reasonable implementations.