Unrecoverable error when referencing external repository

[ilinam]

Hi there,

I've set up a local_repository in our WORKSPACE file, and would like to leverage bind to prevent having to change the BUILD files.

From the documentation, it seems I should be able to do something like the following:

bind(
name = "grpc",
actual = "@grpc//:grpc",
)

local_repository(
name = "grpc",
path = "third_party/google/grpc",
)

And the BUILD dependency should become: "//external:grpc" whereas it was previously "//third_party/google/grpc:gprc".

However, if I try to do this, I hit the following exception:

java.lang.RuntimeException: Unrecoverable error while evaluating node 'CONFIGURED_TARGET://lb:ip-forwarder-main eeab51cc7d46801c1a05a5f646430553 (936318221 375452941)' (requested by nodes 'CONFIGURED_TARGET://lb/testing:ip_forwarder_test eeab51cc7d46801c1a05a5f646430553 (1211325356 375452941)')
    at com.google.devtools.build.skyframe.ParallelEvaluator$Evaluate.run(ParallelEvaluator.java:438)
    at com.google.devtools.build.lib.concurrent.AbstractQueueVisitor$WrappedRunnable.run(AbstractQueueVisitor.java:501)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.NullPointerException
    at com.google.devtools.build.lib.packages.NonconfigurableAttributeMapper.get(NonconfigurableAttributeMapper.java:45)
    at com.google.devtools.build.lib.bazel.rules.BazelRuleClassProvider$BazelPrerequisiteValidator.isTestOnlyRule(BazelRuleClassProvider.java:266)
    at com.google.devtools.build.lib.bazel.rules.BazelRuleClassProvider$BazelPrerequisiteValidator.validateDirectPrerequisiteForTestOnly(BazelRuleClassProvider.java:252)
    at com.google.devtools.build.lib.bazel.rules.BazelRuleClassProvider$BazelPrerequisiteValidator.validate(BazelRuleClassProvider.java:196)
    at com.google.devtools.build.lib.analysis.RuleContext$Builder.validateDirectPrerequisite(RuleContext.java:1954)
    at com.google.devtools.build.lib.analysis.RuleContext$Builder.createTargetMap(RuleContext.java:1626)
    at com.google.devtools.build.lib.analysis.RuleContext$Builder.build(RuleContext.java:1465)
    at com.google.devtools.build.lib.analysis.ConfiguredTargetFactory.createRule(ConfiguredTargetFactory.java:245)
    at com.google.devtools.build.lib.analysis.ConfiguredTargetFactory.createConfiguredTarget(ConfiguredTargetFactory.java:176)
    at com.google.devtools.build.lib.skyframe.SkyframeBuildView.createConfiguredTarget(SkyframeBuildView.java:491)
    at com.google.devtools.build.lib.skyframe.ConfiguredTargetFunction.createConfiguredTarget(ConfiguredTargetFunction.java:1094)
    at com.google.devtools.build.lib.skyframe.ConfiguredTargetFunction.compute(ConfiguredTargetFunction.java:251)
    at com.google.devtools.build.skyframe.ParallelEvaluator$Evaluate.run(ParallelEvaluator.java:373)
    ... 4 more

Environment info

• Operating System: Ubuntu 14.04

• Bazel version (output of bazel info release): up to git commit ID 54223b0

[kchodorow]

You cannot use the same name twice (this should work if you call the local repository grpc_repo or something) but generally we advise not using bind() anyway.

This is bug, though, in that Bazel shouldn't throw an exception.

[Helcaraxan]

I'm also encountering this issue currently... surprisingly also when using gRPC as a dependency. Might this be because of some things they do internally in their BUILD files? I'm currently upgrading from gRPC 1.0.1 to 1.1.4 and I did not have this issue up to now.

The most troubling thing with this is that it looks impossible to find out what the offending twice-used name is (none of my own's project targets conflict with the gRPC ones). @kchodorow is there a, potentially hackyish, way to find that out?

Also on a secondary note I don't really understand your point stating that:

but generally we advise not using bind() anyway.

Could you be more explicit as to why you discourage the use of bind? As far as I can see bind is very often required to fix up naming differences of the targets of the same second-degree dependency between two different first-degree dependencies. Or with an example:

• Your project X lists projects A, B and C as dependencies in your WORKSPACE
• Both A and B depend both on C (for example an openssl library: boringssl or other) but they refer to the main target of C by two different names (for example //external:ssl and //external:libssl).

This looks to me like the perfect situation for me to use bind.

I do understand that you could solve that by copying the BUILD files of both A and B in to your project X, switch to a new_git_repository or new_http_archive rule and just modify the names of the dependencies to be the same. However with a large scale project this is just not viable. This would imply that I need to maintain 10+ locally copied BUILD files of external dependencies that are all modified in slightly different and subtle ways. Regularly upgrading these external dependencies would just become a nightmare with the need to each time go and fetch the external BUILD file and then parse it manually to find the right places where to rename dependencies.

NB: This is not meant to be a "rant" but I do would like to know if I am maybe missing an important design pattern somewhere that would take all my worries away. 🙂

[kchodorow]

The most troubling thing with this is that it looks impossible to find out what the offending twice-used name is (none of my own's project targets conflict with the gRPC ones). @kchodorow is there a, potentially hackyish, way to find that out?

It should be a dependency of the target named in the error. E.g., above, the error message says Unrecoverable error while evaluating node 'CONFIGURED_TARGET://lb:ip-forwarder-main, so you'd look at //lb:ip-forwarder-main's deps. Does that help narrow it down?

Could you be more explicit as to why you discourage the use of bind? As far as I can see bind is very often required to fix up naming differences of the targets of the same second-degree dependency between two different first-degree dependencies.

There is a fairly long discussion on #1952.

[Helcaraxan]

Thank you very much for the quick answer @kchodorow! I was able to trace it back that way. It was an issue that I had not spotted earlier on with a third party dependency.

Also thanks for the link to #1952. I do understand better now that there is a growing consensus behind the deprecation of bind() but it would indeed, as noted in the discussion, require a great number of existing Bazel users to change their approach to external dependency referencing.

[dslomov]

Dupilcate of #3676