stub_finds_runfiles_test failing on nightly

[katre]

Error:

home/ci/.cache/bazel/_bazel_ci/65ed5a6a0442d902ec235fee508231f8/bazel-sandbox/7343738837734816838/execroot/io_bazel/bazel-out/k8-fastbuild/bin/src/test/shell/integration/stub_finds_runfiles_test.runfiles/io_bazel/src/test/shell/bin
INFO[stub_finds_runfiles_test 2018-02-01 18:43:45 (+0000)] setting up client in /home/ci/.cache/bazel/_bazel_ci/65ed5a6a0442d902ec235fee508231f8/bazel-sandbox/7343738837734816838/execroot/io_bazel/_tmp/6c0c311f6b4ee77de13913fa4bb530f0/workspace
INFO[stub_finds_runfiles_test 2018-02-01 18:43:53 (+0000)] setting up client in /home/ci/.cache/bazel/_bazel_ci/65ed5a6a0442d902ec235fee508231f8/bazel-sandbox/7343738837734816838/execroot/io_bazel/_tmp/6c0c311f6b4ee77de13913fa4bb530f0/workspace
/home/ci/.cache/bazel/_bazel_ci/65ed5a6a0442d902ec235fee508231f8/bazel-sandbox/7343738837734816838/execroot/io_bazel/bazel-out/k8-fastbuild/bin/src/test/shell/integration/stub_finds_runfiles_test.runfiles/io_bazel/src/test/shell/bin/bazel: line 20: /io_bazel/src/bazel: No such file or directory
-- Test log: -----------------------------------------------------------
------------------------------------------------------------------------
FAILED: terminated because this command returned a non-zero status:
/home/ci/.cache/bazel/_bazel_ci/65ed5a6a0442d902ec235fee508231f8/bazel-sandbox/7343738837734816838/execroot/io_bazel/bazel-out/k8-fastbuild/bin/src/test/shell/integration/stub_finds_runfiles_test.runfiles/io_bazel/src/test/shell/integration/stub_finds_runfiles_test:100: in call to main

Failing in many different configurations, see here: https://ci.bazel.build/blue/organizations/jenkins/Global%2Fbazel-tests/detail/bazel-tests/482/tests

[philwo]

Umm.. why did you assign this to me? This test is horrible - doesn't redirect anything to $TEST_log and doesn't even encapsulate its code in a test_* function so that the test framework probably won't work at all. This is why we don't get any good log or stack trace from the test.

Can you find out who the original owner of this test is via the internal file history (public Git history only shows that it was basically not touched after open-sourcing some time in 2016) and assign it to them?

I think before we can get anything valuable out of the test we have to fix it, alternatively try to reproduce locally and if that succeeds git bisect it.

[katre]

Sorry, I was looking for people with insight into runfiles. I'll reassign to myself and start investigating as you suggest.

[philwo]

Oh, OK! Sorry, I don't know much about runfiles :(

Pinging @laszlocsomor just in case he has any clue what this might be about.

[katre]

I can reproduce this, but only by running the test with Bazel built from HEAD. So it's some kind of interaction in the test environment, I suspect.

First try at fixing will be to make this a real sh_test, I guess.

[laszlocsomor]

This is the source of the error:

bazel/src/test/shell/integration/stub_finds_runfiles_test.sh

Line 100 in b534e6b

bazel build //pkg:javabin

The error message says "/io_bazel/src/bazel: No such file or directory", so the test cannot find bazel. That's strange given that //src:bazel is a data-dependency of it:

bazel/src/test/shell/bazel/BUILD

Line 29 in b534e6b

"//src:bazel",

@katre : if you want to investigate, make sure the runfile is there ("io_bazel/src/bazel" is in bazel-bin/src/test/shell/integration/stub_finds_runfiles_test.runfiles_manifest and that bazel-bin/src/test/shell/integration/stub_finds_runfiles_test.runfiles/io_bazel/src/bazel exists).

[katre]

Yeah, looks like the "unset TEST_SRCDIR" might be the problem. I have a fix (basically just removes that), but I'm going to bisect this morning and figure out what change in Bazel actually causes this.

[katre]

Okay, here's the issue, in @laszlocsomor's change 8130287, in tools/test/test-setup.sh, where it changes to using TEST_SRCDIR in the implementation of rlocation. Since this test unsets TEST_SRCDIR, that leaves the path to bazel as "/io_bazel/src/bazel".

[katre]

Going ahead and importing https://bazel-review.googlesource.com/c/bazel/+/35850 to fix this.

[laszlocsomor]

Ouch. Thanks for figuring that out!

[katre]

Fixed in ea2d4c4

[katre]

Fixed in 0.11.0rc4.