Paths to Param Files in Command Lines Refer to Host Execroot Paths (instead of sandbox execroot paths) With --incompatible_sandbox_hermetic_tmp=true
#20515
Assignees
Labels
Comments
|
Thanks for the highly detailed report! I will look into this tomorrow. Cc @lberki |
iancha1992
added
the
team-Local-Exec
|
Woah, that was a very detailed report, thanks for investing the time and energy into it :) It helped me to zero in on the problem: I can reproduce this. It's due to this snippet: It doesn't recognize |
|
The good news is that this tiny patch: fixes the issue with the params files. The bad news is that it's still not working, but the deploy jar creation (which is the action that fails for me) now fails with a different error. I probably won't have time to debug this any further today. |
|
Reproduction: |
|
@fmeum I think you're right -- I didn't get even far enough to find out that the missing file is a symlink but it indeed is. |
|
BTW: I'm running it either this issue or a very similar one showing up when Every path starting with
|
|
@bazel-io flag |
bazel-io
added
the
potential release blocker
|
@bazel-io fork 7.1.0 |
bazel-io
removed
the
potential release blocker
|
@iancha1992 Forget to say that we should really have this target 7.0.1 - maybe not the full fix, but something that makes the default configuration work with it. |
bazel-io
pushed a commit
to bazel-io/bazel
that referenced
this issue
Jan 2, 2024
…sted themselves when the output base was under `/tmp`: 1. Virtual action inputs didn't work. This was a fairly simple omission in the path transformation logic. 2. Artifacts which are resolved symlinks (i.e. declared using `declare_file`) did not work. This is fixed by keeping track of the realpath of the symlink in `FileArtifactValue` / `TreeArtifactValue`. Fixes bazelbuild#20515 and bazelbuild#20518. RELNOTES: None. PiperOrigin-RevId: 595145191 Change-Id: I833025928374c78bc719d8e3be1efb2b2950b9f1
bazel-io
pushed a commit
to bazel-io/bazel
that referenced
this issue
Jan 2, 2024
…sted themselves when the output base was under `/tmp`: 1. Virtual action inputs didn't work. This was a fairly simple omission in the path transformation logic. 2. Artifacts which are resolved symlinks (i.e. declared using `declare_file`) did not work. This is fixed by keeping track of the realpath of the symlink in `FileArtifactValue` / `TreeArtifactValue`. Fixes bazelbuild#20515 and bazelbuild#20518. RELNOTES: None. PiperOrigin-RevId: 595145191 Change-Id: I833025928374c78bc719d8e3be1efb2b2950b9f1
iancha1992
pushed a commit
that referenced
this issue
Jan 3, 2024
…manifested themselves when the output base was under /tmp (#20718) 1. Virtual action inputs didn't work. This was a fairly simple omission in the path transformation logic. 2. Artifacts which are resolved symlinks (i.e. declared using `declare_file`) did not work. This is fixed by keeping track of the realpath of the symlink in `FileArtifactValue` / `TreeArtifactValue`. Fixes #20515 and #20518. RELNOTES: None. Commit fb6658c PiperOrigin-RevId: 595145191 Change-Id: I833025928374c78bc719d8e3be1efb2b2950b9f1 Co-authored-by: Googler <lberki@google.com>
bazel-io
pushed a commit
to bazel-io/bazel
that referenced
this issue
Jan 5, 2024
…sted themselves when the output base was under `/tmp`: 1. Virtual action inputs didn't work. This was a fairly simple omission in the path transformation logic. 2. Artifacts which are resolved symlinks (i.e. declared using `declare_file`) did not work. This is fixed by keeping track of the realpath of the symlink in `FileArtifactValue` / `TreeArtifactValue`. Fixes bazelbuild#20515 and bazelbuild#20518. RELNOTES: None. PiperOrigin-RevId: 595145191 Change-Id: I833025928374c78bc719d8e3be1efb2b2950b9f1
github-merge-queue bot
pushed a commit
that referenced
this issue
Jan 8, 2024
…manifested themselves when the output base was under /tmp (#20766) 1. Virtual action inputs didn't work. This was a fairly simple omission in the path transformation logic. 2. Artifacts which are resolved symlinks (i.e. declared using `declare_file`) did not work. This is fixed by keeping track of the realpath of the symlink in `FileArtifactValue` / `TreeArtifactValue`. Fixes #20515 and #20518. RELNOTES: None. Commit fb6658c PiperOrigin-RevId: 595145191 Change-Id: I833025928374c78bc719d8e3be1efb2b2950b9f1 Co-authored-by: Googler <lberki@google.com>
|
A fix for this issue has been included in Bazel 7.1.0 RC1. Please test out the release candidate and report any issues as soon as possible. Thanks! |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Issue
Building Bazel at HEAD with Bazel 7.0.0 (with
--incompatible_sandbox_hermetic_tmp=true, as is the default in Bazel 7) and an output base on/tmpyields the following error about a.paramsfile not being found:If
--incompatible_sandbox_hermetic_tmp=falseis passed, the above command succeeds.Details
Running with
--sandbox_debugproduces the following:Click to expand
Inspecting the mounts and their ordering above confirms that contents in Bazel source roots and execroot should be visible: these mounts are bind mounted into the hermetic tmp location (not
/tmpdirectly) before/tmpis pivoted to the hermetic tmp location (the finalbind mount:in the above).By altering the
linux-sandboxinvocation above we can confirm that thebazel-out/k8-opt/bin/src/main/java/com/google/devtools/build/lib/analysis/libanalysis_cluster-hjar.jar-0.paramsfile is a dangling symlink, within the sandbox:/tmp/bazel-rrbutani/install/89a68939cbf63eb54205fdf943a58b15/linux-sandbox \ -W /tmp/bazel-working-directory/_main \ -t 15 \ -w /tmp/bazel-execroot/_main \ -w /tmp \ -w /dev/shm \ -M /tmp/bazel-rrbutani/705c9771f27ae01ebf0993be3b99a716/execroot \ -m /tmp/bazel-rrbutani/705c9771f27ae01ebf0993be3b99a716/sandbox/linux-sandbox/3031/_hermetic_tmp/bazel-execroot \ -M /tmp/bazel-rrbutani/705c9771f27ae01ebf0993be3b99a716/sandbox/linux-sandbox/3031/execroot \ -m /tmp/bazel-rrbutani/705c9771f27ae01ebf0993be3b99a716/sandbox/linux-sandbox/3031/_hermetic_tmp/bazel-working-directory \ -M /tmp/bazel-rrbutani/705c9771f27ae01ebf0993be3b99a716/external/rules_java~7.3.1~toolchains~remote_java_tools_linux \ -m /tmp/bazel-rrbutani/705c9771f27ae01ebf0993be3b99a716/sandbox/linux-sandbox/3031/_hermetic_tmp/bazel-source-roots/0 \ -M /tmp/bazel-rrbutani/705c9771f27ae01ebf0993be3b99a716/external/rules_java~7.3.1~toolchains~remotejdk21_linux \ -m /tmp/bazel-rrbutani/705c9771f27ae01ebf0993be3b99a716/sandbox/linux-sandbox/3031/_hermetic_tmp/bazel-source-roots/1 \ -M /src/dev/bazel \ -m /tmp/bazel-rrbutani/705c9771f27ae01ebf0993be3b99a716/sandbox/linux-sandbox/3031/_hermetic_tmp/bazel-source-roots/2 \ -M /tmp/bazel-rrbutani/705c9771f27ae01ebf0993be3b99a716/sandbox/linux-sandbox/3031/_hermetic_tmp \ -m /tmp \ -S /tmp/bazel-rrbutani/705c9771f27ae01ebf0993be3b99a716/sandbox/linux-sandbox/3031/stats.out \ -D /dev/null \ -- ls --color -l bazel-out/k8-opt/bin/src/main/java/com/google/devtools/build/lib/analysis/libanalysis_cluster-hjar.jar-0.paramsInspecting
bazel-out/k8-opt/bin/src/main/java/com/google/devtools/build/lib/analysis/libanalysis_cluster-hjar.jar-0.paramswithin this action's execroot (/tmp/bazel-rrbutani/705c9771f27ae01ebf0993be3b99a716/sandbox/linux-sandbox/3031/execroot/) confirms that the file is a symlink that refers to the host's execroot path rather than that of the sandbox (i.e./tmp/bazel-execroot/):(note how the other files in the above are symlinks to
/tmp/bazel-execroot/)Cause
Poking around a bit, I think this is the relevant part of
rules_java:bazel/src/main/java/com/google/devtools/build/lib/rules/java/JavaHeaderCompileAction.java
Lines 486 to 489 in 379ee5f
Notably
rules_ccgenerated param files for link actions do not seem to have this issue (note the/tmp/bazel-execrootin the destinations):I think that's because
rules_ccgenerates its param files (for link actions) on its own using separate actions + files that are added as dependencies and doesn't use theCommandLines.Buildermachinery:bazel/src/main/java/com/google/devtools/build/lib/rules/cpp/CppLinkActionBuilder.java
Lines 950 to 971 in 0aacc74
I am not familiar with this part of the Bazel codebase but it seems likely the problematic bit (placing host-execroot paths for the params file onto the command line) lives here:
bazel/src/main/java/com/google/devtools/build/lib/actions/CommandLines.java
Lines 121 to 154 in 29f1db2
And here:
bazel/src/main/java/com/google/devtools/build/lib/analysis/actions/SpawnAction.java
Lines 348 to 360 in 41ca39a
(note the use of
getPrimaryOutput().getExecPath()forparamFileBasePath)The text was updated successfully, but these errors were encountered: