Windows: break the dependency on Bash #4319

laszlocsomor · 2017-12-18T14:21:24Z

Bazel should not require the presence of Bash, nor the presence of the BAZEL_SH envvar.

Bash should be just another tool(chain) like Python.

The text was updated successfully, but these errors were encountered:

laszlocsomor · 2018-04-03T12:58:19Z

Design doc: https://docs.google.com/document/d/e/2PACX-1vRCSB_n3vctL6bKiPkIa_RN_ybzoAccSe0ic8mxdFNZGNBJ3QGhcKjsL7YKf-ngVyjRZwCmhi_5KhcX/pub

Bazel automatically detects the local Bash and creates a custom toolchain rule for it. Later, rules that use Bash will require this toolchain and retrieve Bash's path from it instead of relying on hardcoded paths or the --shell_executable flag. See bazelbuild#4319 Change-Id: Idd8242a20d202b1f5a56cddac95b625c6c08ede9

Bazel automatically detects the local Bash and creates a custom toolchain rule for it. Later, rules that use Bash will require this toolchain and retrieve Bash's path from it instead of relying on hardcoded paths or the `--shell_executable` flag. See #4319 Change-Id: Idd8242a20d202b1f5a56cddac95b625c6c08ede9 Closes #4980. Change-Id: Ic2406a4da260b284e15852070d58472ca18340af PiperOrigin-RevId: 193022708

SpawnAction.setShellCommand(String) now expects the shell interpreter's path as an argument. This change enables two things: - rules can report an error if the shell is missing - SpawnAction no longer has to know about a default shell The new ShToolchain class will later also be responsible for retrieving the active shell toolchain (added in 81ed3ad). This change brings Bazel closer to not depend on the shell unless it has to (e.g. to run shell scripts). See #4319 RELNOTES: none PiperOrigin-RevId: 193885943

Fix error reporting in the path conversion methods of the Bazel client. Previously the error reporting logic used GetLastErrorString in places where it was not appropriate (i.e. it was not a failed Windows API call that caused an error). This cleanup prepares removing the concept of the MSYS root from the Bazel client, since MSYS paths are no longer supported and we want to cut Bazel's dependency on Bash (thus MSYS) completely. See bazelbuild#4319 Change-Id: Ie50a20e0ee0c572592f637340a2f2948c7f53088

Fix error reporting in the path conversion methods of the Bazel client. Previously the error reporting logic used GetLastErrorString in places where it was not appropriate (i.e. it was not a failed Windows API call that caused an error). This cleanup prepares removing the concept of the MSYS root from the Bazel client, since MSYS paths are no longer supported and we want to cut Bazel's dependency on Bash (thus MSYS) completely. See #4319 Change-Id: Ie50a20e0ee0c572592f637340a2f2948c7f53088 Closes #5072. Change-Id: Ie50a20e0ee0c572592f637340a2f2948c7f53088 PiperOrigin-RevId: 194052665

The Bazel client no longer supports MSYS paths. The only exception is "/dev/null" which the client treats as "NUL". After this change you can no longer pass MSYS paths as Bazel flag values on Windows. See bazelbuild#4319 Change-Id: I39d81843015c5a4014dd5953bac2e1c29dcd5bed

The Bazel client no longer supports MSYS paths. The only exception is "/dev/null" which the client treats as "NUL". After this change you can no longer pass MSYS paths as Bazel flag values on Windows. See #4319 Change-Id: I39d81843015c5a4014dd5953bac2e1c29dcd5bed PiperOrigin-RevId: 194372504

This new test rule compares two files and passes if the files match. On Linux/macOS/non-Windows, the test compares files using 'diff'. On Windows, the test compares files using 'fc.exe'. This utility is available on all Windows versions I tried (Windows 2008 Server, Windows 2016 Datacenter Core). See bazelbuild/bazel#5508 See bazelbuild/bazel#4319

TestActionBuilder now only looks up the shell (via ShToolchain.getPathOrError) when the shell is required. Now, when using the Windows-native test wrapper with --shell_toolchain="" (and without a shell-command-looking --run_under argument) the TestActionBuilder won't depend on Bash. Related: bazelbuild#4319

test_wrapper_test asserts that Bazel can run tests with the Windows-native test wrapper, i.e. without Bash. Yet test_wrapper_test itself required Bash for its mock sh_test rules. This PR replaces those sh_test rules with simple Starlark test rules. Related: bazelbuild#4319

test_wrapper_test asserts that Bazel can run tests with the Windows-native test wrapper, i.e. without Bash. Yet test_wrapper_test itself required Bash for its mock sh_test rules. This PR replaces those sh_test rules with simple Starlark test rules. Related: #4319 Closes #8165. Change-Id: I6a3d6641d0998f5a7f02def61902e3fb8187e0c9 PiperOrigin-RevId: 245688126

TestActionBuilder now only looks up the shell (via ShToolchain.getPathOrError) when the shell is required. Now, when using the Windows-native test wrapper with --shell_toolchain="" (and without a shell-command-looking --run_under argument) the TestActionBuilder won't depend on Bash. Related: bazelbuild#4319

TestActionBuilder now only looks up the shell (via ShToolchain.getPathOrError) when the shell is required. Now, when using the Windows-native test wrapper with --shell_toolchain="" (and without a shell-command-looking --run_under argument) the TestActionBuilder won't depend on Bash. Related: #4319 Closes #8161. PiperOrigin-RevId: 246275913

laszlocsomor · 2019-07-29T13:36:22Z

Status:

in danger for 1.0 : depends on incompatible_windows_native_test_wrapper: enables Bash-less test execution on Windows #6622 which is in danger (incompatible_windows_native_test_wrapper: enables Bash-less test execution on Windows #6622 (comment))
Windows: "bazel run" requires Bash [blocking #4319] #8229 won't be ready for Bazel 1.0

laszlocsomor · 2019-10-17T15:21:00Z

Bazel 1.0 is released, and both #6622 and #8229 made it into the release.

The CommandHelper no longer looks up the shell interpreter's path itself. Instead its ctor takes the path as an argument. This change will allow incrementally migrating rules that use CommandHelper to start depending on the shell toolchain. Shell-using rules today only use the ShellConfiguration config fragment to look up the shell's path. In the future, more and more rules will also retrieve the active shell toolchain, and be able to: 1. use the auto-detected local shell interpreter's location, especially when it's not the default /bin/bash 2. use define different shell toolchains (different interpreter paths) for remote builds 3. gracefully fail the build if the machine has no shell installed but the action graph included a shell action See bazelbuild/bazel#4319 Change-Id: I4da4e77e7d1fe57e8e4f5eb8820d03a840915e20 Closes #5283. Change-Id: I4da4e77e7d1fe57e8e4f5eb8820d03a840915e20 PiperOrigin-RevId: 198298315

SpawnAction.setShellCommand(String) now expects the shell interpreter's path as an argument. This change enables two things: - rules can report an error if the shell is missing - SpawnAction no longer has to know about a default shell The new ShToolchain class will later also be responsible for retrieving the active shell toolchain (added in bazelbuild/bazel@81ed3ad). This change brings Bazel closer to not depend on the shell unless it has to (e.g. to run shell scripts). See bazelbuild/bazel#4319 RELNOTES: none PiperOrigin-RevId: 193885943

maprule() is an improved version of native.genrule(), with the following advantages: - Maprule can process source files in parallel, creating separate actions for each of them. - Maprule does not require declaring all output files. Instead you declare templates for the output files yielded for each source. Therefore N source files and M templates yield N*M outputs. - Maprule supports both Bash and cmd.exe syntax for its commands via the specialized rules bash_maprule and cmd_maprule. - Maprule's cmd attribute does deliberately not support $(location) expression nor Make Variables, in order to avoid issues and challenges with quoting. (In case of cmd.exe passing empty arguments is impossible). These paths can be passed as envvars instead. - Maprule's add_env attribute does support $(location) expressions (and some extra placeholders) and is the idiomatic way to pass execpaths of labels in "tools" or "srcs" (the shared sources available for all actions) to the command. See bazelbuild/bazel#4319

This PR adds two new rules: copy_file and copy_xfile. Both rules solve a common problem: to copy one file to another location. The problem is routinely solved using a genrule. That however requires Bash, since genrules execute Bash commands. Requiring Bash is a problem on Windows. The new rules do not require Bash on Windows (only on other platforms). The only difference between the rules is that copy_xfile creates an executable file while copy_file doesn't. See bazelbuild/bazel#4319

This PR adds two new rules: write_file and write_xfile. Both rules solve a common problem: to write a text file with user-defined contents. The problem is routinely solved using a genrule. That however requires Bash, since genrules execute Bash commands. Requiring Bash is a problem on Windows. The new rules do not require any shell. The only difference between the rules is that write_xfile creates an executable file while write_file doesn't. See bazelbuild/bazel#4319

This new test rule compares two files and passes if the files match. On Linux/macOS/non-Windows, the test compares files using 'diff'. On Windows, the test compares files using 'fc.exe'. This utility is available on all Windows versions I tried (Windows 2008 Server, Windows 2016 Datacenter Core). See bazelbuild/bazel#5508 See bazelbuild/bazel#4319

laszlocsomor self-assigned this Dec 18, 2017

laszlocsomor added platform: windows P1 I'll work on this now. (Assignee required) type: feature request windows-q1-2018-maybe labels Dec 18, 2017

buchgr added the category: misc > misc label Mar 21, 2018

laszlocsomor mentioned this issue Mar 28, 2018

Cut all dependencies on bash [duplicate of #4319] #931

Closed

laszlocsomor mentioned this issue Apr 9, 2018

bash: Add a toolchain for local Bash. #4980

Closed

This was referenced Apr 17, 2018

validate --workspace_status_command's value #5034

Closed

cannot pass script/binary in root of repository to --workspace_status_command with bazel 0.12 #5002

Closed

laszlocsomor mentioned this issue Apr 23, 2018

windows,client: fix error reporting #5072

Closed

laszlocsomor mentioned this issue May 3, 2018

Docker build rule with deb packages tries to use xzcat which might not exist #611

Closed

aherrmann mentioned this issue Apr 24, 2019

Windows: BAZEL_SH breaks remote caching #8131

Closed

laszlocsomor mentioned this issue Apr 26, 2019

Windows, testing: only look up shell if needed #8161

Closed

laszlocsomor mentioned this issue Apr 26, 2019

Windows, testing: replace sh_tests with Starlark #8165

Closed

laszlocsomor mentioned this issue May 3, 2019

Windows: "bazel run" requires Bash [blocking #4319] #8229

Closed

meteorcloudy mentioned this issue May 14, 2019

incompatible_use_native_patch: Use native patch in http_archive and git_repository rules #8316

Closed

laszlocsomor removed bazel 1.0 platform: windows labels Jul 29, 2019

laszlocsomor closed this as completed Oct 17, 2019

philwo added the team-OSS Issues for the Bazel OSS team: installation, release processBazel packaging, website label Jun 15, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Windows: break the dependency on Bash #4319

Windows: break the dependency on Bash #4319

laszlocsomor commented Dec 18, 2017

laszlocsomor commented Apr 3, 2018

laszlocsomor commented Jul 29, 2019

laszlocsomor commented Oct 17, 2019

Windows: break the dependency on Bash #4319

Windows: break the dependency on Bash #4319

Comments

laszlocsomor commented Dec 18, 2017

laszlocsomor commented Apr 3, 2018

laszlocsomor commented Jul 29, 2019

laszlocsomor commented Oct 17, 2019