Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Windows, msys-less: support other shells (cygwin, powershell) #2725

Closed
laszlocsomor opened this issue Mar 22, 2017 · 20 comments
Closed

Windows, msys-less: support other shells (cygwin, powershell) #2725

laszlocsomor opened this issue Mar 22, 2017 · 20 comments
Assignees
Labels
P2 We'll consider working on this in future. (Assignee optional) platform: windows type: feature request
Milestone

Comments

@laszlocsomor
Copy link
Contributor

laszlocsomor commented Mar 22, 2017

Description of the problem / feature request / question:

The MSYS-less version should, by definition, work under other shells.

At c34320d it works under cmd.exe, Powershell 5.0.10586.672, MSYS2 20161025, and potentially other versions of these too, but these are the ones I tried.

It however doesn't work under Cygwin, especially if I do:

export BAZEL_SH=c:\\cygwin64\\bin\\bash.exe

Since the MSYS-less version should only require a valid BAZEL_SH for now (complete independence from MSYS is a bit further away), these scenarios should work.

@laszlocsomor laszlocsomor added platform: windows P3 We're not considering working on this, but happy to review a PR. (No assignee) type: feature request labels Mar 22, 2017
@laszlocsomor laszlocsomor self-assigned this Mar 22, 2017
@laszlocsomor laszlocsomor added this to the 0.6 milestone Mar 22, 2017
@laszlocsomor laszlocsomor added P2 We'll consider working on this in future. (Assignee optional) and removed P3 We're not considering working on this, but happy to review a PR. (No assignee) labels Mar 22, 2017
bazel-io pushed a commit that referenced this issue Mar 23, 2017
Came up while working on #2725

--
Change-Id: I923690642d0fc93dcdb5050837b5ddaaa2a1d494
Reviewed-on: https://cr.bazel.build/9469
PiperOrigin-RevId: 150880961
MOS_MIGRATED_REVID=150880961
bazel-io pushed a commit that referenced this issue Mar 23, 2017
See #2725
Related to #2447

--
Change-Id: I5cc7fba43ae46bc9ef47bcaf9efaf2a0b9911bc4
Reviewed-on: https://cr.bazel.build/9468
PiperOrigin-RevId: 150881196
MOS_MIGRATED_REVID=150881196
bazel-io pushed a commit that referenced this issue Mar 23, 2017
See #2725
Related to #2447

--
Change-Id: I723764ee1b41caf62a7d71abcdcdfb704521f206
Reviewed-on: https://cr.bazel.build/9511
PiperOrigin-RevId: 150882100
MOS_MIGRATED_REVID=150882100
bazel-io pushed a commit that referenced this issue Mar 23, 2017
Make it more robust: it now works with Cygwin too,
e.g. BAZEL_SH=c:/cygwin64/bin/bash.exe

See #2725
Related to #2447

--
Change-Id: I911f09acd3e39c7cd0fe0750774fa0a900ffd844
Reviewed-on: https://cr.bazel.build/9510
PiperOrigin-RevId: 150885982
MOS_MIGRATED_REVID=150885982
@laszlocsomor
Copy link
Contributor Author

Update: cygwin doesn't have zip installed and I haven't found it among the packages, and genrules need it (e.g. //src:embedded_tools). We should look for a BAZEL_ZIP envvar that specifies the path to a zip utility.

@laszlocsomor
Copy link
Contributor Author

...or better yet, have a hermetic zip utility in //third_party and use that.

@davido
Copy link
Contributor

davido commented Apr 11, 2017

Update: cygwin doesn't have zip installed

Just define zip to be a Cygwin Bazel pre-requisite then? Here what we are requiring for LibreOffice cygwin based build tool chain on Windows: [1].

setup-x86_64.exe -P autoconf -P automake -P bison -P cabextract -P doxygen -P flex -P gcc-g++ ^
                -P git -P gnupg -P gperf -P make -P mintty ^
                -P nasm -P openssh -P openssl -P patch -P perl -P python -P python3 ^
                -P pkg-config -P rsync -P unzip -P vim -P wget -P zip -P perl-Archive-Zip

@laszlocsomor
Copy link
Contributor Author

Neat! Thanks for the tip. I didn't find zip/unzip in the installer's package selector's search bar before, but the -P <package> approach works.

@laszlocsomor
Copy link
Contributor Author

Thanks again @davido , now it works like a charm -- I can now build the previously failing genrule.

We still cannot build //src:bazel.exe because //src/main/native:windows_jni fails but we're now further than before having zip installed.

laszlocsomor@laszlocsomor0-w /cygdrive/c/work/bazel
$ export PATH="/usr/local/bin:/usr/bin"  # to avoid accidentally using MSYS binaries

laszlocsomor@laszlocsomor0-w /cygdrive/c/work/bazel
$ export BAZEL_SH="c:/cygwin64/bin/bash.exe"

laszlocsomor@laszlocsomor0-w /cygdrive/c/work/bazel
$ export BAZEL_PYTHON=c:/python27/python.exe

laszlocsomor@laszlocsomor0-w /cygdrive/c/work/bazel
$ c:/work/bazel-msvc.exe --output_user_root=c:/tmp-cyg build --{host_,}copt=-w --{host_,}cpu=x64_windows_msvc --verbose_failures //src:embedded_tools
...
Target //src:embedded_tools up-to-date:
  C:/tmp-cyg/o6hbs7n0/execroot/bazel/bazel-out/msvc_x64-fastbuild/genfiles/src/embedded_tools.zip
____Elapsed time: 66.175s, Critical Path: 65.27s

@davido
Copy link
Contributor

davido commented Apr 11, 2017

Thanks again @davido , now it works like a charm -- I can now build the previously failing genrule.

@laszlocsomor Thank you for all your hard work to support Windows platform!

@laszlocsomor
Copy link
Contributor Author

Cool, it all works now.

Turns out I made PATH too bare.

From cygwin:

laszlocsomor@laszlocsomor0-w /cygdrive/c/work/bazel
$ env | grep -i 'msys'
BAZEL_SH=C:\tools\msys64\usr\bin\bash.exe

laszlocsomor@laszlocsomor0-w /cygdrive/c/work/bazel
$ export BAZEL_SH=c:\\cygwin64\\bin\\bash.exe

laszlocsomor@laszlocsomor0-w /cygdrive/c/work/bazel
$ env | grep -i 'msys'

laszlocsomor@laszlocsomor0-w /cygdrive/c/work/bazel
$ env | grep -i 'cygwin'
BAZEL_SH=c:\cygwin64\bin\bash.exe

laszlocsomor@laszlocsomor0-w /cygdrive/c/work/bazel
$ c:/work/bazel-msvc.exe --output_user_root=c:/tmp-cyg build --{host_,}copt=-w --{host_,}cpu=x64_windows_msvc --verbose_failures //src:bazel.exe
...
Target //src:bazel.exe up-to-date:
  C:/tmp-cyg/o6hbs7n0/execroot/bazel/bazel-out/msvc_x64-fastbuild/bin/src/bazel.exe
____Elapsed time: 290.273s, Critical Path: 100.20s

From cmd.exe:

c:\work\bazel>set | c:\Windows\System32\find.exe /I "msys"
BAZEL_SH=C:\tools\msys64\usr\bin\bash.exe

c:\work\bazel>set BAZEL_SH=c:\cygwin64\bin\bash.exe

c:\work\bazel>set | c:\Windows\System32\find.exe /I "msys"

c:\work\bazel>c:/work/bazel-msvc.exe --output_user_root=c:/tmp-cmd build --host_copt=-w --copt=-w --host_cpu=x64_windows_msvc --cpu=x64_windows_msvc --verbose_failures //src:bazel.exe
...
Target //src:bazel.exe up-to-date:
  C:/tmp-cmd/o6hbs7n0/execroot/bazel/bazel-out/msvc_x64-fastbuild/bin/src/bazel.exe
INFO: Elapsed time: 175.676s, Critical Path: 91.36s

@laszlocsomor
Copy link
Contributor Author

To be fair this bug isn't yet closed as I haven't tested with Windows bash (my work machine doesn't have it).

@davido Do you have Windows bash installed and would you be interested in trying it?

@laszlocsomor laszlocsomor reopened this Apr 11, 2017
@davido
Copy link
Contributor

davido commented Apr 11, 2017

@laszlocsomor What exactly do you mean by "Windows bash"? Windows Subsystem for Linux feature?

I thought this is tracked in different issue: #1608. Yes, i can reproduce it, see my last comment there. Yes, i have that installed, but I'm not sure, how could I try bazel from master, if bazel cannot be run there? Right now I installed Bazel released version from Ubuntu Bazel repository (0.4.5).

@laszlocsomor
Copy link
Contributor Author

@davido : Yes, that's what I meant, and thanks for referencing the bug, I wasn't aware of it.
I made the assumption that installing the Anniversary Update along with this component meant there would be an actual bash.exe installed somewhere whose path we could set to BAZEL_SH. Is that not how it works?

@davido
Copy link
Contributor

davido commented Apr 11, 2017

@laszlocsomor There are two tiny steps more. You enable developer feature on your machine and select "Windows Subsystem for Linux feature " from the Settings GUI (that I wasn't able to find ;-), so I was able to enable this feature in alternative way, per power shell command. Then you reboot, and voilà, you run bash.exe, and Ubuntu Trusty is installed.

See the step by step installation guide provided here.

Then I added Java8 PPA and installed Java8, both JDK and JRE, and added Bazel repository and installed bazel 0.4.5 and was able to run bazel version. But trying to run bazel build :foo hangs.

@laszlocsomor
Copy link
Contributor Author

Cool, thanks!

microsoft/WSL#620 (comment) goes further in suspecting Java as the culprit, but then in microsoft/WSL#620 (comment) someone shows a completed bazel version in bash.
I'll have to get a machine with recent enough Win10 to investigate.

@davido
Copy link
Contributor

davido commented Apr 11, 2017

@laszlocsomor Thanks!

As I said in another issue, making Bazel work on "Windows bash" would be a huge step forward to support Windows platform for all bazel driven projects. And that would definitely help Gerrit community with Gerrit Code Review, JGit, Gitiles, Prolog-Caffee, 100+ plugins: all those projects are pure Java/JavaScript projects and don't have any native code (so that MSVC is definitely unrelated).

I would be also able to help with investigation, but unfortunately I'm stuck and don't know how to proceed any further.

Do you have any hints for me?

@laszlocsomor
Copy link
Contributor Author

Thanks, your help would be greatly appreciated!

  1. What's the output of java -version in bash?
  2. Do MSVC-built C++ binaries work in bash? A hello world would suffice.
  3. Does bash translate UNIX paths to Windows paths? i.e. Can you pass /tmp/foo.txt to a MSVC-built .exe and call CreateFile on argv[1]?

@davido
Copy link
Contributor

davido commented Apr 11, 2017

I don't think any MSVC related questions make any sense on "Windows Bash".

That's Linux Ubuntu Trusty, that is ran without virtual machine. The whole MSVC topic is entirely unrelated. In fact, you can't run any Windows binary on "Windows Bash". At least that's how I understand it.

@laszlocsomor
Copy link
Contributor Author

I see, thanks. So if anything we could only run the Linux version of bazel, right?
In that case, can you install OpenJDK 8 and see whether java -version works, whether you can run a pre-built Bazel 0.4.5 for Linux, and whether you can bootstrap Bazel from source (using compile.sh)?

@davido
Copy link
Contributor

davido commented Apr 11, 2017

I see, thanks. So if anything we could only run the Linux version of bazel, right?

Yes, that's correct.

In that case, can you install OpenJDK 8 and see whether java -version works,

Yes, it works. javac worked too.

whether you can run a pre-built Bazel 0.4.5 for Linux

Yes, I was able to install pre-built version 0.4.5 Bazel and run bazel version.

and whether you can bootstrap Bazel from source (using compile.sh)

I haven't tried that yet.

@laszlocsomor
Copy link
Contributor Author

whether you can run a pre-built Bazel 0.4.5 for Linux

Yes, I was able to install pre-built version 0.4.5 Bazel and run bazel version.

And that's what hung for "bazel build foo", correct? I think I caught up to you now :)

@davido
Copy link
Contributor

davido commented Apr 11, 2017

And that's what hung for "bazel build foo", correct? I think I caught up to you now :)

Yeah, exactly! ;-)

@laszlocsomor laszlocsomor changed the title Windows, msys-less: support other shells (cygwin, windows bash, powershell) Windows, msys-less: support other shells (cygwin, powershell) Apr 11, 2017
@laszlocsomor
Copy link
Contributor Author

In light of the latest comments here I'm closing this bug. Let's continue on #1608.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
P2 We'll consider working on this in future. (Assignee optional) platform: windows type: feature request
Projects
None yet
Development

No branches or pull requests

2 participants