-
-
Notifications
You must be signed in to change notification settings - Fork 312
MSYS2 Bash shell seems to override PATH settings #293
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Comments
What happens if you add another step right after the setup-ruby step that adds build to the env? Some thing like: echo "$PWD/build" >> $GITHUB_PATH bash - run: echo "$PWD/build" >> $GITHUB_PATH ps1 - run: echo "$pwd\build" | Out-File -FilePath $env:GITHUB_PATH -Encoding utf8 -Append |
Forget the above. The issue seems to be the bash shell. So, add the following to your above workflow, which will use the Git bash shell instead of the msys2 bash shell: - run: echo "C:/Program Files/Git/bin" | Out-File -FilePath $env:GITHUB_PATH -Encoding utf8 -Append Also, Ruby 3.1.0 will install the MSYS2 ucrt64 gcc tools, where 3.0.3 will install the MSYS2 mingw64 gcc tools. See workflow run here |
Hey, thanks for looking at this! I confess I remain baffled as to what is fundamentally happening here. I'd actually tried the I had an idea overnight which I wanted to try, which was to explicitly use the "old" Git, i.e., - uses: ruby/setup-ruby@v1
with:
ruby-version: 3.1.0
- run: |
git version
git-lfs version
git lfs version
echo "============"
where git
echo "============"
where git-lfs
echo "============"
echo "Building Git LFS demo ..."
cd build
go build -o git-lfs.exe
cd ..
PATH="$(pwd)/build:$PATH"
export PATH
git-lfs version
git lfs version
'/c/Program Files/Git/mingw64/bin/git' lfs version
echo "============"
where git-lfs
echo "============"
shell: bash and here's the output:
So it almost seems as though the issue might lie in the Git binary and/or the binary in combination with the Bash shell. As a non-Windows developer I'm in deep waters here and can't yet see how a different Git binary (of the same version of Git!) could end up searching different paths when running a command, but I thought I'd try to trace the code a bit and see if anything jumps out. (Trying with |
One update -- I realized that trying to prepend In the "old" version with
In the "new" version with
I wrote last night:
But as I noted in the previous commit, it actually seems that which Git binary runs makes a big difference (again, for reasons I don't understand). Adding this step after
I'm going to try a full CI job next with that in place. I do wish I had some inkling why this matters, though. |
Ok. First of all, Windows exe's find library files in a much different manner than *nix. There's newer functions that often aren't used, and there's the legacy method which relies on PATH and other things.
Also, I thought I saw a command |
That's not quite correct. There can occur some problems with adding the complete The "wrong"
|
FWIW this is related to git-for-windows/git#3668. |
Just as an update, we're likely to try to bypass this issue in the Git LFS CI suite by dropping the use of ronn, and therefore dodging the need to install Ruby at all. That's take us some time, however, as we'll have to rewrite the manual pages' ronn-format source files. In the meantime, if I find some spare time I'll try to revisit the issue and see if I can make your suggestions work for us; it would be nice to upgrade to the Windows 2022 runners, as we bumped into the problem of not using them again during a recent Git LFS patch release. Anyway, thanks for the advice and I'll report back if I have any success. |
In commit f4ab4e7 of PR git-lfs#4883 we noted that the "gem install ronn" step in our Windows CI workflows currently fails on newer Windows Server 2022 GitHub Actions runners, due to possibly some missing C headers or libraries. While this could be resolved by upgrading to the "ruby/setup-ruby" action from the deprecated "actions/setup-ruby" one, that introduced a series of issues with PATH lookups as documented in ruby/setup-ruby#293. We therefore chose to continue using Windows 2019 runners until a resolution or workaround was found. Fortunately, as of commit 912b607 in PR git-lfs#4992 we are now using the ronn-ng Ruby gem instead of the unmaintained ronn gem, and the "gem install ronn-ng" workflow step succeeds on Windows 2022 while still using the "actions/setup-ruby" action to install a Ruby build environment. This allows us to upgrade to the latest Windows runners in GitHub Actions, although we can't yet upgrade to the "ruby/setup-ruby" workflow action. h/t to @bk2204 for finding ronn-ng!
A small update: we're upgrading to ronn-ng (see git-lfs/git-lfs#4992 and git-lfs/git-lfs#4997) which is going to allow us to stick with the deprecated We tried also upgrading to the |
In commit dd03e52 [formerly f4ab4e7445c04fe5d663fc79ddb2ebf30a5a870f] of PR #4883 we noted that the "gem install ronn" step in our Windows CI workflows currently fails on newer Windows Server 2022 GitHub Actions runners, due to possibly some missing C headers or libraries. While this could be resolved by upgrading to the "ruby/setup-ruby" action from the deprecated "actions/setup-ruby" one, that introduced a series of issues with PATH lookups as documented in ruby/setup-ruby#293. We therefore chose to continue using Windows 2019 runners until a resolution or workaround was found. Fortunately, as of commit ccefc90 [formerly 912b607acb162ea324d0ff00fea1a1f47657e1d4] in PR #4992 we are now using the ronn-ng Ruby gem instead of the unmaintained ronn gem, and the "gem install ronn-ng" workflow step succeeds on Windows 2022 while still using the "actions/setup-ruby" action to install a Ruby build environment. This allows us to upgrade to the latest Windows runners in GitHub Actions, although we can't yet upgrade to the "ruby/setup-ruby" workflow action. h/t to @bk2204 for finding ronn-ng! Former-commit-id: 6c234a29a2a2b646b4b3596cd0a386b0df7a4f32
Avoid ruby/setup-ruby because it messes with $PATH, see ruby/setup-ruby#293
Avoid ruby/setup-ruby because it messes with $PATH, see ruby/setup-ruby#293
Avoid ruby/setup-ruby because it messes with $PATH, see ruby/setup-ruby#293
A couple of followup notes: First, I realized that the Actions step suggested in #293 (comment) appends, not prepends, the provided path to the Git binaries to the
That "works" in the sense that the Git binary in the However, @dscho noted in #293 (comment) that:
Unfortunately, the In the same commit, @dscho's suggested that an alternative would be to remove the system-installed
So that might be what we end up using. Unfortunately, though, that's not quite the end of the story; there are also problems caused by MSYS2 mangling command arguments due to its automatic path translation. One example is that the legitimate Git LFS command So we're experimenting with setting a global
In brief, it would be great to have a way to retain the Bash shell and environment used in |
First, sorry about that mix up between prepend & append. Anyway, re the which Re the bash issue, can you rename As to the reasons for the MSYS2 path changes, this is messy. Ruby needs MSYS2 build tools, which are installed in directories within But, some CI scripts use bash commands. Also, some 'platform agnostic' build tools are located in We certainly would strongly prefer to not add an input for this, but I suppose we could add a switch based on an ENV setting that could affect it. |
Thanks for the thoughts!
We could try that, but just removing the system-installed (To recap the original issue: we build a local
Great idea, thank you! 🙇 Trying that with the following step works really well, and leaves just a couple of tests (out of about 700) failing:
It does indeed let the Bash shell in Well, that's definitely progress -- thank you for the great suggestion! |
Glad things are working a bit better.
The symlink might fix that. Windows 'location resolution' for library files (*.dll) has a few mechanisms that take precedence over Two things. First, Ruby 3.1 and later (3.2 will be released this month) are compiled with the MSYS2 ucrt tool chain. This could be causing issues. If so, you could rename Secondly, things may get strange when MSYS2 switches to OpenSSL 3.0.x, as it's currently using 1.1.1. Ping me if that causes problems. |
One other thing I've run into is the need to unset the For us, that produced some CI failures because directories created using Using |
Is this still an issue or could we close this? |
We've worked around it for Git LFS, but so far as I know it's still an issue. |
One option would be to add an input (or env setting) that turns off the MSYS2 additions to PATH. Many workflows would break if it was changed to 'opt-in', but allowing 'opt-out' would be ok... |
I think we should close this as not planned. |
Just to make clear for non-Windows users:
Because of the above, and to make Windows CI easier for non-Windows users, the MSYS2 build tools (gcc & packages) needed to build Ruby are installed and added to PATH, and the MSYS2 bash shell is also added to PATH. Note that some of build tools are located in the bash shell path ( These PATH additions should remain the default. Whether an option to 'opt-out' of the PATH additions is added, I don't have a strong opinion... |
On the Git LFS project we have an older Windows CI workflow which uses the legacy
actions/setup-ruby@v1
action, and which we might like to upgrade toruby/setup-ruby@v1
as it would address some build problems on Windows Server 2022.However, simply replacing one for the other results in many CI test failures, apparently because a different Bash shell is used and our
PATH
environment variable settings are not respected, the result being that when our tests rungit lfs
a system default Git LFS binary is found by Git and executed instead of the newly-builtgit-lfs.exe
which we want to test, leading to lots of unexpected results (new tests run by an older binary which then, of course, fail).Here are two versions of a small demo CI workflow, one which runs
actions/setup-ruby@v1
and one which runsruby/setup-ruby@v1
. In both cases the workflow then builds a small demo Go executable in a subdirectory; we name this executablegit-lfs.exe
. By default, if we run eithergit-lfs version
orgit lfs version
, one of the system's defaultgit-lfs.exe
binaries is found and run.If we prepend our build subdirectory's path to the
PATH
environment variable, we expect that our demogit-lfs.exe
should be run instead, and that is what happens in both cases if we run it directly withgit-lfs version
, which can be seen in the workflow's logs. However, only in theactions/setup-ruby@v1
script doesgit lfs version
find and run our demo executable; in theruby/setup-ruby@v1
case thegit.exe
binary instead finds only the system'sgit-lfs.exe
. This may be due to some interaction with the MSYS2 installation and its Bash shell, and possibly theMSYS2_PATH_TYPE
configuration.First, here's the full workflow on a
windows-2022
Actions runner which demonstrates howactions/setup-ruby@v1
does not causePATH
to be ignored, which is as we would expect:The output looks like:
Notice how when we run
git-lfs version
andgit lfs version
the first time, before we changePATH
, one of the system defaultgit-lfs.exe
binaries runs and outputsgit-lfs/3.0.2 (GitHub; windows amd64; go 1.17.2)
. However, once we build our demogit-lfs.exe
and put it in thePATH
, both of those commands now returnGit LFS demo
because our demo binary runs instead. That makes sense, given that it's now found first bywhere
, as shown.Next, here is the another workflow, again on a
windows-2022
runner, where all we've changed is to replaceactions/setup-ruby@v1
withruby/setup-ruby@v1
and specifiedruby-version: 3.1.0
(and we've also tried 3.0.3 and seen the same result):The output looks like:
With this second workflow, notice that now, after changing
PATH
, we only see our demogit-lfs.exe
run by thegit-lfs version
command, whereas thegit lfs version
one finds and runs a system defaultgit-lfs.exe
. This can be seen by the difference in output; the former returnsGit LFS demo
but the latter returnsgit-lfs/3.0.2 (GitHub; windows amd64; go 1.17.2)
. And that's despite the fact thatwhere
correctly finds our demogit-lfs.exe
first.A few things are also different, of course. A different
bash.exe
is run; the first workflow runsC:\Program Files\Git\bin\bash.EXE
but the second workflow runsC:\msys64\usr\bin\bash.EXE
, and that reports a set of environment variables, includingMSYS2_PATH_TYPE: inherit
and aPATH
in whichC:\Program Files\Git\bin
is ahead ofC:\Program Files\Git\mingw64\bin
.That accounts for another obvious difference, which is that the first workflow finds
C:\Program Files\Git\mingw64\bin\git.exe
when runninggit
while the second findsC:\Program Files\Git\bin\git.exe
and runs that instead. Whichevergit.exe
is run, though, should not make a difference; both are version2.35.1.windows.2
. Both should respond to the change inPATH
which puts our local build subdirectory first.We are suspicious that there may be some way in which the
MSYS2_PATH_TYPE: inherit
setting is not taking effect here. We're not familiar with MSYS2 at all, but skimming through the code it appears that this is intended to make it work the "normal" way, withPATH
being inherited by any processes invoked by a parent process. But the default appears to be to not follow this convention, and msys2/MSYS2-packages#2140 suggests that perhaps there are ways this setting reverts to the default in various situations. Is it possible that that is what is occurring here, and the Git process we invoke gets itsPATH
reset back to the one Bash is started with? That would certainly account for Git not finding our demogit-lfs.exe
.At any rate, the overall situation is one which seems akin to the issue raised in #19 and #20, namely, that highly unexpected behaviour results from running
ruby/setup-ruby@v1
. This issue has taken two days of time to debug just to the point where we could report it in a somewhat detailed fashion, and we still don't understand exactly what's going on.For the time being we're going to stick with
actions/setup-ruby@v1
because it continues to work without breaking our entire Git LFS test suite, which is what happens if we replace it withruby/setup-ruby@v1
due to this issue of localPATH
settings apparently being ignored by child processes.The text was updated successfully, but these errors were encountered: