Avoid sharing cygheaps across Cygwin versions #48

dscho · 2023-01-30T22:56:59Z

It frequently leads to problems when trying, say, to call from Git for Windows' Bash into Cygwin's or MSYS2's, merely because sharing that data is pretty finicky.

For example, using the Git for Windows that is current at time of writing, trying to call MSYS2's Bash from Git for Windows' Bash fails somewhere in _cmalloc(), without any error message, and with the rather misleading exit code 127 (a code which is reserved to indicate that a command was not found).

Let's just treat these as completely incompatible with one another, by virtue of using a different CHILD_INFO_MAGIC constant.

One consequence is that spawned MSYS processes using a different MSYS2 runtime will not be visible as such to the parent process, i.e. they cannot share any resources such as pseudo terminals. But that's okay, they are simply treated as if they were regular Win32 programs.

This should also help the scenario where interactions between two different versions of Git for Windows lead to those infamous cygheap base mismatch detected problems mentioned e.g. in git-for-windows/git#4255

This fixes git-for-windows/git#4255

It frequently leads to problems when trying, say, to call from Git for Windows' Bash into Cygwin's or MSYS2's, merely because sharing that data is pretty finicky. For example, using the Git for Windows that is current at time of writing, trying to call MSYS2's Bash from Git for Windows' Bash fails somewhere in `_cmalloc()`, without any error message, and with the rather misleading exit code 127 (a code which is reserved to indicate that a command was not found). Let's just treat these as completely incompatible with one another, by virtue of using a different `CHILD_INFO_MAGIC` constant. One consequence is that spawned MSYS processes using a different MSYS2 runtime will not be visible as such to the parent process, i.e. they cannot share any resources such as pseudo terminals. But that's okay, they are simply treated as if they were regular Win32 programs. This should also help the scenario where interactions between two different versions of Git for Windows lead to those infamous `cygheap base mismatch detected` problems mentioned e.g. in git-for-windows/git#4255 Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>

A recent binutils version introduced `libsframe` and made it a dependency of `libbfd`. Which means that we have to link that library into `dumper.exe`, too, if it exists. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>

dscho · 2023-01-31T13:32:52Z

I tested this a bit and am fairly confident that it 1) solves the problem and 2) does not introduce other problems.

derrickstolee · 2023-01-31T14:04:24Z

winsup/cygwin/dcrt0.cc

@@ -545,7 +545,7 @@ get_cygwin_startup_info ()
  child_info *res = (child_info *) si.lpReserved2;

  if (si.cbReserved2 < EXEC_MAGIC_SIZE || !res
-      || res->intro != PROC_MAGIC_GENERIC || res->magic != CHILD_INFO_MAGIC)
+      || res->intro != PROC_MAGIC_GENERIC || res->magic != (CHILD_INFO_MAGIC ^ CYGWIN_VERSION_DLL_COMBINED))


I was surprised at first at the use of bitwise XOR here, as I would expect this to be a bitwise OR if these were flags. However, CYGWIN_VERSION_DLL_COMBINED is an integer with multiple bits (currently set to 3003006 to represent 3.3.6).

So this "magic" is very magical, it seems (and you do a similar update in sigproc.cc.

But the commit message motivates this well, saying that we should use a new magic value whenever Git for Windows' version of MSYS2 is updated. (It's not accurate that only a new Git version would change this magic, but if two Git for Windows releases have the identical MSYS2 version, then they should be compatible still.)

Indeed. The idea here is to have some kind of unique identifier that won't ever be shared between different MSYS2 runtime versions. Hence the XOR (because the OR would decrease the entropy).

derrickstolee

I can't speak for all of the magic in this PR, but it seems like a reasonable approach. It definitely seems like the compatibility issue will be avoided by further partitioning shared memory, not combining any previously-separated partitions.

derrickstolee · 2023-01-31T14:06:41Z

winsup/cygwin/dcrt0.cc

@@ -545,7 +545,7 @@ get_cygwin_startup_info ()
  child_info *res = (child_info *) si.lpReserved2;

  if (si.cbReserved2 < EXEC_MAGIC_SIZE || !res
-      || res->intro != PROC_MAGIC_GENERIC || res->magic != CHILD_INFO_MAGIC)
+      || res->intro != PROC_MAGIC_GENERIC || res->magic != (CHILD_INFO_MAGIC ^ CYGWIN_VERSION_DLL_COMBINED))


But the commit message motivates this well, saying that we should use a new magic value whenever Git for Windows' version of MSYS2 is updated. (It's not accurate that only a new Git version would change this magic, but if two Git for Windows releases have the identical MSYS2 version, then they should be compatible still.)

…e versions When calling, say, a Cygwin Bash (or even a regular MSYS2 Bash, or even a MinGit Bash of a different Git for Windows version), it is possible that the MSYS2 runtime of the called Bash is incompatible with the MSYS2 runtime of the caller. When that happens, all kinds of chaos can ensue. One possible symptom is that the MSYS2 runtime aborts with the error message "cygheap base mismatch detected". Another possible symptom is that the MSYS2 runtime (or the Cygwin runtime) exits silently, with exit code 127 (which is misleading as that exit code is reserved to imply that an executable could not be found). With this patch, we try harder to prevent such a thing from happening. This is a companion to git-for-windows/msys2-runtime#48 Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>

When the MSYS2-runtime version changed recently, the `msys-2.0.dll` contents of the minimal SDK diverged from the contents of the same file in the Git for Windows installation in `C:\Program Files`. Since we adjust the `PATH` before caching, that matters because the `gzip` executable that is found in the `PATH` by `tar -z` is the version from the minimal SDK, which therefore insists on using the MSYS2 runtime from the minimal SDK, while the `tar` executable used by @actions/cache is the `C:\Program Files\Git\usr\bin\tar.exe` one and hence uses a potentially-incompatible MSYS2 runtime version of the Git for Windows installation. In git-for-windows/msys2-runtime#48, we tried to address this by teaching the MSYS2 runtime to avoid talking to a different MSYS2 runtime version. But the indicator used in that Pull Request is the version of the _Cygwin_ runtime on which it is based. Which means that Git for Windows-only changes in the MSYS2 runtime are not reflected in that indicator, and the MSYS2 runtime _still_ tries to talk to the other version, and the only symptom the user gets is a terse `Child returned status 127`. Let's work around this here (taking the pressure off fixing the MSYS2 runtime logic) simply by adjusting the `PATH` only _after_ letting @actions/cache do its thing. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>

When trying to call Cygwin (or for that matter, MSYS2) programs from Git about a "cygheap"](git-for-windows/git#4255) [now](git-for-windows/msys2-runtime#48) allowing basic interactions. While it is still not possible for, say, _is_ now possible for Cygwin's `zstd.exe` in conjuction with Git for Signed-off-by: gitforwindowshelper[bot] <gitforwindowshelper-bot@users.noreply.github.com>

…ys2-runtime-versions-from-sharing-cygheaps Avoid sharing cygheaps across Cygwin versions

dscho self-assigned this Jan 30, 2023

dscho force-pushed the prevent-different-msys2-runtime-versions-from-sharing-cygheaps branch from a7f7189 to 4704d29 Compare January 31, 2023 09:05

dscho marked this pull request as ready for review January 31, 2023 13:32

derrickstolee reviewed Jan 31, 2023

View reviewed changes

derrickstolee approved these changes Jan 31, 2023

View reviewed changes

dscho merged commit ebcc274 into git-for-windows:main Jan 31, 2023

dscho deleted the prevent-different-msys2-runtime-versions-from-sharing-cygheaps branch January 31, 2023 14:09

dscho mentioned this pull request Jan 31, 2023

Avoid sharing cygheap between msys2 runtime versions git-for-windows/MSYS2-packages#77

Merged

rimrul mentioned this pull request Feb 1, 2023

[New msys2-runtime version] dumper: avoid linker problem when libbfd depends on libsframe git-for-windows/git#4263

Closed

dscho mentioned this pull request Feb 1, 2023

cache trying to use git for windows' tar.exe on self-hosted runner, failing to find correct gzip actions/toolkit#1311

Open

This was referenced Feb 21, 2023

Try harder to avoid sharing cygheaps across MSYS2 runtime versions git-for-windows/git#4305

Closed

Prevent different msys2 runtime versions from sharing cygheaps, take two #49

Merged

dscho added a commit to dscho/msys2-runtime that referenced this pull request May 12, 2023

Merge pull request git-for-windows#48 from dscho/prevent-different-ms…

c427654

…ys2-runtime-versions-from-sharing-cygheaps Avoid sharing cygheaps across Cygwin versions

dscho added a commit to dscho/msys2-runtime that referenced this pull request Jun 16, 2023

Merge pull request git-for-windows#48 from dscho/prevent-different-ms…

dced4e3

…ys2-runtime-versions-from-sharing-cygheaps Avoid sharing cygheaps across Cygwin versions

dscho added a commit to dscho/msys2-runtime that referenced this pull request Jun 19, 2023

Merge pull request git-for-windows#48 from dscho/prevent-different-ms…

2d11bfd

…ys2-runtime-versions-from-sharing-cygheaps Avoid sharing cygheaps across Cygwin versions

dscho added a commit to dscho/msys2-runtime that referenced this pull request Sep 6, 2023

Merge pull request git-for-windows#48 from dscho/prevent-different-ms…

a2255d3

…ys2-runtime-versions-from-sharing-cygheaps Avoid sharing cygheaps across Cygwin versions

dscho mentioned this pull request Nov 22, 2023

Cygwin doesn't work with Git Bash but does with Windows Shell git-for-windows/git#3728

Closed

1 task

dscho added a commit to dscho/msys2-runtime that referenced this pull request Nov 29, 2023

Merge pull request git-for-windows#48 from dscho/prevent-different-ms…

90282d6

…ys2-runtime-versions-from-sharing-cygheaps Avoid sharing cygheaps across Cygwin versions

dscho mentioned this pull request Jul 10, 2024

[New msys2-runtime-package version] msys2-runtime: prevent different msys2 runtime versions from sharing … git-for-windows/git#5051

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Avoid sharing cygheaps across Cygwin versions #48

Avoid sharing cygheaps across Cygwin versions #48

dscho commented Jan 30, 2023

dscho commented Jan 31, 2023

derrickstolee Jan 31, 2023

derrickstolee Jan 31, 2023

dscho Jan 31, 2023

derrickstolee left a comment

derrickstolee Jan 31, 2023

Avoid sharing cygheaps across Cygwin versions #48

Avoid sharing cygheaps across Cygwin versions #48

Conversation

dscho commented Jan 30, 2023

dscho commented Jan 31, 2023

derrickstolee Jan 31, 2023

Choose a reason for hiding this comment

derrickstolee Jan 31, 2023

Choose a reason for hiding this comment

dscho Jan 31, 2023

Choose a reason for hiding this comment

derrickstolee left a comment

Choose a reason for hiding this comment

derrickstolee Jan 31, 2023

Choose a reason for hiding this comment