-
Notifications
You must be signed in to change notification settings - Fork 6.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Unix tools: move away from MSYS2 (if possible) #12636
Comments
The tools in these projects were very old, over 10 years not updated. |
VCPKG could probably build all those tools itself which I am strongly in favor of. Some of those tools are also simply scripts. Depending on MSYS will always occasionally break VCPKG. |
We don't really need up to date tools for these, unless the build system depends on some new features |
And AFAIK win-bash is still active (last update around 2019) so I think it's fine |
Where is it ? |
Whoops I misread something, sorry for that xD |
@longnguyen2004
|
Seems good enough for the job, just need GNU make (Chocolatey has 4.3). @ras0219-msft what do you think? |
Perl would be a problem though, I haven't known yet if the openssl build system can use native Perl or not... |
msys2 download time is relatively long.
I do not understand that building such a small library zlib requires msys2. |
I thought zlib uses CMake, no? That's probably the pkg-config part, we could use a standalone pkg-config for that. |
zlib Can we download standalone |
For No need to run |
About using BusyBox as a substitute for common Unix tools: |
why not, an auxiliary utility, if it really helps in the assembly, then obviously can use it? @longnguyen2004 |
@voskrese nope that's pretty much same as msys2 xD I need fast GNU make for openssl |
Alright, current state of my BusyBox test:
And that's only for common tools. Other ports might require tools such as |
|
For pieces that still require Msys, we can use a technique like in #12626; download the packages directly and manually unpack them together instead of using Obviously other solutions like busybox are fine too (and may be less complicated if the msys version has lots of dependencies!), but I think we can pick-and-choose on a per-tool basis what's best. |
One thing to note: The current busybox binary on the site doesn't have |
What ports are still using sh-based build systems in vcpkg? I wanna do a test on all of them to determine if it's viable to switch to busybox |
@longnguyen2004 You can find using the keyword |
PR #9966 |
Why related to #9966? |
I'll update this comment as I continue my testing vcpkg_configure_make
vcpkg_acquire_msys
|
I'm gonna pick a few random ones and test with |
I do not understand the motivation here. Can someone help? You are moving from a well tested, constantly updated Linux-like distro to some custom builds. This seems like a big effort for little gain because of some perceived slowness of |
The problem is that the 'constantly updated' part results in breaking vcpkg seemingly every week or so. pacman doesn't seem to like being driven in an automated way like we do, and when it prompts for input or otherwise becomes unhappy we waste ~92 hours of compute time. I believe @ras0219 was looking at downloading the packages from msys directly rather than relying on pacman as an alternate fix. |
that is #13019 |
Pacman can be fully automated. Check the help. You can also point it at your own repository if you are unhappy with the stability of msys2 and curate the package updates manually. |
@mingwandroid Currently Microsoft does not host its own repositories for the tools, URLs always point to official download locations. |
an alternative is always needed, but not on a permanent basis. msys2 and pacman work properly. |
The other problem is the size. Although this is somewhat better now due to the use of zstd, but as of today, BusyBox is still way lighter than MSYS2 installer (~500KiB vs ~90MiB) and also works pretty well with the libraries I've tested (see above). |
BusyBox a stripped-down version of the main utilities and commands
and this will be seen in errors during compilation |
Here's my results with MSVC: vcpkg_configure_make
vcpkg_acquire_msys
|
I know #13019 isn't exactly what you asked for here but I think we should merge something like that, and then if you still think that's too slow we can have a continued discussion. Unless we are able to completely eliminate the msys dependency from all ports I don't think it would be a good idea to not have a single source of truth for these unix-y tools. |
Sure, after all MSYS2 is a complete Unix environment, while BusyBox is simply a collection of common Unix tools. The main problem for me is that MSYS2 |
Or port it to CMake , muahahaa . autotools should die. Now seriously. I don't need OpenSSL anymore, but I used to work on that, I think I made my own I'm only writing here to ask why do we need msys to build |
libiconv build also takes forever (6.5 minutes on a fast PC). |
MSYS2 is about to move its msys subsystem ( |
@longnguyen2004 commented on Aug 28, 2020:
The Midipix project might be of some interest here, but it seems to be in pre-Alpha state yet. Then, building it requires a Linux machine at the moment: Everything I do in its' version of MinTTY feels so quick. Eg. recursive enumeration and
The Midipix action:
Thinking about how to benchmark
At least from the latter two runs Midipix seems to be at least 4.6x faster in forking (launching sub-processes). |
It seems unlikely that make is the source of the problem here rather than relative process creation costs on Linux vs. Windows and the scripts that comprise OpenSSL's build system wanting to spawn a zillion subprocesses. |
I too think that it's Also I think @longnguyen2004 was referring not to the slowness of the |
The problem is that the comparison is not vs. a native make implementation, it is vs. Linux, and the cited example is one of the "shell scripts that cause your CPU pain" posterchildren. I'm not saying that a different make would make no difference, only that this isn't really evidence of that. |
I want to add the deadlinks for Msys2 as another benefit of not using msys2 #36502 |
Is your feature request related to a problem? Please describe.
There are several projects like win-bash, UnxUtils and GnuWin32 that provides most of the tools needed for bash/autotools-based build systems. If we can move to one of these projects, we can avoid the slow build time caused by MSYS2
The text was updated successfully, but these errors were encountered: