[vcpkg] Replace uses of msys pacman.exe with direct package downloads #13019
Conversation
JackBoosY
added
category:tool-update
ras0219
force-pushed
the
dev/roschuma/msys
branch
3 times, most recently
from
39396c4
to
eb69df6
Compare
ras0219
force-pushed
the
dev/roschuma/msys
branch
from
eb69df6
to
bef28ec
Compare
IMHO these edits can be made parallel to the main MSYS2? |
ras0219
force-pushed
the
dev/roschuma/msys
branch
from
cac1ad3
to
0ca1b08
Compare
@voskrese We definitely need to enable ports to use new utilities -- that is why I've added the |
ras0219-msft
changed the title
| @@ -12,5 +12,5 @@ Build-Depends: openblas | |||
| Description: Use external optimized BLAS | |||
|
|
|||
| Feature: blas-select | |||
| Build-Depends: lapack-reference[core, noblas](!windows|(windows&!static)) | |||
| Build-Depends: lapack-reference[core, noblas](!windows|!static) | |||
There was a problem hiding this comment.
I sincerely love these diff snippets in a PR, an unnecessary touch of logic and love out of the blue.
ras0219
force-pushed
the
dev/roschuma/msys
branch
from
220ff00
to
95542b8
Compare
|
What about |
| function(tensorflow_try_remove_recurse_wait PATH_TO_REMOVE) | ||
| file(REMOVE_RECURSE ${PATH_TO_REMOVE}) | ||
| if (EXISTS "${PATH_TO_REMOVE}") | ||
| execute_process(COMMAND ${CMAKE_COMMAND} -E sleep 5) | ||
| _execute_process(COMMAND ${CMAKE_COMMAND} -E sleep 5) |
There was a problem hiding this comment.
It enables this sleep to execute during download-only mode
|
Based on https://repo.msys2.org/msys/x86_64/, it looks like packages are kept for at least 5 years. This is sufficient for our current needs, and in the future we will investigate adding mechanisms to mirror remote resources such as these. |
…issues As part of a full rebuild in microsoft#13019, several transitive rebuild issues were found. 1) selene was dynamically detecting the presence of OpenCV, which was not on its dependency list. 2) The dynamic OpenCV was using `find_package()` instead of `find_dependency()` 3) The dynamic OpenCV was always finding all dependencies instead of just those that were used during build. 4) CMake's FindHDF5 module requires C language support 5) ceres was passing `REQUIRED` into find_dependency (not needed -- find_dependency implies REQUIRED) 6) FindLAPACK.cmake was sometimes provided by the meta-port lapack, sometimes by the dependency lapack-reference/clapack 7) The lapack-reference vcpkg-cmake-wrapper was directly including the FindLAPACK, which doesn't respect variables such as QUIET and REQUIRED Interestingly, the particular issue required a cascade of failures to occur to actually cause a build failure; almost any of these fixes would independently fix the build.
|
/azp run |
|
Azure Pipelines successfully started running 1 pipeline(s). |
|
This pull request uses cmake file(DOWNLOAD https://repo.msys2.org/mingw/i686/mingw-w64-i686-pkg-config-0.29.2-1-any.pkg.tar.xz)
file(DOWNLOAD https://repo.msys2.org/mingw/i686/mingw-w64-i686-libwinpthread-git-8.0.0.5906.c9a21571-1-any.pkg.tar.zst)If using Windows 7, downloading from https://repo.msys2.org/ will fail with status: 35: SSL connect error |
|
@linquize |
|
Seems cmake on Windows uses |
This PR completely replaces the approach inside
vcpkg_acquire_msys()with a more direct and streamlined package acquisition stream. Instead of relying on pacman.exe to get packages (which often changes out from underneath us, either by changing keys or otherwise), we instead encode direct references to the archives themselves. This results in a vastly more stable system, since every file is hash checked and goes through our existing download machinery, ensuring everything is properly pinned and robust. It also ensures each unique set of packages gets a unique directory, which ensures that separate builds will avoid destructive interference -- everyone will always get exactly what was requested and no more.However, this also means we lose the ability for arbitrary ports to "simply" use packages we don't already know about (and perhaps didn't even exist at the time of vcpkg_acquire_msys()'s authoring). To handle this case, ports can instead directly fetch packages by adding them to the
DIRECT_PACKAGESparameter; this will simply download the provided URLs and unpack them into the same msys directory. These URLs can be trivially found by perusing https://packages.msys2.org; seeffmpegas an example.