VCPKG_FORCE_SYSTEM_BINARIES cause vcpkg_apply_patches error on Windows #16216

huangqinjin · 2021-02-14T06:57:37Z

Environment

OS: Windows

To Reproduce
Steps to reproduce the behavior:

set VCPKG_FORCE_SYSTEM_BINARIES=1
vcpkg install sqlite3

Failure logs

-- Applying patch fix-arm-uwp.patch
CMake Error at scripts/cmake/vcpkg_apply_patches.cmake:68 (message):
  Applying patch failed.
Call Stack (most recent call first):
  scripts/cmake/vcpkg_extract_source_archive_ex.cmake:145 (vcpkg_apply_patches)
  ports/sqlite3/portfile.cmake:10 (vcpkg_extract_source_archive_ex)
  scripts/ports.cmake:131 (include)

Additional context
Add some logs to vcpkg_apply_patches

    message(STATUS "PATH = $ENV{PATH}")
    message(STATUS "GIT = ${GIT}")

got

-- PATH = C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\Common7\IDE\\Extensions\Microsoft\IntelliCode\CLI;C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Tools\MSVC\14.28.29333\bin\HostX64\x86;C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Tools\MSVC\14.28.29333\bin\HostX64\x64;C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\Common7\IDE\VC\VCPackages;C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\Common7\IDE\CommonExtensions\Microsoft\TestWindow;C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\Common7\IDE\CommonExtensions\Microsoft\TeamFoundation\Team Explorer;C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\MSBuild\Current\bin\Roslyn;C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\Team Tools\Performance Tools;C:\Program Files (x86)\Microsoft Visual Studio\Shared\Common\VSPerfCollectionTools\vs2019\;C:\Program Files (x86)\Microsoft SDKs\Windows\v10.0A\bin\NETFX 4.8 Tools\x64\;C:\Program Files (x86)\HTML Help Workshop;C:\Program Files (x86)\Windows Kits\10\bin\10.0.19041.0\x64;C:\Program Files (x86)\Windows Kits\10\bin\x64;C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\\MSBuild\Current\Bin;C:\Windows\Microsoft.NET\Framework64\v4.0.30319;C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\Common7\IDE\;C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\Common7\Tools\;;C:\WINDOWS\system32;C:\WINDOWS;C:\WINDOWS\system32\Wbem;C:\WINDOWS\system32\WindowsPowerShell\v1.0\;C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\Common7\IDE\VC\Linux\bin\ConnectionManagerExe
-- GIT = GIT-NOTFOUND

PATH doesn't contain git installation path which should be

C:\vcpkg>where git
C:\Program Files\Git\cmd\git.exe

The text was updated successfully, but these errors were encountered:

StarGate-One · 2021-02-14T23:32:30Z

@huangqinjin

From my experience, when VCPKG_FORCE_SYSTEM_BINARIES is set to true or 1, then vcpkg will not download tools such as CMake, Ninja and other tools and forces vcpkg to look for the tool(s) binaries in the PATH environment variable.
If a tool is not in the PATH, then someone would need to find/add/install the tool/location to the PATH, otherwise vcpkg will fail.
It is best to unset VCPKG_FORCE_SYSTEM_BINARIES and let vcpkg download/install the tools it needs as it has been tested to work with the versions it downloads.

huangqinjin · 2021-02-15T01:27:47Z

@StarGate-One I already installed git in system, see the last part of the issuse description, where.exe can find git. Vcpkg seems to change PATH to keep it "clean".

StarGate-One · 2021-02-15T02:35:38Z

Hmm, I do know vcpkg creates a clean environment, but I would think it would bring in some of the environment variables when forcing the system binaries.

PhoebeHui · 2021-02-18T08:51:17Z

@huangqinjin, I can't repro this isssue locally, I installed git 2.30.0, and it appied the patch correctly. what's revision do you use?

PS F:\vcpkg\src> ./vcpkg install sqlite3
Computing installation plan...
The following packages will be built and installed:
    sqlite3[core]:x86-windows -> 3.33.0
Detecting compiler hash for triplet x86-windows...
Could not locate cached archive: C:\Users\phoebe\AppData\Local\vcpkg\archives\42\42326ecbd33a047da662f02a3c499ae37de2d885.zip
Starting package 1/1: sqlite3:x86-windows
Building package sqlite3[core]:x86-windows...
-- Using cached F:/vcpkg/src/downloads/sqlite-amalgamation-3330000.zip
-- Cleaning sources at F:/vcpkg/src/buildtrees/sqlite3/src/3330000-c6f2129a7a.clean. Use --editable to skip cleaning for the packages you specify.
-- Extracting source F:/vcpkg/src/downloads/sqlite-amalgamation-3330000.zip
-- PATH = C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\VC\Tools\MSVC\14.16.27023\bin\HostX64\x86;C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\VC\Tools\MSVC\14.16.27023\bin\HostX64\x64;C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\Common7\IDE\VC\VCPackages;C:\Program Files (x86)\Microsoft SDKs\TypeScript\3.1;C:\Program Files (x86)\Microsoft SDKs\TypeScript\3.1;C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\Common7\IDE\CommonExtensions\Microsoft\TestWindow;C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\Common7\IDE\CommonExtensions\Microsoft\TeamFoundation\Team Explorer;C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\MSBuild\15.0\bin\Roslyn;C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\Team Tools\Performance Tools;C:\Program Files (x86)\Microsoft Visual Studio\Shared\Common\VSPerfCollectionTools\;C:\Program Files (x86)\Microsoft SDKs\Windows\v10.0A\bin\NETFX 4.6.1 Tools\x64\;C:\Program Files (x86)\HTML Help Workshop;C:\Program Files (x86)\Windows Kits\10\bin\10.0.19041.0\x64;C:\Program Files (x86)\Windows Kits\10\bin\x64;C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\\MSBuild\15.0\bin;C:\Windows\Microsoft.NET\Framework64\v4.0.30319;C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\Common7\IDE\;C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\Common7\Tools\;F:\vcpkg\src\downloads\tools\powershell-core-7.1.0-windows;C:\Windows\system32;C:\Windows;C:\Windows\system32\Wbem;C:\Windows\system32\WindowsPowerShell\v1.0\;C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\Common7\IDE\CommonExtensions\Microsoft\CMake\CMake\bin;C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\Common7\IDE\CommonExtensions\Microsoft\CMake\Ninja
-- GIT = C:/Program Files/Git/cmd/git.exe
-- Applying patch fix-arm-uwp.patch
-- Using source at F:/vcpkg/src/buildtrees/sqlite3/src/3330000-c6f2129a7a.clean
-- Configuring x86-windows
-- Building x86-windows-dbg
-- Building x86-windows-rel
-- Performing post-build validation
-- Performing post-build validation done
Stored binary cache: C:\Users\\phoebe\AppData\Local\vcpkg\archives\42\42326ecbd33a047da662f02a3c499ae37de2d885.zip
Building package sqlite3[core]:x86-windows... done
Installing package sqlite3[core]:x86-windows...
Installing package sqlite3[core]:x86-windows... done
Elapsed time for package sqlite3:x86-windows: 15.79 s

Total elapsed time: 17.65 s

The package sqlite3:x86-windows provides CMake targets:

    find_package(unofficial-sqlite3 CONFIG REQUIRED)
    target_link_libraries(main PRIVATE unofficial::sqlite3::sqlite3)

JackBoosY · 2021-02-18T08:56:27Z

Can you please check the Windows region setting about utf-8?
See #12865 (comment).

huangqinjin · 2021-02-18T10:25:59Z

@PhoebeHui C:\Program Files\Git\cmd is not in your PATH, how can cmake find git? Have you performed Repro Step 1?

huangqinjin · 2021-02-18T10:29:57Z

@JackBoosY I don't set Utf8, the locale of my PC is Chinese.

JackBoosY · 2021-02-19T02:54:16Z

Maybe same with #15456.

JackBoosY · 2021-02-19T05:49:40Z

Can't reproduce this issue on Windows 10 Chinese.

PhoebeHui · 2021-02-19T07:02:19Z

@huangqinjin, what's git revision in your local machine? the ' C:/Program Files/Git/cmd/' in the system environment PATH in my local mamachine, and the git tool path should be aquired before applying patch, also before getting the clean PATH. I suggest to reinstall git and try again.

huangqinjin · 2021-02-19T14:01:10Z

I use vcpkg f79c681.

This is neither locale issue nor git installation issue. Without set VCPKG_FORCE_SYSTEM_BINARIES=1, I can install sqlite3 successfully.

Another way to repro the issue. unset(GIT CACHE) before find_program(GIT NAMES git git.cmd). In this case, even unset VCPKG_FORCE_SYSTEM_BINARIES, the issue occurred. Because git.exe in not in clean PATH, so find_program(GIT) failed.

JackBoosY · 2021-02-20T02:48:44Z

cc @ras0219 @ras0219-msft

PhoebeHui · 2021-02-20T03:09:49Z

@huangqinjin, the problem here is vcpkg didn't get the git tool path from system successfully, without settingVCPKG_FORCE_SYSTEM_BINARIES=1, it will use the one that downloaded by vcpkg, so you can install sqlite3 successfully.

JackBoosY · 2021-02-20T06:14:15Z

@huangqinjin Can you use powershell instead of cmd?

PhoebeHui · 2021-02-20T06:23:31Z

It should be a vcpkg bug, after setting ' $env:VCPKG_FORCE_SYSTEM_BINARIES=1', it failed before the 'apply patch steps' with latest source.

PS E:\vcpkg\clean\vcpkg> ./vcpkg install sqlite3:x86-windows-static --debug
[DEBUG] Feature flag 'binarycaching' unset
[DEBUG] Feature flag 'manifests' unset
[DEBUG] Feature flag 'compilertracking' unset
[DEBUG] Feature flag 'registries' unset
[DEBUG] Feature flag 'versions' unset
[DEBUG] BuiltinRegistry initialized with: ""
[DEBUG] Using vcpkg-root: E:\vcpkg\clean\vcpkg
[DEBUG] Using installed-root: E:\vcpkg\clean\vcpkg\installed
[DEBUG] BuiltinRegistry initialized with: ""
[DEBUG] Using buildtrees-root: E:\vcpkg\clean\vcpkg\buildtrees
[DEBUG] Using downloads-root: E:\vcpkg\clean\vcpkg\downloads
[DEBUG] Using packages-root: E:\vcpkg\clean\vcpkg\packages
[DEBUG] Using scripts-root: E:\vcpkg\clean\vcpkg\scripts
[DEBUG] Using ports-root: E:\vcpkg\clean\vcpkg\ports
[DEBUG] Using versions-root: E:\vcpkg\clean\vcpkg\versions
[DEBUG] Default binary cache path is: C:\Users\phoebe\AppData\Local\vcpkg\archives
Computing installation plan...
[DEBUG] CreateProcessW(cmake -DVCPKG_ROOT_DIR=E:/vcpkg/clean/vcpkg -DPACKAGES_DIR=E:/vcpkg/clean/vcpkg/packages -DBUILDTREES_DIR=E:/vcpkg/clean/vcpkg/buildtrees -D_VCPKG_INSTALLED_DIR=E:/vcpkg/clean/vcpkg/installed -DDOWNLOADS=E:/vcpkg/clean/vcpkg/downloads -DVCPKG_MANIFEST_INSTALL=OFF -P "E:\vcpkg\clean\vcpkg\buildtrees\0.vcpkg_tags.cmake")
[DEBUG] cmd_execute_and_stream_data() returned 2 after 2570 us

[DEBUG] C:\A\1\60\s\src\vcpkg\cmakevars.cpp(178)
[DEBUG] Exiting after 186627 us (173362 us)

byronfa · 2021-08-11T16:07:50Z

I'm having this problem as well.. 'did anybody make any headway? I've got git in the path too, but it can't find it after forcing system binaries.

C:\git\vcpkg>where git
C:\Program Files\Git\cmd\git.exe

YenForYang · 2021-08-13T01:03:55Z

I had a similar type of issue with z_vcpkg_apply_patches.cmake:

...
CMake Error at scripts/cmake/z_vcpkg_apply_patches.cmake:38 (find_program):
  Could not find GIT using the following names: git, git.cmd
Call Stack (most recent call first):
  scripts/cmake/vcpkg_extract_source_archive.cmake:230 (z_vcpkg_apply_patches)
  scripts/cmake/vcpkg_extract_source_archive_ex.cmake:32 (vcpkg_extract_source_archive)
  scripts/cmake/vcpkg_from_github.cmake:180 (vcpkg_extract_source_archive_ex)
...

Managed to get git detection working by setting VCPKG_KEEP_ENV_VARS=PATH, but I doubt this requirement is intentional.

This got git and ninja detection working. 7-zip detection fails, however:

-- Performing post-build validation
-- Performing post-build validation done
A suitable version of 7zip was not found (required v19.0.0). Downloading portable 7zip v19.0.0...
Downloading 7zip...
  https://www.7-zip.org/a/7z1900-x64.msi -> Z:\vcpkg\downloads\7z1900-x64.msi
Extracting 7zip...

7z is in PATH. This is probably a different issue altogether.

NancyLi1013 · 2021-08-30T09:59:21Z

Currently, there are two cases about this issue:

It will fail like this Computing installation plan..., the details are from VCPKG_FORCE_SYSTEM_BINARIES cause vcpkg_apply_patches error on Windows #16216 (comment).
It will fail like this Could not find GIT using the following names: git, git.cmd, the details are from VCPKG_FORCE_SYSTEM_BINARIES cause vcpkg_apply_patches error on Windows #16216 (comment).

For case 1, the steps to reproduce:

Precondition: CMake has been installed to my local manually, but the path was not added to environment variables.

set VCPKG_FORCE_SYSTEM_BINARIES=1 in environment variables.
run .\vcpkg install sqlite3

For case 2, the steps to reproduce:

Precondition: CMake has been installed to my local manually(cmake >=3.21), the path C:\Program Files (x86)\CMake\bin was appended to Path environment variables.

set VCPKG_FORCE_SYSTEM_BINARIES=1 in environment variables.
run .\vcpkg install sqlite3

In both case 1 and case 2, git has been installed to the machine and the path C:\Program Files\Git\bin was also appended to Path environment variables.

PhoebeHui · 2021-08-30T10:13:36Z

Two issues here:

Set VCPKG_FORCE_SYSTEM_BINARIES=1, if the system doesn't install cmake, it should use the one downloaded by vcpkg instead, however it crashed now.
Set VCPKG_FORCE_SYSTEM_BINARIES=1, if the system installed git, it should use the one from system, however it can't find it from system.

BillyONeal · 2021-08-31T00:46:01Z

Paraphrasing @ras0219 , the right fix is to:

Remove this if check so that we always set GIT: https://github.com/microsoft/vcpkg-tool/blob/a23237293d0dfc8b9418c36cfc7fb3b98106167f/src/vcpkg/build.cpp#L632
Change tools.cpp to do find_from_PATH for the VCPKG_FORCE_SYSTEM_BINARIES in tools.cpp: https://github.com/microsoft/vcpkg-tool/blob/a23237293d0dfc8b9418c36cfc7fb3b98106167f/src/vcpkg/tools.cpp#L590

BillyONeal · 2022-05-19T22:53:34Z

Fixed in microsoft/vcpkg-tool#540

yekanchi · 2022-05-28T15:21:20Z

Fixed in microsoft/vcpkg-tool#540

problem still there. maybe merge is not released yet?

JackBoosY · 2022-05-30T02:36:26Z

@yekanchi Yeah, waiting for the next release.

@dg0yt

* Update vcpkg-tool to 2022-06-15. See https://github.com/microsoft/vcpkg-tool/releases/tag/2022-06-15 * Add `=pass` entries from microsoft/vcpkg-tool#579 * Cherry pick XUnit output from #23477 which hooks up microsoft/vcpkg-tool#585 (Thanks again @dg0yt !) * Update release process document now that we can do the full tree rebuild to test the tool release before signing. Thank you: @autoantwort @christophe-calmejane @JackBoosY @benmcmorran @hmoffatt @klalumiere @Thomas1664 @Neumann-A @AenBleidd for your contributions! Fixes #22686 . Fixes #21800 . Fixes #16216 . Fixes #15836 . Full tree rebuild testing this update is: https://dev.azure.com/vcpkg/public/_build/results?buildId=73664&view=results

JonLiu1993 assigned JackBoosY Feb 18, 2021

JackBoosY assigned PhoebeHui Feb 18, 2021

PhoebeHui added the requires:repro The issue is not currently repro-able label Feb 18, 2021

JackBoosY assigned ras0219-msft Feb 20, 2021

PhoebeHui added category:vcpkg-bug The issue is with the vcpkg system (including helper scripts in `scripts/cmake/`) and removed requires:repro The issue is not currently repro-able labels Feb 20, 2021

emmedve1 mentioned this issue Feb 20, 2021

x64-windows-static/boost-build: Applying fix_options.patch fails #15456

Closed

huangqinjin closed this as completed Feb 26, 2021

huangqinjin reopened this Feb 26, 2021

PhoebeHui mentioned this issue Sep 1, 2021

[vcpkg] Fix vcpkg crash and fail to apply patch issue when set VCPKG_FORCE_SYSTEM_BINARIES=1 microsoft/vcpkg-tool#181

Closed

JackBoosY mentioned this issue May 9, 2022

Output error text if cmake failed during triplet var detection microsoft/vcpkg-tool#536

Merged

BillyONeal added the requires:tool-release An issue that has been fixed in the microsoft/vcpkg-tool repo and is waiting for a release thereof label May 19, 2022

BillyONeal mentioned this issue Jun 15, 2022

Update vcpkg-tool to 2022-06-15 #25226

Merged

BillyONeal closed this as completed in #25226 Jun 15, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

VCPKG_FORCE_SYSTEM_BINARIES cause vcpkg_apply_patches error on Windows #16216

VCPKG_FORCE_SYSTEM_BINARIES cause vcpkg_apply_patches error on Windows #16216

huangqinjin commented Feb 14, 2021

StarGate-One commented Feb 14, 2021

huangqinjin commented Feb 15, 2021

StarGate-One commented Feb 15, 2021

PhoebeHui commented Feb 18, 2021

JackBoosY commented Feb 18, 2021

huangqinjin commented Feb 18, 2021

huangqinjin commented Feb 18, 2021

JackBoosY commented Feb 19, 2021

JackBoosY commented Feb 19, 2021

PhoebeHui commented Feb 19, 2021 •

edited

Loading

huangqinjin commented Feb 19, 2021 •

edited

Loading

JackBoosY commented Feb 20, 2021

PhoebeHui commented Feb 20, 2021 •

edited

Loading

JackBoosY commented Feb 20, 2021

PhoebeHui commented Feb 20, 2021

byronfa commented Aug 11, 2021

YenForYang commented Aug 13, 2021

NancyLi1013 commented Aug 30, 2021

PhoebeHui commented Aug 30, 2021

BillyONeal commented Aug 31, 2021

BillyONeal commented May 19, 2022

yekanchi commented May 28, 2022

JackBoosY commented May 30, 2022

VCPKG_FORCE_SYSTEM_BINARIES cause vcpkg_apply_patches error on Windows #16216

VCPKG_FORCE_SYSTEM_BINARIES cause vcpkg_apply_patches error on Windows #16216

Comments

huangqinjin commented Feb 14, 2021

StarGate-One commented Feb 14, 2021

huangqinjin commented Feb 15, 2021

StarGate-One commented Feb 15, 2021

PhoebeHui commented Feb 18, 2021

JackBoosY commented Feb 18, 2021

huangqinjin commented Feb 18, 2021

huangqinjin commented Feb 18, 2021

JackBoosY commented Feb 19, 2021

JackBoosY commented Feb 19, 2021

PhoebeHui commented Feb 19, 2021 • edited Loading

huangqinjin commented Feb 19, 2021 • edited Loading

JackBoosY commented Feb 20, 2021

PhoebeHui commented Feb 20, 2021 • edited Loading

JackBoosY commented Feb 20, 2021

PhoebeHui commented Feb 20, 2021

byronfa commented Aug 11, 2021

YenForYang commented Aug 13, 2021

NancyLi1013 commented Aug 30, 2021

PhoebeHui commented Aug 30, 2021

BillyONeal commented Aug 31, 2021

BillyONeal commented May 19, 2022

yekanchi commented May 28, 2022

JackBoosY commented May 30, 2022

PhoebeHui commented Feb 19, 2021 •

edited

Loading

huangqinjin commented Feb 19, 2021 •

edited

Loading

PhoebeHui commented Feb 20, 2021 •

edited

Loading