[liblzma] Unable to download "xz" caused install falied

[zhang49]

D:\vcpkg>vcpkg.exe install opencv2
warning: In the September 2023 release, the default triplet for vcpkg libraries changed from x86-windows to the detected host triplet (x64-windows). For the old behavior, add --triplet x86-windows . To suppress this message, add --triplet x64-windows .
Computing installation plan...
The following packages will be built and installed:

• liblzma:x64-windows -> 5.4.4
• libpng:x64-windows -> 1.6.39#1
  opencv2[core,default-features,eigen,jpeg,png,tiff]:x64-windows -> 2.4.13.7#19
• tiff[core,jpeg,lzma,zip]:x64-windows -> 4.6.0#1
• vcpkg-get-python-packages:x64-windows -> 2023-07-28
• zlib:x64-windows -> 1.3
  Additional packages (*) will be modified to complete this operation.
  Detecting compiler hash for triplet x64-windows...
  Restored 0 package(s) from C:\Users\heyy\AppData\Local\vcpkg\archives in 141 us. Use --debug to see more details.
  Installing 1/6 liblzma:x64-windows...
  Building liblzma:x64-windows...
  -- Downloading https://github.com/tukaani-project/xz/archive/v5.4.4.tar.gz -> tukaani-project-xz-v5.4.4.tar.gz...
  [DEBUG] To include the environment variables in debug output, pass --debug-env
  [DEBUG] Trying to load bundleconfig from D:\vcpkg\vcpkg-bundle.json
  [DEBUG] Failed to open: D:\vcpkg\vcpkg-bundle.json
  [DEBUG] Bundle config: readonly=false, usegitregistry=false, embeddedsha=nullopt, deployment=Git, vsversion=nullopt
  [DEBUG] Metrics enabled.
  [DEBUG] Feature flag 'binarycaching' unset
  [DEBUG] Feature flag 'compilertracking' unset
  [DEBUG] Feature flag 'registries' unset
  [DEBUG] Feature flag 'versions' unset
  [DEBUG] Feature flag 'dependencygraph' unset
  Downloading https://github.com/tukaani-project/xz/archive/v5.4.4.tar.gz
  [DEBUG] Trying to hash D:\vcpkg\downloads\tukaani-project-xz-v5.4.4.tar.gz.16680.part
  [DEBUG] D:\vcpkg\downloads\tukaani-project-xz-v5.4.4.tar.gz.16680.part has hash 77255aac06aad2d93e1778cd9ca3a7df2da6c60b362d12e764055e83eb84b8f29c45b150a9818aa251c82af62233841db10ae6e4a076d6a6a007a60a5a6227a6
  error: Failed to download from mirror set
  error: File does not have the expected hash:
  url: https://github.com/tukaani-project/xz/archive/v5.4.4.tar.gz
  File: D:\vcpkg\downloads\tukaani-project-xz-v5.4.4.tar.gz.16680.part
  Expected hash: c28461123562564e030f3f733f078bc4c840e87598d9f4b718d4bca639120d8133f969c45d7bdc62f33f081d789ec0f14a1791fb7da18515682bfe3c0c7362e0
  Actual hash: 77255aac06aad2d93e1778cd9ca3a7df2da6c60b362d12e764055e83eb84b8f29c45b150a9818aa251c82af62233841db10ae6e4a076d6a6a007a60a5a6227a6
  [DEBUG] D:\a_work\1\s\src\vcpkg\base\downloads.cpp(1051):
  [DEBUG] Time in subprocesses: 0us
  [DEBUG] Time in parsing JSON: 5us
  [DEBUG] Time in JSON reader: 0us
  [DEBUG] Time in filesystem: 729us
  [DEBUG] Time in loading ports: 0us
  [DEBUG] Exiting after 3.1 s (3092955us)

CMake Error at scripts/cmake/vcpkg_download_distfile.cmake:32 (message):

  Failed to download file with error: 1
  If you are using a proxy, please check your proxy setting. Possible causes are:

  1. You are actually using an HTTP proxy, but setting HTTPS_PROXY variable
     to `https://address:port`. This is not correct, because `https://` prefix
     claims the proxy is an HTTPS proxy, while your proxy (v2ray, shadowsocksr
     , etc..) is an HTTP proxy. Try setting `http://address:port` to both
     HTTP_PROXY and HTTPS_PROXY instead.

  2. If you are using Windows, vcpkg will automatically use your Windows IE Proxy Settings
     set by your proxy software. See https://github.com/microsoft/vcpkg-tool/pull/77
     The value set by your proxy might be wrong, or have same `https://` prefix issue.

  3. Your proxy's remote server is out of service.

  If you've tried directly download the link, and believe this is not a temporary
  download server failure, please submit an issue at https://github.com/Microsoft/vcpkg/issues
  to report this upstream download server failure.

Call Stack (most recent call first):
scripts/cmake/vcpkg_download_distfile.cmake:270 (z_vcpkg_download_distfile_show_proxy_and_fail)
scripts/cmake/vcpkg_from_github.cmake:106 (vcpkg_download_distfile)
ports/liblzma/portfile.cmake:1 (vcpkg_from_github)
scripts/ports.cmake:168 (include)

error: building liblzma:x64-windows failed with: BUILD_FAILED
Elapsed time to handle liblzma:x64-windows: 3.2 s
Please ensure you're using the latest port files with git pull and vcpkg update.
Then check for known issues at:
https://github.com/microsoft/vcpkg/issues?q=is%3Aissue+is%3Aopen+in%3Atitle+liblzma
You can submit a new issue at:
https://github.com/microsoft/vcpkg/issues/new?title=[liblzma]+Build+error&body=Copy+issue+body+from+D%3A%2Fvcpkg%2Finstalled%2Fvcpkg%2Fissue_body.md

[zhang49]

how to solve this problem?

[Osyotr]

#37839

[gracicot]

Is it possible to create a new tag for vcpkg with liblzma downgraded? I'm currently waiting on a tag to upgrade vcpkg on nixpkgs.

[dg0yt]

Just choose a commit you trust. AFAICT tags have no meaning with regard to maturity here. They are updated so that nobody downloads even older tags.
And of course if users don't have a historical xz-utils tarball already, reverse dependencies of liblzma won't build.

[jimwang118]

@zhang49 Can you successfully install opencv2 after updating the vcpkg library?

[rezzeted]

@dg0yt The official xz repository has been turned off. Is it possible to have a temporary solution for the vcpkg maintainers to accept the path to use the bminor/xz mirror repository?

[dg0yt]

@rezzeted I'm only a community member... You would better ask in the PR which rolled back the port version (#37841).

If the SHA512 of the other repo matches, you can prefetch the tarball and put it to a local asset source or the downloads directory.

[ijean]

Hello,
Any solution ? There are a lot of closed issues that point to #37893 , but I see no solution here.

[blessio]

@dg0yt The official xz repository has been turned off. Is it possible to have a temporary solution for the vcpkg maintainers to accept the path to use the bminor/xz mirror repository?

This is exactly what is done here: (#37839 (comment)).

You have to not only reinstall the library as explained in the above link but also

1. clear the vcpkg cache ( on windows this is to delete C:\Users<your-user>\AppData\Local\vcpkg\archives - on other platforms please search the internet )
2. in VCPKG_ROOT\buildtrees\versioning_\versions\liblzma<any guild> update the file as explained in the link from above ([liblzma] port uses compromised version #37839 (comment)).

[dsvensson]

It would be great if the freeimage port gained a feature flag to disable the transitive lzma dependency of tiff. Lots of projects can probably survive without it.

[AbdulsalamAmin]

The file is available here

https://sourceforge.net/projects/lzmautils/files/xz-5.4.4.tar.gz/download, but it cannot be installed manually due to a different hash

[LilyWangLL]

The file is available here https://sourceforge.net/projects/lzmautils/files/xz-5.4.4.tar.gz/download, but it cannot be installed manually due to a different hash

cc @JavierMatosD @BillyONeal, can we switch to this URL https://sourceforge.net/projects/lzmautils?

[rjindael]

This has affected our organizations development workflow significantly. Pending a permanent fix, here is a rudimentary patch that seems to work for the most part:

1. Download a tarball of xz v5.4.4 - this is the one vcpkg is currently using, seeing as subsequent versions contain a backdoor. I downloaded mine from the Internet Archive's Wayback Machine. Link: https://web.archive.org/web/20240329183646if_/https://codeload.github.com/tukaani-project/xz/tar.gz/refs/tags/v5.4.4
2. Rename the file to tukaani-project-xz-v5.4.4.tar.gz
3. Place it in <vcpkg_root>/downloads (e.g. C:\vcpkg\downloads)
4. Clear the z_vcpkg_download_distfile_show_proxy_and_fail function in <vcpkg_root>/scripts/cmake/vcpkg_download_distfile.cmake so that it does nothing. From what I have observed, this change seems to be harmless since even if a download does fail it will still eventually error since it cannot find the file for extraction. However, by emptying this function, this bypasses the error caused by not being able to download the xz source tarball.

I haven't been able to figure out how to implement this in our CI/CD solutions, but this serves as a functioning rudimentary patch.

[BillyONeal]

#37841 (comment) The repo should be public again.

[Neustradamus]

@teo-tsirpanis has done a PR about the new XZ version (5.6.2) here:

• [liblzma] Update to version 5.6.2. #39024

cc: @carsten-grimm.

Linked to:

• [liblzma] Unable to download "xz" caused install falied #37893
• Revert [liblzma] update to version 5.6.0 (#37199) #37841
• [liblzma] port uses compromised version #37839
• [liblzma] update to version 5.6.0 #37199
• [liblzma] update to 5.6.0 #37197
• [liblzma] update to version 5.4.4 #34306

Official XZ links:

• https://tukaani.org/
• https://tukaani.org/xz/
• https://tukaani.org/xz/#_stable
• https://tukaani.org/xz-backdoor/
• https://github.com/tukaani-project/
• https://github.com/tukaani-project/xz