Merge bitcoin#17416: Appveyor improvement - text file for vcpkg package list

29eb039 Moves vcpkg list to a text file and updates the appveyor job and readme to use it. (Aaron Clauson)

Pull request description:

  bitcoin#17364 attempted to save a couple of minutes by skipping the `vcpkg` steps if the vcpkg install directory was already cached.

  The discussion in bitcoin#15382 highlights the approach used in bitcoin#17364 does not accommodate adding a new package.

  ~~This PR improves the approach to individually check whether  each vcpg package is installed rather than checking for the existence of the vcpkg install directory.~~

  This PR moves the list of required vcpkg packages into a separate file and uses changes to that file to invalidate the appveyor cache. Whenever the cache is invalidated the vcpkg sources will be updated, the vcpkg binary built and the required packages installed from the latest port files.

ACKs for top commit:
  MarcoFalke:
    ACK 29eb039

Tree-SHA512: 0c2a170f4e4b47ca0f9cef14f1e3892001b441a6d84f50bf5fd8a26bc4cdbd9358dfce7ef180d37150262e849650e9857d6b2bcd686964b963c3de6cd708a2f3

Author: MarcoFalke

.appveyor.yml

@@ -7,41 +7,42 @@ clone_depth: 5
 environment:
   APPVEYOR_SAVE_CACHE_ON_ERROR: true
   CLCACHE_SERVER: 1
-  PACKAGES: berkeleydb boost-filesystem boost-multi-index boost-signals2 boost-test boost-thread libevent openssl rapidcheck zeromq double-conversion
   PATH: 'C:\Python37-x64;C:\Python37-x64\Scripts;%PATH%'
   PYTHONUTF8: 1
   QT_DOWNLOAD_URL: 'https://github.com/sipsorcery/qt_win_binary/releases/download/v1.4/Qt5.9.8_x64_static_vs2019.zip'
   QT_DOWNLOAD_HASH: 'f285cbb02bec3b3f3cc2621e3fa7d5edf0d6a66fa30c57859e583acda954ea80'
   QT_LOCAL_PATH: 'C:\Qt5.9.8_x64_static_vs2019'
   VCPKG_INSTALL_PATH: 'C:\tools\vcpkg\installed'
 cache:
-- C:\tools\vcpkg\installed
+- C:\tools\vcpkg\installed -> build_msvc\vcpkg-packages.txt
 - C:\Users\appveyor\clcache -> .appveyor.yml, build_msvc\**, **\Makefile.am, **\*.vcxproj.in
 - C:\Qt5.9.8_x64_static_vs2019
 install:
 - cmd: pip install --quiet git+https://github.com/frerich/clcache.git@v4.2.0
 # Disable zmq test for now since python zmq library on Windows would cause Access violation sometimes.
 # - cmd: pip install zmq
 # Powershell block below is to install the c++ dependencies via vcpkg. The pseudo code is:
-# 1. If the vcpkg install directory exists assume dependencies are installed and do nothing. To
-#    force a fresh install of the packages delete the job's appveyor cache.
-# 2. Otherwise:
+# 1. Check whether the vcpkg install directory exists (note that updating the vcpkg-packages.txt file
+#    will cause the appveyor cache rules to invalidate the directory)
+# 2. If the directory is missing:
 #    a. Update the vcpkg source (including port files) and build the vcpkg binary,
-#    b. Install the required packages.
+#    b. Install the missing packages.
 - ps: |
-      cd c:\tools\vcpkg
+      $env:PACKAGES = Get-Content -Path build_msvc\vcpkg-packages.txt
+      Write-Host "vcpkg list: $env:PACKAGES"
       if(!(Test-Path -Path ($env:VCPKG_INSTALL_PATH))) {
+          cd c:\tools\vcpkg
           $env:GIT_REDIRECT_STDERR = '2>&1' # git is writing non-errors to STDERR when doing git pull. Send to STDOUT instead.
-          Add-Content "C:\tools\vcpkg\triplets\$env:PLATFORM-windows-static.cmake" "set(VCPKG_BUILD_TYPE release)"
           git pull origin master
           .\bootstrap-vcpkg.bat
+          Add-Content "C:\tools\vcpkg\triplets\$env:PLATFORM-windows-static.cmake" "set(VCPKG_BUILD_TYPE release)"
           .\vcpkg install --triplet $env:PLATFORM-windows-static $env:PACKAGES.split() > $null
+          cd "$env:APPVEYOR_BUILD_FOLDER"
       }
       else {
-        Write-Host "vcpkg packages already installed (to reinstall purge appveyor job's cache)."
+        Write-Host "required vcpkg packages already installed."
       }
-      .\vcpkg integrate install
-      cd "$env:APPVEYOR_BUILD_FOLDER"
+      c:\tools\vcpkg\vcpkg integrate install
 before_build:
 - ps:  clcache -M 536870912
 # Powershell block below is to download and extract the Qt static libraries. The pseudo code is:

build_msvc/README.md

@@ -51,10 +51,10 @@ Building
 The instructions below use `vcpkg` to install the dependencies.
 
 - Install [`vcpkg`](https://github.com/Microsoft/vcpkg).
-- Install the required packages (replace x64 with x86 as required):
+- Install the required packages (replace x64 with x86 as required). The list of required packages can be found in the `build_msvc\vcpkg-packages.txt` file. The PowerShell command below will work if run from the repository root directory and `vcpkg` is in the path. Alternatively the contents of the packages text file can be pasted in place of the `Get-Content` cmdlet.
 
 ```
-PS >.\vcpkg install --triplet x64-windows-static boost-filesystem boost-multi-index boost-signals2 boost-test boost-thread libevent openssl zeromq berkeleydb rapidcheck double-conversion
+PS >.\vcpkg install --triplet x64-windows-static $(Get-Content -Path build_msvc\vcpkg-packages.txt).split()
 ```
 
 - Use Python to generate `*.vcxproj` from Makefile
@@ -68,13 +68,13 @@ PS >py -3 msvc-autogen.py
 - Build with Visual Studio 2017 or msbuild.
 
 ```
-msbuild /m bitcoin.sln /p:Platform=x64 /p:Configuration=Release /t:build
+msbuild /m bitcoin.sln /p:Platform=x64 /p:Configuration=Release /p:PlatformToolset=v141 /t:build
 ```
 
 - Build with Visual Studio 2019 or msbuild.
 
 ```
-msbuild /m bitcoin.sln /p:Platform=x64 /p:Configuration=Release /p:PlatformToolset=v142 /t:build
+msbuild /m bitcoin.sln /p:Platform=x64 /p:Configuration=Release /t:build
 ```
 
 AppVeyor

build_msvc/vcpkg-packages.txt

@@ -0,0 +1 @@
+berkeleydb boost-filesystem boost-multi-index boost-signals2 boost-test boost-thread libevent openssl rapidcheck zeromq double-conversion