How do I change vcpkg source to include --editable even when a dependent port is not on command line?

[StarGate-One]

@JackBoosY cc: @strega-nil @ras0219-msft @ras0219

• How do I change vcpkg source to include --editable even when a dependent port is not on vcpkg install command line?
• When installing ports I always specfiy --editable on command line, but I have discovered when installing ports the --editable option only applies to ports on the command line, not any dependent ports that may need to be installed to support the core and/or features of said port.
• The reason I need all ports in editable is when dealing with financial/payroll data, the auditors request all source programs for review in order to certify them SOX (Sarbanes Oxley Action after the financial fallout in 2007/2008) compliant.
• Companies like Microsoft supply windows and Visual Studio and they are certified SOX compliant programs, but vcpkg is an open source project where anyone can submit changes to the code, so the project is not SOX compliant, thus requiring the technical auditors to review the code, by using automated routines against a copy of all sources to approve/disapprove SOX compliancy.

Examples on a clean vcpkg clone/instance - vanilla no customizations.

1. .\vcpkg.exe install --x-use-aria2 --recurse --keep-going --editable libpq[core,icu,nls,openssl,xml,zlib] results in this:

Computing installation plan...
The following packages will be built and installed:

• gettext[core]:x86-windows
• icu[core]:x86-windows
• libiconv[core]:x86-windows
• liblzma[core]:x86-windows
  libpq[core,icu,nls,openssl,xml,zlib]:x86-windows
• libxml2[core]:x86-windows
• openssl[core]:x86-windows
• zlib[core]:x86-windows
  Starting package 1/8: libiconv:x86-windows
  Building package libiconv[core]:x86-windows...
  -- Using cached D:/Git/vcpkg_x86/downloads/libiconv-1.16.tar.gz
  -- Extracting source D:/Git/vcpkg_x86/downloads/libiconv-1.16.tar.gz
  -- Applying patch 0002-Config-for-MSVC.patch
  -- Applying patch 0003-Add-export.patch
  -- Using source at D:/Git/vcpkg_x86/buildtrees/libiconv/src/1.16-de9101a181.clean
  ...

The install process does this for all the ports with * in front of them, only port libpq is installed in editable mode, but the command line:
2. .\vcpkg.exe install --x-use-aria2 --recurse --keep-going --editable gettext[core] icu[core] libiconv[core] liblzma[core] libpq[core,icu,nls,openssl,xml,zlib] libxml2[core] openssl[core] zlib[core] results in this:

Computing installation plan...
The following packages will be built and installed:
gettext[core]:x86-windows
icu[core]:x86-windows
libiconv[core]:x86-windows
liblzma[core]:x86-windows
libpq[core,icu,nls,openssl,xml,zlib]:x86-windows
libxml2[core]:x86-windows
openssl[core]:x86-windows
zlib[core]:x86-windows
Starting package 1/8: libiconv:x86-windows
Building package libiconv[core]:x86-windows...
-- Using cached D:/Git/vcpkg_x86/downloads/libiconv-1.16.tar.gz
-- Extracting source D:/Git/vcpkg_x86/downloads/libiconv-1.16.tar.gz
-- Applying patch 0002-Config-for-MSVC.patch
-- Applying patch 0003-Add-export.patch
-- Using source at D:/Git/vcpkg_x86/buildtrees/libiconv/src/1.16-de9101a181
...
The install process all ports libpq are installed in editable mode

• Bottom line, some of the ports install take hours to complete and if a port's dependencies change (as they do often) and I remove and install the updated port just to find out a new dependency I added was not on the command line, I have to redo the complete build so I get the source.
• This is also why I have so many ports where I modified the portfile.cmake where the function vcpkg_install_msbuild is used and adding the SKIP_CLEAN in order to keep the sources, example ports scintilla and tbb, as I need them for the SOX auditors.
• There is not much documentation on the editable option, except from the PR [vcpkg] Implement --editable #12200 [vcpkg] Implement --editable.

[strega-nil]

Hrm, this is definitely a thing we should do, then. I think adding --no-clean-after-build, perhaps? cc @ras0219, let's discuss this on Thursday.

[StarGate-One]

Hrm, this is definitely a thing we should do, then. I think adding --no-clean-after-build, perhaps? cc @ras0219, let's discuss this on Thursday.

I think this would be a great idea, a flag to not delete sources regardless of what build method is used to build/compile/link/install the port(s) would be fantastic, if possible and the time/effort would not be too great.

[strega-nil]

@StarGate-One , actually, after looking at the actual behavior, it seems I was wrong -- I think the existing behavior is such that we keep the source around; for example:

> C:\..\projects\work\vcpkg> .\vcpkg install zlib:x64-windows
Computing installation plan...
The following packages will be built and installed:
    zlib[core]:x64-windows
Detecting compiler hash for triplet x64-windows...
Could not locate cached archive: C:\Users\Nicole\AppData\Local\vcpkg\archives\ab\abf03ea09ea8bced3e7b011e4d7c6f4b9a6c179e.zip
Starting package 1/1: zlib:x64-windows
Building package zlib[core]:x64-windows...
-- Using cached C:/Users/Nicole/Documents/projects/work/vcpkg/downloads/zlib1211.tar.gz
-- Cleaning sources at C:/Users/Nicole/Documents/projects/work/vcpkg/buildtrees/zlib/src/1.2.11-63309e48e5.clean. Use --editable to skip cleaning for the packages you specify.
-- Extracting source C:/Users/Nicole/Documents/projects/work/vcpkg/downloads/zlib1211.tar.gz
-- Applying patch cmake_dont_build_more_than_needed.patch
-- Applying patch 0001-Prevent-invalid-inclusions-when-HAVE_-is-set-to-0.patch
-- Applying patch add_debug_postfix_on_mingw.patch
-- Using source at C:/Users/Nicole/Documents/projects/work/vcpkg/buildtrees/zlib/src/1.2.11-63309e48e5.clean
-- Configuring x64-windows
-- Building x64-windows-dbg
-- Building x64-windows-rel
-- Using msys root at C:/Users/Nicole/Documents/projects/work/vcpkg/downloads/tools/msys2/c809757c94447846
-- Fixing pkgconfig file: C:/Users/Nicole/Documents/projects/work/vcpkg/packages/zlib_x64-windows/lib/pkgconfig/zlib.pc
-- Fixing pkgconfig file: C:/Users/Nicole/Documents/projects/work/vcpkg/packages/zlib_x64-windows/debug/lib/pkgconfig/zlib.pc
-- Installing: C:/Users/Nicole/Documents/projects/work/vcpkg/packages/zlib_x64-windows/share/zlib/copyright
-- Performing post-build validation
-- Performing post-build validation done
Stored binary cache: C:\Users\Nicole\AppData\Local\vcpkg\archives\ab\abf03ea09ea8bced3e7b011e4d7c6f4b9a6c179e.zip
Building package zlib[core]:x64-windows... done
Installing package zlib[core]:x64-windows...
Installing package zlib[core]:x64-windows... done
Elapsed time for package zlib:x64-windows: 5.814 s

Total elapsed time: 9.122 s

The package zlib is compatible with built-in CMake targets:

    find_package(ZLIB REQUIRED)
    target_link_libraries(main PRIVATE ZLIB::ZLIB)
> dir C:\..\projects\work\vcpkg> dir .\buildtrees\zlib\src\1.2.11-63309e48e5.clean\


    Directory: C:\Users\Nicole\Documents\projects\work\vcpkg\buildtrees\zlib\src\1.2.11-63309e48e5.clean

Mode                 LastWriteTime         Length Name
----                 -------------         ------ ----
d----          2010-02-13    16:22                amiga
d----          2017-01-15    09:29                contrib
d----          2011-09-10    15:08                doc
d----          2016-12-31    23:37                examples
d----          2015-07-28    22:44                msdos
d----          2009-12-20    22:06                nintendods
d----          2015-07-28    22:44                old
d----          2017-01-15    09:29                os400
d----          2017-01-15    09:29                qnx
d----          2016-12-31    23:37                test
d----          2010-02-13    16:41                watcom
d----          2017-01-15    09:29                win32
-a---          2016-12-31    23:37           5204 adler32.c
-a---          2017-01-15    09:29          78553 ChangeLog
-a---          2020-11-19    14:52           8163 CMakeLists.txt
-a---          2016-12-31    23:37           2699 compress.c
-a---          2016-12-31    10:06          28244 configure
-a---          2016-12-31    23:37          14053 crc32.c
-a---          2012-04-29    16:18          30562 crc32.h
-a---          2017-01-15    09:29          78889 deflate.c
-a---          2016-12-31    23:37          13150 deflate.h
-a---          2011-11-27    14:15          16573 FAQ
-a---          2010-02-13    16:12            678 gzclose.c
-a---          2016-12-31    23:37           6819 gzguts.h
-a---          2017-01-15    09:29          16599 gzlib.c
-a---          2016-12-31    23:37          20428 gzread.c
-a---          2017-01-15    09:29          19253 gzwrite.c
-a---          2012-03-11    10:16           1988 INDEX
-a---          2016-12-31    23:37          22715 infback.c
-a---          2017-01-15    09:29          12978 inffast.c
-a---          2010-04-18    21:16            427 inffast.h
-a---          2011-10-05    20:32           6332 inffixed.h
-a---          2016-12-31    23:37          54800 inflate.c
-a---          2016-12-31    23:37           6618 inflate.h
-a---          2017-01-15    09:29          12999 inftrees.c
-a---          2010-04-18    21:15           2928 inftrees.h
-a---          2012-03-09    18:43          26402 make_vms.com
-a---          2011-09-09    23:33            100 Makefile
-a---          2017-01-15    09:29          13680 Makefile.in
-a---          2017-01-15    09:29           5187 README
-a---          2017-01-15    09:29           3142 treebuild.xml
-a---          2017-01-15    09:07          43761 trees.c
-a---          2010-04-18    10:32           8472 trees.h
-a---          2016-12-31    08:49           2966 uncompr.c
-a---          2020-11-19    14:52          16607 zconf.h.cmakein
-a---          2020-11-19    14:52          16556 zconf.h.in
-a---          2017-01-15    09:29           4477 zlib.3
-a---          2017-01-15    09:29          19318 zlib.3.pdf
-a---          2017-01-15    09:29          96239 zlib.h
-a---          2016-12-31    23:37           1457 zlib.map
-a---          2012-03-11    10:16            294 zlib.pc.cmakein
-a---          2010-04-17    23:48            254 zlib.pc.in
-a---          2006-10-14    09:45           3895 zlib2ansi
-a---          2017-01-15    09:29           7304 zutil.c
-a---          2016-12-31    23:37           7127 zutil.h

as you can see, the source code is still here. Is there anything else you need?

[StarGate-One]

Hmm, I must be losing it 🤔😢 😒
I spent days trying to it so it would not delete the sources, nothing I tried worked, it kept deleting them. Now it seems I can reproduce it deleting the sources.
Thank you for taking a look, I apologize for wasting everyone's time.👍👍✔