Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[vcpkg.cmake] fix Neumann-A problem with find_package #17955

Merged
merged 1 commit into from
May 17, 2021

Conversation

strega-nil
Copy link
Contributor

Basically, traces with vcpkg.cmake are way too long; this halves or quarters the size of traces (and additionally makes some minor improvements like "working automatically on arm64-osx")

@strega-nil-ms strega-nil-ms added category:code-cleanup This PR cleans up code, without fixing any existing bugs nor adding any features. info:world-rebuild info:internal This PR or Issue was filed by the vcpkg team. labels May 15, 2021
Comment on lines +690 to +695
function("${VCPKG_OVERRIDE_FIND_PACKAGE_NAME}" z_vcpkg_fp_package_name)
# these variables, as well as any variables matching `z_vcpkg_fp_*`,
# are not reexported to parent scope
set(z_vcpkg_fp_ignored_variables
"^(ARGS|z_vcpkg_fp_.*|CMAKE_FIND_ROOT_PATH|Boost_USE_STATIC_LIBS|Boost_USE_MULTITHREADED|Boost_USE_STATIC_RUNTIME|Boost_NO_BOOST_CMAKE|Boost_COMPILER)$")

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this should be a macro instead. You can backup the ignored variables and restore them afterwards if necessary or unset them. In the end you are forwarding all the variables any way so why not make it a macro?

else()
list(FILTER z_vcpkg_fp_variable_list EXCLUDE "${z_vcpkg_fp_ignored_variables}")
foreach(z_vcpkg_fp_variable IN LISTS z_vcpkg_fp_variable_list)
set("${z_vcpkg_fp_variable}" "${${z_vcpkg_fp_variable}}" PARENT_SCOPE)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The problem whatever ${${z_vcpkg_fp_variable}} is determines the final size of the log. If it is a very big string it will be set numerous times if it is a nested find_package call. If it would be a macro this would not happen since you don't need to forward!

@Neumann-A
Copy link
Contributor

Neumann-A commented May 15, 2021

Just for metrics:
qt6 pr: qttools with --trace-expand:
Log Size:

  • before: ~ 5 GB
  • after: ~ 550 MB
  • macro(find_package) hack without backup vars: ~42 MB

@Neumann-A
Copy link
Contributor

Just to give an example:

E:/vcpkg_folders/qt6/scripts/buildsystems/vcpkg.cmake(768):  set(Qt6_FIND_VERSION_TWEAK 0 PARENT_SCOPE )
E:/vcpkg_folders/qt6/scripts/buildsystems/vcpkg.cmake(765):  if(z_vcpkg_fp_variable MATCHES ^(ARGS|z_vcpkg_fp_.*|CMAKE_FIND_ROOT_PATH|Boost_USE_STATIC_LIBS|Boost_USE_MULTITHREADED|Boost_USE_STATIC_RUNTIME|Boost_NO_BOOST_CMAKE|Boost_COMPILER)$ )
E:/vcpkg_folders/qt6/scripts/buildsystems/vcpkg.cmake(768):  set(Qt6_FOUND 1 PARENT_SCOPE )
E:/vcpkg_folders/qt6/scripts/buildsystems/vcpkg.cmake(765):  if(z_vcpkg_fp_variable MATCHES ^(ARGS|z_vcpkg_fp_.*|CMAKE_FIND_ROOT_PATH|Boost_USE_STATIC_LIBS|Boost_USE_MULTITHREADED|Boost_USE_STATIC_RUNTIME|Boost_NO_BOOST_CMAKE|Boost_COMPILER)$ )
E:/vcpkg_folders/qt6/scripts/buildsystems/vcpkg.cmake(768):  set(Qt6_VERSION 6.1.0 PARENT_SCOPE )
E:/vcpkg_folders/qt6/scripts/buildsystems/vcpkg.cmake(765):  if(z_vcpkg_fp_variable MATCHES ^(ARGS|z_vcpkg_fp_.*|CMAKE_FIND_ROOT_PATH|Boost_USE_STATIC_LIBS|Boost_USE_MULTITHREADED|Boost_USE_STATIC_RUNTIME|Boost_NO_BOOST_CMAKE|Boost_COMPILER)$ )
E:/vcpkg_folders/qt6/scripts/buildsystems/vcpkg.cmake(768):  set(Qt6_VERSION_COUNT 3 PARENT_SCOPE )
E:/vcpkg_folders/qt6/scripts/buildsystems/vcpkg.cmake(765):  if(z_vcpkg_fp_variable MATCHES ^(ARGS|z_vcpkg_fp_.*|CMAKE_FIND_ROOT_PATH|Boost_USE_STATIC_LIBS|Boost_USE_MULTITHREADED|Boost_USE_STATIC_RUNTIME|Boost_NO_BOOST_CMAKE|Boost_COMPILER)$ )
E:/vcpkg_folders/qt6/scripts/buildsystems/vcpkg.cmake(768):  set(Qt6_VERSION_MAJOR 6 PARENT_SCOPE )
E:/vcpkg_folders/qt6/scripts/buildsystems/vcpkg.cmake(765):  if(z_vcpkg_fp_variable MATCHES ^(ARGS|z_vcpkg_fp_.*|CMAKE_FIND_ROOT_PATH|Boost_USE_STATIC_LIBS|Boost_USE_MULTITHREADED|Boost_USE_STATIC_RUNTIME|Boost_NO_BOOST_CMAKE|Boost_COMPILER)$ )
E:/vcpkg_folders/qt6/scripts/buildsystems/vcpkg.cmake(768):  set(Qt6_VERSION_MINOR 1 PARENT_SCOPE )
E:/vcpkg_folders/qt6/scripts/buildsystems/vcpkg.cmake(765):  if(z_vcpkg_fp_variable MATCHES ^(ARGS|z_vcpkg_fp_.*|CMAKE_FIND_ROOT_PATH|Boost_USE_STATIC_LIBS|Boost_USE_MULTITHREADED|Boost_USE_STATIC_RUNTIME|Boost_NO_BOOST_CMAKE|Boost_COMPILER)$ )
E:/vcpkg_folders/qt6/scripts/buildsystems/vcpkg.cmake(768):  set(Qt6_VERSION_PATCH 0 PARENT_SCOPE )
E:/vcpkg_folders/qt6/scripts/buildsystems/vcpkg.cmake(765):  if(z_vcpkg_fp_variable MATCHES ^(ARGS|z_vcpkg_fp_.*|CMAKE_FIND_ROOT_PATH|Boost_USE_STATIC_LIBS|Boost_USE_MULTITHREADED|Boost_USE_STATIC_RUNTIME|Boost_NO_BOOST_CMAKE|Boost_COMPILER)$ )
E:/vcpkg_folders/qt6/scripts/buildsystems/vcpkg.cmake(768):  set(Qt6_VERSION_TWEAK 0 PARENT_SCOPE )
E:/vcpkg_folders/qt6/scripts/buildsystems/vcpkg.cmake(765):  if(z_vcpkg_fp_variable MATCHES ^(ARGS|z_vcpkg_fp_.*|CMAKE_FIND_ROOT_PATH|Boost_USE_STATIC_LIBS|Boost_USE_MULTITHREADED|Boost_USE_STATIC_RUNTIME|Boost_NO_BOOST_CMAKE|Boost_COMPILER)$ )
E:/vcpkg_folders/qt6/scripts/buildsystems/vcpkg.cmake(768):  set(QtTools_BINARY_DIR E:/vcpkg_folders/qt6/buildtrees/qttools/x64-windows-static-dbg PARENT_SCOPE )
E:/vcpkg_folders/qt6/scripts/buildsystems/vcpkg.cmake(765):  if(z_vcpkg_fp_variable MATCHES ^(ARGS|z_vcpkg_fp_.*|CMAKE_FIND_ROOT_PATH|Boost_USE_STATIC_LIBS|Boost_USE_MULTITHREADED|Boost_USE_STATIC_RUNTIME|Boost_NO_BOOST_CMAKE|Boost_COMPILER)$ )
E:/vcpkg_folders/qt6/scripts/buildsystems/vcpkg.cmake(768):  set(QtTools_BINARY_DIR E:/vcpkg_folders/qt6/buildtrees/qttools/x64-windows-static-dbg PARENT_SCOPE )
E:/vcpkg_folders/qt6/scripts/buildsystems/vcpkg.cmake(765):  if(z_vcpkg_fp_variable MATCHES ^(ARGS|z_vcpkg_fp_.*|CMAKE_FIND_ROOT_PATH|Boost_USE_STATIC_LIBS|Boost_USE_MULTITHREADED|Boost_USE_STATIC_RUNTIME|Boost_NO_BOOST_CMAKE|Boost_COMPILER)$ )
E:/vcpkg_folders/qt6/scripts/buildsystems/vcpkg.cmake(768):  set(QtTools_DESCRIPTION Qt Tools PARENT_SCOPE )
E:/vcpkg_folders/qt6/scripts/buildsystems/vcpkg.cmake(765):  if(z_vcpkg_fp_variable MATCHES ^(ARGS|z_vcpkg_fp_.*|CMAKE_FIND_ROOT_PATH|Boost_USE_STATIC_LIBS|Boost_USE_MULTITHREADED|Boost_USE_STATIC_RUNTIME|Boost_NO_BOOST_CMAKE|Boost_COMPILER)$ )
E:/vcpkg_folders/qt6/scripts/buildsystems/vcpkg.cmake(768):  set(QtTools_HOMEPAGE_URL https://qt.io/ PARENT_SCOPE )
E:/vcpkg_folders/qt6/scripts/buildsystems/vcpkg.cmake(765):  if(z_vcpkg_fp_variable MATCHES ^(ARGS|z_vcpkg_fp_.*|CMAKE_FIND_ROOT_PATH|Boost_USE_STATIC_LIBS|Boost_USE_MULTITHREADED|Boost_USE_STATIC_RUNTIME|Boost_NO_BOOST_CMAKE|Boost_COMPILER)$ )
E:/vcpkg_folders/qt6/scripts/buildsystems/vcpkg.cmake(768):  set(QtTools_SOURCE_DIR E:/vcpkg_folders/qt6/buildtrees/qttools/src/v6.1.0-ea8bcb428c.clean PARENT_SCOPE )
E:/vcpkg_folders/qt6/scripts/buildsystems/vcpkg.cmake(765):  if(z_vcpkg_fp_variable MATCHES ^(ARGS|z_vcpkg_fp_.*|CMAKE_FIND_ROOT_PATH|Boost_USE_STATIC_LIBS|Boost_USE_MULTITHREADED|Boost_USE_STATIC_RUNTIME|Boost_NO_BOOST_CMAKE|Boost_COMPILER)$ )
E:/vcpkg_folders/qt6/scripts/buildsystems/vcpkg.cmake(768):  set(QtTools_SOURCE_DIR E:/vcpkg_folders/qt6/buildtrees/qttools/src/v6.1.0-ea8bcb428c.clean PARENT_SCOPE )
E:/vcpkg_folders/qt6/scripts/buildsystems/vcpkg.cmake(765):  if(z_vcpkg_fp_variable MATCHES ^(ARGS|z_vcpkg_fp_.*|CMAKE_FIND_ROOT_PATH|Boost_USE_STATIC_LIBS|Boost_USE_MULTITHREADED|Boost_USE_STATIC_RUNTIME|Boost_NO_BOOST_CMAKE|Boost_COMPILER)$ )
E:/vcpkg_folders/qt6/scripts/buildsystems/vcpkg.cmake(768):  set(QtTools_VERSION 6.1.0 PARENT_SCOPE )
E:/vcpkg_folders/qt6/scripts/buildsystems/vcpkg.cmake(765):  if(z_vcpkg_fp_variable MATCHES ^(ARGS|z_vcpkg_fp_.*|CMAKE_FIND_ROOT_PATH|Boost_USE_STATIC_LIBS|Boost_USE_MULTITHREADED|Boost_USE_STATIC_RUNTIME|Boost_NO_BOOST_CMAKE|Boost_COMPILER)$ )
E:/vcpkg_folders/qt6/scripts/buildsystems/vcpkg.cmake(768):  set(QtTools_VERSION_MAJOR 6 PARENT_SCOPE )
E:/vcpkg_folders/qt6/scripts/buildsystems/vcpkg.cmake(765):  if(z_vcpkg_fp_variable MATCHES ^(ARGS|z_vcpkg_fp_.*|CMAKE_FIND_ROOT_PATH|Boost_USE_STATIC_LIBS|Boost_USE_MULTITHREADED|Boost_USE_STATIC_RUNTIME|Boost_NO_BOOST_CMAKE|Boost_COMPILER)$ )
E:/vcpkg_folders/qt6/scripts/buildsystems/vcpkg.cmake(768):  set(QtTools_VERSION_MINOR 1 PARENT_SCOPE )

and there are like screens over screens of those settings in the log.

@strega-nil-ms strega-nil-ms merged commit b369b11 into microsoft:master May 17, 2021
@PhoebeHui
Copy link
Contributor

@strega-nil, we saw the opencv4 failure starts from this changes, related PR #17983

Could you please take a look?

CMake Error at cmake/OpenCVGenConfig.cmake:49 (export):
  export Export set "OpenCVModules" not found.
Call Stack (most recent call first):
  CMakeLists.txt:923 (include)

@Neumann-A
Copy link
Contributor

hmm CI seems to not have rebuild the world. Took only 3 mins.

strega-nil added a commit to strega-nil/vcpkg that referenced this pull request May 18, 2021
…oft#17955)"

This reverts commit b369b11.

This change broke opencv, and it was not noticed since vcpkg.cmake
didn't rebuild the world.
strega-nil-ms pushed a commit that referenced this pull request May 18, 2021
…" (#17991)

This reverts commit b369b11.

This change broke opencv, and it was not noticed since vcpkg.cmake
didn't rebuild the world.
@cenit
Copy link
Contributor

cenit commented May 19, 2021

vcpkg.cmake is not considered in binary cache hashing.
both edits, this one and the revert one, didn't invalidate my binary cache, so i'm pretty sure the same happened here

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
category:code-cleanup This PR cleans up code, without fixing any existing bugs nor adding any features. info:internal This PR or Issue was filed by the vcpkg team.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants