[vcpkg_fixup_pkgconfig] Check for more problems, add unit test #23898

dg0yt · 2022-03-31T07:37:39Z

What does your PR fix?

Adds a unit test for vcpkg_fixup_pkgconfig.
Improves vcpkg_fixup_pkgconfig:
- Handle line continuations.
- Replaces ';' with ' '.
- Resolves CMake linking keywords optimized, debug, general (from select_library_configurations)
- Check for invalid Libs content:
  - <namespace>::<target> (from <Pkg>_LIBRARIES)
  - ...-NOTFOUND (from find_library)
  Failed checks are reported as warning, not error, to avoid problems with old port versions in manifest mode.
Which triplets are supported/not supported? Have you updated the CI baseline?

all, no
Does your PR follow the maintainer guide?

yes
If you have added/updated a port: Have you run ./vcpkg x-add-version --all and committed the result?

no

JackBoosY · 2022-03-31T09:59:21Z

dg0yt · 2022-03-31T10:18:02Z

FTR It is only a draft now, to get an idea of the size of the problem.
I'm undecided whether to fix individual portfiles, or to implement fixup in vcpkg_fixup_pkgconfig.

What already popped up is NOTFOUND. In some case it could be just dropped, in other cases it might indicate an actual error.

dg0yt · 2022-04-01T11:37:56Z

Updated 2022-07-03:

allegro5:
    Libs.private: -loptimized -lpng -ldebug -lpng16d -loptimized -lz -ldebug -lz -lm

arrow:
  Libs.private: SNAPPY_LIB-NOTFOUND RE2_LIB-NOTFOUND optimized;D:/installed/x64-windows/lib/bz2.lib;debug;D:/installed/x64-windows/debug/lib/bz2d.lib

dlib:
  Libs: -L${libdir} -ldlib ws2_32 winmm comctl32 gdi32 imm32 optimized  D:/installed/x64-windows/lib/libpng16.lib debug  D:/installed/x64-windows/debug/lib/libpng16d.lib optimized D:/installed/x64-windows/lib/zlib.lib debug D:/installed/x64-windows/debug/lib/zlibd.lib optimized D:/installed/x64-windows/lib/jpeg.lib debug D:/installed/x64-windows/debug/lib/jpeg.lib D:/installed/x64-windows/lib/openblas.lib D:/installed/x64-windows/lib/lapack.lib D:/installed/x64-windows/lib/openblas.lib unofficial::sqlite3::sqlite3 FFTW3::fftw3

fluidsynth:
  'Libs' refer to a CMake target: -lPkgConfig::GLIB

libconfuse:
  Libs: -L${libdir} -lconfuse optimized;D:/installed/x64-windows/lib/intl.lib;debug;D:/installed/x64-windows/debug/lib/intl.lib;D:/installed/x64-windows/lib/iconv.lib;D:/installed/x64-windows/lib/charset.lib

libgd:
  Libs.private: optimized;D:/installed/x64-windows/lib/webp.lib;debug;D:/installed/x64-windows/debug/lib/webpd.lib;optimized;D:/installed/x64-windows/lib/webpdecoder.lib;debug;D:/installed/x64-windows/debug/lib/webpdecoderd.lib;optimized;D:/installed/x64-windows/lib/webpdemux.lib;debug;D:/installed/x64-windows/debug/lib/webpdemuxd.lib;optimized;D:/installed/x64-windows/lib/libwebpmux.lib;debug;D:/installed/x64-windows/debug/lib/libwebpmuxd.lib

libgta:
  Libs.private: -loptimized -lD:/installed/x64-windows/lib/lzma.lib -ldebug -lD:/installed/x64-windows/debug/lib/lzma.lib -loptimized -lD:/installed/x64-windows/lib/bz2.lib -ldebug -lD:/installed/x64-windows/debug/lib/bz2d.lib -loptimized -lD:/installed/x64-windows/lib/zlib.lib -ldebug -lD:/installed/x64-windows/debug/lib/zlibd.lib

libpcap:
  Libs.private: -lws2_32 -loptimized -lD:/installed/x64-windows/lib/libssl.lib -ldebug -lD:/installed/x64-windows/debug/lib/libssl.lib -loptimized -lD:/installed/x64-windows/lib/libcrypto.lib -ldebug -lD:/installed/x64-windows/debug/lib/libcrypto.lib

mlpack:
  Libs: -Loptimized -LD:/installed/x64-windows/lib/boost_serialization-vc140-mt.lib -Ldebug -LD:/installed/x64-windows/debug/lib/boost_serialization-vc140-mt-gd.lib -larmadillo -loptimized -lD:/installed/x64-windows/lib/boost_serialization-vc140-mt.lib -ldebug -lD:/installed/x64-windows/debug/lib/boost_serialization-vc140-mt-gd.lib -LD:/packages/mlpack_x64-windows/lib/ -lmlpack

mongo-c-driver:
  Libs: -L${libdir} -lmongoc-static-1.0 -lsecur32.lib -lcrypt32.lib -lShlwapi.lib -lsecur32.lib -lcrypt32.lib -lBcrypt.lib -lDnsapi -loptimized -LD:/installed/x64-windows-static/lib -lzlib -ldebug -LD:/installed/x64-windows-static/debug/lib -lzlibd

sdl2cpp:
  'Libs' refer to a CMake target: SDL2::SDL2main

Errror classes:

; (from CMake lists) instead of for separating command line arguments
optimized/debug CMake linking keywords
- -loptimized -lrel -ldebug -ldbg (allegro5, libgta, libpcap, mlpack,mongo-c-driver)
- -Loptimized -Lrel -Ldebug -Ldbg (mlpack)
- optimized;librel.a;debug;libdbg.a[;optimized;...] (arrow, libconfuse, libgd)
- optimized librel.a debug libdbg.a (dlib)
<Lib>-NOTFOUND (arrow)
namespaced::target (dlib, fluidsynth, sdl2cpp)

Neumann-A · 2022-04-01T11:45:40Z

Seems like this could be fixed with REGEX MATCH ? at least the list like stuff and the -ldebug -l<lib> .... The rest probably needs manual updating.

Neumann-A · 2022-04-01T11:46:07Z

Also a check for targets might be appropiate: -lOpenSSL::SSL

dg0yt · 2022-04-01T14:12:08Z

Also a check for targets might be appropiate: -lOpenSSL::SSL

Basically I need a vcpg_validate_pkgconfig as much as a vcpg_validate_pkgconfig. I don't even care for symbols at the moment, just validate the linking command...

dg0yt · 2022-04-01T14:16:24Z

BTW, I do have the cmake code to expand the Libs from the target properties. An early version is here in port curl, and an extended version is in GDAL. (Covering common generator expressions, too.) Maybe I should wrap that as a CMake package in a vcpkg port, and patch it in where needed.

github-actions

You have modified or added at least one vcpkg.json where you should check the license field.

If you feel able to do so, please consider adding a "license" field to the following files:

scripts/test_ports/unit-test-cmake/vcpkg.json

Valid values for the license field can be found in the documentation

dg0yt · 2022-11-19T13:48:32Z

unit-test-cmake: Must no longer expect errors.

qtwebengine: Strange!

-- Configuring x64-windows-dbg
-- Configuring x64-windows-rel
-- Building x64-windows-dbg
CMake Error at scripts/cmake/vcpkg_execute_build_process.cmake:131 (message):
    Command failed: D:/downloads/tools/cmake-3.24.0-windows/cmake-3.24.0-windows-i386/bin/cmake.exe --build . --config Debug --target install -- -v -j33
    Working Directory: D:/buildtrees/tmp/x64-windows-dbg
    See logs for more information:
      D:\buildtrees\tmp\install-x64-windows-dbg-out.log

But this log file is not in the failure logs archive. There is also no trace of the config step. I guess it is due to

CMake Warning at ports/qtwebengine/portfile.cmake:83 (message):
  Buildtree path 'D:/buildtrees/qtwebengine' is too long.

  Consider passing --x-buildtrees-root=<shortpath> to vcpkg!

  Trying to use 'D:/buildtrees/qtwebengine/../tmp'

not considered during CI log collection.

dg0yt · 2022-12-01T07:43:04Z

Ping @LilyWangLL

lukester1975 · 2022-12-07T12:41:04Z

Nice, been waiting for this! Would you consider fixing up .private fields when generating shared libs? Whilst they are not relevant it would be nice if they weren't bad. E.g. arrow generates:

Libs.private: ${prefix}/bin/snappy.dll optimized;${prefix}/lib/bz2.lib;debug;${prefix}/debug/lib/bz2d.lib

Or maybe they should be just stripped out completely in a shared build's pc files?

Thanks!

dg0yt · 2022-12-08T07:22:40Z

Or maybe they should be just stripped out completely in a shared build's pc files?

This might be a nice-to-have option for vcpkg, under the assumption that vcpkg should never use static libs from outside.
However, I think it is a topic for another PR.

dg0yt · 2022-12-15T06:02:04Z

Ping for merge or new feedback.
CC @BillyONeal

dg0yt · 2022-12-18T21:39:27Z

The port is ready to merge: Only known baseline regression, no new regressions from this PR's changes.

LilyWangLL · 2022-12-19T02:50:18Z

Depends on: #28352

BillyONeal · 2023-01-20T02:28:06Z

Thank you for:

Adding tests
Thinking about old port versions
Dealing with this for 10 months

<3

github-actions bot approved these changes Mar 31, 2022

View reviewed changes

JonLiu1993 assigned JackBoosY Mar 31, 2022

JackBoosY added the category:tool-update The issue is with build tool or build script, which requires update or should be executed correctly label Mar 31, 2022

dg0yt mentioned this pull request Apr 23, 2022

[harfbuzz] Update to 4.2.0 #24144

Merged

dg0yt mentioned this pull request Jun 12, 2022

[osg] Cleanup, export unofficial config #24720

Merged

github-actions bot previously approved these changes Jul 1, 2022

View reviewed changes

dg0yt dismissed github-actions[bot]’s stale review via 2cbea99 July 3, 2022 00:20

github-actions bot previously approved these changes Jul 3, 2022

View reviewed changes

dg0yt added 12 commits July 23, 2022 17:07

Print stack traces for failed unit tests

9ed5937

Add test for vcpkg_fixup_pkgconfig.cmake

5cc0072

Check for 'optimized' and 'debug' in pc files

470157c

Check for 'NOTFOUND' and for 'ns::target' in pc files

4100f17

Check for, and fix, line continuations

49212b2

Test file path substitutions

df6f595

Move contents processing into separate function

250d3d6

Update and leverage line break normalization

80c4b04

Pass prefix to data processing

4103882

Validate only the collapsed Libs

eabfabe

Test removal of '<field>.private'

85e10cf

Declare unit test license (same as vcpkg)

062b870

dg0yt dismissed github-actions[bot]’s stale review via 062b870 July 23, 2022 15:45

dg0yt force-pushed the fixup-pkgconfig branch from 2cbea99 to 062b870 Compare July 23, 2022 15:45

dg0yt changed the title ~~[vcpkg_fixup_pkgconfig] Check for 'optimized' and 'debug' in pc files~~ [vcpkg_fixup_pkgconfig] Check for more problems, add unit test Jul 23, 2022

Disable unit-testing for fatal errors

74752e2

dg0yt dismissed github-actions[bot]’s stale review via 74752e2 November 20, 2022 17:13

github-actions bot approved these changes Nov 20, 2022

View reviewed changes

LilyWangLL approved these changes Dec 5, 2022

View reviewed changes

LilyWangLL added the info:reviewed Pull Request changes follow basic guidelines label Dec 5, 2022

lukester1975 mentioned this pull request Dec 6, 2022

[libpcap] msvc static builds generate incorrect library name in libpcap.pc #28208

Closed

dan-shaw approved these changes Dec 15, 2022

View reviewed changes

Merge branch 'microsoft:master' into fixup-pkgconfig

691e40f

github-actions bot approved these changes Dec 17, 2022

View reviewed changes

LilyWangLL added the depends:different-pr This PR or Issue depends on a PR which has been filed label Dec 19, 2022

dan-shaw removed the info:reviewed Pull Request changes follow basic guidelines label Dec 22, 2022

Merge branch 'microsoft:master' into fixup-pkgconfig

e8f5ae6

github-actions bot approved these changes Dec 31, 2022

View reviewed changes

lukester1975 mentioned this pull request Dec 31, 2022

[C++] arrow.pc is missing dependencies with Windows static builds apache/arrow#15139

Closed

Merge branch 'microsoft:master' into fixup-pkgconfig

3e91f2e

github-actions bot approved these changes Jan 5, 2023

View reviewed changes

LilyWangLL removed the depends:different-pr This PR or Issue depends on a PR which has been filed label Jan 5, 2023

Merge branch 'microsoft:master' into fixup-pkgconfig

261671c

github-actions bot approved these changes Jan 12, 2023

View reviewed changes

dan-shaw added the info:reviewed Pull Request changes follow basic guidelines label Jan 19, 2023

BillyONeal merged commit edcf949 into microsoft:master Jan 20, 2023

dg0yt deleted the fixup-pkgconfig branch January 20, 2023 06:29

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[vcpkg_fixup_pkgconfig] Check for more problems, add unit test #23898

[vcpkg_fixup_pkgconfig] Check for more problems, add unit test #23898

dg0yt commented Mar 31, 2022 •

edited

Loading

JackBoosY commented Mar 31, 2022

dg0yt commented Mar 31, 2022

dg0yt commented Apr 1, 2022 •

edited

Loading

Neumann-A commented Apr 1, 2022

Neumann-A commented Apr 1, 2022

dg0yt commented Apr 1, 2022

dg0yt commented Apr 1, 2022 •

edited

Loading

github-actions bot left a comment

dg0yt commented Nov 19, 2022

dg0yt commented Dec 1, 2022

lukester1975 commented Dec 7, 2022

dg0yt commented Dec 8, 2022

dg0yt commented Dec 15, 2022

dg0yt commented Dec 18, 2022

LilyWangLL commented Dec 19, 2022

BillyONeal commented Jan 20, 2023

[vcpkg_fixup_pkgconfig] Check for more problems, add unit test #23898

[vcpkg_fixup_pkgconfig] Check for more problems, add unit test #23898

Conversation

dg0yt commented Mar 31, 2022 • edited Loading

What does your PR fix?

Which triplets are supported/not supported? Have you updated the CI baseline?

Does your PR follow the maintainer guide?

If you have added/updated a port: Have you run ./vcpkg x-add-version --all and committed the result?

JackBoosY commented Mar 31, 2022

dg0yt commented Mar 31, 2022

dg0yt commented Apr 1, 2022 • edited Loading

Neumann-A commented Apr 1, 2022

Neumann-A commented Apr 1, 2022

dg0yt commented Apr 1, 2022

dg0yt commented Apr 1, 2022 • edited Loading

github-actions bot left a comment

Choose a reason for hiding this comment

dg0yt commented Nov 19, 2022

dg0yt commented Dec 1, 2022

lukester1975 commented Dec 7, 2022

dg0yt commented Dec 8, 2022

dg0yt commented Dec 15, 2022

dg0yt commented Dec 18, 2022

LilyWangLL commented Dec 19, 2022

BillyONeal commented Jan 20, 2023

dg0yt commented Mar 31, 2022 •

edited

Loading

If you have added/updated a port: Have you run `./vcpkg x-add-version --all` and committed the result?

dg0yt commented Apr 1, 2022 •

edited

Loading

dg0yt commented Apr 1, 2022 •

edited

Loading