Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Consolidate reporting the result of a dependency check #3657

Merged

Conversation

jon-turney
Copy link
Member

Motivated by the observation in #3095 that currently, we don't report if a dependency was not found before trying a fallback subproject.

(at the moment, if a dependency isn't found, find_external_dependency() either (i) logs that it wasn't found, if it is not required, or (ii) raises an exception, if it is required (that exception being held by func_dependency() while it checks if a workable fallback exists))

This is hard to fix straightforwardly because, at the moment, the ExternalDependency-derived constructor called by find_external_dependency() is responsible for reporting success or raising an exception if not found, but required.

This patch series consolidates that reporting and checking in find_external_dependency().

This enables always logging the result of the dependency check, as well as removing some inconsistencies in how those checks are reported.

This might also help #3376 to simply report that "checking for dependency x was skipped, because the requirement for it was disabled"

@jon-turney
Copy link
Member Author

jon-turney commented Jun 1, 2018

Rather than spamming issues, some other things noted in passing:

  • only pkgconfig dependency implements the version: keyword check !?!?
  • config tools are searched for every time they are needed (unlike pkg-config, where we are smart enough to only look for it once)
  • LLVM only reports the first missing component, which could be annoying
  • the version of a pcap dependency starts with "libpcap version", which is probably not what's wanted
  • python3 dependency still uses the anti-pattern identified in dependency('gl') can return an object which claims to be a pkgconfig dependency, but isn't #2905
  • specialized dependency detectors often don't correctly consider if we are looking for a cross or native dependency

@nirbheek
Copy link
Member

nirbheek commented Jun 1, 2018

only pkgconfig dependency implements the version: keyword check !?!?

HAHA yes. We were supposed to consolidate the code for tool searching and version checking across all dependencies instead of duplicating it in each, but we never got around to doing it.

@jon-turney jon-turney force-pushed the consolidate-dependency-check-report branch 2 times, most recently from 0d32b53 to d31c44c Compare June 3, 2018 18:53
@nirbheek nirbheek self-assigned this Jun 3, 2018
@jon-turney jon-turney force-pushed the consolidate-dependency-check-report branch from d31c44c to e31609f Compare June 10, 2018 13:22
@jon-turney
Copy link
Member Author

Rebased to resolve conflict.

@nirbheek
Copy link
Member

Is this ready for review?

@jon-turney
Copy link
Member Author

Is this ready for review?

Sure. I'm not sure it's important for 0.47.0, though, since it's just clean up.

@jon-turney jon-turney force-pushed the consolidate-dependency-check-report branch from e31609f to bf58852 Compare July 4, 2018 13:51
@jpakkane
Copy link
Member

⚔️ conflicts.

@jon-turney jon-turney force-pushed the consolidate-dependency-check-report branch from bf58852 to 39fa94c Compare July 31, 2018 21:14
@jon-turney
Copy link
Member Author

Rebased to resolve conflict.

libwmf and pcap allegedly support method:extraframework, but this does
nothing but return not-found.  Possibly cut-and-pasted from elsewhere?
dub dependencies allegedly support method:pkgconfig, but this is meaningless
as it's only ever used when the dub method is explicitly requested.
Possibly cut-and-pasted from elsewhere?
…_dependency()

find_external_dependency() now makes and iterates over a list of callables
which are constructors with bound arguments for the dependency objects we
are going to attempt to make, so we can consolidate reporting on these
attempts and handling failures in that function.
Give ConfigToolDependency() a finish_init callback, so that tool-specific
initialization can be called from the constructor, rather than after
construction in the factory class.

v2:
finalize -> finish_init for clarity
This is now done by find_external_dependency() in all cases

I can't help but think this perhaps should be in a few more places...
If successful, we should identify the method which was successful
If successful, we should report the version found (if known)
If failing, we should identify the methods we tried

Some dependency detectors which had no reporting now gain it

There's all kinds of complexities, inconsistencies and special cases hidden
in the existing behaviour, e.g.:

- boost reports modules requested, and BOOST_ROOT (if set)
- gtest/gmock report if they are a prebuilt library or header only
- mpi reports the language
- qt reports modules requested, and the config tool used or tried
- configtool reports the config tool used
- llvm reports if missing modules are optional (one per line)

We add some simple hooks to allow the dependency object to expose the
currently reported information into the consolidated reporting

Note that PkgConfigDependency() takes a silent: keyword which is used
internallly to suppress reporting.  This behaviour isn't needed in
find_external_dependency().
If we aren't cross-building, this distinction isn't very interesting

(Most dependencies didn't bother reporting this. pkgconfig did it
unconditionally, and Qt reported as 'native' or 'cross' depending on
is_cross_build(), i.e. ignoring the native: keyword)
Display dependency name with the correct casing, for the (extended) list of
dependencies we have a preferred casing for.
@jon-turney jon-turney force-pushed the consolidate-dependency-check-report branch from 39fa94c to c333561 Compare August 1, 2018 13:26
@nirbheek nirbheek added this to the 0.48.0 milestone Aug 7, 2018
@nirbheek
Copy link
Member

nirbheek commented Aug 8, 2018

This PR is quite difficult to review, could you paste some examples of what the output looks like? I think that should be enough to do a review.

@jon-turney
Copy link
Member Author

This PR is quite difficult to review, could you paste some examples of what the output looks like? I think that should be enough to do a review.

The idea behind this change wasn't really to change the output, I've tried to keep it generally the same, but that it's very hard to make any change to the output, as each ExternalDependency-derived subclass replicates the logging and exception raising code.

The output discussed in #3095 changes like this:

before:

The Meson build system
Version: 0.48.0.dev1
Source dir: /home/jon/src/valum
Build dir: /home/jon/src/valum/_build
Build type: native build
Project name: valum
Project version: 0.4.0-dev
Native C compiler: cc (gcc 7.3.0 "cc (Ubuntu 7.3.0-16ubuntu3) 7.3.0")
Native Vala compiler: valac (valac 0.40.4)
Build machine cpu family: x86_64
Build machine cpu: x86_64
Found pkg-config: /usr/bin/pkg-config (0.29.1)
Native dependency glib-2.0 found: YES 2.56.1
Native dependency gobject-2.0 found: YES 2.56.1
Native dependency gio-2.0 found: YES 2.56.1
Native dependency libsoup-2.4 found: YES 2.62.1
Looking for a fallback subproject for the dependency vsgi
Cloning into 'vsgi'...
remote: Counting objects: 5754, done.
remote: Total 5754 (delta 0), reused 0 (delta 0), pack-reused 5754
Receiving objects: 100% (5754/5754), 1.76 MiB | 2.41 MiB/s, done.
Resolving deltas: 100% (3900/3900), done.
Already on 'master'
Your branch is up-to-date with 'origin/master'.

|
|Executing subproject vsgi 
|
|Project name: vsgi
|Project version: 0.4.0-dev
|Native C compiler: cc (gcc 7.3.0 "cc (Ubuntu 7.3.0-16ubuntu3) 7.3.0")
|Native Vala compiler: valac (valac 0.40.4)
|Dependency glib-2.0 found: YES (cached)
|Dependency gobject-2.0 found: YES (cached)
|Dependency gio-2.0 found: YES (cached)
|Native dependency gio-unix-2.0 found: YES 2.56.1
|Native dependency gmodule-2.0 found: YES 2.56.1
|Dependency libsoup-2.4 found: YES (cached)
|Dependency libsystemd found: NO
Couldn't use fallback subproject in subprojects/vsgi for the dependency vsgi 
Reason: subprojects/vsgi/meson.build:20: Native dependency 'openssl' not found

meson.build:17:0: ERROR:  Native dependency 'vsgi' not found

A full log can be found at /home/jon/src/valum/_build/meson-logs/meson-log.txt

after:

The Meson build system
Version: 0.48.0.dev1
Source dir: /home/jon/src/valum
Build dir: /home/jon/src/valum/_build
Build type: native build
Project name: valum
Project version: 0.4.0-dev
Native C compiler: cc (gcc 7.3.0 "cc (Ubuntu 7.3.0-16ubuntu3) 7.3.0")
Native Vala compiler: valac (valac 0.40.4)
Build machine cpu family: x86_64
Build machine cpu: x86_64
Found pkg-config: /usr/bin/pkg-config (0.29.1)
Dependency glib-2.0 found: YES 2.56.1
Dependency gobject-2.0 found: YES 2.56.1
Dependency gio-2.0 found: YES 2.56.1
Dependency libsoup-2.4 found: YES 2.62.1
Dependency vsgi found: NO (tried pkgconfig)
Looking for a fallback subproject for the dependency vsgi
Cloning into 'vsgi'...
remote: Counting objects: 5754, done.
remote: Total 5754 (delta 0), reused 0 (delta 0), pack-reused 5754
Receiving objects: 100% (5754/5754), 1.76 MiB | 2.41 MiB/s, done.
Resolving deltas: 100% (3900/3900), done.
Already on 'master'
Your branch is up-to-date with 'origin/master'.

|
|Executing subproject vsgi 
|
|Project name: vsgi
|Project version: 0.4.0-dev
|Native C compiler: cc (gcc 7.3.0 "cc (Ubuntu 7.3.0-16ubuntu3) 7.3.0")
|Native Vala compiler: valac (valac 0.40.4)
|Dependency glib-2.0 found: YES (cached)
|Dependency gobject-2.0 found: YES (cached)
|Dependency gio-2.0 found: YES (cached)
|Dependency gio-unix-2.0 found: YES 2.56.1
|Dependency gmodule-2.0 found: YES 2.56.1
|Dependency libsoup-2.4 found: YES (cached)
|Dependency libsystemd found: NO (tried pkgconfig)
|Dependency openssl found: NO (tried pkgconfig)
Couldn't use fallback subproject in subprojects/vsgi for the dependency vsgi 
Reason: subprojects/vsgi/meson.build:20: Dependency "openssl" not found, tried pkgconfig

meson.build:17:0: ERROR:  Dependency "vsgi" not found, tried pkgconfig

A full log can be found at /home/jon/src/valum/_build/meson-logs/meson-log.txt

diff:

--- before	2018-08-08 21:24:13.567000000 +0100
+++ after	2018-08-08 21:14:14.003000000 +0100
@@ -10,10 +10,11 @@
 Build machine cpu family: x86_64
 Build machine cpu: x86_64
 Found pkg-config: /usr/bin/pkg-config (0.29.1)
-Native dependency glib-2.0 found: YES 2.56.1
-Native dependency gobject-2.0 found: YES 2.56.1
-Native dependency gio-2.0 found: YES 2.56.1
-Native dependency libsoup-2.4 found: YES 2.62.1
+Dependency glib-2.0 found: YES 2.56.1
+Dependency gobject-2.0 found: YES 2.56.1
+Dependency gio-2.0 found: YES 2.56.1
+Dependency libsoup-2.4 found: YES 2.62.1
+Dependency vsgi found: NO (tried pkgconfig)                                       <--- added
 Looking for a fallback subproject for the dependency vsgi
 Cloning into 'vsgi'...
 remote: Counting objects: 5754, done.
@@ -33,14 +34,15 @@
 |Dependency glib-2.0 found: YES (cached)
 |Dependency gobject-2.0 found: YES (cached)
 |Dependency gio-2.0 found: YES (cached)
-|Native dependency gio-unix-2.0 found: YES 2.56.1
-|Native dependency gmodule-2.0 found: YES 2.56.1
+|Dependency gio-unix-2.0 found: YES 2.56.1
+|Dependency gmodule-2.0 found: YES 2.56.1
 |Dependency libsoup-2.4 found: YES (cached)
-|Dependency libsystemd found: NO
+|Dependency libsystemd found: NO (tried pkgconfig)
+|Dependency openssl found: NO (tried pkgconfig)
 Couldn't use fallback subproject in subprojects/vsgi for the dependency vsgi 
-Reason: subprojects/vsgi/meson.build:20: Native dependency 'openssl' not found
+Reason: subprojects/vsgi/meson.build:20: Dependency "openssl" not found, tried pkgconfig
 
-meson.build:17:0: ERROR:  Native dependency 'vsgi' not found
+meson.build:17:0: ERROR:  Dependency "vsgi" not found, tried pkgconfig
 
 A full log can be found at /home/jon/src/valum/_build/meson-logs/meson-log.txt

It also suppresses cross/native reporting as uninteresting when we are not cross-building, corrects it for Qt, and extends it to all dependency classes.

It also should make it much more straightforward to extend the implementation of the version: keyword to all dependency classes.

Copy link
Member

@nirbheek nirbheek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I reviewed this manually with git log -p because github's UI sucks for doing a commit-by-commit review.

All changes LGTM. This might cause regressions, but they will likely be minor.

@nirbheek nirbheek merged commit 41bb594 into mesonbuild:master Aug 11, 2018
@nirbheek
Copy link
Member

Merged with a merge commit; should have no functionality changes or new features.

@jon-turney
Copy link
Member Author

Thanks.

I'll probably take a look at seeing if I can factor out version: so it applies to everything sometime this week, if I have time.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants