[RFC] add prefer_static built-in option #9603

Dudemanguy · 2021-11-21T20:34:09Z

By default, meson will try to look for shared libraries first before
static ones. In the meson.build itself, one can use the static keyword
to control if a static library will be tried first but there's no simple
way for an end user performing a build to switch back and forth at will.
Let's cover this usecase by adding an option that allows a user to
specify if they want dependency lookups to try static or shared
libraries first. The writer of the meson.build can manually specify the
static keyword where appropriate which will override the value of this
option.

I suppose this needs test cases to go with it? I was a bit lost on where exactly tests should be made/created so that was left out for now (guidance would be appreciated).

Fixes #2816
Fixes #4276
Fixes #6109
Fixes #6629
Fixes #7621

codecov · 2021-11-21T20:42:21Z

Codecov Report

Merging #9603 (2a12518) into master (acef5a9) will increase coverage by 0.00%.
The diff coverage is 75.00%.

@@           Coverage Diff           @@
##           master    #9603   +/-   ##
=======================================
  Coverage   67.80%   67.80%           
=======================================
  Files         400      400           
  Lines       85522    85532   +10     
  Branches    18825    18827    +2     
=======================================
+ Hits        57987    57997   +10     
+ Misses      23035    23033    -2     
- Partials     4500     4502    +2

Impacted Files	Coverage Δ
mesonbuild/coredata.py	`83.74% <ø> (ø)`
mesonbuild/dependencies/__init__.py	`100.00% <ø> (ø)`
mesonbuild/dependencies/misc.py	`55.30% <0.00%> (+0.15%)`	⬆️
mesonbuild/mesonlib/universal.py	`81.61% <ø> (+0.23%)`	⬆️
mesonbuild/interpreter/compiler.py	`93.27% <50.00%> (-0.38%)`	⬇️
mesonbuild/dependencies/base.py	`85.07% <100.00%> (ø)`
mesonbuild/dependencies/hdf5.py	`85.98% <100.00%> (ø)`
mesonbuild/dependencies/scalapack.py	`38.15% <100.00%> (+1.67%)`	⬆️
mesonbuild/scripts/vcstagger.py	`87.50% <0.00%> (-4.17%)`	⬇️
interpreter/compiler.py	`93.27% <0.00%> (-0.38%)`	⬇️
... and 6 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update acef5a9...2a12518. Read the comment docs.

eli-schwartz · 2021-11-21T20:44:42Z

This would probably solve:

#2816
#4276
#6109
#6629
#7621

Dudemanguy · 2021-11-22T00:43:50Z

Fixed the tests.

eli-schwartz

Overall looks like a nice change to make.

While we are here, though, there are a couple things that should be cleaned up on general principle, and if you do clean them up, it means actually implementing this new feature changes fewer lines. With that in mind...

eli-schwartz · 2022-01-10T21:44:24Z

mesonbuild/dependencies/hdf5.py

@@ -51,7 +51,8 @@ def __init__(self, name: str, environment: 'Environment', kwargs: T.Dict[str, T.
        newinc = []  # type: T.List[str]
        for arg in self.compile_args:
            if arg.startswith('-I'):
-                stem = 'static' if kwargs.get('static', False) else 'shared'
+                static_opt =  environment.coredata.get_option(OptionKey('prefer_static'))
+                stem = 'static' if kwargs.get('static', static_opt) else 'shared'


kwargs.get('static', False) seems sub-optimal and as a general cleanup we can probably use self.static here (as anywhere that inherits from ExternalDependency).

eli-schwartz · 2022-01-10T21:45:12Z

mesonbuild/dependencies/hdf5.py

        args = self.get_config_value(['-show', '-c'], 'args')[1:]
-        args += self.get_config_value(['-show', '-noshlib' if kwargs.get('static', False) else '-shlib'], 'args')[1:]
+        args += self.get_config_value(['-show', '-noshlib' if kwargs.get('static', static_opt) else '-shlib'], 'args')[1:]


Likewise here.

eli-schwartz · 2022-01-10T21:46:06Z

mesonbuild/dependencies/misc.py

@@ -175,7 +175,7 @@ def __init__(self, name: str, environment: 'Environment', kwargs: T.Dict[str, T.
            return

        self.name = 'python3'
-        self.static = kwargs.get('static', False)
+        self.static = kwargs.get('static', environment.coredata.get_option(mesonlib.OptionKey('prefer_static')))


This seems odd! Why does this line exist, it seems to be literally overriding the existing self.static attribute...

Do you mean that this is attribute is already inherited from base.py? Yeah, I can just delete the line in that case.

That is exactly what I mean. :)

eli-schwartz · 2022-01-10T21:46:58Z

mesonbuild/dependencies/misc.py

@@ -537,7 +537,7 @@ def shaderc_factory(env: 'Environment',
        shared_libs = ['shaderc']
        static_libs = ['shaderc_combined', 'shaderc_static']

-        if kwargs.get('static', False):
+        if kwargs.get('static', env.coredata.get_option(mesonlib.OptionKey('prefer_static'))):


This, however, is a function, not a class, so kwargs.get() is correct as is your modification.

Dudemanguy · 2022-01-10T23:56:51Z

mesonbuild/dependencies/__init__.py

-        static = Mesonlib.LibType.STATIC if kwargs.get('static', False) else Mesonlib.LibType.SHARED
+        get_option = environment.coredata.get_option
+        static_opt = kwargs.get('static', get_option(Mesonlib.OptionKey('prefer_static'))
+        static = Mesonlib.LibType.STATIC if static_opt else Mesonlib.LibType.SHARED


Should this change still be here? All of this is comments so I guess it doesn't matter too much, but I don't want to add wrong things in here.

Yeah, this documentation is, in part, showing how it all fits together. So I believe it is correct to demonstrate the underlying implementation. IIRC a major motivation in adding this comment section was so that we could move from one core dev understanding how the mesonbuild/dependencies/ directory works, to multiple core devs understanding.

After all, there is also a self.libtype which can technically be used too, but that doesn't obsolete the comment tutorial either.

lgtm-com · 2022-01-11T00:00:59Z

This pull request introduces 1 alert when merging 92dd836 into e77582b - view on LGTM.com

new alerts:

1 for Unused import

Dudemanguy · 2022-01-11T03:12:15Z

Split this into two commits now and rebased.

xclaesse · 2022-01-20T15:45:00Z

I haven't looked at code, does it cover both dependency() and find_library()? This new option is better than nothing but unfortunately it's a all-or-nothing option. Ideally we should have more fine grained way of telling which deps should be static and which should be shared.

A long time ago I made that attempt: #4276, and a few others, but we never agreed on a good solution.

The use-case I still have is when you build an app (e.g. GStreamer) for a platform that provide shared libs for some basic deps (e.g. zlib). In that case I want to use the shared libs for stuff provided by the device's base platform and static link everything else into my app.

xclaesse · 2022-01-20T15:47:15Z

That said, since we had that use-case for years and we never found an elegant enough solution, I wouldn't block this PR that already covers a good part of the use-case.

Dudemanguy · 2022-01-20T16:12:41Z

I haven't looked at code, does it cover both dependency() and find_library()?

That's a really good point. This does not cover find_library() and it really should. I'll fix that.

Dudemanguy · 2022-01-21T03:53:09Z

I added support to find_library now. I noticed that static works differently in find_library than it does in dependency. In dependency, it's merely just a preference and always tries the fallback if it fails. However in find_library, explicitly setting the static kwarg makes it a hard value (i.e. it will either only try static or only try shared). If you just don't set anything (i.e. the default), then it acts as a prefer shared. So the only thing that was missing was a prefer static (but fallback to shared) setting. That was just another elif statement.

In a couple of spots, kwargs.get('static', False) was being unneccesarily used. In these spots, we can just use self.static instead which is already inherited from the ExternalDependency. In additional, the python system dependency oddly has a kwargs.get('static', False) line which overrides the self.static in that dependency for no real reason. Delete this line too.

By default, meson will try to look for shared libraries first before static ones. In the meson.build itself, one can use the static keyword to control if a static library will be tried first but there's no simple way for an end user performing a build to switch back and forth at will. Let's cover this usecase by adding an option that allows a user to specify if they want dependency lookups to try static or shared libraries first. The writer of the meson.build can manually specify the static keyword where appropriate which will override the value of this option.

ePirat · 2022-02-23T20:44:28Z

This would be an incredibly useful to have feature for VLC's WIP meson port, currently we clutter every dependency and find_library with the static kwarg to get this ability.

xclaesse · 2022-02-24T02:06:05Z

Throwing just an idea: what I often want is use shared libraries from the system, and if not found build a static fallback subproject. Not sure how we could try to express that use-case, -Dprefer_static=subprojects ?

eli-schwartz · 2022-02-24T02:29:13Z

Perhaps the prefer_static option should cause all subprojects to default to static lookup? IIRC we already special case dependency('foo', static: true) to pass default_library=static as a subproject default option. Perhaps we should check this option there too in the case that the static kwarg is None.

xclaesse · 2022-02-24T13:10:05Z

IIRC we already special case dependency('foo', static: true) to pass default_library=static as a subproject default option

Yes we do that, so if I understand correctly prefer_static as proposed would already build static subprojects. However my use-case is a little bit different: I build GStreamer against a platform SDK that contains a few basic libraries (e.g. zlib, openssl) and I want to use the shared libraries provided by the platform for them. But everything else that is not part of the SDK is built as subproject and those would be nice to be static.

So dependency('zlib', fallback: 'zlib') with -Dprefer_static=subprojects would return the shared lib from system, or static lib from fallback subproject. If we make that option a combo [true, false, subprojects] instead of boolean, it should actually be pretty easy to implement. That said, it can also be extended later.

Dudemanguy · 2022-02-24T22:31:13Z

Yes we do that, so if I understand correctly prefer_static as proposed would already build static subprojects.

Well this PR currently only touches code related to dependency and find_library. I'm not sure if that's all that is required and the subproject case automatically inherits the method or if more changes are required.

So dependency('zlib', fallback: 'zlib') with -Dprefer_static=subprojects would return the shared lib from system, or static lib from fallback subproject.

If such an option is added, would it make sense to turn the static kwarg to a combo option like that as well? The main intent of this PR was to simply mimic the static kwarg but on a global level, so I would think any new options should also propagate downwards.

Dudemanguy · 2022-03-18T15:52:55Z

@xclaesse: Just revisiting this one, but if I understand correctly, would your usecase be covered if prefer_static is allowed to be set per-subproject like default_library? That way you could just set per subproject options for those to be static and everything else would do shared lookups. I can add a commit for that. It's not complicated.

As a sidenote, I finally got around to confirming that yes, subprojects do inherit the prefer_static argument so they all do static lookups as well (as expected).

Tachi107 · 2022-04-30T19:46:02Z

Throwing just an idea: what I often want is use shared libraries from the system, and if not found build a static fallback subproject. Not sure how we could try to express that use-case, -Dprefer_static=subprojects ?

Doesn't dependency('name', default_options: 'default_library=static') already cover this use case?

xclaesse · 2022-04-30T20:04:41Z

Yes, but it means editing entry dependency() calls. I would like instead a global switch for that. Preferring static or shared is often a user choice rather than a project choice.

xclaesse · 2022-05-03T19:59:47Z

Doesn't dependency('name', default_options: 'default_library=static') already cover this use case?

Re-thinking about this, subprojects inherit default_library=static from main project by default, so as long as it's fine to also have the main project built with default_library=static my use-case is already covered. More fine grained would still be nice, but probably out of scope for this PR.

Ultimately, we could add a list of libs/deps in native/cross file that tells exactly which one should be static.

Dudemanguy · 2022-05-04T01:28:37Z

Anything else that this PR needs?

eli-schwartz · 2022-05-04T03:03:50Z

No, I don't think so. And it seems we're all in agreement that this option is useful as it stands.

Dudemanguy · 2022-05-04T03:22:34Z

Thanks!

Also see mesonbuild/meson#9603

sertonix · 2024-07-29T11:13:00Z

Could this be added to the documentation? The reference manual doesn't mention prefer_static at all: https://mesonbuild.com/Reference-manual_functions.html#dependency

Ref #3084

Dudemanguy · 2024-07-29T13:18:52Z

It's a built-in option. You can see it here. https://mesonbuild.com/Builtin-options.html#core-options

eli-schwartz · 2024-07-29T15:02:48Z

I have no objection to also adding a reference to the builtin option, in the documentation for dependency(). ;)

@sertonix would you like to submit a patch for this?

Ref mesonbuild#9603

Ref #9603

Ref mesonbuild#9603

Dudemanguy requested a review from jpakkane as a code owner November 21, 2021 20:34

Dudemanguy force-pushed the prefer_static branch 2 times, most recently from e19f693 to af20ebe Compare November 21, 2021 23:09

Dudemanguy mentioned this pull request Nov 22, 2021

meson: need to handle linking to static libass mpv-player/mpv#9478

Closed

eli-schwartz mentioned this pull request Jan 10, 2022

Unable to compile a statically linked executable with valac or gcc #1715

Closed

eli-schwartz reviewed Jan 10, 2022

View reviewed changes

Dudemanguy force-pushed the prefer_static branch from af20ebe to 92dd836 Compare January 10, 2022 23:48

Dudemanguy commented Jan 10, 2022

View reviewed changes

Dudemanguy force-pushed the prefer_static branch from 92dd836 to 85e11b5 Compare January 11, 2022 01:04

Dudemanguy force-pushed the prefer_static branch 2 times, most recently from 8392fe9 to b183005 Compare January 21, 2022 03:47

Dudemanguy added 2 commits January 20, 2022 22:16

Dudemanguy force-pushed the prefer_static branch from b183005 to 2a12518 Compare January 21, 2022 04:18

eli-schwartz merged commit 557680f into mesonbuild:master May 4, 2022

Dudemanguy deleted the prefer_static branch May 4, 2022 03:22

Tachi107 mentioned this pull request May 6, 2022

Can we get rid of chroot and containers? probonopd/static-tools#18

Open

leotaku added a commit to leotaku/tomie that referenced this pull request Jul 14, 2022

Commit hack to build statically before Meson v63.0

055d420

Also see mesonbuild/meson#9603

sertonix added a commit to sertonix/meson that referenced this pull request Jul 30, 2024

docs: document prefer_static effect on dependency()

32a6638

Ref mesonbuild#9603

sertonix mentioned this pull request Jul 30, 2024

docs: document prefer_static effect on dependency() #13489

Merged

sertonix added a commit to sertonix/meson that referenced this pull request Jul 31, 2024

docs: document prefer_static effect on dependency()

60a143f

Ref mesonbuild#9603

jpakkane pushed a commit that referenced this pull request Aug 4, 2024

docs: document prefer_static effect on dependency()

839fef4

Ref #9603

serebit pushed a commit to serebit/meson that referenced this pull request Feb 19, 2025

docs: document prefer_static effect on dependency()

aefd789

Ref mesonbuild#9603

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[RFC] add prefer_static built-in option #9603

[RFC] add prefer_static built-in option #9603

Dudemanguy commented Nov 21, 2021 •

edited by eli-schwartz

Loading

codecov bot commented Nov 21, 2021 •

edited

Loading

eli-schwartz commented Nov 21, 2021

Dudemanguy commented Nov 22, 2021

eli-schwartz left a comment

eli-schwartz Jan 10, 2022

eli-schwartz Jan 10, 2022

eli-schwartz Jan 10, 2022

Dudemanguy Jan 10, 2022

eli-schwartz Jan 10, 2022

eli-schwartz Jan 10, 2022

Dudemanguy Jan 10, 2022

eli-schwartz Jan 11, 2022

lgtm-com bot commented Jan 11, 2022

Dudemanguy commented Jan 11, 2022

xclaesse commented Jan 20, 2022

xclaesse commented Jan 20, 2022

Dudemanguy commented Jan 20, 2022

Dudemanguy commented Jan 21, 2022 •

edited

Loading

ePirat commented Feb 23, 2022

xclaesse commented Feb 24, 2022 •

edited

Loading

eli-schwartz commented Feb 24, 2022

xclaesse commented Feb 24, 2022 •

edited

Loading

Dudemanguy commented Feb 24, 2022 •

edited

Loading

Dudemanguy commented Mar 18, 2022

Tachi107 commented Apr 30, 2022

xclaesse commented Apr 30, 2022 •

edited

Loading

xclaesse commented May 3, 2022

Dudemanguy commented May 4, 2022

eli-schwartz commented May 4, 2022

Dudemanguy commented May 4, 2022

sertonix commented Jul 29, 2024 •

edited

Loading

Dudemanguy commented Jul 29, 2024 •

edited

Loading

eli-schwartz commented Jul 29, 2024

[RFC] add prefer_static built-in option #9603

[RFC] add prefer_static built-in option #9603

Conversation

Dudemanguy commented Nov 21, 2021 • edited by eli-schwartz Loading

codecov bot commented Nov 21, 2021 • edited Loading

Codecov Report

eli-schwartz commented Nov 21, 2021

Dudemanguy commented Nov 22, 2021

eli-schwartz left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

lgtm-com bot commented Jan 11, 2022

Dudemanguy commented Jan 11, 2022

xclaesse commented Jan 20, 2022

xclaesse commented Jan 20, 2022

Dudemanguy commented Jan 20, 2022

Dudemanguy commented Jan 21, 2022 • edited Loading

ePirat commented Feb 23, 2022

xclaesse commented Feb 24, 2022 • edited Loading

eli-schwartz commented Feb 24, 2022

xclaesse commented Feb 24, 2022 • edited Loading

Dudemanguy commented Feb 24, 2022 • edited Loading

Dudemanguy commented Mar 18, 2022

Tachi107 commented Apr 30, 2022

xclaesse commented Apr 30, 2022 • edited Loading

xclaesse commented May 3, 2022

Dudemanguy commented May 4, 2022

eli-schwartz commented May 4, 2022

Dudemanguy commented May 4, 2022

sertonix commented Jul 29, 2024 • edited Loading

Dudemanguy commented Jul 29, 2024 • edited Loading

eli-schwartz commented Jul 29, 2024

Dudemanguy commented Nov 21, 2021 •

edited by eli-schwartz

Loading

codecov bot commented Nov 21, 2021 •

edited

Loading

Dudemanguy commented Jan 21, 2022 •

edited

Loading

xclaesse commented Feb 24, 2022 •

edited

Loading

xclaesse commented Feb 24, 2022 •

edited

Loading

Dudemanguy commented Feb 24, 2022 •

edited

Loading

xclaesse commented Apr 30, 2022 •

edited

Loading

sertonix commented Jul 29, 2024 •

edited

Loading

Dudemanguy commented Jul 29, 2024 •

edited

Loading