examples are hard-coded to use the shared form of the library

[iskunk]

I see that the CMakeLists.txt file for libsoundio has two separate options for building dynamic vs. static libraries.

I take it this was inspired by Libtool, which in its usual way of working compiles both static and shared libraries in one pass.

However, in CMake, it is conventional to build either one or the other, controlled by the special variable BUILD_SHARED_LIBS. I've put together a patch that simplifies the listfile accordingly.

(I don't currently have a proper setup to prepare a Git pull request, and I see that GitHub still doesn't allow issue attachments, so I've uploaded my patch here.)

The immediate issue that spurred this was breakage when I disabled the dynamic lib and attempted to build the examples. The examples are hard-coded to use the shared form of the library. It would be possible to code the listfile so the examples/tests link with one or the other as appropriate, but because this approach is foreign to CMake anyway, I believe reducing the complexity is preferable.

Oh, and as an extra, I added a bit to set CMAKE_INSTALL_RPATH. This allows the executables linked to a shared libsoundio to work in the install tree (without needing to set LD_LIBRARY_PATH) as well as the build tree.

[tresf]

GitHub still doesn't allow issue attachments

FYI, yes-it-does.zip

Edit: It is a relatively new feature (2015).

[iskunk]

Okay, I stand corrected :-]

(Must have been thinking of GitLab, then... I recalled hearing that one of the two has that limitation)

[andrewrk]

This seems correct. Let me play around with it to make sure I understand and then we'll schedule it for next API breakage.

[andrewrk]

cc @Naftoreiclag @IceDragon200 @ligfx @hasufell @diogocp

Everyone I just cc'd, you made a pull request that had to do with the cmake build script. Everyone OK with this change?

[ligfx]

TL;DR: LOOKS GOOD

Seems legit, that's how I've seen other CMake projects set up. While we're on the topic, there's plenty of other possible improvements to the CMakeLists.txt, but that's at least another issue or two's worth 😄

[IceDragon200]

Looks fine to me, reduces some of the code churn from before

[hasufell]

👎

this is removing useful functionality. It doesn't really matter what is "conventional" in cmake. Being able to build both dynamic and shared libraries in one go is conventional in linux. Just because it requires a bit more code doesn't mean it's not useful.

Also, the breakage you encountered is simply a bug that can be fixed and unrelated to the decision you are proposing.

[ligfx]

@hasufell I agree it's a feature removal, but how useful is it really? Seems like almost everything on Linux packages/builds against shared libraries anyways, and the building-both-in-one-go feature is more a quirk of auto tools.

Let's step back a sec. It comes down to, is this feature useful enough to merit more effort on the project's part, vs. pushing that effort to the users trying to do something out-of-the-ordinary?

[iskunk]

Most end users would only want one form (shared or static) of the library. The time when you would want both is generally when you're building the library for others, e.g. as a distro maintainer or packager. In that scenario, you're typically dealing with a large number of packages, and your job is made easier when all those packages follow their respective build-system conventions. It's not much of a favor for us to build both libraries in one go if packagers already have standard scripts for CMake that do the two separate builds.

I've been looking a bit more closely at the listfile, and am preparing a revised patch. The #include directories are not in the right order, and something's up with the find_package(JACK) logic on my system. Hope to have that up here soon.

[ligfx]

@iskunk Probably better to fix those issues in a different patch, to keep this one on-focus.

Also, if you haven't gotten Git set up yet, GitHub lets you make edits in your browser:

[iskunk]

Revised patch

Okay, so the JACK problem just turned out to be issue #11. My system has that older version of JACK without jack_set_port_rename_callback(). This can reasonably be addressed in FindJACK.cmake, so I added a test-compile in there after the normal package-finding logic. It could use some testing, especially if anyone is using a non-system-install of the JACK software.

I fixed up the #include directories in the main listfile. The order should start with the build directory, then the source directory, then dependency-library directories, then (implicitly) the system includes---generally, a header file that is closer to the build should override a file farther away. (One benefit of this approach is that if a library/system header is buggy, you can copy it into the build tree, fix it up, and be on your way.)

One small thing that I didn't address here: config.h.in should really live in the top-level directory of the source tree, given that it is instantiated in the top-level directory of the build tree. Going from $srcdir/src to $builddir basically amounts to moving the file, and that can be confusing to someone examining the trees.

@ligfx: I could open a new issue for these... they are relatively minor. Would the house prefer lots of little issues over consolidating them by area?

Editing in the browser... hmm... that's a new one on me. I'll have to play with that....

[ligfx]

@ligfx: I could open a new issue for these... they are relatively minor. Would the house prefer lots of little issues over consolidating them by area?

The house is @andrewrk, but my personal preference is to try to keep changes small enough that new failures can be properly root-caused. Changing from shared+static libraries could introduce one type of build failure (dunno why it would, but just in case...), and changing how JACK is searched for could introduce a different one.

re: the latest revision:

1. find_package_handle_standard_args needs to stay at the end of FindJACK.cmake. You would add HAVE_jack_set_port_rename_callback to the variables to check, like this:

   find_package_handle_standard_args(JACK DEFAULT_MSG JACK_LIBRARY JACK_INCLUDE_DIR HAVE_jack_set_port_rename_callback)

2. It's good defensive programming to unset the CMAKE_REQUIRED_* variables after using them

3. Instead of compiling a custom C program, you should probably be using the CheckLibraryExists or CheckFunctionExists modules, like this:

   check_library_exists(jack "jack_set_port_rename_callback" "${JACK_LIBRARY}" HAVE_jack_set_port_rename_callback)

[iskunk]

@ligfx: Yes, being able to point to the root cause is desirable... if the changes are relatively minor, wouldn't a single issue be specific enough?

I concede the three points you made. check_library_exists() is a better solution, and doesn't need CMAKE_REQUIRED_*. Here's the patch, updated accordingly.

(Someone needs to check the success case, as I can only test the fail case.)

[hasufell]

Seems like almost everything on Linux packages/builds against shared libraries anyways, and the building-both-in-one-go feature is more a quirk of auto tools.

That really depends on the distro. Gentoo likes to give the choice to enable the user to build certain parts of the system statically. Alpine Linux also has special needs wrt static libs. Do we really want to make assumptions here that may or may not be correct? It's not really a whole lot of complicated code, is it? The report looks like an exaggeration to me.

[iskunk]

@hasufell, nothing that has been discussed here will prevent distributions from building this library in the form they want. All it takes to build libsoundio statically after these changes is

$ cmake /path/to/libsoundio-src -DBUILD_SHARED_LIBS=OFF

This is consistent with other CMake-based projects, many of which are already handled by the specialty distributions you mentioned. Whatever limitations CMake and its conventions may impose on a project have already been worked around eons ago.

That said, it may be worth revisiting the topic of which form of the library (shared or static) should be the default. My patch has the former, as that was the preferred one for the examples and tests. However, I think that end users building this library will generally want a static build (to avoid the specter of LD_LIBRARY_PATH on Unix or missing DLLs on Windows), and that distributions/packagers will tend to be the ones wanting the shared form. The former group is much more likely to accept the defaults than the latter, so IMO the defaults would be better aimed at them.

[hasufell]

All it takes to build libsoundio statically after these changes is

You didn't get what I was saying: a lot of distros want both at the same time, not one or the other.

This is consistent with other CMake-based projects

That's irrelevant. It's relevant what people/distributors expect of a build system. What you are proposing is a step backwards. And the fact that other cmake projects do annoying stuff too isn't a particularly appealing argument. In fact, I know quite a few cmake based projects that had to work around this cmake oddity.

[iskunk]

You didn't get what I was saying: a lot of distros want both at the same time, not one or the other.

$ (mkdir shared && cd shared && cmake /path/to/libsoundio-src -DBUILD_SHARED_LIBS=ON)
$ (mkdir static && cd static && cmake /path/to/libsoundio-src -DBUILD_SHARED_LIBS=OFF)

[hasufell]

oh dear.

So you...

1. want people/packagers to do more work to get the same result (and it is more than you think when it's about actual packaging... your code above there is not enough)
2. double the compile time
3. make it possible to get more things wrong (e.g. async build options between the two builds)

Great step forward.

[tresf]

Albeit off topic; curious; lurker; etc... what is the use-case for the shared + static?

I do static packaging for a fairly large project and occasionally we have Debian packagers that chime in with very packaging-specific commits. We always discuss them but in the end tend to honor them because the static packagers are already written; no harm done. The distro packagers however tend to have quite a few packages they manage, so reducing their work as a trade-off to a slight increase in build logic is usually favored.

In regards to this "non-PR" PR, although the original diff simplified the CMakeLists.txt file, the latest is about the same amount of code it seems before/after.

Note, I too haven't yet seen the shared + static in a single CMakeLists.txt file, so that is a new concept to me as well, I too am interested in knowing the reasoning behind this (asking in an effort to learn from those needing it, rather than to criticize) 👍

[iskunk]

Hi @tresf,

The use case for shared+static is straightforward: Build both forms of the library so that make install puts both into $PREFIX/lib/. If you are building for a distribution, you'll typically want both, though they usually don't go together (shared goes into the base library package, static goes into the development package).

The idea of doing shared+static builds in a single pass mainly comes from Libtool, which achieves this using a compiler wrapper. Every source file is compiled into an .o and an .lo object, which are eventually linked into libfoo.a and libfoo.so libraries respectively. (The makefiles actually have libfoo.la as the target name, which is a small text file containing library metadata. You can't even build the static or shared form separately without reconfiguring; the two are considered a unit.)

CMake, however, does not use a compiler wrapper; it drives the compiler directly with the appropriate flags. So if you want to build both shared and static libraries, you have to have separate targets for them (as was done here). Then you have to worry about which of the two forms are used in later parts of the build; Libtool defaults to the shared library, but if you configure with --disable-shared, it will transparently shift to use the static form. Also, there's the nit that on Windows, both builds will yield a foo.lib file (static library or DLL import library), so the names can't be the same.

All this logic would have to be re-created for CMake, because this is not how CMake normally works. To give an analogy, this would be akin to setting up a GNU Autotools-based project to build both libfoo_debug and libfoo_release in the same tree, approximating the behavior of a Visual Studio solution. Which is to say, it can be done, and some folks may find it useful/familiar, but most are going to be annoyed that this is not handled as a configure-time option [like every other Autotools-based project out there]. And the project is saddled with maintaining the additional complexity of that build approach in perpetuity.

Making a $FRAMEWORK-based build system operate like other $FRAMEWORK-based build systems is ultimately the path of least resistance. It's easier to maintain, it's easier on packagers (who have scripts to address the most common cases), and it's easier for casual users who have never seen the project before.

In regards to this "non-PR" PR, although the original diff simplified the CMakeLists.txt file, the latest is about the same amount of code it seems before/after.

I'm addressing multiple things in my patch. Less armor-piercing round, more birdshot :-)

[tresf]

@iskunk thanks for the detailed explanation.

but most are going to be annoyed that this is not handled as a configure-time option [like every other Autotools-based project out there]

Wouldn't -DBUILD_STATIC_LIBS=False qualify as "configure-time option"?

[iskunk]

Wouldn't -DBUILD_STATIC_LIBS=False qualify as "configure-time option"?

Yes, but CMake uses BUILD_SHARED_LIBS. (It actually looks at this variable and changes the behavior of add_library() accordingly.)

[hasufell]

shared goes into the base library package, static goes into the development package

Depends on the distro. It's better to not make too many assumptions there.

The idea of doing shared+static builds in a single pass mainly comes from Libtool

People have been doing that with all sorts of build system for years, including hand-written Makefiles. It's pretty much the de-facto standard. Libtool is only special with regards to libtool files (.la), but that's a completely different topic.

It would save a lot of time if you do less assumptions and accept the fact that the feature is useful.

[tresf]

Yes, but CMake uses BUILD_SHARED_LIBS

New switch name perhaps? 🍻

[fsateler]

Fedora explicitly recommends not building them.

With my debian downstream maintainer hat on, I wouldn't mind losing the static library (I normally only build shared ones), and if required add the couple of commands needed to build the static one too

[iskunk]

New switch name perhaps?

Talk to the CMake folks ^_^

[mdsitton]

I don't have an opinion either way, but if you don't want to recompile the code twice and want a shared and static lib you could do something like this:

add_library(projectObj OBJECT ${SOURCE})
set_target_properties(projectObj PROPERTIES POSITION_INDEPENDENT_CODE TRUE)

add_library(project-shared SHARED $<TARGET_OBJECTS:projectObj>)
add_library(project-static STATIC $<TARGET_OBJECTS:projectObj>)

[hasufell]

Fedora explicitly recommends not building them.

With my debian downstream maintainer hat on, I wouldn't mind losing the static library (I normally only build shared ones), and if required add the couple of commands needed to build the static one too

Neither of them being source distros, where the impact is quite different. Both in terms of leaving the choice of building static libraries out and in terms of doubling compile time, because someone thought it's "simpler".

[ligfx]

@hasufell Petulantly repeating yourself and referring to other arguments in scare quotes is not contributing to the conversation! You've offered no concrete data to support your position, instead relying on generalities such as "lots of distros" and "source distros." We've heard from people involved in the actual packaging process who've offered their thoughts and experiences, and who actually respect the idea that making code simpler and following convention might be a reasonable project goal.

If you have any specific data to add or point to (e.g. "Distro X requires both static and shared in the same package" or "CMake person Y thinks shared and static should be built at the same time"), contribute that, otherwise please hold your peace.

[andrewrk]

Alright everyone. Easy on the name calling. We're all on the same team here, trying to make software work better for the open source community.

I'm going to step in and make a judgement call here.

However, in CMake, it is conventional to build either one or the other, controlled by the special variable BUILD_SHARED_LIBS. I've put together a patch that simplifies the listfile accordingly.

I think this is the key point here - convention. cmake makes some unfortunate decisions and doesn't completely solve the problem of building software optimally, but at the end of the day it's useful because of its widespread use. Follow the cmake conventions, and it becomes easy for package managers of various systems to build the software without having to individually tweak it. Breaking away from cmake convention would cause package maintainers to have to create patch the build system.

I appreciate the counterpoints in this discussion, and you're not wrong. But especially given the fact that libsoundio builds in sub ten seconds, I think it makes more sense to optimize for using cmake in the simple, expected, standard way than to try to do something clever to save time, for some use cases.

Just wait for the zig build system to come out and everyone can be happy :-)

[hasufell]

We've heard from people involved in the actual packaging process who've offered their thoughts and experiences, and who actually respect the idea that making code simpler and following convention might be a reasonable project goal.

I've been a gentoo developer for 5 years and am still working on several source distros as a collaborator now and then (gentoo, exherbo...). Thanks for the reminder though.

I've been trying to fix broken cmake build systems for quite some time to make things work properly on the distributions I've been packaging for years, including libraries like mbedtls. Now, seeing some people walking through the ecosystem and reverting things isn't something that I find particularly appealing.

I also have no time or interest in educating you about source distros, what they do, why they do it and so on.

If you have any specific data to add or point to (e.g. "Distro X requires both static and shared in the same package" or "CMake person Y thinks shared and static should be built at the same time"), contribute that

Since you seem unable to do your own research, here's an example that this "simplifcation" will break: https://gitweb.gentoo.org/repo/gentoo.git/tree/media-libs/libsoundio/libsoundio-1.1.0.ebuild

[andrewrk]

here's an example that this "simplifcation" will break: https://gitweb.gentoo.org/repo/gentoo.git/tree/media-libs/libsoundio/libsoundio-1.1.0.ebuild

hmm. Now I'm less sure about the best approach here. Status quo does seem reasonable. The proposal this issue makes is one of simplification, but does not fix behavior for any specific use cases. This makes me lean in favor of status quo.

Side note: you could enable JACK support in that package by patching 2 lines in libsoundio to match the function prototype from the version of JACK on the system. (This prototype is incorrect according to the official jack.h header which is why this problem exists in the first place). Either way, problem should go away after next jack2 release.

[hasufell]

I think this is the key point here - convention.

CMake also has the convention to not provide uninstall rules, so LFS users (or anyone else, temporarily in need to install to /usr/local) can go screw themselves. That's a pretty good starting point to understand why you shouldn't care about cmake convention. Another point is that the de-facto standard pkg-config is rather broken in "conventional cmake".

I guess we'll just have to go with any convention a random cmake dev imposes on us, so the rest of the users can go screw themselves...

[mdsitton]

Are these things really as mutually exclusive as people are making it out to be? Could both camps be satisfied with an extra options flag?

[andrewrk]

I agree that not providing an uninstall target is a missing feature in cmake. However, in favor of the convention argument - cmake does provide install_manifest.txt, which a distribution could use to provide an uninstall target, solving the problem system-wide. cmake users would need to do nothing for this to work. Versus if cmake users tried to solve the problem themselves we would end up with competing conventions and overall more complexity.

I guess we'll just have to go with any convention a random build system author imposes on us, so the rest of the users can go screw themselves...

At the end of the day I'm going to go with the thing that is best for the users. This means people building libsoundio from source directly and it also includes folks such as yourself who are maintaining packages. I'm on your side. Let's figure out what is best for users together.

[hasufell]

@mdsitton there's no need for that, the real problem here was:

The examples are hard-coded to use the shared form of the library.

which means there is a bug. It can be fixed...

[andrewrk]

I agree. I'll fix the problem of the examples requiring the shared form. Then I will close this issue, since it is not clear what the actual issue is. Anyone is free to open new issues with specific problems and use cases.

[iskunk]

My original report was basically "It doesn't have to be that complicated" :-]

I would point out that if shared+static libraries in a single CMake build are desirable, then this should be implemented as a CMake module and pulled in (e.g. include(BuildSharedAndStatic)). Otherwise, you're not only re-creating Libtool-like functionality, you're also maintaining it as an integral part of libsoundio from here on out.

Re uninstall rule: This is addressed in the CMake FAQ. Keep in mind that Unix makefiles are only one of the many build mechanisms supported by CMake (in addition to Visual Studio solutions, Xcode projects, etc.).

There's plenty more critique that can be leveled at CMake-generated makefiles, not least that they are baroquely complex and don't respect subdirectory scoping (a side effect of having the same semantics as IDE projects). At the end of the day, if you want a build system that acts like Autotools, then you're much better off going with Autotools proper rather than trying to beat something else into a similar shape.

(That would actually be feasible, as that handily addresses all systems except for [non-MinGW] Windows. And I could submit a straightforward NMAKE makefile for that.)

[0joshuaolson1]

@andrewrk I too would appreciate the ability to build examples statically. I'm having problems trying to use libsoundio.a and the .o files with it from another programming language, so the examples are the only way I can think of to determine whether the problem is with the language's compiler options or my libsoundio build.

[0joshuaolson1]

I'm not seeing any combination of cmake flags that prevents make from building libsoundio.so* and having the example programs depend on it. Is -DBUILD_SHARED_LIBS=0 -DBUILD_STATIC_LIBS=1 not the way to invoke this fix?

[andrewrk]

-DBUILD_SHARED_LIBS=no will only build static libs and example programs will depend on static libs.

[0joshuaolson1]

Still not working for me. Another git pull of the v2 branch shows my download is up to date. I was using rsoundio's flags as the example. The Arch User Repository flags use =OFF. Even using no, the very first thing make does is Scanning dependencies of target libsoundio_shared. If I omit -DBUILD_STATIC_LIBS=yes etc., It doesn't print that but still goes into libsoundio_shared.dir.

[andrewrk]

sorry, the option is called BUILD_DYNAMIC_LIBS not BUILD_SHARED_LIBS:

option(BUILD_DYNAMIC_LIBS "Build dynamic libraries" ON)

so you want -DBUILD_SHARED_LIBS=OFF

[0joshuaolson1]

Thanks a bunch! Do you think the available/used flags should be added to the documentation? For people like me, or package managers.

[andrewrk]

They're all kinda at the top of CMakeLists.txt which is where I would expect package managers to look:

option(BUILD_STATIC_LIBS "Build static libraries" ON)
option(BUILD_DYNAMIC_LIBS "Build dynamic libraries" ON)
option(BUILD_EXAMPLE_PROGRAMS "Build example programs" ON)
option(BUILD_TESTS "Build tests" ON)
option(ENABLE_JACK "Enable JACK backend" ON)
option(ENABLE_PULSEAUDIO "Enable PulseAudio backend" ON)
option(ENABLE_ALSA "Enable ALSA backend" ON)
option(ENABLE_COREAUDIO "Enable CoreAudio backend" ON)
option(ENABLE_WASAPI "Enable WASAPI backend" ON)

where would you put this documentation?

[0joshuaolson1]

Changelog? You're right though that they might check CMakeLists again at a major release.

[hasufell]

where would you put this documentation?

A file called INSTALL. Some projects have an even more specific file PACKAGING (or similar).

[andrewrk]

Yeah we can fix that. I'll take a look soon. Feel free to make a PR if you
beat me to it.

On Oct 20, 2016 10:37 PM, "Joshua Olson" notifications@github.com wrote:

@andrewrk https://github.com/andrewrk I'd also appreciate the ability
to build examples statically. I'm having problems trying to use
libsoundio.a and the .o files with it from another programming language
http://chapel.cray.com/, so the examples are the only way I can think
of to demonstrate that the problem is with the language and not my
libsoundio build.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#106 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAGgD9KgFwNkTriH-aWyAM65_xQebxbtks5q2CVRgaJpZM4KIT9K
.