Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add MUE_COMPILE_USE_SYSTEM_OPUS cmake option to help prevent library conflicts #22243

Merged

Conversation

Chrisimx
Copy link
Contributor

@Chrisimx Chrisimx commented Apr 6, 2024

This PR creates a simple way for using the system library version of opus which solves the problem of at same time dynamically linking to system libopus.so through libMuseSamplerCore.so and statically linking to a different version in an elegant way. Using this option can prevent conflicts arising because of this leading to hard-to-diagnose bugs/crashes (e.g. #21838)

As has been shown with #21838 using two different versions of libopus can lead and has led to hard-to-diagnose problems as explained in #21838 (comment)
As there is API and ABI compatibility between different versions of libopus, there is nothing preventing us from both times linking to the same library instead of different versions by both times linking to the system opus lib.
This is often a more sensible option as it resolves the following issue: In the current state, each time a new opus sys lib is packaged the musescore packager would have to update/replace the libopus under src/framework/audio/thirdparty/opus with the system version and release a new package. While building they also need to make sure that the compile options always match to avoid crashes/bugs/breakage.
Relying on such exact timing and precision and always updating the opus and musescore package together is unfeasible, at the very least unnecessary and will likely lead to breakage on other distros (other than Arch Linux) as well in the future when they update their opus system library. Therefore, I propose adding the cmake variable to MUE_COMPILE_USE_SYSTEM_OPUS to make the job of package maintainers of MuseScore easier and more failsafe.

Resolves: #21838 (the issue is closed but there was no elegant and sustainable solution brought into musescore; this PR aims to be this)

  • I signed the CLA
  • The title of the PR describes the problem it addresses
  • Each commit's message describes its purpose and effects, and references the issue it resolves
  • If changes are extensive, there is a sequence of easily reviewable commits
  • The code in the PR follows the coding rules
  • There are no unnecessary changes
  • The code compiles and runs on my machine, preferably after each commit individually

@Chrisimx Chrisimx force-pushed the add-use-system-opus-cmake-option branch 3 times, most recently from 35aa39a to 1c5ef4a Compare April 6, 2024 18:15
@Chrisimx
Copy link
Contributor Author

Chrisimx commented Apr 6, 2024

I assume there's a problem with the pipeline because other PRs fail with the same error.

@Jojo-Schmitz
Copy link
Contributor

Yes, there are. Not your fault

@cbjeukendrup
Copy link
Contributor

I was wondering if it's possible to use pkg_check_modules's IMPORTED_TARGET option (https://cmake.org/cmake/help/latest/module/FindPkgConfig.html), in combination with an alias target (https://cmake.org/cmake/help/latest/manual/cmake-buildsystem.7.html#alias-targets).

The idea would be that if system opus is being used and is found, then you create an alias target pointing to the imported PkgConfig::OPUS target; and otherwise, you would use the opus target that arises from add_subdirectory.
Then you can unconditionally pass this target to MODULE_LINK in src/framework/audio/thirdparty/opusenc/CMakeLists.txt.

The advantage would be that you only need to pass this target to MODULE_LINK (from where it will eventually be used for target_link_libraries), and you don't need to worry about specifying include directories or compile options because that should go automatically if you pass a target to target_link_libraries.

I'm not completely sure whether that works immediately, and it might need some tweaking, but if it does work that would be quite nice.

@cbjeukendrup cbjeukendrup requested review from igorkorsukov and removed request for Jojo-Schmitz April 6, 2024 23:54
@Chrisimx
Copy link
Contributor Author

Chrisimx commented Apr 7, 2024

@cbjeukendrup I think it's definitely a good idea and makes things a little nicer. I tried to make it work but it would always not compile because it can't find the header files when I just add the target to MODULE_LINK and not add anything to MODULE_INCLUDE:

FAILED: src/framework/audio/CMakeFiles/audio.dir/Unity/unity_4_cxx.cxx.o 
/usr/bin/g++ -DDECODE_ON_THE_FLY -DFLAC__NO_DLL -DHAVE_CONFIG_H -DHAVE_MPGLIB -DHAVE_STDINT_H -DKORS_LOGGER_QT_SUPPORT -DKORS_PROFILER_ENABLED -DMUE_BUILD_APPSHELL_MODULE -DMUE_BUILD_CONVERTER_MODULE -DMUE_BUILD_CRASHPAD_CLIENT -DMUE_BUILD_DIAGNOSTICS_MODULE -DMUE_BUILD_IMAGESEXPORT_MODULE -DMUE_BUILD_IMPORTEXPORT_MODULE -DMUE_BUILD_INSPECTOR_MODULE -DMUE_BUILD_INSTRUMENTSSCENE_MODULE -DMUE_BUILD_NOTATION_MODULE -DMUE_BUILD_PALETTE_MODULE -DMUE_BUILD_PLAYBACK_MODULE -DMUE_BUILD_PROJECT_MODULE -DMUE_BUILD_UPDATE_MODULE -DMUE_COMPILE_USE_QTFONTMETRICS -DMUSE_MODULE_GLOBAL_LOGGER_DEBUGLEVEL -DNO_GLIB -DNO_THREADS -DOUTSIDE_SPEEX -DQT_CONCURRENT_LIB -DQT_CORE5COMPAT_LIB -DQT_CORE_LIB -DQT_DBUS_LIB -DQT_GUI_LIB -DQT_NETWORKAUTH_LIB -DQT_NETWORK_LIB -DQT_OPENGL_LIB -DQT_PRINTSUPPORT_LIB -DQT_QMLMODELS_LIB -DQT_QML_DEBUG -DQT_QML_LIB -DQT_QUICKCONTROLS2_LIB -DQT_QUICKTEMPLATES2_LIB -DQT_QUICKWIDGETS_LIB -DQT_QUICK_LIB -DQT_STATEMACHINE_LIB -DQT_SUPPORT -DQT_SVG_LIB -DQT_WIDGETS_LIB -DQT_XML_LIB -DRANDOM_PREFIX=opusenc_prefix -DSCRIPT_INTERFACE -DSTDC_HEADERS -DTAKEHIRO_IEEE754_HACK -DUSE_FAST_LOG -Daudio_QML_IMPORT=\"/home/chrisimx/CLionProjects/MuseScore/src/framework/audio/qml\" -Dflac_QML_IMPORT=\"\" -Dfluidsynth_QML_IMPORT=\"\" -Dglobal_EXPORTS=1 -Dglobal_QML_IMPORT=\"\" -Dieee754_float32_t=float -Dlame_QML_IMPORT=\"\" -Dopusenc_QML_IMPORT=\"\" -I/home/chrisimx/CLionProjects/MuseScore/cmake-build-debug/src/framework/audio -I/home/chrisimx/CLionProjects/MuseScore/src/framework/audio -I/home/chrisimx/CLionProjects/MuseScore/cmake-build-debug/src/framework/audio/audio_autogen/include -I/home/chrisimx/CLionProjects/MuseScore/cmake-build-debug -I/home/chrisimx/CLionProjects/MuseScore/src -I/home/chrisimx/CLionProjects/MuseScore -I/home/chrisimx/CLionProjects/MuseScore/framework -I/home/chrisimx/CLionProjects/MuseScore/framework/global -I/home/chrisimx/CLionProjects/MuseScore/framework/testing/thirdparty/googletest/googletest/include -I/home/chrisimx/CLionProjects/MuseScore/src/framework -I/home/chrisimx/CLionProjects/MuseScore/src/framework/global -I/home/chrisimx/CLionProjects/MuseScore/src/framework/testing/thirdparty/googletest/googletest/include -I/home/chrisimx/CLionProjects/MuseScore/src/framework/audio/thirdparty/fluidsynth/fluidsynth-2.3.3/include -I/home/chrisimx/CLionProjects/MuseScore/src/framework/audio/thirdparty/fluidsynth/fluidsynth-2.3.3/src -I/home/chrisimx/CLionProjects/MuseScore/src/framework/audio/thirdparty/fluidsynth/fluidsynth-2.3.3/src/external -I/home/chrisimx/CLionProjects/MuseScore/src/framework/audio/thirdparty/fluidsynth/fluidsynth-2.3.3/src/utils -I/home/chrisimx/CLionProjects/MuseScore/src/framework/audio/thirdparty/fluidsynth/fluidsynth-2.3.3/src/midi -I/home/chrisimx/CLionProjects/MuseScore/src/framework/audio/thirdparty/fluidsynth/fluidsynth-2.3.3/src/rvoice -I/home/chrisimx/CLionProjects/MuseScore/src/framework/audio/thirdparty/fluidsynth/fluidsynth-2.3.3/src/sfloader -I/home/chrisimx/CLionProjects/MuseScore/src/framework/audio/thirdparty/fluidsynth/fluidsynth-2.3.3/src/bindings -I/home/chrisimx/CLionProjects/MuseScore/src/framework/audio/thirdparty/fluidsynth/fluidsynth-2.3.3/src/synth -I/home/chrisimx/CLionProjects/MuseScore/src/framework/audio/thirdparty/fluidsynth/fluidsynth-2.3.3/src/drivers -I/home/chrisimx/CLionProjects/MuseScore/cmake-build-debug/src/framework/global -I/home/chrisimx/CLionProjects/MuseScore/cmake-build-debug/src/framework/audio/fluidsynth -I/home/chrisimx/CLionProjects/MuseScore/cmake-build-debug/src/framework/audio/lame -I/home/chrisimx/CLionProjects/MuseScore/src/framework/audio/thirdparty/lame/include -I/home/chrisimx/CLionProjects/MuseScore/src/framework/audio/thirdparty/lame/libmp3lame -I/home/chrisimx/CLionProjects/MuseScore/src/framework/audio/thirdparty/lame/mpglib -I/home/chrisimx/CLionProjects/MuseScore/src/framework/audio/thirdparty/lame -I/home/chrisimx/CLionProjects/MuseScore/cmake-build-debug/src/framework/audio/opusenc -I/home/chrisimx/CLionProjects/MuseScore/src/framework/audio/thirdparty/opusenc/libopusenc-0.2.1/include -I/home/chrisimx/CLionProjects/MuseScore/src/framework/audio/thirdparty/opusenc/libopusenc-0.2.1/src -I/home/chrisimx/CLionProjects/MuseScore/src/framework/audio/thirdparty/opusenc/libopusenc-0.2.1 -I/home/chrisimx/CLionProjects/MuseScore/cmake-build-debug/src/framework/audio/flac -I/home/chrisimx/CLionProjects/MuseScore/src/framework/audio/thirdparty/flac/flac-1.4.3 -I/home/chrisimx/CLionProjects/MuseScore/src/framework/audio/thirdparty/flac/flac-1.4.3/include -I/home/chrisimx/CLionProjects/MuseScore/src/framework/audio/thirdparty/flac/flac-1.4.3/src -I/home/chrisimx/CLionProjects/MuseScore/src/framework/audio/thirdparty/flac/flac-1.4.3/src/libFLAC/include -I/home/chrisimx/CLionProjects/MuseScore/src/framework/audio/thirdparty/flac/flac-1.4.3/src/libFLAC++/include -isystem /home/chrisimx/Qt/6.2.4/gcc_64/include/QtCore -isystem /home/chrisimx/Qt/6.2.4/gcc_64/include -isystem /home/chrisimx/Qt/6.2.4/gcc_64/include/QtGui -isystem /home/chrisimx/Qt/6.2.4/gcc_64/include/QtDBus -isystem /home/chrisimx/Qt/6.2.4/gcc_64/include/QtWidgets -isystem /home/chrisimx/Qt/6.2.4/gcc_64/include/QtNetwork -isystem /home/chrisimx/Qt/6.2.4/gcc_64/include/QtNetworkAuth -isystem /home/chrisimx/Qt/6.2.4/gcc_64/include/QtQml -isystem /home/chrisimx/Qt/6.2.4/gcc_64/include/QtQuick -isystem /home/chrisimx/Qt/6.2.4/gcc_64/include/QtQmlModels -isystem /home/chrisimx/Qt/6.2.4/gcc_64/include/QtOpenGL -isystem /home/chrisimx/Qt/6.2.4/gcc_64/include/QtQuickControls2 -isystem /home/chrisimx/Qt/6.2.4/gcc_64/include/QtQuickTemplates2 -isystem /home/chrisimx/Qt/6.2.4/gcc_64/include/QtQuickWidgets -isystem /home/chrisimx/Qt/6.2.4/gcc_64/include/QtXml -isystem /home/chrisimx/Qt/6.2.4/gcc_64/include/QtSvg -isystem /home/chrisimx/Qt/6.2.4/gcc_64/include/QtPrintSupport -isystem /home/chrisimx/Qt/6.2.4/gcc_64/include/QtCore5Compat -isystem /home/chrisimx/Qt/6.2.4/gcc_64/include/QtStateMachine -isystem /home/chrisimx/Qt/6.2.4/gcc_64/include/QtConcurrent -isystem /home/chrisimx/Qt/6.2.4/gcc_64/mkspecs/linux-g++ -g -std=gnu++17 -fdiagnostics-color=always -Wall -Wextra -fPIC -Winvalid-pch -include /home/chrisimx/CLionProjects/MuseScore/cmake-build-debug/src/framework/global/CMakeFiles/global.dir/cmake_pch.hxx -MD -MT src/framework/audio/CMakeFiles/audio.dir/Unity/unity_4_cxx.cxx.o -MF src/framework/audio/CMakeFiles/audio.dir/Unity/unity_4_cxx.cxx.o.d -o src/framework/audio/CMakeFiles/audio.dir/Unity/unity_4_cxx.cxx.o -c /home/chrisimx/CLionProjects/MuseScore/cmake-build-debug/src/framework/audio/CMakeFiles/audio.dir/Unity/unity_4_cxx.cxx
In file included from /home/chrisimx/CLionProjects/MuseScore/src/framework/audio/internal/encoders/oggencoder.cpp:25,
                 from /home/chrisimx/CLionProjects/MuseScore/cmake-build-debug/src/framework/audio/CMakeFiles/audio.dir/Unity/unity_4_cxx.cxx:4:
/home/chrisimx/CLionProjects/MuseScore/src/framework/audio/thirdparty/opusenc/libopusenc-0.2.1/include/opusenc.h:58:10: fatal error: opus.h: No such file or directory
   58 | #include <opus.h>
      |          ^~~~~~~~
compilation terminated.

I tried first implementing everything you mentioned in another branch of my fork in this commit: Chrisimx@1831ec9
Then, in another branch, I just removed the include directory of the opus lib from the MODULE_INCLUDE in this commit: 6473431
In any case it doesn't work, but I don't no why because the opus target clearly adds public include dirs:

target_include_directories(
opus
PUBLIC $<BUILD_INTERFACE:${CMAKE_CURRENT_SOURCE_DIR}/include>
$<INSTALL_INTERFACE:${CMAKE_INSTALL_INCLUDEDIR}>
PRIVATE ${CMAKE_CURRENT_BINARY_DIR}
${CMAKE_CURRENT_SOURCE_DIR}
celt
silk)

This creates a simple way for using the system library version of opus which solves the problem of at same time dynamically linking to system libopus.so through libMuseSamplerCore.so and statically linking to a different version in an elegant way. Using this option can prevent conflicts arising because of this leading to hard-to-diagnose bugs/crashes (e.g. musescore#21838)
@Chrisimx Chrisimx force-pushed the add-use-system-opus-cmake-option branch from 1c5ef4a to 8089082 Compare April 7, 2024 09:48
@cbjeukendrup
Copy link
Contributor

I commented on Chrisimx@1831ec9 that it contains a typo; maybe that is causing problems. Also, note that target_link_libraries only manages include directories when you pass another target to it (and that target has the correct public include dirs), and not when you pass a library path to it, so I think that's why this PR currently doesn't work.

@Chrisimx
Copy link
Contributor Author

Chrisimx commented Apr 8, 2024

@cbjeukendrup I've updated the branch to fix the typo. Unfortunately, it still does not work.

Isn't "opus" the name of the target that is created by the CMakeLists.txt in the thirdparty/opus directory here:

add_library(opus ${opus_sources} ${opus_sources_float})

I also tried instead creating an alias target pointing to it, so it would report if this target doesn't exist (and not think it's a lib name or path). This has shown that it correctly recognises the target. And I also think the right public include directory is added by the opus CMakeLists.txt:
target_include_directories(
opus
PUBLIC $<BUILD_INTERFACE:${CMAKE_CURRENT_SOURCE_DIR}/include>
$<INSTALL_INTERFACE:${CMAKE_INSTALL_INCLUDEDIR}>
PRIVATE ${CMAKE_CURRENT_BINARY_DIR}
${CMAKE_CURRENT_SOURCE_DIR}
celt
silk)

Finally, I've printed the INCLUDE_DIRECTORIES property of the opus target. It contained a path to the thidparty/opus/include dir but only with a CMake generator expression (BUILD_INTERFACE and INSTALL_INTERFACE in it). Therefore, I used target_include_directories to just added a simple path to the opus target for testing and it all the same didn't work.
I honestly have no idea yet what could be the reason. I'll maybe try again tomorrow or friday but we might have to consider just leaving the PR as is

@igorkorsukov igorkorsukov merged commit be54d1f into musescore:master Apr 9, 2024
10 of 11 checks passed
cbjeukendrup added a commit to cbjeukendrup/MuseScore that referenced this pull request Jun 9, 2024
cbjeukendrup added a commit to cbjeukendrup/MuseScore that referenced this pull request Jun 9, 2024
cbjeukendrup added a commit to cbjeukendrup/MuseScore that referenced this pull request Jun 9, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4.2.1 crashes upon loading sounds (Linux, when Opus lib updated to version 1.5)
4 participants