Building for MacOS fails

[emlynmac]

Describe the bug
Following the instructions here: https://github.com/jamulussoftware/jamulus/blob/master/COMPILING.md#macos leads to an error at the xcodebuild phase:

PhaseScriptExecution Qt\ Qmake .xcode/Jamulus.build/Release/Qt\ Preprocess.build/Script-5E618435888B9D49F8540165.sh
    cd /Volumes/Data/Users/emlyn/code/jamulus
    /bin/sh -c \"/Volumes/Data/Users/emlyn/code/jamulus/.xcode/Jamulus.build/Release/Qt\ Preprocess.build/Script-5E618435888B9D49F8540165.sh\"
make: `Jamulus.xcodeproj/project.pbxproj' is up to date.

PhaseScriptExecution Qt\ Preprocessors .xcode/Jamulus.build/Release/Qt\ Preprocess.build/Script-7EF0942E79C014DCEC8976BC.sh
    cd /Volumes/Data/Users/emlyn/code/jamulus
    /bin/sh -c \"/Volumes/Data/Users/emlyn/code/jamulus/.xcode/Jamulus.build/Release/Qt\ Preprocess.build/Script-7EF0942E79C014DCEC8976BC.sh\"
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang -msse  -I. -Isrc -Ilibs/opus/include -Ilibs/opus/celt -Ilibs/opus/silk -Ilibs/opus/silk/float -Ilibs/opus/silk/fixed -Ilibs/opus -I/usr/local/Cellar/qt@5/5.15.2/lib/QtWidgets.framework/Headers -I/usr/local/Cellar/qt@5/5.15.2/lib/QtMacExtras.framework/Headers -I/usr/local/Cellar/qt@5/5.15.2/lib/QtGui.framework/Headers -I/usr/local/Cellar/qt@5/5.15.2/lib/QtNetwork.framework/Headers -I/usr/local/Cellar/qt@5/5.15.2/lib/QtXml.framework/Headers -I/usr/local/Cellar/qt@5/5.15.2/lib/QtConcurrent.framework/Headers -I/usr/local/Cellar/qt@5/5.15.2/lib/QtCore.framework/Headers -I. -I/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.3.sdk/System/Library/Frameworks/OpenGL.framework/Headers -I/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.3.sdk/System/Library/Frameworks/AGL.framework/Headers -I. -I/usr/local/Cellar/qt@5/5.15.2/mkspecs/macx-clang -F/usr/local/Cellar/qt@5/5.15.2/lib -c libs/opus/celt/x86/pitch_sse.c -o pitch_sse.o
In file included from libs/opus/celt/x86/pitch_sse.c:34:
libs/opus/celt/stack_alloc.h:39:2: error: "Opus requires one of VAR_ARRAYS, USE_ALLOCA, or NONTHREADSAFE_PSEUDOSTACK be defined to select the temporary allocation mode."
#error "Opus requires one of VAR_ARRAYS, USE_ALLOCA, or NONTHREADSAFE_PSEUDOSTACK be defined to select the temporary allocation mode."
 ^
1 error generated.
make: *** [pitch_sse.o] Error 1

To Reproduce
Follow the instructions on Xcode 12.5 / macOS Big Sur 11.4

Expected behavior
Clean build, no errors

Operating system
macOS 11.4, Xcode 12.5, qt5 from home-brew

Version of Jamulus
master as of 9am PDT June 7 2021

Additional context
Looks like some OPUS compiler flags need to be defined somewhere

[npostavs]

Running make clean (EDIT: or whatever XCode equivalent is) first might help (assuming this isn't a totally fresh checkout).

Otherwise I'm not sure how USE_ALLOCA isn't getting defined, because it gets added to OPUS_DEFINES unconditionally.

The bit where x86 gets added to CONFIG for macOS does look a little suspicious though.

[emlynmac]

A totally fresh checkout unfortunately.
I'm looking at the generated qt_preprocess.mak and see the right DEFINES at the top. However, this probably should be CFLAGS and not DEFINES?
It builds fine then...

[emlynmac]

Adding:
CFLAGS += DEFINES
at line 179 in Jamulus.pro seems to help, but I'm not sure if that is how this should be fixed...

[hoffie]

Thanks for the report.

Just a short note: The official Mac binaries are currently built with Xcode 11.7 (maybe with 12.1 soon) as those are the last versions which still ship SDK 10.15. The latter is the highest version which is supported by Qt5 right now.

It would still be interesting to find the cause for this issue though. However, even if found and fixed, Qt5 may show problems with newer SDKs.

Any specific reason for building yourself -- development?

cc @dcorson-ticino-com who has recently worked on Qt6 compatibility and @softins who has most-recently handled the Xcode version pinnings.

Adding:
CFLAGS += DEFINES
at line 179 in Jamulus.pro seems to help, but I'm not sure if that is how this should be fixed...

Yeah, the log you posted does not seem to contain any defines in the clang call, which is strange. I could imagine that this is indeed caused by a Qt/qmake incompatibility with SDK 11.3?

[emlynmac]

Interesting, I'll try with latest QT and see if that works. - never mind, I'll not do that yet now I see that is a WiP.

As for why, I'm hoping to dig into this: #1840

I want to make multi-track input more flexible.
About a year ago, I was helping with getting multi-channel mixers working and I'd really like to have the possibility to have them as discrete mixable channels.

[dcorson-ticino-com]

I made the necessary changes to get Jamulus to compile under Qt6.1.0 without errors locally.
The github autobuild for Mac is made without errors, but as I have no Mac I can't test the results.
You can download the Mac binary at https://github.com/dcorson-ticino-com/jamulus/suites/2929013840/artifacts/65759590
For the other platforms I have not yet been able to get the autobuilds using Qt6 to run without errors.

[emlynmac]

Thanks @dcorson-ticino-com - I'll pull your repo and work with that one then for now.

are there plans for qt6 to get merged soon or later?

[ghost]

I think it is better to drop Qt, especially for MacOS, and use native programming for MacOS. I watched a Apple seminar where they said not to use wrappers around their libraries (Qt yeah). I think that keeping up with MacOS using Qt will be bad for Jamulus as a whole. Better to drop Qt, or stay with Qt5, than move to Qt6.

[dcorson-ticino-com]

Programming just for a minority portion of the computer users out there, or maintaining multiple code bases, no thank you.
If not Qt, then another platform that runs at least on Win10, Linux and Mac.

[dcorson-ticino-com]

are there plans for qt6 to get merged soon or later?

@emlynmac For the moment the idea is to see if Qt6 solves the very opaque problems we are now seeing on Macs. If that is the case I'm sure transferring to Qt6 will go quickly. The problem is that Qt6 does not support much of the legacy Mac hardware we are used to supporting.

If you get the Qt6 version running please let us know how it behaves, both from the UI problems we have seen and the degradation of audio quality over time some users are seeing. It is very strange that this is only manifesting itself on newer Mac units and not at all on the other platforms.

[emlynmac]

Qt6 version compiled happily for me and seems to run. Need to move the build to my main music Mac next time we do band and see how it behaves.

Update- I've got it looping back my local audio via the server and will let it run for the day while I work. So far it's going fine.

[henkdegroot]

I had a similar problem on my system when manually running the xcodebuild build command.
As a knew the "autobuild" is working fine, I thought about running that script and that works for me.

So instead of running "xcodebuild build", I just run ./mac/deploy_mac.sh. (from the folder I downloaded the source code into)

Guess somewhere in that script a variable is set correct which makes things work.

[dcorson-ticino-com]

Update- I've got it looping back my local audio via the server and will let it run for the day while I work. So far it's going fine.

The problem might be related to if Jamulus or its windows are open in the background.
If you can, please try if there are audio problems or increasing memory usage with:
settings window open in foreground
settings window open in background
Jamulus in the background
Jamulus in the foreground.

[ann0see]

Closing since this issue is related to missing Qt6 support.

[ann0see]

Re-opening as I can reproduce it with Qt 6.3.1 on Xcode 13.3.1

[ann0see]

Ok. git bisect gave this as bad commit: 28a66e2

Tagging
@npostavs

[pljones]

@emlynmac @gilgongo @ann0see Is this still an issue? If not, please close.

[ann0see]

Yes. Still an issue

[ann0see]

I think this might be related to #2806

Opus wants that USE_ALLOCA is set, but somehow this doesn't get passed through if I build with Xcode.

[ann0see]

Ok. Small update: It could be related to the compiler flag - it needs to be checked what clang needs for the optimizations.

At least it builds now, if msvc and Mac-xcode use the simple case

jamulus/Jamulus.pro

Lines 1132 to 1136 in e45ad67

	msvc {
	# According to opus/win32/config.h, "no special compiler
	# flags necessary" when using msvc. It always supports
	# SSE intrinsics, but does not auto-vectorize.
	SOURCES += $$SOURCES_OPUS_ARCH

(but crashes with an error that it doesn't find the cocoa plugin (https://forum.qt.io/topic/98816/qt-could-not-find-the-platform-plugin-cocoa))

[ann0see]

@npostavs First of all thank you for your answer:

I don't know much about macOS, and I definitely have no idea of what the difference between macx and macx-xcode is.

These flags are to enable specific hardware instructions & corresponding instrinsic functions for various generations of x86 chips (except the neon one which is for some version of ARM chips, of course). They are listed in the gcc manual here: https://gcc.gnu.org/onlinedocs/gcc-13.1.0/gcc/x86-Options.html#index-mmmx

Are you perhaps able to run autoreconf && ./configureunder thelibs/opus` directory with the macx-xcode toolchain to see what flags it selects? (assuming whatever this xcode thing is, it's able to run autotools)

What flags are being used by the current Jamulus.pro, and what sort of errors is the macx-xcode compiler emitting?

autoreconf doesn't even run on macOS - as autoconf doesn't seem to be installed by default. However, I don't think this is the issue.

./configure

gives

... removed ...
  opus 1.3.1:  Automatic configuration OK.

    Compiler support:

      C99 var arrays: ................ yes
      C99 lrintf: .................... yes
      Use alloca: .................... no (using var arrays)

    General configuration:

      Floating point support: ........ yes
      Fast float approximations: ..... no
      Fixed point debugging: ......... no
      Inline Assembly Optimizations: . No inline ASM for your platform, please send patches
      External Assembly Optimizations: 
      Intrinsics Optimizations: ...... x86 SSE SSE2 SSE4.1 AVX
      Run-time CPU detection: ........ x86 AVX
      Custom modes: .................. no
      Assertion checking: ............ no
      Hardening: ..................... yes
      Fuzzing: ....................... no
      Check ASM: ..................... no

      API documentation: ............. yes
      Extra programs: ................ yes

Which seems ok.

As mentioned above, clicking "play" in Xcode gives

/jamulus/libs/opus/celt/stack_alloc.h:39:2 "Opus requires one of VAR_ARRAYS, USE_ALLOCA, or NONTHREADSAFE_PSEUDOSTACK be defined to select the temporary allocation mode."

Even though USE_ALLOCA is set via Jamulus.pro and also appears in the xcodeproj bundle in qt_preprocess.mak

DEFINES       = -DAPP_VERSION=\"3.9.1dev-d7465f55-dirty\" -DCUSTOM_MODES -D_REENTRANT -DQT_NO_DEPRECATED_WARNINGS -D_OSX -DOPUS_X86_MAY_HAVE_SSE -DOPUS_X86_MAY_HAVE_SSE2 -DOPUS_X86_MAY_HAVE_SSE4_1 -DOPUS_X86_PRESUME_SSE=1 -DOPUS_X86_PRESUME_SSE2=1 -DCPU_INFO_BY_C -DOPUS_BUILD=1 -DUSE_ALLOCA=1 -DOPUS_HAVE_RTCD=1 -DHAVE_LRINTF=1 -DHAVE_LRINT=1 -DQT_NO_DEBUG -DQT_WIDGETS_LIB -DQT_MULTIMEDIA_LIB -DQT_GUI_LIB -DQT_NETWORK_LIB -DQT_XML_LIB -DQT_CONCURRENT_LIB -DQT_CORE_LIB

[npostavs]

Even though USE_ALLOCA is set via Jamulus.pro and also appears in the xcodeproj bundle in qt_preprocess.ma

Okay, looking over this thread, it seems like the issue is that DEFINES is not being included into CFLAGS in this configuration for whatever reason.

Just to confirm, in the Makefile generated on my Linux box I have

# Command: /usr/lib/qt5/bin/qmake -o Makefile ../Jamulus.pro CONFIG+=headless
[...]
CFLAGS        = -pipe -O2 -D_REENTRANT -Wall -W -fPIC $(DEFINES)

[ann0see]

CFLAGS doesn't even seem to occur in the make files included in the Xcode file. However, in another file there's the following, which looks good.

OTHER_CFLAGS = (
					"-pipe",
					"-O2",
					"-Wall",
					"-Wextra",
					"-DOPUS_BUILD",
					"-DUSE_ALLOCA",
					"-DAPP_VERSION=\\\"3.9.1dev-d7465f55-dirty\\\"",
					"-DCUSTOM_MODES",
					"-D_REENTRANT",
					"-DQT_NO_DEPRECATED_WARNINGS",
					"-DQT_NO_DEBUG",
					"-DQT_WIDGETS_LIB",
					"-DQT_MULTIMEDIA_LIB",
					"-DQT_GUI_LIB",
					"-DQT_NETWORK_LIB",
					"-DQT_XML_LIB",
					"-DQT_CONCURRENT_LIB",
					"-DQT_CORE_LIB");

[ann0see]

Ok. Something unrelated (?) I succeeded to build the latest build by basically using a conditional "not macx" all your changes in Jamulus.pro and moving the PlugIns folder from an existing Qt6 build to the Jamulus.app/Contents folder - otherwise the app wouldn't start. I believe something bigger is wrong. This was to fix #1841 (comment)

[ann0see]

Maybe the files are included in a wrong order?

[ann0see]

I believe I have a working version now. Probably it's like Windows - but I'm not sure if the optimizations are still in.

[pljones]

Doesn't

Use alloca: .................... no (using var arrays)

mean "don't says "-DUSE_ALLOCA", say something else"?

[ann0see]

Doesn't

Use alloca: .................... no (using var arrays)

mean "don't says "-DUSE_ALLOCA", say something else"?

I think we use ALLOCA while the default might not?

[npostavs]

mean "don't says "-DUSE_ALLOCA", say something else"?

Yes, it turns that libopus' ./configure on macx-xcode happens to make a different choice than we're currently using. However, the compile error is due the missing DEFINES/CFLAGS which prevents any choice from being made at all.

[ann0see]

I mean see my PR #3076 I think I've at least partly fixed it. The app now compiles.