Popular formulae with bottling trouble on 10.12

[zmwangx]

Here's a list of popular formulae (among the 500 most installed according to our analytics) that had trouble being bottled for Sierra. The checklist is followed by a table with more details. Please help us fix your favorite formula(e)!

• xctool (Project status inquiry w.r.t. Xcode 8 facebookarchive/xctool#716, asked Sep 30, 2016)
• swiftlint (Build fails on macOS 10.12 Sierra GM realm/SwiftLint#815)
• pypy (upstream issue)
• sbcl (sbcl on macOS Sierra #5125 release planned)
• erlang
• elixir
• ilmbase
• rust
• pyqt (no Sierra for you)
• android-sdk (failure not related to 10.12)
• gdal
• libspatialite (caveat: check_sql_stmt check fails — happens on 10.10 and 10.11 too — and we ignored that)
• libressl
• haskell-stack (GHC panic when building Stack on macOS Sierra commercialhaskell/stack#2577)
• packer (failure not related to 10.12)
• valgrind (not Sierra-ready at the moment)
• postgis
• bazel
• boot2docker
• cocoapods
• wxpython
• mpg123

Qt shitshow:

• qt (no Sierra for you)
• qt5
• qscintilla2
• pyqt5
• qwt

Details

Note:

1. Build numbers are for https://bot.brew.sh/job/Homebrew%20Sierra%20Testing/.
2. The table is also in this gist where you can search in the tsv.

Rank	Formula	Build #	Deps satisfied	Failure stage	Comment
90	qt5	744	✔︎	install
144	xctool	744	✔︎	install
226	swiftlint	750	✔︎	install
240	qscintilla2	-	✗	-	depends on qt5
317	packer	751	✔︎	test
338	pypy	746	✔︎	install	ld: library not found for -lrt
414	pyqt5	-	✗	-	depends on qt5
445	qwt	-	✗	-	depends on qt

[DomT4]

0e99665 - That's the qt problem solved.

[ilovezfs]

@damienmg is upstream aware of the bazel build issue on Sierra?

[zmwangx]

@DomT4 For the goddamn qt tree:

$ brew uses --recursive qt
automoc4          coin              ezlupdate         gnuradio          liblastfm         pyqt              pyside            qjson             qwtpolar          shiboken          valkyrie        
binwalk           cuty_capt         frescobaldi       gpsbabel          libqglviewer      pyqwt             qbzr              quazip            qxmpp             sqliteman         weboob          
bzr-explorer      djview4           git-cola          libechonest       puddletag         pyside-tools      qca               qwt               rcssserver        treeline

Is it necessary to specify the maximum OS requirement individually? (Of course I hope some of those qt dependencies could be made optional.)

[ilovezfs]

@zmwangx see all of the great progress? #1705

[damienmg]

@ilovezfs Yes we got a bug report earlier this week.

[zmwangx]

@ilovezfs You forgot the "in progress" label apparently.

[ilovezfs]

@damienmg OK, thanks for the quick response! Hopefully, it can be fixed soon since the Sierra GM is in the wild :)

[DomT4]

For the goddamn qt tree:

As soon as we get Qt5 working on Sierra quite a lot of those will likely be migrated in a wild rush. MMOSR drips down automatically unless you already have qt installed prior to the Requirement coming into effect.

[zmwangx]

MMOSR drips down automatically

Oh cool, good to know.

[zmwangx]

Marking pyqt as resolved since apparently it won't get any love.

[ilovezfs]

@zmwangx seems like maybe we should turn make check off on libspatialite and ship a bottle ...

[zmwangx]

@ilovezfs It has that awkward --with-test/--without-test option... IMO we should remove that option entirely.

[ilovezfs]

I'm curious if we change it just without-test -> with-test if it can pass the rest of CI and then ditto for postgis and gdal which need it

[zmwangx]

I'm curious if we change it just without-test -> with-test if it can pass the rest of CI and then ditto for postgis and gdal which need it

Probably, but I'd rather just kill the option because no one in their right mind will ever use it (and indeed no one did).

#4845

[zmwangx]

@ilovezfs libspatialite, gdal, postgis done.

[ilovezfs]

@zmwangx nice work!

[zmwangx]

mpg123 issue has been resolved.

[zmwangx]

cocoapods issue has been resolved too. Making good progress!

[zmwangx]

valgrind now resolved, erlang and elixir along with honorable mention wxmac under way (I believe).

[ilovezfs]

@zmwangx great work!

[ilovezfs]

@zmwangx incidentally, the make check failure for libspatialite is fixed in upstream HEAD.

[zmwangx]

erlang and elixir resolved.

[zmwangx]

@ilovezfs Were you able to back port a single commit?

[ilovezfs]

@zmwangx nope, I didn't bisect it. But I'm even less worried about it now since it means upstream knows about it, fixed it, and didn't think it warranted a new release.

[LRFLEW]

Did some research into what's going on with the qt5 build, and it's less about not compiling on Sierra itself, and more that it won't compile with the 10.12 SDK at all (so I bet it won't build on 10.11 using Xcode 8).

The error in the compilation is not finding an implementation for the symbol _dyld_register_image_state_change_handler. From what I can tell, this was a private function used in the dyld library that some devs have used directly. The problem is that 10.12 seems to have changed the back-end of the dyld library a lot, including dropping that function. Comparing the libdyld.tbd files in the 10.11 and 10.12 SDK's shows this difference.

I'm having a hard time pinpointing the source file correlating to the object file that's failing to link (otest-shim.o), but I'll keep looking into it, and I'll see if the upstream has already made a patch for it.

EDIT: I think I may have been looking at the wrong build number. Either way, that may help for whatever project that issue is affecting.

EDIT: Yep, had the wrong build in that massive build. Turns out my description of the problem applies to xctool instead. There's an issue open for it on the project already.

[LRFLEW]

Took a look at the qt5 build (for real this time). The only thing in the logs I could see that could make the install fail was this:

./osx/osxbtcentralmanager_p.h:91:33: error: use of undeclared identifier 'CBService'; did you mean 'QBluetoothDeviceInfo::NoService'?
typedef QHash<QLowEnergyHandle, CBService *> ServiceHash;
                                ^~~~~~~~~

I can't find a previous build log to see if this error appeared there, so I'm not sure if this is specific to the 10.12 build. If this is new, then this could cause one of the modules to fail to build, which would cause the install script to report that it failed due to the missing file. I'm not sure why it can't find the identifier, though.

[UdjinM6]

If you are like me and you just need qt5 to be installed asap even though it would miss few modules you are not using currently:

diff --git a/Formula/qt5.rb b/Formula/qt5.rb
index f63a40f..3be6244 100644
--- a/Formula/qt5.rb
+++ b/Formula/qt5.rb
@@ -89,6 +89,9 @@ class Qt5 < Formula
       -qt-freetype
       -qt-pcre
       -nomake tests
+      -skip qtconnectivity
+      -skip qtwebengine
+      -skip qtwebview
       -no-rpath
     ]

[LRFLEW]

@UdjinM6 There are problems with QtWebEngine on Sierra?

If there are, that might be harder to simply fix. I remember that in the discussion in #2087, there was the idea that once Sierra was released and we stopped bottling for Mavericks, we could push the update, despite being unable to build on the old version. If back porting fixes to support Sierra is too problematic, we may need to go a route like this.

[ilidar]

@UdjinM6 thanks, saved my day.

[LRFLEW]

I would just like to point out that trying to build Qt 5.6.1-1 on my local machine fails at the configuration stage (due to xcrun -find xcrun failing for no obvious reason, but the line was changed to /usr/bin/xcrun -find xcodebuild in the 5.7 git branch)

EDIT: I figured out what exactly needed to be changed to fix the issue. Here is the diff:

diff --git a/qtbase/configure b/qtbase/configure
--- a/qtbase/configure
+++ b/qtbase/configure
@@ -543,7 +543,7 @@
         exit 2
     fi

-    if ! /usr/bin/xcrun -find xcrun >/dev/null 2>&1; then
+    if ! /usr/bin/xcrun -find xcodebuild >/dev/null 2>&1; then
         echo >&2
         echo "   Xcode not set up properly. You may need to confirm the license" >&2
         echo "   agreement by running /usr/bin/xcodebuild without arguments." >&2
diff --git a/qtbase/mkspecs/features/mac/default_pre.prf b/qtbase/mkspecs/features/mac/default_pre.prf
--- a/qtbase/mkspecs/features/mac/default_pre.prf
+++ b/qtbase/mkspecs/features/mac/default_pre.prf
@@ -12,7 +12,7 @@
         error("Xcode is not installed in $${QMAKE_XCODE_DEVELOPER_PATH}. Please use xcode-select to choose Xcode installation path.")

     # Make sure Xcode is set up properly
-    isEmpty($$list($$system("/usr/bin/xcrun -find xcrun 2>/dev/null"))): \
+    isEmpty($$list($$system("/usr/bin/xcrun -find xcodebuild 2>/dev/null"))): \
         error("Xcode not set up properly. You may need to confirm the license agreement by running /usr/bin/xcodebuild.")
 }

[zmwangx]

boot2docker resolved in #5264.

[zmwangx]

haskell-stack at least half-resolved in #5195.

[zmwangx]

packer resolved in #5058.

[zmwangx]

bazel resolved in #5270.

[zmwangx]

android-sdk resolved in 8e0acdf because no one cares about the broken dylib links.

[zmwangx]

rust resolved in #5392.

[zmwangx]

sbcl resolved in #5433.

[zmwangx]

This issue has served its purpose. Closing in favor of #5488 (which is now ordered by popularity too).

[mfukar]

#4841 (comment) states valgrind is resolved, and brew install valgrind reports:

valgrind: This formula either does not compile or function as expected on macOS
versions newer than El Capitan due to an upstream incompatibility.

Is there any more information on this incompatibility?

[zmwangx]

@mfukar Valgrind isn't supported on macOS 10.12, see #4879. You can prove me wrong by submitting a PR to fix it.