go 1.12 binaries rejected from mac app store due to ___getdirentries64

[zx2c4]

The release notes mention something clever with regards to iOS, but not with regards to macOS. Here's a rejection I just received from Apple:

Guideline 2.5.1 - Performance - Software Requirements

Your app uses or references the following non-public APIs:

Symbol: ___getdirentries64
From framework: /usr/lib/libSystem.B.dylib
In binary: Contents/PlugIns/WireGuardNetworkExtension.appex/Contents/MacOS/WireGuardNetworkExtension

The use of non-public APIs is not permitted on the App Store, because it can lead to a poor user experience should these APIs change.

We are constantly reevaluating and identifying non-public APIs that you may have been using for an extended period of time. You should always use public APIs and frameworks and ensure they are up-to-date to prevent this issue in the future.

It then goes on with the usual boiler plate.

Note that this is for the macOS app store.

As I'm not using getdirentries64 (at least, I don't think I am...), I'll probably patch the runtime in some grotesque and heavy-handed way to make the problem go away in the short term. But Go really needs a proper solution to this. I imagine they're something like one of these:

• Find a different syscall that isn't getdirentries, or
• When being built for the app store, make the actual syscall instead of going through libsystem, or
• Load the symbol with dlsym lazily as needed, rather than declaring it in the ELF, so that only users who use this get flagged, or
• Remove this from go all together.

[zx2c4]

These are somewhat grotesque, but I just resubmitted using these:

https://git.zx2c4.com/wireguard-ios/tree/wireguard-go-bridge/goruntime-syscall-remove-getdirentries.diff
https://git.zx2c4.com/wireguard-ios/tree/wireguard-go-bridge/sys-unix-remove-getdirentries.diff

[zx2c4]

This appears to be an unforeseen extension of #28984 . CC @eliasnaur

[eliasnaur]

It seems that my fix for iOS needs to cover all of Darwin. I don't have the bandwidth right now, so I'm hoping someone else will take this up. Perhaps @randall77 can take a look.

[zx2c4]

Was your iOS fix something beyond just ENOSYS'ing those syscalls? If so, those two patches I linked above basically do that, and I can submit them to gerrit. Or did you actually wind up wrapping libc's readdir as a replacement?

[eliasnaur]

Yes, I had to replace getdirentries in the directory listing code in the runtime. See https://go-review.googlesource.com/c/go/+/153338/. There was also an issue with the "64" suffixed system calls, but that might be different for macOS.

[randall77]

I think we'll need to do the same thing as CL 153338 for macOS.

[zx2c4]

Thanks for making it as cherrypickcandidate. I assume that means it'll wind up in 1.12.2, as this is sort of a significant regression.

[gopherbot]

Change https://golang.org/cl/168479 mentions this issue: syscall: deprecate Getdirentries on darwin

[randall77]

@zx2c4, if you could patch the above CL in and try it, that would help.

[zx2c4]

I'll give it a go and report back. Note that you'll also need a corresponding one in x/sys, because that also links to the symbol.

[randall77]

@zx2c4: right.

[andybons]

@zx2c4 any word on trying the patch?

[zx2c4]

The CL still has crashes, last I read. Meanwhile I'm just ENOSYSing it out, which is suitable for my purposes.

[randall77]

@gopherbot please consider this for backport to 1.12.

[gopherbot]

Backport issue(s) opened: #31244 (for 1.12).

Remember to create the cherry-pick CL(s) as soon as the patch is submitted to master, according to https://golang.org/wiki/MinorReleases.

[mikepulaski]

Thanks @randall77! I've just received a rejection from the Mac App Store for this very issue. I'll use version 1.11 until this makes it into an official release.

[constantinevassil]

If you submitted to AppStore for review was it with BITCODE = YES?

I cannot use gomobile bind -target=ios ... in a way when the framework is imported
in Xcode to compile with BITCODE = YES

How we solve this issue?