make android redownloads ndk on every build

[mikemorris]

This makes testing any Android build system changes quite slow.

[incanus]

Currently looking into this as part of #2344.

[incanus]

From what I can see, the various Mason scripts involved use mason_download, which does in fact check mason_packages/.cache. Digging further.

[incanus]

Checking this list of NDK downloads and hashes — I have a hunch we have a bad hash and that is causing the re-download every time.

https://gist.github.com/incanus/016ac1d43fdc4fae8978

[incanus]

And the scripts are re-downloading arm-9-r10e.tar.gz which isn't mentioned in Mason...

[incanus]

@mikemorris Are you on 10.10 or 10.11? I'm on 10.11.

[incanus]

I think the problem is no osx-10.11 cache:

$ aws s3 ls s3://mason-binaries/ | grep osx
                           PRE osx-10.10/
                           PRE osx-10.9/

[incanus]

Not necessarily the problem, as this version is based on:

xcrun --sdk macosx --show-sdk-version

Which returns 10.10 even on 10.11, depending on Xcode version currently in use.

[incanus]

This may however be a discrepancy between the SDK version and the host version, the latter of which is used for the cache path.

[incanus]

Mason seems to be downloading nested binaries, for example trying to write the NDK to here:

.../gl-native/mason_packages/.build/geojson-vt-cpp-1.1.0/mason_packages/.binaries/osx-10.10/android-ndk/arm-9-r10e.tar.gz

Now this seems related to the geojson-vt changes in #2348.

[incanus]

I'm going to wait until the npm-related changes are reverted in master in #2349 before pursuing this any further.

[incanus]

Confirmed — once #2349 landed, the NDK is only downloaded once.

[ljbade]

On Linux the NDK is cached fine, so never noticed this issue.

[ljbade]

Oh, @incanus actually yes I remember seeing NDK redownload yesterday and put it down to random network glitch (that was before I was aware of #2348)

[mikemorris]

I think the problem is no osx-10.11 cache

I was seeing this on OS X 10.10.5, xcrun --sdk macosx --show-sdk-version returns 10.11 though. That directory nesting does look wrong.

[incanus]

Yeah, this was related to the now-reverted geojson-vt change using it's own nested Mason and installing the NDK multiple times. That's gone now.

[mikemorris]

So the initial bug is that there isn't a matching binary published yet...

* Building for Platform 'android/arm-v7-9'...
* Loading install script 'https://github.com/mapbox/mason/blob/android-ndk-arm-9-r10e/script.sh'...
* Downloading binary package osx-10.11/android-ndk/arm-9-r10e.tar.gz...
curl: (22) The requested URL returned error: 403 Forbidden
* Binary not available yet for osx-10.11/android-ndk/arm-9-r10e.tar.gz
* Downloading http://dl.google.com/android/ndk/android-ndk-r10e-darwin-x86_64.bin...

But after downloading the ndk source, everything gets moved into the geojson-vt-cpp directory for some reason and then starts building for the wrong platform?

* Building for Platform 'osx/10.11'...
Copying prebuilt binaries...
Copying sysroot headers and libraries...
Copying c++ runtime headers and libraries (with libc++abi)...
Copying files to: /Users/mikemorris/Projects/mapbox-gl-native/mason_packages/.build/geojson-vt-cpp-1.1.0/mason_packages/osx-10.11/android-ndk/arm-9-r10e

[incanus]

I think we are talking about two different things.

1. What I was seeing was with a 10.10 SDK, so the osx-10.10/android-ndk/arm-9-r10e.tar.gz binary did exist, and was getting downloaded, but geojson-vt was nesting it one or more times, so it kept re-downloading it.
2. What you are seeing @mikemorris is due to the osx-10.11/android-ndk/arm-9-r10e.tar.gz binary not existing, getting downloaded from dl.google.com instead of S3, but also having some problem relating to geojsonvt copying files.

[incanus]

FWIW with a 10.10 SDK, I no longer see my problem as of #2349.

[mikemorris]

Looks like geojsonvt-1.1.0 is overwriting the MASON_BUILD_PATH at https://github.com/mapbox/mason/blob/geojsonvt-1.1.0/script.sh#L16? Although other packages seem to do this as well...

[ljbade]

Does this still happen?

[mikemorris]

Think this was "fixed" by adding a mason binary.

[ljbade]

Ah yeah that's the usual way to prevent recompiling.