add compiler package script

[cosmicexplorer]

The new script ./build-compiler-5.0.1.sh will package an archive named compiler.tar.gz for both MacOS and Linux when executed in a Linux environment. This archive currently contains the clang compiler and the C standard library, which is sufficient to compile the python_dist() test project -- this can be tested by running ./pants run testprojects/src/python/python_distribution/fasthello_with_install_requires:main_with_no_conflict.

Notes

1. This script pulls down a binary release of clang+llvm from the llvm downloads page for MacOS, but we have to build the Linux package from a source release (downloaded from the same page). This means that the script should be executed in a Linux environment to produce both MacOS and Linux packages.
2. LLVM really likes .tar.xz files for some reason, so xz is needed to unpack their archives -- you will need to run yum install -y xz if using the CentOS Dockerfile available in the pants codebase.
3. This script executes the build-cmake-3.9.5.sh script already available in this repo to produce a cmake binary that can be used to build LLVM from source. This is necessary because LLVM requires cmake >= 3.4, which is otherwise not available in the CentOS Dockerfile from the pants codebase. Because build-cmake-3.9.5.sh only builds for Linux x86_64, this script only builds a compiler for 64-bit Linux hosts. If 32-bit support becomes necessary, I can probably modify both scripts to do so, but I haven't investigated whether this is supported for either cmake or LLVM yet.

[stuhood]

Thanks, looks good. I've just updated the docker instructions a bit to synchronize them with pantsbuild/pants: #58. Please use those instructions to build the binaries in centos6 (rather than wheezy).

[stuhood]

compiler is too generic a name... clang, probably?

[cosmicexplorer]

clang is definitely better than compiler, but thinking about it now I might actually suggest to call it clang-llvm, or even clang+llvm, because the LLVM version (5.0.1) is what's used in the release download url from the LLVM downloads page, not the clang version. Also, the MacOS binary release download unpacks into an archive named clang+llvm-5.0.1-<platform>.tar.xz, so calling it clang+llvm would match that convention. It could be clang-llvm too, but I prefer clang+llvm because that makes it clear which part of clang+llvm-5.0.1.tar.gz is the package name and which part is the version.

[stuhood]

Are you planning to package something in here other than clang itself?

[cosmicexplorer]

Yes, the C and C++ standard library headers. The C standard library headers are already included -- I hadn't tested the C++ standard library headers because we don't have a python_dist() target with C++ code in testprojects/ yet in the pants repo. These would be provided with the compiler so that they are compatible with the compiler, and so we don't have to rely on headers existing in standard locations (which is a requirement for an internal consumer of python_dist() with native code).

I'm not planning to package anything that's not considered part of clang, however. It might make sense to just call it clang, by that logic. Since we're not packaging any other LLVM tools, it might be confusing/surprising to call it clang+llvm.

After concentrating on this for a bit, my vote goes to just calling it clang.

[cosmicexplorer]

Changed to build-clang-5.0.1.sh, producing clang.tar.gz in supportdir build-support/bin/clang in ad67ff4.

[stuhood]

clang

[cosmicexplorer]

Changed to clang in ad67ff4.

[benjyw]

Mod stu's suggestions.

[illicitonion]

LGTM modulo Stu's comments