-
Notifications
You must be signed in to change notification settings - Fork 37
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
add compiler package script #57
add compiler package script #57
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks, looks good. I've just updated the docker instructions a bit to synchronize them with pantsbuild/pants: #58. Please use those instructions to build the binaries in centos6
(rather than wheezy).
build-compiler-5.0.1.sh
Outdated
@@ -0,0 +1,141 @@ | |||
#!/bin/bash | |||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
compiler
is too generic a name... clang
, probably?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
clang
is definitely better than compiler
, but thinking about it now I might actually suggest to call it clang-llvm
, or even clang+llvm
, because the LLVM version (5.0.1
) is what's used in the release download url from the LLVM downloads page, not the clang version. Also, the MacOS binary release download unpacks into an archive named clang+llvm-5.0.1-<platform>.tar.xz
, so calling it clang+llvm
would match that convention. It could be clang-llvm
too, but I prefer clang+llvm
because that makes it clear which part of clang+llvm-5.0.1.tar.gz
is the package name and which part is the version.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Are you planning to package something in here other than clang
itself?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, the C and C++ standard library headers. The C standard library headers are already included -- I hadn't tested the C++ standard library headers because we don't have a python_dist()
target with C++ code in testprojects/
yet in the pants repo. These would be provided with the compiler so that they are compatible with the compiler, and so we don't have to rely on headers existing in standard locations (which is a requirement for an internal consumer of python_dist()
with native code).
I'm not planning to package anything that's not considered part of clang, however. It might make sense to just call it clang
, by that logic. Since we're not packaging any other LLVM tools, it might be confusing/surprising to call it clang+llvm
.
After concentrating on this for a bit, my vote goes to just calling it clang
.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Changed to build-clang-5.0.1.sh
, producing clang.tar.gz
in supportdir build-support/bin/clang
in ad67ff4.
build-compiler-5.0.1.sh
Outdated
CORRESPONDING_CLANG_BIN_VERSION='5.0' | ||
LLVM_RELEASE_BUILD_DIRNAME='llvm-tmp' | ||
LLVM_PANTS_ARCHIVE_NAME='compiler.tar.gz' | ||
COMPILER_SUPPORTDIR='build-support/bin/compiler' |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
clang
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Changed to clang
in ad67ff4.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Mod stu's suggestions.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM modulo Stu's comments
Add buildozer 0.6.0-80c7f0d45d7e40fa1f7362852697d4a03df557b3 (pantsbuild#56) Synchronize docker instructions with pantsbuild/pants. (pantsbuild#58) Add clang (pantsbuild#57) Switch to a workflow without committed binaries (pantsbuild#59) * Update the README to indicate that no new binaries should be committed here. * Formalize the script location to improve maintainability. Add Go 1.9.4 and 1.10 (pantsbuild#61) Here we add `build.sh` scripts for Go 1.9.4 and 1.10, and a helper script for generating the `build.sh` scripts. We also clarify in the docs what platforms are supported (i386 and macOS pre 10.8 are not supported going forward). update some of the commands described in the README (pantsbuild#63) add gcc 7.3.0 and clang 6.0.0 cross platform, binutils 2.30 for linux, edit clang 5.0.1 to conform to pantsbuild#59 (pantsbuild#62) ``` $ ./sync-s3.sh upload: bin/clang/linux/x86_64/6.0.0/build-cmake.sh to s3://binaries.pantsbuild.org/bin/clang/linux/x86_64/6.0.0/build-cmake.sh upload: bin/clang/linux/x86_64/6.0.0/build.sh to s3://binaries.pantsbuild.org/bin/clang/linux/x86_64/6.0.0/build.sh upload: bin/binutils/linux/x86_64/2.30/build.sh to s3://binaries.pantsbuild.org/bin/binutils/linux/x86_64/2.30/build.sh upload: bin/binutils/linux/x86_64/2.30/build-binutils.sh to s3://binaries.pantsbuild.org/bin/binutils/linux/x86_64/2.30/build-binutils.sh upload: bin/binutils/linux/x86_64/2.30/binutils.tar.gz to s3://binaries.pantsbuild.org/bin/binutils/linux/x86_64/2.30/binutils.tar.gz upload: bin/clang/mac/10.10/6.0.0/build-clang.sh to s3://binaries.pantsbuild.org/bin/clang/mac/10.10/6.0.0/build-clang.sh upload: bin/clang/mac/10.10/6.0.0/build.sh to s3://binaries.pantsbuild.org/bin/clang/mac/10.10/6.0.0/build.sh upload: bin/clang/linux/x86_64/6.0.0/clang.tar.gz to s3://binaries.pantsbuild.org/bin/clang/linux/x86_64/6.0.0/clang.tar.gz upload: bin/clang/mac/10.11/6.0.0/build-clang.sh to s3://binaries.pantsbuild.org/bin/clang/mac/10.11/6.0.0/build-clang.sh upload: bin/clang/mac/10.11/6.0.0/build.sh to s3://binaries.pantsbuild.org/bin/clang/mac/10.11/6.0.0/build.sh upload: bin/clang/linux/x86_64/6.0.0/build-clang.sh to s3://binaries.pantsbuild.org/bin/clang/linux/x86_64/6.0.0/build-clang.sh upload: bin/clang/linux/x86_64/6.0.0/build-clang-for-linux-with-cmake.sh to s3://binaries.pantsbuild.org/bin/clang/linux/x86_64/6.0.0/build-clang-for-linux-with-cmake.sh upload: bin/clang/mac/10.12/6.0.0/build-clang.sh to s3://binaries.pantsbuild.org/bin/clang/mac/10.12/6.0.0/build-clang.sh upload: bin/clang/mac/10.12/6.0.0/build.sh to s3://binaries.pantsbuild.org/bin/clang/mac/10.12/6.0.0/build.sh upload: bin/clang/mac/10.10/6.0.0/clang.tar.gz to s3://binaries.pantsbuild.org/bin/clang/mac/10.10/6.0.0/clang.tar.gz upload: bin/clang/mac/10.12/6.0.0/clang.tar.gz to s3://binaries.pantsbuild.org/bin/clang/mac/10.12/6.0.0/clang.tar.gz upload: bin/clang/mac/10.13/6.0.0/build-clang.sh to s3://binaries.pantsbuild.org/bin/clang/mac/10.13/6.0.0/build-clang.sh upload: bin/clang/mac/10.13/6.0.0/build.sh to s3://binaries.pantsbuild.org/bin/clang/mac/10.13/6.0.0/build.sh upload: bin/clang/mac/10.11/6.0.0/clang.tar.gz to s3://binaries.pantsbuild.org/bin/clang/mac/10.11/6.0.0/clang.tar.gz upload: bin/clang/mac/10.8/6.0.0/build.sh to s3://binaries.pantsbuild.org/bin/clang/mac/10.8/6.0.0/build.sh upload: bin/clang/mac/10.8/6.0.0/build-clang.sh to s3://binaries.pantsbuild.org/bin/clang/mac/10.8/6.0.0/build-clang.sh upload: bin/clang/mac/10.13/6.0.0/clang.tar.gz to s3://binaries.pantsbuild.org/bin/clang/mac/10.13/6.0.0/clang.tar.gz upload: bin/clang/mac/10.9/6.0.0/build-clang.sh to s3://binaries.pantsbuild.org/bin/clang/mac/10.9/6.0.0/build-clang.sh upload: bin/clang/mac/10.9/6.0.0/build.sh to s3://binaries.pantsbuild.org/bin/clang/mac/10.9/6.0.0/build.sh upload: bin/clang/mac/10.8/6.0.0/clang.tar.gz to s3://binaries.pantsbuild.org/bin/clang/mac/10.8/6.0.0/clang.tar.gz upload: bin/gcc/linux/x86_64/7.3.0/build.sh to s3://binaries.pantsbuild.org/bin/gcc/linux/x86_64/7.3.0/build.sh upload: bin/gcc/linux/x86_64/7.3.0/build-gcc.sh to s3://binaries.pantsbuild.org/bin/gcc/linux/x86_64/7.3.0/build-gcc.sh upload: bin/clang/mac/10.9/6.0.0/clang.tar.gz to s3://binaries.pantsbuild.org/bin/clang/mac/10.9/6.0.0/clang.tar.gz upload: bin/gcc/mac/10.10/7.3.0/build-gcc.sh to s3://binaries.pantsbuild.org/bin/gcc/mac/10.10/7.3.0/build-gcc.sh upload: bin/gcc/mac/10.10/7.3.0/build.sh to s3://binaries.pantsbuild.org/bin/gcc/mac/10.10/7.3.0/build.sh upload: bin/gcc/mac/10.11/7.3.0/build-gcc.sh to s3://binaries.pantsbuild.org/bin/gcc/mac/10.11/7.3.0/build-gcc.sh upload: bin/gcc/linux/x86_64/7.3.0/gcc.tar.gz to s3://binaries.pantsbuild.org/bin/gcc/linux/x86_64/7.3.0/gcc.tar.gz upload: bin/gcc/mac/10.11/7.3.0/build.sh to s3://binaries.pantsbuild.org/bin/gcc/mac/10.11/7.3.0/build.sh upload: bin/gcc/mac/10.12/7.3.0/build-gcc.sh to s3://binaries.pantsbuild.org/bin/gcc/mac/10.12/7.3.0/build-gcc.sh upload: bin/gcc/mac/10.12/7.3.0/build.sh to s3://binaries.pantsbuild.org/bin/gcc/mac/10.12/7.3.0/build.sh upload: bin/gcc/mac/10.10/7.3.0/gcc.tar.gz to s3://binaries.pantsbuild.org/bin/gcc/mac/10.10/7.3.0/gcc.tar.gz upload: bin/gcc/mac/10.11/7.3.0/gcc.tar.gz to s3://binaries.pantsbuild.org/bin/gcc/mac/10.11/7.3.0/gcc.tar.gz upload: bin/gcc/mac/10.13/7.3.0/build-gcc.sh to s3://binaries.pantsbuild.org/bin/gcc/mac/10.13/7.3.0/build-gcc.sh upload: bin/gcc/mac/10.13/7.3.0/build.sh to s3://binaries.pantsbuild.org/bin/gcc/mac/10.13/7.3.0/build.sh upload: bin/gcc/mac/10.12/7.3.0/gcc.tar.gz to s3://binaries.pantsbuild.org/bin/gcc/mac/10.12/7.3.0/gcc.tar.gz upload: bin/gcc/mac/10.8/7.3.0/build-gcc.sh to s3://binaries.pantsbuild.org/bin/gcc/mac/10.8/7.3.0/build-gcc.sh upload: bin/gcc/mac/10.8/7.3.0/build.sh to s3://binaries.pantsbuild.org/bin/gcc/mac/10.8/7.3.0/build.sh upload: bin/gcc/mac/10.13/7.3.0/gcc.tar.gz to s3://binaries.pantsbuild.org/bin/gcc/mac/10.13/7.3.0/gcc.tar.gz upload: bin/gcc/mac/10.9/7.3.0/build-gcc.sh to s3://binaries.pantsbuild.org/bin/gcc/mac/10.9/7.3.0/build-gcc.sh upload: bin/gcc/mac/10.9/7.3.0/build.sh to s3://binaries.pantsbuild.org/bin/gcc/mac/10.9/7.3.0/build.sh upload: bin/gcc/mac/10.8/7.3.0/gcc.tar.gz to s3://binaries.pantsbuild.org/bin/gcc/mac/10.8/7.3.0/gcc.tar.gz upload: bin/gcc/mac/10.9/7.3.0/gcc.tar.gz to s3://binaries.pantsbuild.org/bin/gcc/mac/10.9/7.3.0/gcc.tar.gz ``` add xz package (pantsbuild#66) cloc and isort are now fetched from `bin`, but without the platform-specific path component. (pantsbuild#68) add --disable-rpath when configuring xz (pantsbuild#71) This removes the need to set `DYLD_LIBRARY_PATH` or `LD_LIBRARY_PATH` in pantsbuild/pants#5936. The `xz` binary will use the version of `xz` and `liblzma` in the installation directory when invoked. add xz wrapper script to set the library path (pantsbuild#72) See pantsbuild#71, where we added `--disable-rpath` to `xz` to stop it from using a hardcoded library path at runtime. However, that still fails in Travis in pantsbuild/pants#5936 with the output: ``` INFO] Fetched llvm.tar.xz binary from: https://releases.llvm.org/6.0.0/clang+llvm-6.0.0-x86_64-linux-gnu-ubuntu-16.04.tar.xz . INFO] Extracting /home/travis/.cache/pants/bin/llvm/linux/x86_64/6.0.0/llvm.tar.xz to /home/travis/.cache/pants/bin/llvm/linux/x86_64/6.0.0/llvm . xz: /lib/x86_64-linux-gnu/liblzma.so.5: version `XZ_5.2' not found (required by xz) ``` While we had removed the query for the hardcoded path, we still need to actually set our own path. We had been doing that on the pants side -- instead, this PR adds a (very short) python script at the path of the real `xz` binary (a suggestion from @illicitonion in that PR) which sets `LD_LIBRARY_PATH` or `DYLD_LIBRARY_PATH` appropriately, not clobbering any existing values. This works with the current version of pantsbuild/pants#5936 for me on osx and linux. statically link xz (pantsbuild#74) See pantsbuild#71, pantsbuild#72, pantsbuild#73. Our `xz` is picking up travis's `xz` installation and using its (incompatible) `liblzma` shared library. On the suggestion of @stuhood in pantsbuild/pants#5936, we statically link `xz` here to avoid ever having this problem ever again. Proof: *linux*: ``` > ldd ./xz-5.2.4-linux/xz-install/bin/xz linux-vdso.so.1 (0x00007ffc865cd000) libpthread.so.0 => /usr/lib/libpthread.so.0 (0x00007f1fa385f000) libc.so.6 => /usr/lib/libc.so.6 (0x00007f1fa34a3000) /lib64/ld-linux-x86-64.so.2 => /usr/lib64/ld-linux-x86-64.so.2 (0x00007f1fa3cb4000) ``` *osx*: ``` > otool -L ./xz-5.2.4-linux/xz-install/bin/xz ./xz-5.2.4-linux/xz-install/bin/xz: /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1252.50.4) ``` add llvm 6.0.0 package. (pantsbuild#75) See pantsbuild/pants#5991. This adds the binary I should have provided when we added the ability to download `BinaryTool`s from external urls, in the exact same way we do it within Pants. Upgrade isort to 4.3.4. (pantsbuild#76) This adds support for more python3 stdlib symbols. Also introduce a build script. Fixup isort path structure. Node.js 10.7.0 and Yarnpkg 1.7.0 (pantsbuild#80) Four small scripts, each named "build.sh" have been added. These will build a specific version of Node.js or Yarnpkg in the current directory. Symlinks from older macOS to 10.13 are included. Add build scripts and symlinks for Node 8.11.3 and Yarnpkg 1.6.0 (pantsbuild#81) Adding Node v10.13.0 build scripts for Mac 10.8-10.13 and Linux X64 (pantsbuild#86) Add go 1.10.5 (pantsbuild#87) Add build script for Thrift 0.11.0 (pantsbuild#83) Add build scripts and symlinks for Yarnpkg 1.12.3 (pantsbuild#89) Add updated go versions to pants binaries (pantsbuild#90) * Add updated go versions to pants binaries * Expanded the mac_versions into a map to encode the OSX min version requirements of the latest builds of go. Add build scripts and symlinks for Yarnpkg 1.16.0 (pantsbuild#91) * Add build scripts and symlinks for Yarnpkg 1.16.0 Add go 1.12.7 (pantsbuild#92) * Add go 1.12.7 Upgrade to cloc 1.82 (pantsbuild#93) Add cloc 1.80 (pantsbuild#94) The current version of cloc (version 1.82) has a bug where the --ignored flag is not working properly ( AlDanial/cloc#401 ). The pants test for the cloc goal tests this behavior, so as long as this bug is not fixed upstream, pants will fail this test. I've confirmed that version 1.80 of cloc does not have this bug, so we should be able to use it until upstream cloc fixes the bug in a new verson. Add go 1.13.8 for linux and mac 10.11+
Add buildozer 0.6.0-80c7f0d45d7e40fa1f7362852697d4a03df557b3 (pantsbuild#56) Synchronize docker instructions with pantsbuild/pants. (pantsbuild#58) Add clang (pantsbuild#57) Switch to a workflow without committed binaries (pantsbuild#59) * Update the README to indicate that no new binaries should be committed here. * Formalize the script location to improve maintainability. Add Go 1.9.4 and 1.10 (pantsbuild#61) Here we add `build.sh` scripts for Go 1.9.4 and 1.10, and a helper script for generating the `build.sh` scripts. We also clarify in the docs what platforms are supported (i386 and macOS pre 10.8 are not supported going forward). update some of the commands described in the README (pantsbuild#63) add gcc 7.3.0 and clang 6.0.0 cross platform, binutils 2.30 for linux, edit clang 5.0.1 to conform to pantsbuild#59 (pantsbuild#62) ``` $ ./sync-s3.sh upload: bin/clang/linux/x86_64/6.0.0/build-cmake.sh to s3://binaries.pantsbuild.org/bin/clang/linux/x86_64/6.0.0/build-cmake.sh upload: bin/clang/linux/x86_64/6.0.0/build.sh to s3://binaries.pantsbuild.org/bin/clang/linux/x86_64/6.0.0/build.sh upload: bin/binutils/linux/x86_64/2.30/build.sh to s3://binaries.pantsbuild.org/bin/binutils/linux/x86_64/2.30/build.sh upload: bin/binutils/linux/x86_64/2.30/build-binutils.sh to s3://binaries.pantsbuild.org/bin/binutils/linux/x86_64/2.30/build-binutils.sh upload: bin/binutils/linux/x86_64/2.30/binutils.tar.gz to s3://binaries.pantsbuild.org/bin/binutils/linux/x86_64/2.30/binutils.tar.gz upload: bin/clang/mac/10.10/6.0.0/build-clang.sh to s3://binaries.pantsbuild.org/bin/clang/mac/10.10/6.0.0/build-clang.sh upload: bin/clang/mac/10.10/6.0.0/build.sh to s3://binaries.pantsbuild.org/bin/clang/mac/10.10/6.0.0/build.sh upload: bin/clang/linux/x86_64/6.0.0/clang.tar.gz to s3://binaries.pantsbuild.org/bin/clang/linux/x86_64/6.0.0/clang.tar.gz upload: bin/clang/mac/10.11/6.0.0/build-clang.sh to s3://binaries.pantsbuild.org/bin/clang/mac/10.11/6.0.0/build-clang.sh upload: bin/clang/mac/10.11/6.0.0/build.sh to s3://binaries.pantsbuild.org/bin/clang/mac/10.11/6.0.0/build.sh upload: bin/clang/linux/x86_64/6.0.0/build-clang.sh to s3://binaries.pantsbuild.org/bin/clang/linux/x86_64/6.0.0/build-clang.sh upload: bin/clang/linux/x86_64/6.0.0/build-clang-for-linux-with-cmake.sh to s3://binaries.pantsbuild.org/bin/clang/linux/x86_64/6.0.0/build-clang-for-linux-with-cmake.sh upload: bin/clang/mac/10.12/6.0.0/build-clang.sh to s3://binaries.pantsbuild.org/bin/clang/mac/10.12/6.0.0/build-clang.sh upload: bin/clang/mac/10.12/6.0.0/build.sh to s3://binaries.pantsbuild.org/bin/clang/mac/10.12/6.0.0/build.sh upload: bin/clang/mac/10.10/6.0.0/clang.tar.gz to s3://binaries.pantsbuild.org/bin/clang/mac/10.10/6.0.0/clang.tar.gz upload: bin/clang/mac/10.12/6.0.0/clang.tar.gz to s3://binaries.pantsbuild.org/bin/clang/mac/10.12/6.0.0/clang.tar.gz upload: bin/clang/mac/10.13/6.0.0/build-clang.sh to s3://binaries.pantsbuild.org/bin/clang/mac/10.13/6.0.0/build-clang.sh upload: bin/clang/mac/10.13/6.0.0/build.sh to s3://binaries.pantsbuild.org/bin/clang/mac/10.13/6.0.0/build.sh upload: bin/clang/mac/10.11/6.0.0/clang.tar.gz to s3://binaries.pantsbuild.org/bin/clang/mac/10.11/6.0.0/clang.tar.gz upload: bin/clang/mac/10.8/6.0.0/build.sh to s3://binaries.pantsbuild.org/bin/clang/mac/10.8/6.0.0/build.sh upload: bin/clang/mac/10.8/6.0.0/build-clang.sh to s3://binaries.pantsbuild.org/bin/clang/mac/10.8/6.0.0/build-clang.sh upload: bin/clang/mac/10.13/6.0.0/clang.tar.gz to s3://binaries.pantsbuild.org/bin/clang/mac/10.13/6.0.0/clang.tar.gz upload: bin/clang/mac/10.9/6.0.0/build-clang.sh to s3://binaries.pantsbuild.org/bin/clang/mac/10.9/6.0.0/build-clang.sh upload: bin/clang/mac/10.9/6.0.0/build.sh to s3://binaries.pantsbuild.org/bin/clang/mac/10.9/6.0.0/build.sh upload: bin/clang/mac/10.8/6.0.0/clang.tar.gz to s3://binaries.pantsbuild.org/bin/clang/mac/10.8/6.0.0/clang.tar.gz upload: bin/gcc/linux/x86_64/7.3.0/build.sh to s3://binaries.pantsbuild.org/bin/gcc/linux/x86_64/7.3.0/build.sh upload: bin/gcc/linux/x86_64/7.3.0/build-gcc.sh to s3://binaries.pantsbuild.org/bin/gcc/linux/x86_64/7.3.0/build-gcc.sh upload: bin/clang/mac/10.9/6.0.0/clang.tar.gz to s3://binaries.pantsbuild.org/bin/clang/mac/10.9/6.0.0/clang.tar.gz upload: bin/gcc/mac/10.10/7.3.0/build-gcc.sh to s3://binaries.pantsbuild.org/bin/gcc/mac/10.10/7.3.0/build-gcc.sh upload: bin/gcc/mac/10.10/7.3.0/build.sh to s3://binaries.pantsbuild.org/bin/gcc/mac/10.10/7.3.0/build.sh upload: bin/gcc/mac/10.11/7.3.0/build-gcc.sh to s3://binaries.pantsbuild.org/bin/gcc/mac/10.11/7.3.0/build-gcc.sh upload: bin/gcc/linux/x86_64/7.3.0/gcc.tar.gz to s3://binaries.pantsbuild.org/bin/gcc/linux/x86_64/7.3.0/gcc.tar.gz upload: bin/gcc/mac/10.11/7.3.0/build.sh to s3://binaries.pantsbuild.org/bin/gcc/mac/10.11/7.3.0/build.sh upload: bin/gcc/mac/10.12/7.3.0/build-gcc.sh to s3://binaries.pantsbuild.org/bin/gcc/mac/10.12/7.3.0/build-gcc.sh upload: bin/gcc/mac/10.12/7.3.0/build.sh to s3://binaries.pantsbuild.org/bin/gcc/mac/10.12/7.3.0/build.sh upload: bin/gcc/mac/10.10/7.3.0/gcc.tar.gz to s3://binaries.pantsbuild.org/bin/gcc/mac/10.10/7.3.0/gcc.tar.gz upload: bin/gcc/mac/10.11/7.3.0/gcc.tar.gz to s3://binaries.pantsbuild.org/bin/gcc/mac/10.11/7.3.0/gcc.tar.gz upload: bin/gcc/mac/10.13/7.3.0/build-gcc.sh to s3://binaries.pantsbuild.org/bin/gcc/mac/10.13/7.3.0/build-gcc.sh upload: bin/gcc/mac/10.13/7.3.0/build.sh to s3://binaries.pantsbuild.org/bin/gcc/mac/10.13/7.3.0/build.sh upload: bin/gcc/mac/10.12/7.3.0/gcc.tar.gz to s3://binaries.pantsbuild.org/bin/gcc/mac/10.12/7.3.0/gcc.tar.gz upload: bin/gcc/mac/10.8/7.3.0/build-gcc.sh to s3://binaries.pantsbuild.org/bin/gcc/mac/10.8/7.3.0/build-gcc.sh upload: bin/gcc/mac/10.8/7.3.0/build.sh to s3://binaries.pantsbuild.org/bin/gcc/mac/10.8/7.3.0/build.sh upload: bin/gcc/mac/10.13/7.3.0/gcc.tar.gz to s3://binaries.pantsbuild.org/bin/gcc/mac/10.13/7.3.0/gcc.tar.gz upload: bin/gcc/mac/10.9/7.3.0/build-gcc.sh to s3://binaries.pantsbuild.org/bin/gcc/mac/10.9/7.3.0/build-gcc.sh upload: bin/gcc/mac/10.9/7.3.0/build.sh to s3://binaries.pantsbuild.org/bin/gcc/mac/10.9/7.3.0/build.sh upload: bin/gcc/mac/10.8/7.3.0/gcc.tar.gz to s3://binaries.pantsbuild.org/bin/gcc/mac/10.8/7.3.0/gcc.tar.gz upload: bin/gcc/mac/10.9/7.3.0/gcc.tar.gz to s3://binaries.pantsbuild.org/bin/gcc/mac/10.9/7.3.0/gcc.tar.gz ``` add xz package (pantsbuild#66) cloc and isort are now fetched from `bin`, but without the platform-specific path component. (pantsbuild#68) add --disable-rpath when configuring xz (pantsbuild#71) This removes the need to set `DYLD_LIBRARY_PATH` or `LD_LIBRARY_PATH` in pantsbuild/pants#5936. The `xz` binary will use the version of `xz` and `liblzma` in the installation directory when invoked. add xz wrapper script to set the library path (pantsbuild#72) See pantsbuild#71, where we added `--disable-rpath` to `xz` to stop it from using a hardcoded library path at runtime. However, that still fails in Travis in pantsbuild/pants#5936 with the output: ``` INFO] Fetched llvm.tar.xz binary from: https://releases.llvm.org/6.0.0/clang+llvm-6.0.0-x86_64-linux-gnu-ubuntu-16.04.tar.xz . INFO] Extracting /home/travis/.cache/pants/bin/llvm/linux/x86_64/6.0.0/llvm.tar.xz to /home/travis/.cache/pants/bin/llvm/linux/x86_64/6.0.0/llvm . xz: /lib/x86_64-linux-gnu/liblzma.so.5: version `XZ_5.2' not found (required by xz) ``` While we had removed the query for the hardcoded path, we still need to actually set our own path. We had been doing that on the pants side -- instead, this PR adds a (very short) python script at the path of the real `xz` binary (a suggestion from @illicitonion in that PR) which sets `LD_LIBRARY_PATH` or `DYLD_LIBRARY_PATH` appropriately, not clobbering any existing values. This works with the current version of pantsbuild/pants#5936 for me on osx and linux. statically link xz (pantsbuild#74) See pantsbuild#71, pantsbuild#72, pantsbuild#73. Our `xz` is picking up travis's `xz` installation and using its (incompatible) `liblzma` shared library. On the suggestion of @stuhood in pantsbuild/pants#5936, we statically link `xz` here to avoid ever having this problem ever again. Proof: *linux*: ``` > ldd ./xz-5.2.4-linux/xz-install/bin/xz linux-vdso.so.1 (0x00007ffc865cd000) libpthread.so.0 => /usr/lib/libpthread.so.0 (0x00007f1fa385f000) libc.so.6 => /usr/lib/libc.so.6 (0x00007f1fa34a3000) /lib64/ld-linux-x86-64.so.2 => /usr/lib64/ld-linux-x86-64.so.2 (0x00007f1fa3cb4000) ``` *osx*: ``` > otool -L ./xz-5.2.4-linux/xz-install/bin/xz ./xz-5.2.4-linux/xz-install/bin/xz: /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1252.50.4) ``` add llvm 6.0.0 package. (pantsbuild#75) See pantsbuild/pants#5991. This adds the binary I should have provided when we added the ability to download `BinaryTool`s from external urls, in the exact same way we do it within Pants. Upgrade isort to 4.3.4. (pantsbuild#76) This adds support for more python3 stdlib symbols. Also introduce a build script. Fixup isort path structure. Node.js 10.7.0 and Yarnpkg 1.7.0 (pantsbuild#80) Four small scripts, each named "build.sh" have been added. These will build a specific version of Node.js or Yarnpkg in the current directory. Symlinks from older macOS to 10.13 are included. Add build scripts and symlinks for Node 8.11.3 and Yarnpkg 1.6.0 (pantsbuild#81) Adding Node v10.13.0 build scripts for Mac 10.8-10.13 and Linux X64 (pantsbuild#86) Add go 1.10.5 (pantsbuild#87) Add build script for Thrift 0.11.0 (pantsbuild#83) Add build scripts and symlinks for Yarnpkg 1.12.3 (pantsbuild#89) Add updated go versions to pants binaries (pantsbuild#90) * Add updated go versions to pants binaries * Expanded the mac_versions into a map to encode the OSX min version requirements of the latest builds of go. Add build scripts and symlinks for Yarnpkg 1.16.0 (pantsbuild#91) * Add build scripts and symlinks for Yarnpkg 1.16.0 Add go 1.12.7 (pantsbuild#92) * Add go 1.12.7 Upgrade to cloc 1.82 (pantsbuild#93) Add cloc 1.80 (pantsbuild#94) The current version of cloc (version 1.82) has a bug where the --ignored flag is not working properly ( AlDanial/cloc#401 ). The pants test for the cloc goal tests this behavior, so as long as this bug is not fixed upstream, pants will fail this test. I've confirmed that version 1.80 of cloc does not have this bug, so we should be able to use it until upstream cloc fixes the bug in a new verson. Add go 1.13.8 for linux and mac 10.11+
Add buildozer 0.6.0-80c7f0d45d7e40fa1f7362852697d4a03df557b3 (pantsbuild#56) Synchronize docker instructions with pantsbuild/pants. (pantsbuild#58) Add clang (pantsbuild#57) Switch to a workflow without committed binaries (pantsbuild#59) * Update the README to indicate that no new binaries should be committed here. * Formalize the script location to improve maintainability. Add Go 1.9.4 and 1.10 (pantsbuild#61) Here we add `build.sh` scripts for Go 1.9.4 and 1.10, and a helper script for generating the `build.sh` scripts. We also clarify in the docs what platforms are supported (i386 and macOS pre 10.8 are not supported going forward). update some of the commands described in the README (pantsbuild#63) add gcc 7.3.0 and clang 6.0.0 cross platform, binutils 2.30 for linux, edit clang 5.0.1 to conform to pantsbuild#59 (pantsbuild#62) ``` $ ./sync-s3.sh upload: bin/clang/linux/x86_64/6.0.0/build-cmake.sh to s3://binaries.pantsbuild.org/bin/clang/linux/x86_64/6.0.0/build-cmake.sh upload: bin/clang/linux/x86_64/6.0.0/build.sh to s3://binaries.pantsbuild.org/bin/clang/linux/x86_64/6.0.0/build.sh upload: bin/binutils/linux/x86_64/2.30/build.sh to s3://binaries.pantsbuild.org/bin/binutils/linux/x86_64/2.30/build.sh upload: bin/binutils/linux/x86_64/2.30/build-binutils.sh to s3://binaries.pantsbuild.org/bin/binutils/linux/x86_64/2.30/build-binutils.sh upload: bin/binutils/linux/x86_64/2.30/binutils.tar.gz to s3://binaries.pantsbuild.org/bin/binutils/linux/x86_64/2.30/binutils.tar.gz upload: bin/clang/mac/10.10/6.0.0/build-clang.sh to s3://binaries.pantsbuild.org/bin/clang/mac/10.10/6.0.0/build-clang.sh upload: bin/clang/mac/10.10/6.0.0/build.sh to s3://binaries.pantsbuild.org/bin/clang/mac/10.10/6.0.0/build.sh upload: bin/clang/linux/x86_64/6.0.0/clang.tar.gz to s3://binaries.pantsbuild.org/bin/clang/linux/x86_64/6.0.0/clang.tar.gz upload: bin/clang/mac/10.11/6.0.0/build-clang.sh to s3://binaries.pantsbuild.org/bin/clang/mac/10.11/6.0.0/build-clang.sh upload: bin/clang/mac/10.11/6.0.0/build.sh to s3://binaries.pantsbuild.org/bin/clang/mac/10.11/6.0.0/build.sh upload: bin/clang/linux/x86_64/6.0.0/build-clang.sh to s3://binaries.pantsbuild.org/bin/clang/linux/x86_64/6.0.0/build-clang.sh upload: bin/clang/linux/x86_64/6.0.0/build-clang-for-linux-with-cmake.sh to s3://binaries.pantsbuild.org/bin/clang/linux/x86_64/6.0.0/build-clang-for-linux-with-cmake.sh upload: bin/clang/mac/10.12/6.0.0/build-clang.sh to s3://binaries.pantsbuild.org/bin/clang/mac/10.12/6.0.0/build-clang.sh upload: bin/clang/mac/10.12/6.0.0/build.sh to s3://binaries.pantsbuild.org/bin/clang/mac/10.12/6.0.0/build.sh upload: bin/clang/mac/10.10/6.0.0/clang.tar.gz to s3://binaries.pantsbuild.org/bin/clang/mac/10.10/6.0.0/clang.tar.gz upload: bin/clang/mac/10.12/6.0.0/clang.tar.gz to s3://binaries.pantsbuild.org/bin/clang/mac/10.12/6.0.0/clang.tar.gz upload: bin/clang/mac/10.13/6.0.0/build-clang.sh to s3://binaries.pantsbuild.org/bin/clang/mac/10.13/6.0.0/build-clang.sh upload: bin/clang/mac/10.13/6.0.0/build.sh to s3://binaries.pantsbuild.org/bin/clang/mac/10.13/6.0.0/build.sh upload: bin/clang/mac/10.11/6.0.0/clang.tar.gz to s3://binaries.pantsbuild.org/bin/clang/mac/10.11/6.0.0/clang.tar.gz upload: bin/clang/mac/10.8/6.0.0/build.sh to s3://binaries.pantsbuild.org/bin/clang/mac/10.8/6.0.0/build.sh upload: bin/clang/mac/10.8/6.0.0/build-clang.sh to s3://binaries.pantsbuild.org/bin/clang/mac/10.8/6.0.0/build-clang.sh upload: bin/clang/mac/10.13/6.0.0/clang.tar.gz to s3://binaries.pantsbuild.org/bin/clang/mac/10.13/6.0.0/clang.tar.gz upload: bin/clang/mac/10.9/6.0.0/build-clang.sh to s3://binaries.pantsbuild.org/bin/clang/mac/10.9/6.0.0/build-clang.sh upload: bin/clang/mac/10.9/6.0.0/build.sh to s3://binaries.pantsbuild.org/bin/clang/mac/10.9/6.0.0/build.sh upload: bin/clang/mac/10.8/6.0.0/clang.tar.gz to s3://binaries.pantsbuild.org/bin/clang/mac/10.8/6.0.0/clang.tar.gz upload: bin/gcc/linux/x86_64/7.3.0/build.sh to s3://binaries.pantsbuild.org/bin/gcc/linux/x86_64/7.3.0/build.sh upload: bin/gcc/linux/x86_64/7.3.0/build-gcc.sh to s3://binaries.pantsbuild.org/bin/gcc/linux/x86_64/7.3.0/build-gcc.sh upload: bin/clang/mac/10.9/6.0.0/clang.tar.gz to s3://binaries.pantsbuild.org/bin/clang/mac/10.9/6.0.0/clang.tar.gz upload: bin/gcc/mac/10.10/7.3.0/build-gcc.sh to s3://binaries.pantsbuild.org/bin/gcc/mac/10.10/7.3.0/build-gcc.sh upload: bin/gcc/mac/10.10/7.3.0/build.sh to s3://binaries.pantsbuild.org/bin/gcc/mac/10.10/7.3.0/build.sh upload: bin/gcc/mac/10.11/7.3.0/build-gcc.sh to s3://binaries.pantsbuild.org/bin/gcc/mac/10.11/7.3.0/build-gcc.sh upload: bin/gcc/linux/x86_64/7.3.0/gcc.tar.gz to s3://binaries.pantsbuild.org/bin/gcc/linux/x86_64/7.3.0/gcc.tar.gz upload: bin/gcc/mac/10.11/7.3.0/build.sh to s3://binaries.pantsbuild.org/bin/gcc/mac/10.11/7.3.0/build.sh upload: bin/gcc/mac/10.12/7.3.0/build-gcc.sh to s3://binaries.pantsbuild.org/bin/gcc/mac/10.12/7.3.0/build-gcc.sh upload: bin/gcc/mac/10.12/7.3.0/build.sh to s3://binaries.pantsbuild.org/bin/gcc/mac/10.12/7.3.0/build.sh upload: bin/gcc/mac/10.10/7.3.0/gcc.tar.gz to s3://binaries.pantsbuild.org/bin/gcc/mac/10.10/7.3.0/gcc.tar.gz upload: bin/gcc/mac/10.11/7.3.0/gcc.tar.gz to s3://binaries.pantsbuild.org/bin/gcc/mac/10.11/7.3.0/gcc.tar.gz upload: bin/gcc/mac/10.13/7.3.0/build-gcc.sh to s3://binaries.pantsbuild.org/bin/gcc/mac/10.13/7.3.0/build-gcc.sh upload: bin/gcc/mac/10.13/7.3.0/build.sh to s3://binaries.pantsbuild.org/bin/gcc/mac/10.13/7.3.0/build.sh upload: bin/gcc/mac/10.12/7.3.0/gcc.tar.gz to s3://binaries.pantsbuild.org/bin/gcc/mac/10.12/7.3.0/gcc.tar.gz upload: bin/gcc/mac/10.8/7.3.0/build-gcc.sh to s3://binaries.pantsbuild.org/bin/gcc/mac/10.8/7.3.0/build-gcc.sh upload: bin/gcc/mac/10.8/7.3.0/build.sh to s3://binaries.pantsbuild.org/bin/gcc/mac/10.8/7.3.0/build.sh upload: bin/gcc/mac/10.13/7.3.0/gcc.tar.gz to s3://binaries.pantsbuild.org/bin/gcc/mac/10.13/7.3.0/gcc.tar.gz upload: bin/gcc/mac/10.9/7.3.0/build-gcc.sh to s3://binaries.pantsbuild.org/bin/gcc/mac/10.9/7.3.0/build-gcc.sh upload: bin/gcc/mac/10.9/7.3.0/build.sh to s3://binaries.pantsbuild.org/bin/gcc/mac/10.9/7.3.0/build.sh upload: bin/gcc/mac/10.8/7.3.0/gcc.tar.gz to s3://binaries.pantsbuild.org/bin/gcc/mac/10.8/7.3.0/gcc.tar.gz upload: bin/gcc/mac/10.9/7.3.0/gcc.tar.gz to s3://binaries.pantsbuild.org/bin/gcc/mac/10.9/7.3.0/gcc.tar.gz ``` add xz package (pantsbuild#66) cloc and isort are now fetched from `bin`, but without the platform-specific path component. (pantsbuild#68) add --disable-rpath when configuring xz (pantsbuild#71) This removes the need to set `DYLD_LIBRARY_PATH` or `LD_LIBRARY_PATH` in pantsbuild/pants#5936. The `xz` binary will use the version of `xz` and `liblzma` in the installation directory when invoked. add xz wrapper script to set the library path (pantsbuild#72) See pantsbuild#71, where we added `--disable-rpath` to `xz` to stop it from using a hardcoded library path at runtime. However, that still fails in Travis in pantsbuild/pants#5936 with the output: ``` INFO] Fetched llvm.tar.xz binary from: https://releases.llvm.org/6.0.0/clang+llvm-6.0.0-x86_64-linux-gnu-ubuntu-16.04.tar.xz . INFO] Extracting /home/travis/.cache/pants/bin/llvm/linux/x86_64/6.0.0/llvm.tar.xz to /home/travis/.cache/pants/bin/llvm/linux/x86_64/6.0.0/llvm . xz: /lib/x86_64-linux-gnu/liblzma.so.5: version `XZ_5.2' not found (required by xz) ``` While we had removed the query for the hardcoded path, we still need to actually set our own path. We had been doing that on the pants side -- instead, this PR adds a (very short) python script at the path of the real `xz` binary (a suggestion from @illicitonion in that PR) which sets `LD_LIBRARY_PATH` or `DYLD_LIBRARY_PATH` appropriately, not clobbering any existing values. This works with the current version of pantsbuild/pants#5936 for me on osx and linux. statically link xz (pantsbuild#74) See pantsbuild#71, pantsbuild#72, pantsbuild#73. Our `xz` is picking up travis's `xz` installation and using its (incompatible) `liblzma` shared library. On the suggestion of @stuhood in pantsbuild/pants#5936, we statically link `xz` here to avoid ever having this problem ever again. Proof: *linux*: ``` > ldd ./xz-5.2.4-linux/xz-install/bin/xz linux-vdso.so.1 (0x00007ffc865cd000) libpthread.so.0 => /usr/lib/libpthread.so.0 (0x00007f1fa385f000) libc.so.6 => /usr/lib/libc.so.6 (0x00007f1fa34a3000) /lib64/ld-linux-x86-64.so.2 => /usr/lib64/ld-linux-x86-64.so.2 (0x00007f1fa3cb4000) ``` *osx*: ``` > otool -L ./xz-5.2.4-linux/xz-install/bin/xz ./xz-5.2.4-linux/xz-install/bin/xz: /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1252.50.4) ``` add llvm 6.0.0 package. (pantsbuild#75) See pantsbuild/pants#5991. This adds the binary I should have provided when we added the ability to download `BinaryTool`s from external urls, in the exact same way we do it within Pants. Upgrade isort to 4.3.4. (pantsbuild#76) This adds support for more python3 stdlib symbols. Also introduce a build script. Fixup isort path structure. Node.js 10.7.0 and Yarnpkg 1.7.0 (pantsbuild#80) Four small scripts, each named "build.sh" have been added. These will build a specific version of Node.js or Yarnpkg in the current directory. Symlinks from older macOS to 10.13 are included. Add build scripts and symlinks for Node 8.11.3 and Yarnpkg 1.6.0 (pantsbuild#81) Adding Node v10.13.0 build scripts for Mac 10.8-10.13 and Linux X64 (pantsbuild#86) Add go 1.10.5 (pantsbuild#87) Add build script for Thrift 0.11.0 (pantsbuild#83) Add build scripts and symlinks for Yarnpkg 1.12.3 (pantsbuild#89) Add updated go versions to pants binaries (pantsbuild#90) * Add updated go versions to pants binaries * Expanded the mac_versions into a map to encode the OSX min version requirements of the latest builds of go. Add build scripts and symlinks for Yarnpkg 1.16.0 (pantsbuild#91) * Add build scripts and symlinks for Yarnpkg 1.16.0 Add go 1.12.7 (pantsbuild#92) * Add go 1.12.7 Upgrade to cloc 1.82 (pantsbuild#93) Add cloc 1.80 (pantsbuild#94) The current version of cloc (version 1.82) has a bug where the --ignored flag is not working properly ( AlDanial/cloc#401 ). The pants test for the cloc goal tests this behavior, so as long as this bug is not fixed upstream, pants will fail this test. I've confirmed that version 1.80 of cloc does not have this bug, so we should be able to use it until upstream cloc fixes the bug in a new verson. Add go 1.13.8 for linux and mac 10.11+
The new script
./build-compiler-5.0.1.sh
will package an archive namedcompiler.tar.gz
for both MacOS and Linux when executed in a Linux environment. This archive currently contains the clang compiler and the C standard library, which is sufficient to compile thepython_dist()
test project -- this can be tested by running./pants run testprojects/src/python/python_distribution/fasthello_with_install_requires:main_with_no_conflict
.Notes
.tar.xz
files for some reason, soxz
is needed to unpack their archives -- you will need to runyum install -y xz
if using the CentOS Dockerfile available in the pants codebase.build-cmake-3.9.5.sh
script already available in this repo to produce a cmake binary that can be used to build LLVM from source. This is necessary because LLVM requires cmake >= 3.4, which is otherwise not available in the CentOS Dockerfile from the pants codebase. Becausebuild-cmake-3.9.5.sh
only builds for Linux x86_64, this script only builds a compiler for 64-bit Linux hosts. If 32-bit support becomes necessary, I can probably modify both scripts to do so, but I haven't investigated whether this is supported for either cmake or LLVM yet.