Skip to content

Commit

Permalink
Auto merge of #132 - malbarbo:android-test, r=malbarbo
Browse files Browse the repository at this point in the history
Add limited android test support

We could provide android test support by launching an android emulator and running tests in the emulator (like how it is done in libc and rust), but there is licenses issues in redistributing the sdk (see 3.4 of https://developer.android.com/studio/terms.html). Off course, I maybe be wrong about this.

So we download and build bionic libc from source, which I think allows we to distribute the binaries. Having the android libc allows running native binaries using qemu, which can be enough for projects like, nix, ripgrep, etc.

Related #82
  • Loading branch information
homunkulus committed Oct 20, 2017
2 parents 359c3bb + 140aff3 commit 3de557c
Show file tree
Hide file tree
Showing 9 changed files with 253 additions and 18 deletions.
10 changes: 5 additions & 5 deletions .travis.yml
Original file line number Diff line number Diff line change
Expand Up @@ -27,11 +27,11 @@ matrix:
- env: TARGET=x86_64-unknown-linux-musl STD=1 OPENSSL=0.5.5 RUN=1 DEPLOY=1

# Android
- env: TARGET=aarch64-linux-android CPP=1 STD=1 OPENSSL=0.5.5
- env: TARGET=arm-linux-androideabi CPP=1 STD=1 OPENSSL=0.5.5
- env: TARGET=armv7-linux-androideabi CPP=1 STD=1 OPENSSL=0.5.5
- env: TARGET=i686-linux-android CPP=1 STD=1 OPENSSL=0.5.5
- env: TARGET=x86_64-linux-android CPP=1 STD=1 OPENSSL=0.5.5
- env: TARGET=aarch64-linux-android CPP=1 STD=1 OPENSSL=0.5.5 RUN=1
- env: TARGET=arm-linux-androideabi CPP=1 STD=1 OPENSSL=0.5.5 RUN=1
- env: TARGET=armv7-linux-androideabi CPP=1 STD=1 OPENSSL=0.5.5 RUN=1
- env: TARGET=i686-linux-android CPP=1 STD=1 OPENSSL=0.5.5 RUN=1
- env: TARGET=x86_64-linux-android CPP=1 STD=1 OPENSSL=0.5.5 RUN=1

# OSX
- env: TARGET=i686-apple-darwin DYLIB=1 STD=1 RUN=1
Expand Down
16 changes: 11 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -189,16 +189,16 @@ worst, "hang" (never terminate).

| Target | libc | GCC | OpenSSL | C++ | QEMU | `test` |
|--------------------------------------|--------|---------|---------|:---:|-------|:------:|
| `aarch64-linux-android` | N/A | 4.9 | 1.0.2k || N/A | |
| `aarch64-linux-android` [5] | N/A | 4.9 | 1.0.2k || N/A | |
| `aarch64-unknown-linux-gnu` | 2.19 | 4.8.2 | 1.0.2k || 2.8.0 ||
| `arm-linux-androideabi` | N/A | 4.9 | 1.0.2k || N/A | |
| `arm-linux-androideabi` [5] | N/A | 4.9 | 1.0.2k || N/A | |
| `arm-unknown-linux-gnueabi` | 2.19 | 4.8.2 | 1.0.2k || 2.8.0 ||
| `arm-unknown-linux-musleabi` | 1.1.15 | 5.3.1 | N/A | | 2.8.0 ||
| `armv7-linux-androideabi` | N/A | 4.9 | 1.0.2k || N/A | |
| `armv7-linux-androideabi` [5] | N/A | 4.9 | 1.0.2k || N/A | |
| `armv7-unknown-linux-gnueabihf` | 2.15 | 4.6.2 | 1.0.2k || 2.8.0 ||
| `armv7-unknown-linux-musleabihf` | 1.1.15 | 5.3.1 | N/A | | 2.8.0 ||
| `asmjs-unknown-emscripten` [4] | 1.1.15 | 1.37.13 | N/A || N/A ||
| `i686-linux-android` | N/A | 4.9 | 1.0.2k || N/A | |
| `i686-linux-android` [5] | N/A | 4.9 | 1.0.2k || N/A | |
| `i686-pc-windows-gnu` | N/A | 6.2.0 | N/A || N/A ||
| `i686-unknown-freebsd` [1] | 10.2 | 5.3.0 | 1.0.2k | | N/A | |
| `i686-unknown-linux-gnu` | 2.15 | 4.6.2 | 1.0.2k || N/A ||
Expand All @@ -218,7 +218,7 @@ worst, "hang" (never terminate).
| `thumbv7em-none-eabihf` [3] | 2.2.0 | 5.3.1 | N/A | | N/A | |
| `thumbv7m-none-eabi` [3] | 2.2.0 | 5.3.1 | N/A | | N/A | |
| `wasm32-unknown-emscripten` [4] | 1.1.15 | 1.37.13 | N/A || N/A ||
| `x86_64-linux-android` | N/A | 4.9 | 1.0.2k || N/A | |
| `x86_64-linux-android` [5] | N/A | 4.9 | 1.0.2k || N/A | |
| `x86_64-pc-windows-gnu` | N/A | 6.2.0 | N/A || N/A ||
| `x86_64-sun-solaris` [1] | 2.11 | 5.3.0 | 1.0.2k | | N/A | |
| `x86_64-unknown-dragonfly` [1] [2] | 4.6.0 | 5.3.0 | 1.0.2k | | N/A ||
Expand All @@ -237,6 +237,12 @@ where libc was extracted.
[4] libc = musl, gcc = emcc; Some projects that use libc may fail due to wrong
definitions (will be fixed by https://github.com/rust-lang/libc/pull/610)

[5] Only works with native tests, that is, tests that do not depends on the
Android Runtime. For i686 some tests may fails with the error `assertion
failed: signal(libc::SIGPIPE, libc::SIG_IGN) != libc::SIG_ERR`, see
[issue #140](https://github.com/japaric/cross/issues/140) for more
information.

## Debugging

### QEMU_STRACE (v0.1.9+)
Expand Down
12 changes: 11 additions & 1 deletion docker/aarch64-linux-android/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -12,16 +12,23 @@ RUN apt-get update && \
COPY xargo.sh /
RUN bash /xargo.sh

COPY qemu.sh /
RUN bash /qemu.sh aarch64 android

COPY android-ndk.sh /
RUN bash /android-ndk.sh arm64 21
ENV PATH=$PATH:/android-ndk/bin

COPY openssl.sh /
RUN bash /openssl.sh linux-generic64 aarch64-linux-android- -mandroid -fomit-frame-pointer

COPY android-system.sh /
RUN bash /android-system.sh arm64

# Libz is distributed in the android ndk, but for some unknown reason it is not
# found in the build process of some crates, so we explicit set the DEP_Z_ROOT
ENV CARGO_TARGET_AARCH64_LINUX_ANDROID_LINKER=aarch64-linux-android-gcc \
CARGO_TARGET_AARCH64_LINUX_ANDROID_RUNNER=qemu-aarch64 \
CC_aarch64_linux_android=aarch64-linux-android-gcc \
CXX_aarch64_linux_android=aarch64-linux-android-g++ \
DEP_Z_ROOT=/android-ndk/sysroot/usr/ \
Expand All @@ -30,4 +37,7 @@ ENV CARGO_TARGET_AARCH64_LINUX_ANDROID_LINKER=aarch64-linux-android-gcc \
OPENSSL_INCLUDE_DIR=/openssl/include \
OPENSSL_LIB_DIR=/openssl/lib \
RUST_TEST_THREADS=1 \
HOME=/tmp/
HOME=/tmp/ \
TMPDIR=/tmp/ \
ANDROID_ROOT=/system \
ANDROID_DATA=/
102 changes: 102 additions & 0 deletions docker/android-system.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,102 @@
set -ex

main() {
local arch=$1
local td=$(mktemp -d)
pushd $td

local dependencies=(
ca-certificates
curl
gcc-multilib
git
g++-multilib
make
python
)

# fake java and javac, it is not necessary for what we build, but the build
# script ask for it
cat << EOF > /usr/bin/java
#!/bin/bash
echo "java version \"1.7.0\""
echo "OpenJDK Runtime Environment (IcedTea 2.6.9)"
echo "OpenJDK 64-Bit Server VM (build 24.131-b00, mixed mode)"
EOF

cat << EOF > /usr/bin/javac
#!/bin/bash
echo "javac 1.7.0"
EOF

chmod +x /usr/bin/java
chmod +x /usr/bin/javac

# more faking
export ANDROID_JAVA_HOME=/tmp
mkdir /tmp/lib/
touch /tmp/lib/tools.jar

apt-get update
local purge_list=(default-jre)
for dep in ${dependencies[@]}; do
if ! dpkg -L $dep; then
apt-get install --no-install-recommends -y $dep
purge_list+=( $dep )
fi
done

curl -O https://storage.googleapis.com/git-repo-downloads/repo
chmod +x repo

# this is the minimum set of modules that are need to build bionic
# this was created by trial and error
./repo init -u https://android.googlesource.com/platform/manifest -b android-5.0.0_r1
./repo sync bionic
./repo sync build
./repo sync external/compiler-rt
./repo sync external/jemalloc
./repo sync external/libcxx
./repo sync external/libcxxabi
./repo sync external/stlport
./repo sync prebuilts/clang/linux-x86/host/3.5
./repo sync system/core
case $arch in
arm)
./repo sync prebuilts/gcc/linux-x86/arm/arm-linux-androideabi-4.8
;;
arm64)
./repo sync prebuilts/gcc/linux-x86/arm/arm-linux-androideabi-4.8
./repo sync prebuilts/gcc/linux-x86/aarch64/aarch64-linux-android-4.9
;;
x86)
./repo sync prebuilts/gcc/linux-x86/x86/x86_64-linux-android-4.8
;;
x86_64)
./repo sync prebuilts/gcc/linux-x86/x86/x86_64-linux-android-4.8
;;
esac

# avoid build tests
rm bionic/linker/tests/Android.mk bionic/tests/Android.mk

source build/envsetup.sh
lunch aosp_$arch-user
mmma bionic/

if [ $arch = "arm" ]; then
mv out/target/product/generic/system/ /
else
mv out/target/product/generic_$arch/system/ /
fi

# clean up
apt-get purge --auto-remove -y ${purge_list[@]}

popd

rm -rf $td
rm $0
}

main "${@}"
12 changes: 11 additions & 1 deletion docker/arm-linux-androideabi/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -12,16 +12,23 @@ RUN apt-get update && \
COPY xargo.sh /
RUN bash /xargo.sh

COPY qemu.sh /
RUN bash /qemu.sh arm android

COPY android-ndk.sh /
RUN bash /android-ndk.sh arm 21
ENV PATH=$PATH:/android-ndk/bin

COPY openssl.sh /
RUN bash /openssl.sh android arm-linux-androideabi-

COPY android-system.sh /
RUN bash /android-system.sh arm

# Libz is distributed in the android ndk, but for some unknown reason it is not
# found in the build process of some crates, so we explicit set the DEP_Z_ROOT
ENV CARGO_TARGET_ARM_LINUX_ANDROIDEABI_LINKER=arm-linux-androideabi-gcc \
CARGO_TARGET_ARM_LINUX_ANDROIDEABI_RUNNER=qemu-arm \
CC_arm_linux_androideabi=arm-linux-androideabi-gcc \
CXX_arm_linux_androideabi=arm-linux-androideabi-g++ \
DEP_Z_ROOT=/android-ndk/sysroot/usr/ \
Expand All @@ -30,4 +37,7 @@ ENV CARGO_TARGET_ARM_LINUX_ANDROIDEABI_LINKER=arm-linux-androideabi-gcc \
OPENSSL_INCLUDE_DIR=/openssl/include \
OPENSSL_LIB_DIR=/openssl/lib \
RUST_TEST_THREADS=1 \
HOME=/tmp/
HOME=/tmp/ \
TMPDIR=/tmp/ \
ANDROID_ROOT=/system \
ANDROID_DATA=/
12 changes: 11 additions & 1 deletion docker/armv7-linux-androideabi/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -12,16 +12,23 @@ RUN apt-get update && \
COPY xargo.sh /
RUN bash /xargo.sh

COPY qemu.sh /
RUN bash /qemu.sh arm android

COPY android-ndk.sh /
RUN bash /android-ndk.sh arm 21
ENV PATH=$PATH:/android-ndk/bin

COPY openssl.sh /
RUN bash /openssl.sh android-armv7 arm-linux-androideabi-

COPY android-system.sh /
RUN bash /android-system.sh arm

# Libz is distributed in the android ndk, but for some unknown reason it is not
# found in the build process of some crates, so we explicit set the DEP_Z_ROOT
ENV CARGO_TARGET_ARMV7_LINUX_ANDROIDEABI_LINKER=arm-linux-androideabi-gcc \
CARGO_TARGET_ARMV7_LINUX_ANDROIDEABI_RUNNER=qemu-arm \
CC_armv7_linux_androideabi=arm-linux-androideabi-gcc \
CXX_armv7_linux_androideabi=arm-linux-androideabi-g++ \
DEP_Z_ROOT=/android-ndk/sysroot/usr/ \
Expand All @@ -30,4 +37,7 @@ ENV CARGO_TARGET_ARMV7_LINUX_ANDROIDEABI_LINKER=arm-linux-androideabi-gcc \
OPENSSL_INCLUDE_DIR=/openssl/include \
OPENSSL_LIB_DIR=/openssl/lib \
RUST_TEST_THREADS=1 \
HOME=/tmp/
HOME=/tmp/ \
TMPDIR=/tmp/ \
ANDROID_ROOT=/system \
ANDROID_DATA=/
22 changes: 20 additions & 2 deletions docker/i686-linux-android/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -12,16 +12,31 @@ RUN apt-get update && \
COPY xargo.sh /
RUN bash /xargo.sh

# We could supposedly directly run i686 binaries like we do for x86_64, but
# doing so generates an assertion failure:
# ... assertion failed: signal(libc::SIGPIPE, libc::SIG_IGN) != libc::SIG_ERR
# ... src/libstd/sys/unix/mod.rs
# fatal runtime error: failed to initiate panic, error 5
#
# Running with qemu works as expected
COPY qemu.sh /
RUN bash /qemu.sh i386 android

COPY android-ndk.sh /
RUN bash /android-ndk.sh x86 21
ENV PATH=$PATH:/android-ndk/bin

# Build with no-asm to make openssl linked binaries position-independent (PIE)
COPY openssl.sh /
RUN bash /openssl.sh android-x86 i686-linux-android-
RUN bash /openssl.sh android-x86 i686-linux-android- no-asm

COPY android-system.sh /
RUN bash /android-system.sh x86

# Libz is distributed in the android ndk, but for some unknown reason it is not
# found in the build process of some crates, so we explicit set the DEP_Z_ROOT
ENV CARGO_TARGET_I686_LINUX_ANDROID_LINKER=i686-linux-android-gcc \
CARGO_TARGET_I686_LINUX_ANDROID_RUNNER=qemu-i386 \
CC_i686_linux_android=i686-linux-android-gcc \
CXX_i686_linux_android=i686-linux-android-g++ \
DEP_Z_ROOT=/android-ndk/sysroot/usr/ \
Expand All @@ -30,4 +45,7 @@ ENV CARGO_TARGET_I686_LINUX_ANDROID_LINKER=i686-linux-android-gcc \
OPENSSL_INCLUDE_DIR=/openssl/include \
OPENSSL_LIB_DIR=/openssl/lib \
RUST_TEST_THREADS=1 \
HOME=/tmp/
HOME=/tmp/ \
TMPDIR=/tmp/ \
ANDROID_ROOT=/system \
ANDROID_DATA=/
Loading

0 comments on commit 3de557c

Please sign in to comment.