Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Package request: Android Open Source Project #21222

Closed
spacekitteh opened this issue Dec 17, 2016 · 17 comments
Closed

Package request: Android Open Source Project #21222

spacekitteh opened this issue Dec 17, 2016 · 17 comments
Labels
9.needs: package (new) This needs a new package to be added

Comments

@spacekitteh
Copy link
Contributor

Complete overview/guide for downloading AOSP source (including a custom kernel), building and running a QEMU image: https://www.collabora.com/news-and-blog/blog/2016/09/02/building-android-for-qemu-a-step-by-step-guide/

Build environment requirements: https://source.android.com/source/initializing.html
Source downloading: https://source.android.com/source/downloading.html
Building: https://source.android.com/source/building.html
(Included) compiler toolchain: https://source.android.com/source/jack.html
Selecting build device targets: https://source.android.com/source/devices.html
Kernel-only building: https://source.android.com/source/building-kernels.html
Running builds: https://source.android.com/source/running.html

Relevant for nixos (containers/nixops images):
QEMU kernel build script: https://android.googlesource.com/platform/prebuilts/qemu-kernel/+/master/build-kernel.sh

@spacekitteh
Copy link
Contributor Author

spacekitteh commented Dec 18, 2016 via email

@Mic92
Copy link
Member

Mic92 commented Dec 18, 2016

Sounds to me like a usecase for #21243

@vcunat vcunat added the 9.needs: package (new) This needs a new package to be added label Dec 18, 2016
@moretea
Copy link
Contributor

moretea commented Dec 18, 2016

@spacekitteh ah, I misread! Apologies. In that case 👍. I'd like to build a custom build for my Samsung tablet as well ;)

@abbradar
Copy link
Member

abbradar commented Dec 20, 2016

FWIW, I suceeded building Android on NixOS using an FHS environment:

{ pkgs ? import <nixpkgs> {} }:

let
  jdk = pkgs.jdk8;
  fhs = pkgs.buildFHSUserEnv {
    name = "android-env";
    targetPkgs = pkgs: with pkgs;
      [ git
        gitRepo
        gnupg1
        python2
        curl
        procps
        openssl
        gnumake
        nettools
        androidenv.platformTools
        ccache
        ncurses
        jdk
        schedtool
        utillinux
        m4
        gperf
        perl
        libxml2
        zip
        unzip
        bison
        flex
        lzop
        bc
        which
        gcc
      ];
    multiPkgs = pkgs: with pkgs;
      [ zlib
      ];
    extraOutputsToInstall = [ "dev" ];
    runScript = "bash";
    profile = ''
      export USE_CCACHE=1
      export ANDROID_JAVA_HOME=${jdk.home}
    '';
  };
in fhs.env

Patching all the Android source to get rid of FHS assumptions is a gargantuan and hopeless task IMHO.

@spacekitteh
Copy link
Contributor Author

spacekitteh commented Dec 22, 2016

Hmm, are you sure that one works? It can't find ncurses, it seems.

Edit: Ahh, it wants ncurses 5. Now it can't find libtinfo... ncurses should create a symlink for libtinfo. Submitted in #21350.

@spacekitteh
Copy link
Contributor Author

So this is what I currently have:

{ pkgs ? import <nixpkgs> {} }:

let
  jdk = pkgs.jdk8;
  current-repo = "nougat-mr1-release";
  fhs = pkgs.buildFHSUserEnv {
    name = "android-env";
    targetPkgs = pkgs: with pkgs;
      [ git
        gitRepo
        gnupg1compat
        python2
        curl
        procps
        openssl
        gnumake
        nettools
        rsync
        androidenv.buildTools
        androidenv.platformTools
        ccache
        ncurses5
        jdk
        schedtool
        utillinux
        m4
        gperf
        perl
        libxml2
        zip
        unzip
        bison
        flex
        lzop
        bc
        which
        gcc
      ];
    multiPkgs = pkgs: with pkgs;
      [ zlib
      ];
    extraOutputsToInstall = [ "dev" ];
    runScript = "bash";
    profile = ''
      export USE_CCACHE=1
      export ANDROID_JAVA_HOME=${jdk.home}
      export LANG=C
      unset _JAVA_OPTIONS
      export BUILD_NUMBER=$(date --utc +%Y.%m.%d.%H.%M.%S)
      export DISPLAY_BUILD_NUMBER=true
      export USER=copperhead-os-build

      source build/envsetup.sh
    '';
  };
in pkgs.stdenv.mkDerivation {
  name = "android-env-shell";
  nativeBuildInputs = [fhs];
  shellHook = ''
    cd ${current-repo}
    chrt -b -p 0 $$
    exec android-env
  '';
}

(note: requires #21368)

This gets me part of the way there. I run grsecurity, however, so I have to invoke it with sudo; this results in contamination of my /root folder by jack, among other things.

So, I think the next step from here is to containerise it, because AOSP is garbage and I don't want it touching anything on my system.

@spacekitteh
Copy link
Contributor Author

@abbradar did you ever run into a problem with getting "Unknown cipher suite supported by native code" errors when signing APKs during building?

@abbradar
Copy link
Member

@spacekitteh Sadly no. FWIW I built CyanogenMod, not vanilla AOSP -- maybe some differences are in play.

Anyway, do you think we can close this now? I'm not sure Android can/needs to be packaged any further than a chroot environment.

@spacekitteh
Copy link
Contributor Author

Yeah, I guess.

@spacekitteh
Copy link
Contributor Author

spacekitteh commented Dec 26, 2016

This needs to be reopened, actually. There are other issues getting it compiling on NixOS, and they probably should be figured out.

The current stumbling block is what appears to be an error with OpenSSL(and LibreSSL, but I'll just say OpenSSL for short) behaving differently on NixOS compared to Debian/Arch/Ubuntu. Specifically, the conscrypt Java Crypto Provider doesn't seem to be able to work properly with OpenSSL as nixpkgs provides. It gives the following error:

[ 25% 10601/42050] target Package: BasicDreams (out/targe...ct/marlin/obj/APPS/BasicDreams_intermediates/package.apk)


FAILED: /bin/bash -c "(touch out/target/product/marlin/obj/APPS/BasicDreams_intermediates/zipdummy ) && ((cd out/target/product/marlin/obj/APPS/BasicDreams_intermediates/ && jar cf package.apk zipdummy) ) && (zip -qd out/target/product/marlin/obj/APPS/BasicDreams_intermediates/package.apk zipdummy ) && (rm out/target/product/marlin/obj/APPS/BasicDreams_intermediates/zipdummy ) && (out/host/linux-x86/bin/aapt package -u -z  --pseudo-localize -c en_US,en_US,cs_CZ,da_DK,de_AT,de_CH,de_DE,de_LI,el_GR,en_AU,en_CA,en_GB,en_NZ,en_SG,eo_EU,es_ES,fr_CA,fr_CH,fr_BE,fr_FR,it_CH,it_IT,ja_JP,ko_KR,nb_NO,nl_BE,nl_NL,pl_PL,pt_PT,ru_RU,sv_SE,tr_TR,zh_CN,zh_HK,zh_TW,am_ET,hi_IN,en_US,en_AU,en_IN,fr_FR,it_IT,es_ES,et_EE,de_DE,nl_NL,cs_CZ,pl_PL,ja_JP,zh_TW,zh_CN,zh_HK,ru_RU,ko_KR,nb_NO,es_US,da_DK,el_GR,tr_TR,pt_PT,pt_BR,sv_SE,bg_BG,ca_ES,en_GB,fi_FI,hi_IN,hr_HR,hu_HU,in_ID,iw_IL,lt_LT,lv_LV,ro_RO,sk_SK,sl_SI,sr_RS,uk_UA,vi_VN,tl_PH,ar_EG,fa_IR,th_TH,sw_TZ,ms_MY,af_ZA,zu_ZA,am_ET,en_XA,ar_XB,fr_CA,km_KH,lo_LA,ne_NP,si_LK,mn_MN,hy_AM,az_AZ,ka_GE,my_MM,mr_IN,ml_IN,is_IS,mk_MK,ky_KG,eu_ES,gl_ES,bn_BD,ta_IN,kn_IN,te_IN,uz_UZ,ur_PK,kk_KZ,sq_AL,gu_IN,pa_IN,be_BY,bs_BA,normal,xlarge,large --preferred-density 560dpi -M packages/screensavers/Basic/AndroidManifest.xml -S packages/screensavers/Basic/res  -I out/target/common/obj/APPS/framework-res_intermediates/package-export.apk --min-sdk-version 25 --target-sdk-version 25 --product nosdcard --version-code 25 --version-name 7.1.1   --skip-symbols-without-default-localization -F out/target/product/marlin/obj/APPS/BasicDreams_intermediates/package.apk ) && (find out/target/common/obj/APPS/BasicDreams_intermediates/ -maxdepth 1 -name \"classes*.dex\" | sort | xargs zip -qjX out/target/product/marlin/obj/APPS/BasicDreams_intermediates/package.apk ) && (if [ -d out/target/common/obj/APPS/BasicDreams_intermediates/jack-rsc ] ; then find out/target/common/obj/APPS/BasicDreams_intermediates/jack-rsc -type f | sort | sed -e \"s?^out/target/common/obj/APPS/BasicDreams_intermediates/jack-rsc/? -C \\\"out/target/common/obj/APPS/BasicDreams_intermediates/jack-rsc\\\" \\\"?\" -e\"s/\$/\\\"/\" > out/target/product/marlin/obj/APPS/BasicDreams_intermediates/jack_res_jar_flags; if [ -s out/target/product/marlin/obj/APPS/BasicDreams_intermediates/jack_res_jar_flags ] ; then jar uf out/target/product/marlin/obj/APPS/BasicDreams_intermediates/package.apk @out/target/product/marlin/obj/APPS/BasicDreams_intermediates/jack_res_jar_flags; fi; fi ) && (mv out/target/product/marlin/obj/APPS/BasicDreams_intermediates/package.apk out/target/product/marlin/obj/APPS/BasicDreams_intermediates/package.apk.unsigned ) && (java -Djava.library.path=out/host/linux-x86/lib64 -jar out/host/linux-x86/framework/signapk.jar --min-sdk-version \$((out/host/linux-x86/bin/aapt dump badging out/target/product/marlin/obj/APPS/BasicDreams_intermediates/package.apk.unsigned 2>&1 | grep '^sdkVersion' || echo \"sdkVersion:'0'\") | cut -d\"'\" -f2 | sed -e s/^.*[^0-9].*\$/25/) build/target/product/security/testkey.x509.pem build/target/product/security/testkey.pk8  out/target/product/marlin/obj/APPS/BasicDreams_intermediates/package.apk.unsigned out/target/product/marlin/obj/APPS/BasicDreams_intermediates/package.apk.signed ) && (mv out/target/product/marlin/obj/APPS/BasicDreams_intermediates/package.apk.signed out/target/product/marlin/obj/APPS/BasicDreams_intermediates/package.apk )"

Exception in thread "main" java.lang.ExceptionInInitializerError
        at org.conscrypt.OpenSSLBIOInputStream.<init>(OpenSSLBIOInputStream.java:34)
        at org.conscrypt.OpenSSLX509Certificate.fromX509PemInputStream(OpenSSLX509Certificate.java:119)
        at org.conscrypt.OpenSSLX509CertificateFactory$1.fromX509PemInputStream(OpenSSLX509CertificateFactory.java:220)
        at org.conscrypt.OpenSSLX509CertificateFactory$1.fromX509PemInputStream(OpenSSLX509CertificateFactory.java:216)
        at org.conscrypt.OpenSSLX509CertificateFactory$Parser.generateItem(OpenSSLX509CertificateFactory.java:94)
        at org.conscrypt.OpenSSLX509CertificateFactory.engineGenerateCertificate(OpenSSLX509CertificateFactory.java:272)
        at java.security.cert.CertificateFactory.generateCertificate(CertificateFactory.java:339)
        at com.android.signapk.SignApk.readPublicKey(SignApk.java:182)
        at com.android.signapk.SignApk.main(SignApk.java:1087)
Caused by: java.lang.IllegalArgumentException: Unknown cipher suite supported by native code: DH-DSS-AES256-GCM-SHA384
        at org.conscrypt.NativeCrypto.<clinit>(NativeCrypto.java:750)
        ... 9 more
[ 25% 10601/42050] target thumb C++: libLLVMSelectionDAG_...<= external/llvm/lib/CodeGen/SelectionDAG/DAGCombiner.cpp
ninja: build stopped: subcommand failed.
make: *** [build/core/ninja.mk:149: ninja_wrapper] Error 1

At first I thought it was a bug in the way OpenSSL filters worked which NixOS exposes (see #21408) but apparently not.

Issue upstream in conscrypt is here: google/conscrypt#20

@evrim
Copy link
Contributor

evrim commented Aug 7, 2017

ping @spacekitteh @abbradar

{ pkgs ? import  {} }:

let
  jdk =  pkgs.callPackage <nixpkgs/pkgs/development/compilers/openjdk/8.nix> {
    bootjdk = pkgs.callPackage <nixpkgs/pkgs/development/compilers/openjdk/bootstrap.nix> { version = "8"; };
    inherit (pkgs.gnome2) GConf gnome_vfs;
    minimal = true;
  };
  fhs = pkgs.buildFHSUserEnv {
    name = "android-env";
    targetPkgs = pkgs: with pkgs;
      [ git
        gitRepo
        gnupg1compat
        python2
        curl
        procps
	openssl
	gnumake
	nettools
	rsync
	androidenv.buildTools
	androidenv.platformTools
	ccache
	ncurses5
	jdk
	schedtool
	utillinux
	m4
        gperf
	perl
	libxml2
	zip
	unzip
	bison
	flex
	lzop
	bc
        which
	gcc
	binutils
	coreutils
	gdb
      ];
    multiPkgs = pkgs: with pkgs;
      [ zlib
      ];
    extraOutputsToInstall = [ "dev" ];
    runScript = "bash";
    profile = ''
      export USE_CCACHE=1
      export ANDROID_JAVA_HOME=${jdk.home}
      export LANG=C
      unset _JAVA_OPTIONS
      export BUILD_NUMBER=$(date --utc +%Y.%m.%d.%H.%M.%S)
      export DISPLAY_BUILD_NUMBER=true
    '';
  };
in fhs.env
bash-4.4$ ldd $(which java)
	linux-vdso.so.1 (0x000075585ff92000)
	libpthread.so.0 => /usr/lib/libpthread.so.0 (0x000075585fb53000)
	libz.so.1 => /usr/lib/libz.so.1 (0x000075585f93c000)
	libjli.so => /nix/store/xbr1i9c1gmz4if9l8g75pw6xzj7s1d9n-openjdk-8u141b15-jre/lib/openjdk/jre/lib/amd64/jli/libjli.so (0x000075585f72c000)
	libdl.so.2 => /usr/lib/libdl.so.2 (0x000075585f528000)
	libc.so.6 => /usr/lib/libc.so.6 (0x000075585f189000)
	/nix/store/l48biijfr1j6d5kdg911051x2phfjrz7-glibc-2.25/lib/ld-linux-x86-64.so.2 (0x000075585fd71000)

@evrim
Copy link
Contributor

evrim commented Aug 10, 2017

[aosp/out/repo/host/linux-x86/bin]# paxctl -vQ ./dex2oatd
PaX control v0.9
Copyright 2004,2005,2006,2007,2009,2010,2011,2012,2014 PaX Team 

- PaX flags: -p-s-m-x-e-r [./dex2oatd]

@evrim
Copy link
Contributor

evrim commented Aug 10, 2017

For 6.0.x

{ pkgs ? import  {} }:

let
  jdk =  pkgs.callPackage  {
    bootjdk = pkgs.callPackage  { version = "7"; };
    minimal = true;
  };
  fhs = pkgs.buildFHSUserEnv {
    name = "android-env";
    targetPkgs = pkgs: with pkgs;
      [ git gitRepo gnupg1compat python2 curl procps openssl gnumake nettools
        rsync androidenv.buildTools androidenv.platformTools ccache ncurses5 jdk schedtool utillinux
        m4 gperf perl libxml2 zip unzip bison flex lzop bc which gcc binutils coreutils gdb python
        emacs vim
      ];
    multiPkgs = pkgs: with pkgs;
      [ zlib
      ];
    extraOutputsToInstall = [ "dev" ];
    runScript = "bash";
    profile = ''
      export USE_CCACHE=1
      export ANDROID_JAVA_HOME=${jdk.home}
      export LANG=C
      unset _JAVA_OPTIONS
      export BUILD_NUMBER=$(date --utc +%Y.%m.%d.%H.%M.%S)
      export DISPLAY_BUILD_NUMBER=true
    '';
  };
in fhs.env

@Ekleog
Copy link
Member

Ekleog commented Aug 24, 2018

(triage) Is this still a problem? I know someone who recently built an Android OS using the instructions at https://nixos.wiki/wiki/Android.

@ajs124
Copy link
Member

ajs124 commented Feb 12, 2019

This is definitely not a problem anymore. I have successfully built LineageOS 15.1 and 16.0, so Android 8.1 and 9.0 on NixOS.

@Ekleog
Copy link
Member

Ekleog commented Feb 12, 2019

Closing for now, then.

@Ekleog Ekleog closed this as completed Feb 12, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
9.needs: package (new) This needs a new package to be added
Projects
None yet
Development

No branches or pull requests

8 participants