Feature request: Handle Termux PREFIX

[cognifloyd]

I'm on Android, trying to use this in Termux.

One issue I've found so far is best summarized by https://github.com/termux/termux-exec:

A lot of Linux software is written with the assumption that /bin/sh, /usr/bin/env and similar file exists. This is not the case on Android where neither /bin/ nor /usr/ exists.

When building packages for Termux those hard-coded assumptions are patched away - but this does not help with installing scripts and programs from other sources than Termux packages.

I ran into this issue when attempting to run this in a clone of the pantsbuild/pants repo:

~/git/pants $ pants version
Error: Failed to exec process.

Caused by:
    ENOENT: No such file or directory

Which appears to be triggered here:

scie-pants/src/main.rs

Lines 130 to 145 in 9646e81

	Some(pants_bootstrap) if self != Self::BootstrapTools && pants_bootstrap.is_file() => {
	Process {
	exe: "/usr/bin/env".into(),
	args: vec![
	"bash".into(),
	"-c".into(),
	format!(
	r#"set -eou pipefail; source {bootstrap}; exec {scie} "$0" "$@""#,
	bootstrap = Self::quote(pants_bootstrap)?,
	scie = Self::quote(scie)?
	)
	.into(),
	],
	env,
	}
	}

scie-pants/src/main.rs

Lines 70 to 72 in 9646e81

	execv(&c_exe, &c_args)
	.map(|_| 0)
	.context("Failed to exec process.")

I suspect rust does not use LD_PRELOAD which is how https://github.com/termux/termux-exec fixes this error.

Is there something else scie-pants could do to detect it is running in a prefix, and so should use PATH or PREFIX vars?

~/git/pants $ grep PATH ~/.bashrc
export PATH="${HOME}/.local/bin:${PATH}"
~/git/pants $ uname -a
Linux localhost 4.14.295-g3547aebef7e0-ab9438649 #1 SMP PREEMPT Tue Dec 27 07:28:00 UTC 2022 aarch64 Android
~/git/pants $ env
SHELL=/data/data/com.termux/files/usr/bin/bash
COLORTERM=truecolor
HISTCONTROL=ignoreboth
PREFIX=/data/data/com.termux/files/usr
TERMUX_IS_DEBUGGABLE_BUILD=1
TERMUX_MAIN_PACKAGE_FORMAT=debian
PWD=/data/data/com.termux/files/home/git/pants
TERMUX_VERSION=0.118.0
EXTERNAL_STORAGE=/sdcard
LD_PRELOAD=/data/data/com.termux/files/usr/lib/libtermux-exec.so
TERMUX_API_VERSION=0.50.1
HOME=/data/data/com.termux/files/home
LANG=en_US.UTF-8
TERMUX_APK_RELEASE=GITHUB
DEX2OATBOOTCLASSPATH=/apex/com.android.art/javalib/core-oj.jar:/apex/com.android.art/javalib/core-libart.jar:/apex/com.android.art/javalib/okhttp.jar:/apex/com.android.art/javalib/bouncycastle.jar:/apex/com.android.art/javalib/apache-xml.jar:/system/framework/framework.jar:/system/framework/framework-graphics.jar:/system/framework/ext.jar:/system/framework/telephony-common.jar:/system/framework/voip-common.jar:/system/framework/ims-common.jar:/apex/com.android.i18n/javalib/core-icu4j.jar
TMPDIR=/data/data/com.termux/files/usr/tmp
ANDROID_DATA=/data
TERM=xterm-256color
ANDROID_I18N_ROOT=/apex/com.android.i18n
DISPLAY=:1
SHLVL=1
ANDROID_ROOT=/system
BOOTCLASSPATH=/apex/com.android.art/javalib/core-oj.jar:/apex/com.android.art/javalib/core-libart.jar:/apex/com.android.art/javalib/okhttp.jar:/apex/com.android.art/javalib/bouncycastle.jar:/apex/com.android.art/javalib/apache-xml.jar:/system/framework/framework.jar:/system/framework/framework-graphics.jar:/system/framework/ext.jar:/system/framework/telephony-common.jar:/system/framework/voip-common.jar:/system/framework/ims-common.jar:/apex/com.android.i18n/javalib/core-icu4j.jar:/apex/com.android.adservices/javalib/framework-adservices.jar:/apex/com.android.adservices/javalib/framework-sdksandbox.jar:/apex/com.android.appsearch/javalib/framework-appsearch.jar:/apex/com.android.btservices/javalib/framework-bluetooth.jar:/apex/com.android.conscrypt/javalib/conscrypt.jar:/apex/com.android.ipsec/javalib/android.net.ipsec.ike.jar:/apex/com.android.media/javalib/updatable-media.jar:/apex/com.android.mediaprovider/javalib/framework-mediaprovider.jar:/apex/com.android.ondevicepersonalization/javalib/framework-ondevicepersonalization.jar:/apex/com.android.os.statsd/javalib/framework-statsd.jar:/apex/com.android.permission/javalib/framework-permission.jar:/apex/com.android.permission/javalib/framework-permission-s.jar:/apex/com.android.scheduling/javalib/framework-scheduling.jar:/apex/com.android.sdkext/javalib/framework-sdkextensions.jar:/apex/com.android.tethering/javalib/framework-connectivity.jar:/apex/com.android.tethering/javalib/framework-connectivity-t.jar:/apex/com.android.tethering/javalib/framework-tethering.jar:/apex/com.android.uwb/javalib/framework-uwb.jar:/apex/com.android.wifi/javalib/framework-wifi.jar
ANDROID_TZDATA_ROOT=/apex/com.android.tzdata
TERMUX_APP_PID=580
PATH=/data/data/com.termux/files/home/.local/bin:/data/data/com.termux/files/usr/bin
ANDROID_ART_ROOT=/apex/com.android.art
OLDPWD=/data/data/com.termux/files/home/git             _=/data/data/com.termux/files/usr/bin/env

note: there are probably other issues running with Termux. I will file separate issues.

[cognifloyd]

This also seems to cause networking issues.

I ran pants --version in the st2 repo and got this:

 ~/git/st2 $ pants --version
 Downloading https://github.com/indygreg/python-build-standalone/releases/download/20221106/cpython-3.9.15+20221106-aarch64-unknown-linux-gnu-install_only.tar.gz...
 [snip duplicate Downloading messages]
 Downloading https://github.com/indygreg/python-build-standalone/releases/download/20221106/cpython-3.9.15+20221106-aarch64-unknown-linux-gnu-install_only.tar.gz...
 Failed to source file cpython-3.9.15+20221106-aarch64-unknown-linux-gnu-install_only.tar.gz: Failed to fetch https://github.com/indygreg/python-build-standalone/releases/download/20221106/cpython-3.9.15+20221106-aarch64-unknown-linux-gnu-install_only.tar.gz: [6] Couldn't resolve host name (Could not resolve host: github.com)
 Error: Isolates your Pants from the elements.
 
 Please select from the following boot commands:
  
 [snip boot commands]
 
 You can select a boot command by passing it as the 1st argument or else by setting the SCIE_BOOT environment variable.
 
 ERROR: Failed to establish atomic directory /data/data/com.termux/files/home/.cache/nce/8bd259181e05963da9f88937eb792e9f1b3dd0205c500ad1e0c86685e31bdb1d/locks/configure-521fac2917ca5b3aa1b53bfcfe140da3fd6afdea57eb50986d03bb0ab54826b3. Population of work directory failed: Failed to establish atomic directory /data/data/com.termux/files/home/.cache/nce/52a8c0a67fb919f80962d992da1bddb511cdf92faf382701ce7673e10a8ff98f/cpython-3.9.15+20221106-aarch64-unknown-linux-gnu-install_only.tar.gz. Population of work directory failed: The tar.gz destination /data/data/com.termux/files/home/.cache/nce/52a8c0a67fb919f80962d992da1bddb511cdf92faf382701ce7673e10a8ff98f/cpython-3.9.15+20221106-aarch64-unknown-linux-gnu-install_only.tar.gz of size 0 had unexpected hash: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855

Then I used strace pants version to see why it was failing.

Looking at the strace, it looked all over for ssl/tls stuff, eventually finding it in the PREFIX. It also tried and failed to access /etc/resolve.conf:

 14233 openat(AT_FDCWD, "/etc/resolv.conf", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = -1 ENOENT (No such file or directory)

So, it doesn't know how to look up dns settings. It was able to find the SSL files after looking all over the place.

Looking at the strace for wget (which succeeds), it looks under the prefix to find the file:

/data/data/com.termux/files/usr/etc/resolv.conf
 19411 openat(AT_FDCWD, "/data/data/com.termux/files/usr/etc/resolv.conf", O_RDONLY) = 12

[cognifloyd]

This is probably going to take quite a bit of effort to get working. I'll poke at it every now and then to try and make progress. Today's progress was identifying the core incompatibilities. Next time I will try building scie-pants for Termux. I suspect I will run into issues with PBS since that does not have any of the Termux-specific patches.

Anyway, I'm assigning this to myself.

[cognifloyd]

I'm closer. Apparently, Termux has proot which is like chroot but uses ptrace to rewrite file access to use the prefix files as root.

https://wiki.termux.com/wiki/Differences_from_Linux
https://wiki.termux.com/wiki/PRoot

~/git/st2 $ pants version
Error: Isolates your Pants from the elements.

Please select from the following boot commands:

<default>: Detects the current Pants installation and launches it.
bootstrap-tools: Introspection tools for the Pants bootstrap process.
pants: Runs a hermetic Pants installation.
pants-debug: Runs a hermetic Pants installation with a debug server for debugging Pants code.
update: Update scie-pants.

You can select a boot command by passing it as the 1st argument or else by setting the SCIE_BOOT environment variable.
ERROR: Failed to establish atomic directory /home/.cache/nce/8bd259181e05963da9f88937eb792e9f1b3dd0205c500ad1e0c86685e31bdb1d/locks/configure-c213ec16f6484639fd3986c211e2cac58d9c3067ef46f4445539ed5a24762323. Population of work directory failed: Failed to launch boot binding: Failed to spawn "/home/.cache/nce/52a8c0a67fb919f80962d992da1bddb511cdf92faf382701ce7673e10a8ff98f/cpython-3.9.15+20221106-aarch64-unknown-linux-gnu-install_only.tar.gz/python/bin/python3.9" ["/home/.cache/nce/78d6778ce890b1d6a79b6cc36815f8e2af1501dff97b0e17378c943892026a44/tools.pex", "configure-pants", "--ptex-path", "/home/.cache/nce/826c8b0f361ddc31bd9c38683d1a9e6f07267162e31fcb9a0a48b92b2670e50b/ptex", "--pants-version", "2.16.0rc0", "--pants-sha", "", "--pants-config", "", "--github-api-bearer-token", "", "/home/.cache/nce/8bd259181e05963da9f88937eb792e9f1b3dd0205c500ad1e0c86685e31bdb1d/bindings"]: No such file or directory (os error 2)

So, using proot allowed DNS resolution and downloading python to succeed. Now python is having issues starting:

~/git/st2 $ ~/.cache/nce/52a8c0a67fb919f80962d992da1bddb511cdf92faf382701ce7673e10a8ff98f/cpython-3.9.15+20221106-aarch64-unknown-linux-gnu-install_only.tar.gz/python/bin/python3.9 version
bash: /home/.cache/nce/52a8c0a67fb919f80962d992da1bddb511cdf92faf382701ce7673e10a8ff98f/cpython-3.9.15+20221106-aarch64-unknown-linux-gnu-install_only.tar.gz/python/bin/python3.9: cannot execute: required file not found

strace didn't help (it had to be outside of proot with:

$ strace -fo .git/strace-py ~/.cache/nce/52a8c0a67fb919f80962d992da1bddb511cdf92faf382701ce7673e10a8ff98f/cpython-3.9.15+20221106-aarch64-unknown-linux-gnu-install_only.tar.gz/python/bin/python3.9 version
strace: exec: No such file or directory