Discussion: dropping android 5&6 support

[Grimler91]

So, lets officially start the discussion about dropping android 5 support.
This concerns both the apps and termux-packages but lets gather the discussion here.

What about all the packages?

I suggest that we create an "android 5" branch which won't receive updates but might be useful for people stuck on android 5. We could then also set up a separate android 5 repo or (preferably I guess) let someone who's actually using android 5 host and maintain it.

If we create a separate android 5 repo, how do we inform all android 5 users and simplify the transition between? Posts in all community groups? A temporary new motd message?

What about the apps?

Should we create android 5 branches for the apps as well? Would it be possible to have termux-app as well as termux-app5 on playstore (or would it somehow conflict since both apps are installed into com.termux)?

Do we even want to bump the minimum android version for the apps as well or is it enough to bump the API_LEVEL in build-package.sh?

What should we so when we've bumped the required android version?

We have at least one if version == lollipop that can be removed.
There's probably some patches for packages that can be dropped (identifying which ones might be tedious though).

Most of the cool changes, like the possibility to stop using LD_LIBRARY_PATH will have to wait until we drop android 6 support.
@monoidic did some experiments with this in #2429.

What other changes/benefits have I not thought about?

[hyperpallium]

First, what benefit are there to dropping android 5? It seems most of the benefits would come from dropping 5, 5.1 and 6? Why not defer all the trauma until then, when it's more worth it?

BTW You can have distinct names on playstore (e.g. termux-app and termux-app5). That's how people do "lite" versions.
I have com.termuz on the same device (using aapt .... --rename-manifest-package com.termuz ....) - though there's a bunch of other stuff you need to change too, if you want to install both on the same device.

[fornwall]

I think we should target Android 7 directly.

We could create some kind of old-android branch of termux-packages (for supporting android 6&7) which is only maintained for critical bug fixes and as best effort.

For the package repo at termux.net perhaps the best thing is to leave the current one as is, to avoid people from updating and breaking their environment, and add a new repo alongside it. This would be in combination with updating the Termux app so that new installations get that repo automatically, and it could automatically replace the old repo with the new in etc/apt/sources.list if the device is running Android 7 or later.

For the apps, I think we should bump the required api version to avoid having Android 5 and 6 users installing the app and get a subpar experience with non-supported packages. Those knowledgeable enough can always work around it by finding an earlier app version on e.g. F-Droid if they really need it.

Thoughts?

What other changes/benefits have I not thought about?

One small thing that can be dropped is the libutil package, since the functions implemented there are built into bionic as of Android 6.0.

[tomty89]

I think we should target Android 7 directly.

Couldn't agree more. Android 6 seems to be on half way of many stuff. Doesn't seem worth bumping if that's our target.

[vishalbiswas]

I'm curious about third party repos.
@its-pointless @xeffyr How will you guys handle the API Level change?

[ghost]

Perhaps y'all could handle them apt-level, like:

deb https://termux.net/lollipop stable main
deb https://termux.net/marshmellow stable main
deb https://termux.net/nougat stable main
deb https://termux.net/oreo stable main
deb https://termux.net/pie stable main

or

deb https://termux.net/l stable main
deb https://termux.net/m stable main
deb https://termux.net/n stable main
deb https://termux.net/o stable main
deb https://termux.net/p stable main

[hyperpallium]

leave the current one as is, to avoid people from updating and breaking their environment, and add a new repo alongside it.

Seems the right approach.

earlier app version on e.g. F-Droid

Seems risky... I'm not sure people on older android versions are more likely to be across the android ecosystem...

How to check that https://f-droid.org/en/packages/com.termux/ has not been altered?

Is there some other way to keep the present app version available?

One solution is to leave the app as it is, and create a new version with a new name (com.termux2). Of course, this loses the ratings and download stats - and maybe people on 7, 8 etc wouldn't know to download the new one. But won't the play store only allow compatible downloads, according to the manifest? The description can note the other version.

Or, download the ols version directly from termux.net? (since it's already trusted for packages).

[Grimler91]

Targeting android 7 straight away sounds good to me!

@fornwall how is the current user distribution? How many percent are using android 5 or 6?

I have com.termuz on the same device (using aapt .... --rename-manifest-package com.termuz ....)

@hyperpallium Cool, thanks for that information. I guess all packages need to be recompiled with this new PREFIX if it is to be usuable.

For the package repo at termux.net perhaps the best thing is to leave the current one as is, to avoid people from updating and breaking their environment, and add a new repo alongside it. This would be in combination with updating the Termux app so that new installations get that repo automatically, and it could automatically replace the old repo with the new in etc/apt/sources.list if the device is running Android 7 or later.

I think the other way around sounds more reasonable. We can update apt and have a script replace "termux.net stable main" with "termux.net android-old main" if the android version is 5 or 6. We then wait a few months and hope that the wast majority of old android users have had their repo changed.
Feels more reasonable to have the main termux.net repo up to date, targeting newer android versions.

One solution is to leave the app as it is, and create a new version with a new name (com.termux2). Of course, this loses the ratings and download stats - and maybe people on 7, 8 etc wouldn't know to download the new one. But won't the play store only allow compatible downloads, according to the manifest? The description can note the other version.

Same here then really, feels like the main com.termux app should target up-to-date devices, but we could perhaps have another app targeting android 5 & 6.

Perhaps y'all could handle them apt-level, like:

@aiisuika The differences between the android versions aren't that big, so this might be slightly overkill, taking up 5x as much diskspace on the server. Also, people might not bother to build/test for different android versions, it would require quite a lot of extra time spent in emulators, it's a neat idea though!

[ghost]

I'm curious about third party repos.
@its-pointless @xeffyr How will you guys handle the API Level change?

@vishalbiswas I can handle it in this way:

1. Android 5.0-compatible repository will be accessible from default URL, so sources list will not be changed:

deb https://termux-x11.ml x11 main

2. Android 7.0 will require a different sources list line:

deb https://termux-x11.ml x11-nougat main

[hyperpallium]

@Grimler91 I guess all packages need to be recompiled with this new PREFIX if it is to be usuable.

Yes (it's also hardcoded in the app).

A key advance of termux over other terminals is packages modified for android's directory structure. But they are now compiled to termux's directory structure (literally, /data/data/com.termux/ etc). This could be an opportunity for all packages to dynamically looku the currrent $PREFIX instead of hard-coding it.

This enables different termux's to use the same repository without recompilation. It also enables "self-hosting", in the sense of developing termux within termux itself. The version being developed having a different name (and $PREFIX) from the termux doing the developing... so it can be installed without replacing the first termux, and all packages can be installed too!

[ghost]

This could be an opportunity for all packages to dynamically looku the currrent $PREFIX instead of hard-coding it.

@hyperpallium This is task is not easy and may have some bad effects (at least if PREFIX is long). See sample for bash how this is done: https://github.com/xeffyr/Alpine-Term/blob/master/env-packages/packages/bash/bash-4.4_config-top.h.patch.

It also enables "self-hosting", in the sense of developing termux within termux itself.

Unfortunately, it does not enable "self-hosting". At least because of difference between NDK's clang and one provided in Termux. It also not possible to compile big packages on device (out of ram at least on LLVM).
However if you want to just compile packages on device, take a look on https://github.com/xeffyr/termux-makepkg.

[hyperpallium]

dynamically looku the currrent $PREFIX instead of hard-coding it.

@hyperpallium This is task is not easy and may have some bad effects (at least if PREFIX is long). See sample for bash how this is done.

@xeffyr There's many packages to change, but... it seems to me the places needing the $PREFIX are already identified, and the additional snippet (to lookup and default if not found) would be the same for each. It seems feasible to even write a sed script to transform the present patch scripts, to automate this, based on the occurance of @TERMUX_PREFIX@. Some cases would be tricky. And such automatic transforms are problem-prone and thus more than a little scary!

There could be a test for the length of $PREFIX, in the termux-app, and an informative error message.

It also enables "self-hosting", in the sense of developing termux within termux itself.

Unfortunately, it does not enable "self-hosting".

I beg to differ, as I have compiled and installed the termux app, in termux (no PC). For the app, there's only a little C, and clang worked fine (invoked as gcc). I used this to experiment with horizontal scrolling. That's what I meant by in the sense of developing termux within termux itself.

The main work was a bash script to replace the gradle files. There's still some rough edges, but it basically works e.g. vim and curl are installed and work, after s///g the hardcoded $PREFIX. I agree you probably can't compile every package in the repository.

[fornwall]

@fornwall how is the current user distribution? How many percent are using android 5 or 6?

This is the user distribution of active installs of the Termux app from the Play store:

Android version	Installations (%)
8.0	20.0
7.0	18.4
6.0	18.1
7.1	14.3
8.1	12.0
5.1	9.8
5.0	3.8

Here active means: Number of Android devices that have been active in the previous 30 days on which your app is installed.

[fornwall]

@xeffyr There's many packages to change, but... it seems to me the places needing the $PREFIX are already identified, and the additional snippet (to lookup and default if not found) would be the same for each.

Actually, we only patch cases where the path is hard-coded into source files directly. Often paths end up in binaries after configuration from the build system (as in --prefix=$TERMUX_PREFIX for configure), so having dynamic prefix would probably require more patches.

That said, dynamic prefix would be interesting, I've created #2902 for that!

[hyperpallium]

@fornwall strikingly close to 1/3 each to 5/6, 7 and 8. A third is a lot...

@Grimler91 My first question would have been better as: What are the benefits to users?

[ Making development easier is important. But, like businesses that start placing operations ahead of customers, it can be the beginning of the end (unless it's a monopoly). Maybe there are great benefits to users, e.g. maybe fullscreen mode? ]

[ghost]

Inb4 libtermuxglibbionicc.so :^)

[Dmole]

...user distribution...

My (google, oneplus, LineageOS) devices are all on 9, is the % of Android 9 users unknown?
Those stats are only from people who opted in to reporting to Google, and did not install from f-droid.

I'd drop support for anything older than 6, because LineageOS is a free upgrade and supporting botnets on insecure devices is not cool;
Android 8 (from 2017-08-21) is still supported by Google.
Android 7,6 (from 2015-08-05) "security updates only" (no bug fixing)
Android 5 (from 2014-09-12) is EOL
And I'd drop anything older than 8 is some old Android bug consumed any development time.

[monoidic]

LineageOS is a free upgrade

One that is not an option on all devices and for some other reasons.

I'd drop anything older than 8

So only around ⅕ of the current users can keep using Termux...?

Don't get me wrong, I'm all for getting rid of the LD_{LIBRARY_PATH,PRELOAD} annoyances, being able to actually use patchelf and whatever else Android 7 brings, but "screw 'em" probably isn't the way to go about this.

[ghost]

I'd drop anything older than 8

Only Android 5/6 may be 'dropped'. There also plans for creating separate repositories for Android 5 and 7. Android 5 will receive only critical updates for packages, i.e. bug fixes but nothing new.

[ghost]

what bout an "emulated" libc.so that's actually proot with some changes

that'd also allow for users, groups, binfmt, ect...

[ghost]

what bout an "emulated" libc.so that's actually proot with some changes

@Bakaika

1. Proot is not "emulated" libc.
2. Usage of proot will at least degrade performance of the environment. Why not to use a QEMU then ?

[ghost]

i mean, libc.so where i/o operations are replaced with translated ones, faking of users, mounting, ect... like proot.
degration of performace is a big thing... perhaps a custom dpkg could modify packages as they're installed?
a qemu-proot deb compiler?

[ghost]

libc.so where i/o operations are replaced with translated ones,

@Bakaika No, this won't work here as it is impossible to intercept system calls with this way. You have to use ptrace() here so program like proot is needed.

mounting, ect... like proot.

proot does not do mounting and I'm not sure that there are people that can implement a userspace EXT4 driver usable with proot (even in read-only mode).

degration of performace is a big thing... perhaps a custom dpkg could modify packages as they're installed?
a qemu-proot deb compiler?

QEMU is not proot and I'm not talking about user-mode emulation. I'm about system emulation which actually gives much more than proot: full Linux distribution with real root permissions (on VM level) will provide multiuser environment, mounting and everything else but only on VM level.

The main point of Termux is to provide an environment similar to Linux distributions. It does not provide root, faking of users, mounting, but command line tools. Everything else is secondary: want root - then root your device, need classical unix fs layout - use termux-chroot (from package 'proot'), need something else - use linux chroot (proot) or QEMU (from my x11-packages repo).

[hyperpallium]

Turning it around, there are benefits for 5/6, for having a separate app, as changes have already been made for 7 and 8 that adversely affect 5/6. Examples I know of (I expect there are more):

1. fullscreen mode removed
2. termux-elf-cleaner leaving things that cause warnings
3. ecj needing java 8 features (ecj4.6 workaround)

[ghost]

Fine suggestion because support for things like DT_RPATH can be added. Thus I think stuff like LinuxBrew and third party relocatable binaries will work.

Whatever, keep a termux-old on fdroid.

[joakim-noah]

I don't think this would be a good idea at all: plenty of people still use old devices that stopped getting Android updates years ago, as @fornwall's stats show. Those devices are plenty fast enough to use Termux: I actually build LLVM on a 2014-vintage 32-bit Android/ARM device with 3 GB of RAM and that's stuck on Android 6, without a problem.

The benefits discussed here are fairly small and could probably be done now by changing the Termux build and scripts to be more customized for the different Android versions. Of course, I don't support these old Android versions for Termux, and you may be sick of doing it: I don't know how much work it is. But I think it'd be a mistake to cut the userbase like this anytime soon.

[tomty89]

I suppose it won't be possible to do (local) audio output with such setup? (In other words, we cannot make use of our own libraries and the ones in Android at the same time like now?)

[twaik]

@xeffyr Can you please make bootstrap & repo for armeabi-v7a arch? I have no aarch64 devices.

@tomty89 Audio support will be available after adding libhybris support.

[benyaminl]

I've lollipop aarch64 device. Can you send the APK for me to test? Sent from My Asus Pegasus X005 Pada tanggal Rab, 19 Des 2018 19.05 Twaik Yont <notifications@github.com menulis:

…

@xeffyr <https://github.com/xeffyr> Can you please make bootstrap & repo for armeabi-v7a arch? I have no aarch64 devices. @tomty89 <https://github.com/tomty89> Audio support will be available after adding libhybris support. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#2874 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ACr9NLAcA1n9rb_hnozujj1N1G6Mw5o9ks5u6iuZgaJpZM4W068h> .

[twaik]

@benyaminl Read it again: #2874 (comment)

[ghost]

repo for armeabi-v7a arch?

@twaik Now added support for ARM. Links are updated.

[ghost]

@xeffyr what is wrong with glibc then? Many packages are well supported on glibc than musl or others at the cost of being heavyweight.

I have run some stuff w/ glibc loader. Especially nano and patchelf appear to work. But there needs to be some more investigation on LD_LIBRARY_PATH. Perhaps all binaries should be just built / patched with custom rpath and loader ld-linux-armhf.so.3.

Interestingly it allows to use fakechroot as a preload library while running unmodified binaries from, say, debian repositories. Example I have run nano w/ /etc/nanorc in some custom base.

So glibc is interesting at least as a package.

Well, if all this experiment goes well we may see a musl branch, a glibc branch and bionic of course. :P

Also, I think gentoo prefix for android used their glibc in prefix dir. They probably failed because they didn't have a terminal nor built binaries for one explicitly (although there was a sourceforge repo for terminal ide). I am unsure how it is at the current state.

[twaik]

@extremenerd mussl is lightweight libc. That is important on mobile devices. Nothing in termux-packages changes: packages are still built from source using build.sh script and packed to debs. The only change is libc.

[ghost]

what is wrong with glibc then?

It's really hard to understand glibc's internals to make right changes in it.

Many packages are well supported on glibc than musl or others at the cost of being heavyweight.

Yes, many packages were developed relying on glibc as standard.

Glibc won't significantly reduce the amount of patches needed to compile software in Termux. It also won't allow you to normally use binaries copied from Linux distributions - don't forget that Termux's file system layout is far different.

Also, musl-based programs don't need rpath/runpath/ld_libary_path.

So glibc is interesting at least as a package.

Then patch & compile it yourself 😁.

a glibc branch

No, I won't do glibc branch.

Also, I think gentoo prefix for android used their glibc in prefix dir.

Gentoo prefix is broken by design. They even didn't changed the paths and some progs searched in /bin for some binaries but on Android this directory is not exist.

[benyaminl]

I compiled a basic packages (specifically bootstrap-ones) with musl-libc and did some experiments to check features what it gives.

Termux app that uses a custom bootstrap archive can be found here: https://github.com/xeffyr/termux-musl/releases/tag/app-v1.0-1.

So, what features we may have from custom (musl) libc, but that's likely not a complete list:

• No more LD_LIBRARY_PATH. Termux and Android will automatically detect correct library paths.
• Libc API/ABI will be same on all Android versions, so we don't need patches that implement features missing on older Android version.
• Termux will have own "system-wide" /etc/resolv.conf and /etc/hosts, so resolver configuration will be same for all packages. This will likely deal with issues like host (lookup tool) uses unknown resolver #3124 + adds possibility of configuring vhosts for nginx and similar.

Also, interesting if musl-based packages will work correctly on Android 5. If so, then it may be a solution for Android 5/6 devices. But that needs testing, as I have tried packages only on Android 8.

Seems your build working on my ASUS Pegasus X005 with Lollipop 5.1
Thanks God. Anyway how to port the rest of the termux package now? I see there's not a lot of it anyway. I also never gone into that stuff, but I want to learn and invest my time, any suggestion?

No warning error on Lollipop 5.1 ( For Android 5.0, I've one but it's not broken because the architecture is x86, dunno why)

One with Official Termux

And this's yours.

I just tried dumb thing, maybe another thing to test? Thanks

[Grimler91]

Let's perhaps take the bionic/musl/glibc discussion somewhere else (new issue?) as I believe it is only partly related to the android 5&6 drop. Such a transition would need way more testing before it can happen compared to a split of the repos into android 5&6 and android >=7 I guess.

Anyways, so roughly 35 % of users use android 5 or 6 which is a quite large part of the users. When would a android 5&6 drop feel justified? At 20 %? 10 %?

[benyaminl]

10% off them I think. Because until today many company still produce died Android device. 😑 Sent from My Asus Pegasus X005 Pada tanggal Jum, 4 Jan 2019 22.07 Henrik Grimler <notifications@github.com menulis:

…

Let's perhaps take the bionic/musl/glibc discussion somewhere else (new issue?) as I believe it is only partly related to the android 5&6 drop. Such a transition would need way more testing before it can happen compared to a split of the repos into android 5&6 and android >=7 I guess. Anyways, so roughly 35 % of users use android 5 or 6 which is a quite large part of the users. When would a android 5&6 drop feel justified? At 20 %? 10 %? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#2874 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ACr9NA4DQ5ynr71a38Yh4-n-bd30b-jeks5u_24tgaJpZM4W068h> .

[samoht0]

Anyways, so roughly 35 % of users use android 5 or 6 which is a quite large part of the users. When would a android 5&6 drop feel justified? At 20 %? 10 %?

As I commented above, I think, that depends on the maintenance of the legacy branch.

If it's just a freeze of feature/packages with updates still provided, I wouldn't mind doing it even at current distribution level.
But if a "real drop" is in question (leaving packages unmaintained in a separate repo), I would consider 10 % as suitable mark.

[Wetitpig]

@fornwall how is the current user distribution? How many percent are using android 5 or 6?

This is the user distribution of active installs of the Termux app from the Play store:

Android version Installations (%)
8.0 20.0
7.0 18.4
6.0 18.1
7.1 14.3
8.1 12.0
5.1 9.8
5.0 3.8
Here active means: Number of Android devices that have been active in the previous 30 days on which your app is installed.

Is there any new update? There may be a number of users who have upgraded their Android version already.

[benyaminl]

Could be the option isn't dropping lollipop please?

[Wetitpig]

@benyaminl It is the problem with the timing rather than whether doing so. Anyway we need to drop Lollipop support later as Lollipop is already outdated.

[ghost]

@Wetitpig Once we will find a different hosting, we will create a separate repository. Then Android 5/6 drop. So question is not in "time".

[ghost]

Could be the option isn't dropping lollipop please?

No. Lollipop repo will continue exist some time.

[Grimler91]

Once we will find a different hosting, we will create a separate repository.

This was discussed in the termux development sessions, summaries of the discussions can be find on the wiki here: https://wiki.termux.com/wiki/Dev:Development_Sessions for those interested (full logs of conversations are found on gitter in the termux/dev channel and they are also sent to the termux-dev mailing list: https://groups.io/g/termux-dev).

[fornwall]

The master branch is now for Android 7.0 and later and is available for alpha testing on bintray. Join https://gitter.im/termux/dev or the mirrorred IRC channel #termux/development on freenode if you want to help testing (aarch64 devices only initially).

The termux.net repo will continue to be updated as previously, only that the packages will now come from the android-5 branch.