Need to be able to create App Bundles (.aab) files in addition to APK files

[mtthws]

Google Play now prefers to have app bundles (.aab) files uploaded instead of apk files. It would be nice if there was a way to have this tool produce app bundles instead of apks.

Versions

• Python: N/A
• OS: N/A
• Kivy: N/A
• Cython: N/A

Description

I would like to be able to produce App Bundles that have all of the desired architectures in them to upload to the play store. It can then take care of key management and split the bundles up for the device to be installed on. This then means only needing to produce a single artifact rather than an apk for ever desired target.

buildozer.spec

N/A

Command:
N/A

Spec file:
N/A

Logs

N/A

[alexgrusu]

Have you found a solution until now?

[xloem]

This looks pretty relevant: https://github.com/google/bundletool

[tshirtman]

As we currently use gradle internaly in p4a, https://developer.android.com/studio/build/building-cmdline#bundle_build_gradle might be a good place to start.

[kengoon]

I currently create .aab using android studio after building the apk with bulldozer(actually bulldozer will create an android studio project style) though I have not uploaded it to google playstore and download it on different android arch to see if it works

[PetroGit]

I have just developed an app for android with kivy and found the same issue. Google play store requires since months ago an aab including apks for 64bits architecture and buildozer only generates apk for each architecture 32bits and 64bits. This is a kind of relevant show stopper for kivy and python for android when it comes to 32 bits architecture eg. armeabiv7a instruction set e.g. Motorola Google Nexus 6 6.0.1. Thre is a need to create Need to be able to create App Bundles (.aab) files in addition to APK files.

[obfusk]

Google play still accepts uploading both a 32bit and 64bit apk. Though this of course has drawbacks.

It seems that running ./gradlew bundleDebug or ./gradlew bundleRelease in the build directory after p4a is done already works. Not that that is ideal.

[LionReal]

Have you found a new solution for this problem?

[tshirtman]

Switching to a completely different framework, when you can run one (undocumented, but explained in previous messages) command here, seems a bit extreme.
Sure, it would be nice if someone took the time to expose this command through p4a and/or buildozer, to make life a little bit easier, but it doesn’t seems like it’s a roadblock right now. as per #2084 (comment) (through android studio) and #2084 (comment) (through CLI).

[xloem]

alexgrusu, are you angry at this app bundle issue that you reference flutter so curtly? It doesn't look like flutter uses python as its language, although I haven't explored in depth.

[alexgrusu]

I'm not angry at all. I explored and used kivy a lot and it's a good framework but with some limitations. One of this limitations is explained inhere. If you go further and target to build large scale projects, that include various features, you will encounter other barriers.

[xloem]

Sorry that kivy has frustrated you. We all need workflows that function smoothly. For me that sometimes means using python. Are you able to reference issues for the other barriers you encountered so people can know what needs work?

It looks like the bundleDebug and bundleRelease gradle targets, to close this issue, might be added around

python-for-android/pythonforandroid/toolchain.py

Line 1079 in bf76add

gradle_task = "assembleDebug"

although I'm not at a system to test right now.

[obfusk]

A downside (for some) of app bundles is that you cannot use a private signing key that Google doesn't have access to.

[misl6]

I'm not primarily working on Android, so it wasn't on my top list until a week ago, but this one seems pretty relevant in 2021.

The following one is the statement that Google made about the new requirements for new apps (targeting August 2021) and updates (targeting November 2021):

Requirements for new apps

From August 2021, the Google Play Console will require all new apps to:

Publish with the Android App Bundle format.
Utilize Play Asset Delivery or Play Feature Delivery to deliver assets or features that exceed download size of 150MB. Expansion files (OBBs) will no longer be supported for new apps.
Target API level 30 (Android 11) or above and adjust for behavioral changes.
Requirements for updates to existing apps

From November 2021, updates to existing apps will be required to target API level 30 or above and adjust for behavioral changes in Android 11. Existing apps that are not receiving updates are unaffected and can continue to be downloaded from the Play Store.
Source: https://android-developers.googleblog.com/2020/11/new-android-app-bundle-and-target-api.html?m=1

I analyzed the python-for-android source code, the Android App Bundle specification, and the https://github.com/google/bundletool repo.

Apart from some minor changes in order to support a new distributing format, the major issue is how We ship the "user app" + _python_bundle.

Right now, python-for-android, packages these two (or more, if specified), folders into a private.mp3 file, which is then placed into the assets folder of the apk.

The private.mp3 is essentially an ABI-specific tar file, that is unpacked when starting the app the first time or when it changes (ex: when updating).

Unfortunately, shipping different (ABI-specific) files into the assets folder isn't supported. (See google/bundletool#190)

So, We need to check if the workaround of placing a non-lib, but ".so-extensioned" file is fine also for our use case or not.

@tshirtman , @AndreMiras , any thoughts?

[xloem]

@misl6 so you know, I got really confused reading that python for android packages folders into an mp3 file, a format which is used to make audio files small. I infer it's a typo other software engineers will understand, but it might be clear to fix it for all of us to understand and help together.

[tshirtman]

@misl6 so you know, I got really confused reading that python for android packages folders into an mp3 file, a format which is used to make audio files small. I infer it's a typo other software engineers will understand, but it might be clear to fix it for all of us to understand and help together.

Actually, it's named mp3, although it's a tar.gz (similar to zip) archive, i don't think if it's necessary anymore, but early in p4a history, it was, as the system restricted the type of files that could be shipped in an app, and mp3 was one of the unrestricted types. It stayed that way because there was no real reason to change it.

@misl6 yeah, that deadline is starting to loom, and it's certainly time to look into it more seriously. I've looked at the bundletool issue, and indeed, it might be time to reconsider the current way,

If we ship python as part of the main bundle, wouldn't that make aab packages a lot bigger? or can we still have compression if we go this route?

Should user python files still be part of assets, or is it better to put them in lib as well? (one thing to consider could be Cython and other native extensions, that works as python modules, and hence should be together as python files). Unless bundles are installed by being extracted in the same directory and hence can mix their files in same target directories?

Also, could this mean we could pre package aab files for python and other recipes, and only build an asset aab for user code without requiring any compilation of their code?

[obfusk]

I'm not expecting to use app bundles any time soon myself and don't really know all the details, but I'd be happy to help with this a bit if needed.

If the only problems are

1. running the correct gradle command to create a bundle instead of an APK
2. no longer being able to put architecture-dependent files in the tarball (.mp3)

then 1) seems easy to fix

and 2) is hopefully easy to "fix" short-term by

• skipping all .so files from being included in the tarball
• making a list of those files, adding it to the tarball
• copying those .so files to lib/ instead (e.g. as <sha1-hash-of-relative-file-path>.so to prevent having a lib prefix and duplicate file names)
• copying/symlinking those files to the right place (using the list) after extracting the tarball

NB: this doesn't address the lack of support in p4a for building for multiple architectures at the same time.

[aleonov224]

Hi,
I verified aab bundle file created by Android studio. This way not works properly.

In this image: left side is apk created from bundle using bundle tool, right side is buildozer generated apk.
So, apk from aab is missed next items:

• assets dir
• res dir
• classes.dex file
• resources.arsc file
  And different manifest file.
  When I try to upload aab to the Google PlayMarket I not see warnings related to aab.
  But, of course, application is empty and not works properly.

AAB issue will be show stopper to continue using Kivy+Buildozer.

[aleonov224]

Hi,
I verified aab bundle file created by Android studio. This way not works properly.

In this image: left side is apk created from bundle using bundle tool, right side is buildozer generated apk.
So, apk from aab is missed next items:

* assets dir

* res dir

* classes.dex file

* resources.arsc file
  And different manifest file.
  When I try to upload aab to the Google PlayMarket I not see warnings  related to aab.
  But, of course, application is empty and not works properly.

AAB issue will be show stopper to continue using Kivy+Buildozer.

Looks like I missed one parameter, I tried again like
java -jar bundletool-all-1.6.1.jar build-apks --bundle=file.aab --output=apk-output.apks --ks=keystore.jks --ks-pass=pass:password --ks-key-alias=keyalias --key-pass=pass:password --mode=default

And I have universal.apk in the apks archive.
--- I will verify it again with GooglePlay.
Verified, my application cannot start - down while startup - for AAB created by android studio and deployed to Google Play market.

[LionReal]

@misl6
> Task :compileReleaseJavaWithJavac
[DEBUG]: /home/admin/build/android/platform/build-arm64-v8a/dists/test/src/main/java/org/kivy/android/PythonActivityUtil.java:92: error: method extractTar in class AssetExtract cannot be applied to given types;
[DEBUG]: if (!ae.extractTar(resource + ".mp3", target.getAbsolutePath())) {
[DEBUG]: ^
[DEBUG]: required: String,String,String
[DEBUG]: found: String,String
[DEBUG]: reason: actual and formal argument lists differ in length
[DEBUG]: Note: Some input files use or override a deprecated API.
[DEBUG]: Note: Recompile with -Xlint:deprecation for details.
[DEBUG]: Note: Some input files use unchecked or unsafe operations.
[DEBUG]: Note: Recompile with -Xlint:unchecked for details.
[DEBUG]: 1 error
[DEBUG]:
[DEBUG]: > Task :compileReleaseJavaWithJavac FAILED

// If the disk data is out of date, extract it and write the version file.
if (! dataVersion.equals(diskVersion)) {
Log.v(TAG, "Extracting " + resource + " assets.");

        recursiveDelete(target);
        target.mkdirs();

        AssetExtract ae = new AssetExtract(mActivity);
        if (!ae.extractTar(resource + ".mp3", target.getAbsolutePath())) {
            toastError("Could not extract " + resource + " data.");
        }

[misl6]

@LionReal I don't get why you mentioned me, can you please better explain?

[LionReal]

@misl6
I tried your branch for testing; I got this message, but I cleared the build directory and I didn't get this message.

[LionReal]

Problems I had with the p4a aab trial:
The configuration I am using is Ubuntu 20.04 x86_64
1- I got this error when I selected "ndk-api=23" "configure: error: C compiler cannot create executables"
I was able to get past this error with "ndk-api=21".
2- In the system, you need to create the "python-3.8.6-reproducible-buildinfo.diff" file according to the relevant version.
3-Cannot create multi arch(armeabi-v7a,arm64-v8a,x86_64,x86) aab in one go.
If you try to create aab by selecting arch one by one, the arch you selected also creates aab and since the files it created while creating this aab file remain in the "python-for-android/dists" folder, when you try later, multi arch( armeabi-v7a,arm64-v8a, x86_64,x86) You can create aab.

[grummble-ai]

Any updates? Thanks.

[misl6]

HI @grummble-ai and @LionReal, the PR #2467 is now been updated.
Feel free to test it, and if you're encountering any issues while using it, post your results on the PR comments.

[Sahil-pixel]

is there any update ?

[Gurami30]

Why does every one posting how to generate .apk from existing .aab file? Every answer I check to questions like these, answer are generate .apk file from .aab. Personally I made Python/Kivy app using Pycharm and want to build Android App Bundle(aab). So basically is there any way to build .aab file from Python/Kivy?

[kengoon]

@misl6 I noticed some abnormalities from the method you used in building .aab files:

1. the aab when installed is not shareable with other devices using apps like xender or esfile-explorer; Ok let me make it more clear, The app can be shared but the lib folder in the apk is not shared together with it thereby making it non-installable on any device.
2. When the app is being installed from the google play store it installs both the arm64-v8a and armeabi-v7a this increases the use of storage space (or causes bloatware if I am correct)

The picture below shows an apk separated from a kivy app using an online apk Downloader:

The picture below shows two apks separated from a facebook app using an online apk Downloader:

Judging from both apk you can see clearly that p4a does not package the aab in such a way the app will separate the apk in two ways

This is a kivy apk after installation from play store and trying to share with xender
PicDF_V1.0.1.zip
NB: I renamed this to .zip so as to be able to upload it here on github

you can also try to download the original from here https://apkcombo.com/apk-downloader/?q=org.picdf.picdf

Please do not get me wrong, you have done a great job so far but I believe this might be an issue later in the future where an app might be shared due to no internet connection in a particular area, Also the app seems to unpack every time I start up the app(judging from ADB logcat) this prolongs the app startup a little bit unlike before

[Guhan-SenSam]

@kengoon. I am not really sure about the first point, but the second point I believe that play store only sends the correct apk based on architecture. After deploying my app picdf i tested it in the internal testing track with a 32 bit phone and a 64 bit phone. The 32 bit phone received a download apk of size around 32 mb while the 64 bit one got a download of 34.5 mb. Which means the proper apk is installed based on the architecture of the device

[kengoon]

@Guhan-SenSam thanks for the reply and also, I apologize for tearing up your app in public without your permission.

Please I have one question for you:

• what was the original size of the aab file

[Guhan-SenSam]

@kengoon The original size of the aab was 60.7 mb

[Guhan-SenSam]

Here is the breakdown

[kengoon]

@Guhan-SenSam Ok that is better because, I was scared that they were packed in one. Whenever you have time you can check up on my first point to see if you get the same result as me

[misl6]

Fixed via #2467