App crashing on startup- ImportError: dlopen failed: _imaging.so is 64-bit

[zoemaestra]

EDIT: https://github.com/kivy/kivy/issues/4095 seems to be the same issue, but nobody has gotten an answer from there.

Hi, when I run an apk which has the pillow imported in it (from PIL import Image) it will crash on startup citing this, and force closing:
ImportError: dlopen failed: "/data/data/org.benjimaestro.hypertext/files/app/lib/python2.7/site-packages/PIL/_imaging.so" is 64-bit instead of 32-bit

My buildozer.spec is this:

# (str) Title of your application
title = HyperText

# (str) Package name
package.name = hypertext

# (str) Package domain (needed for android/ios packaging)
package.domain = org.benjimaestro

# (str) Source code where the main.py live
source.dir = .

# (list) Source files to include (let empty to include all the files)
source.include_exts = py,png,jpg,kv,atlas

# (list) List of inclusions using pattern matching
#source.include_patterns = assets/*,images/*.png

# (list) Source files to exclude (let empty to not exclude anything)
#source.exclude_exts = spec

# (list) List of directory to exclude (let empty to not exclude anything)
#source.exclude_dirs = tests, bin

# (list) List of exclusions using pattern matching
#source.exclude_patterns = license,images/*/*.jpg

# (list) Application requirements
# comma seperated e.g. requirements = sqlite3,kivy
requirements = kivy,hostpython2,plyer,android,pil,pillow

# (str) Custom source folders for requirements
# Sets custom source for any requirements with recipes
# requirements.source.kivy = ../../kivy

# (list) List of service to declare
#services = NAME:ENTRYPOINT_TO_PY,NAME2:ENTRYPOINT2_TO_PY

#
# OSX Specific
#

#
# author = © Copyright Info

# change the major version of python used by the app
osx.python_version = 3

# Kivy version to use
osx.kivy_version = 1.9.1

#
# Android specific
#

# (bool) Indicate if the application should be fullscreen or not
fullscreen = 0

# (list) Permissions
android.permissions = INTERNET,VIBRATE,BIND_NOTIFICATION_LISTENER_SERVICE,STATUS_BAR

# (int) Android API to use
android.api = 21

# (int) Minimum API required
android.minapi = 9

# (int) Android SDK version to use
android.sdk = 20

# (str) Android NDK version to use
android.ndk = 9c

# (str) Android entry point, default is ok for Kivy-based app
#android.entrypoint = org.renpy.android.PythonActivity

# (str) python-for-android branch to use, defaults to master
#p4a.branch = stable

# (str) OUYA Console category. Should be one of GAME or APP
# If you leave this blank, OUYA support will not be enabled
#android.ouya.category = GAME

# (str) Filename of OUYA Console icon. It must be a 732x412 png image.
#android.ouya.icon.filename = %(source.dir)s/data/ouya_icon.png

# (str) XML file to include as an intent filters in <activity> tag
#android.manifest.intent_filters =
# (bool) Use --private data storage (True) or --dir public storage (False)
#android.private_storage = True

# (str) Android NDK directory (if empty, it will be automatically downloaded.)
#android.ndk_path =

# (str) Android SDK directory (if empty, it will be automatically downloaded.)
#android.sdk_path =

# (str) ANT directory (if empty, it will be automatically downloaded.)
#android.ant_path =

# (bool) If True, then skip trying to update the Android sdk
# This can be useful to avoid excess Internet downloads or save time
# when an update is due and you just want to test/build your package
# android.skip_update = False

# (str) Android entry point, default is ok for Kivy-based app
#android.entrypoint = org.renpy.android.PythonActivity

# (list) Pattern to whitelist for the whole project
#android.whitelist =

# (str) Path to a custom whitelist file
#android.whitelist_src =

# (str) Path to a custom blacklist file
#android.blacklist_src =

# (list) List of Java .jar files to add to the libs so that pyjnius can access
# their classes. Don't add jars that you do not need, since extra jars can slow
# down the build process. Allows wildcards matching, for example:
# OUYA-ODK/libs/*.jar
#android.add_jars = foo.jar,bar.jar,path/to/more/*.jar

# (list) List of Java files to add to the android project (can be java or a
# directory containing the files)
#android.add_src =

# (list) Android AAR archives to add (currently works only with sdl2_gradle
# bootstrap)
#android.add_aars =

# (list) Gradle dependencies to add (currently works only with sdl2_gradle
# bootstrap)
#android.gradle_dependencies =

# (str) python-for-android branch to use, defaults to master
#p4a.branch = stable

# (str) OUYA Console category. Should be one of GAME or APP
# If you leave this blank, OUYA support will not be enabled
#android.ouya.category = GAME

# (str) Filename of OUYA Console icon. It must be a 732x412 png image.
#android.ouya.icon.filename = %(source.dir)s/data/ouya_icon.png

# (str) XML file to include as an intent filters in <activity> tag
#android.manifest.intent_filters =

# (list) Android additionnal libraries to copy into libs/armeabi
android.add_libs_armeabi = libs/android/*.so
android.add_libs_armeabi_v7a = libs/android-v7/*.so
#android.add_libs_x86 = libs/android-x86/*.so
#android.add_libs_mips = libs/android-mips/*.so

# (str) The Android arch to build for, choices: armeabi-v7a, arm64-v8a, x86
android.arch = armeabi-v7a

# (str) python-for-android git clone directory (if empty, it will be automatically cloned from github)
p4a.source_dir = /home/benjamin/python-for-android

# (str) Bootstrap to use for android builds
# p4a.bootstrap = sdl2

I am using Python 3.5 64bit on Ubuntu 64bit. The Pillow module was installed with pip.

[inclement]

Thanks for the report, it sounds like the PIL recipe has a bug.

[AndreMiras]

Well it could have a bug indeed, but your buildozer.spec seems wrong to me. Since you've added both pil and pillow in here:

requirements = kivy,hostpython2,plyer,android,pil,pillow

But pillow does not (yet) have a recipe, see #786.
So the error you're having would definitely happen if you try to build/install an extension which has no recipe. Even if the pil recipe was not broken ;)

Edit:

I could successfully test the PIL module on my install. Since you installed pillow once, you need to make sure it's completely cleared before trying again with PIL. I remember I had some issues with that where the previous pillow install was remaining. You could try to manually delete the Android application as well as running:

buildozer android p4a -- clean_recipe_build pil

Before doing your build and deployment again, good luck :)

[inclement]

Thanks @AndreMiras, I'll close this. If there are specific issues with the pil recipe, feel free to open a new issue about them.

For what it's worth, I intend to look at fixing up the recipes after the next release.

[010130521]

I've been trying to fix this error for a pretty long time. I did what you said and now my app is running on Android. Thank you @AndreMiras You are my personal jesus christ :)

[Subdue0]

11-29 14:35:39.919 28341 28361 I python : Traceback (most recent call last):
11-29 14:35:39.920 28341 28361 I python : File "/home/kivydev/test/py2apk/main.py", line 6, in
11-29 14:35:39.920 28341 28361 I python : File "/home/kivydev/test/py2apk/autologin.py", line 27, in
11-29 14:35:39.920 28341 28361 I python : File "/home/kivydev/.local/share/python-for-android/dists/py2dist/private/lib/python2.7/site-packages/PIL/Image.py", line 64, in
11-29 14:35:39.920 28341 28361 I python : ImportError: dlopen failed: "/data/data/com.myapp.test/files/app/lib/python2.7/site-packages/PIL/_imaging.so" is 64-bit instead of 32-bit
11-29 14:35:39.944 28341 28361 I python : Python for android ended.

[Subdue0]

How to solve the question?I I haven't found a way to find it for a long time.

@Subdue0 you need to add the exact variant that has a p4a recipe to your --requirements as in the p4a option, or if you use buildozer, the buildozer.spec requirements entry. Adding it to your project's requirements.txt or similar is not sufficient. For a python 2 app, both pil and Pillow should work, for Python 3 use Pillow. Also, I'm not sure the recipes are in working order in p4a's stable, so you may need to use the git master development version. (someone else can possibly correct me on this) You can install the p4a dev version using pip install -U https://github.com/kivy/python-for-android/archive/master.zip

And try what @AndreMiras suggested above, which is:

1. Remove your distribution folder (usually named unnamed_dist_1 or similar in the p4a build subfolder, you should be able to find it on Linux using sudo updatedb && locate unnamed_dist) such that it is recreated from scratch next time
2. Use buildozer android p4a -- clean_recipe_build pil

Hope that helps. You can also drop by on the discord where we might be able to help you better

[Subdue0]

@Jonast I have tried all of the above, but none of them can be solved. My p4a 's 'recipes' contains Pillow and pil, no pillow, and if i use 'pillow' in the requirements, it can be successfully packaged, but the APK cannot be run ('_imaging.so" is 64-bit instead of 32 -bit'), the other will have errors when packaging.

if i use 'pillow' in the requirements

On recent stable this might give you no recipe. That will obviously build but not anything useful (as evidenced by the error at runtime).

the other will have errors when packaging

If you don't give us the detailed errors, nobody can help you 😢

[Subdue0]

[INFO]: Prebuilding sdl2 for armeabi-v7a
[INFO]: sdl2 has no prebuild_armeabi_v7a, skipping
[INFO]: Applying patches for sdl2[armeabi-v7a]
[INFO]: Applying patch add_nativeSetEnv.patch
[INFO]: -> running patch -t -d /home/kivydev/....(and 190 more)
[INFO]: -> running touch /home/kivydev/.local/...(and 84 more)
[INFO]: Prebuilding setuptools for armeabi-v7a
[INFO]: setuptools has no prebuild_armeabi_v7a, skipping
[INFO]: Prebuilding six for armeabi-v7a
[INFO]: six has no prebuild_armeabi_v7a, skipping
[INFO]: Prebuilding Pillow for armeabi-v7a
[INFO]: Pillow has no prebuild_armeabi_v7a, skipping
[INFO]: Applying patches for Pillow[armeabi-v7a]
[INFO]: Applying patch patches/fix-docstring.patch
[INFO]: -> running patch -t -d /home/kivydev/....(and 199 more)
[INFO]: Applying patch patches/fix-setup.patch
[INFO]: -> running patch -t -d /home/kivydev/....(and 195 more)
[INFO]: -> running touch /home/kivydev/.local/...(and 86 more)
[INFO]: Prebuilding pyjnius for armeabi-v7a
[INFO]: pyjnius has no prebuild_armeabi_v7a, skipping
[INFO]: Applying patches for pyjnius[armeabi-v7a]
[INFO]: Applying patch sdl2_jnienv_getter.patch
[INFO]: -> running patch -t -d /home/kivydev/....(and 204 more)
[INFO]: -> running touch /home/kivydev/.local/...(and 93 more)
[INFO]: Prebuilding requests for armeabi-v7a
[INFO]: requests has no prebuild_armeabi_v7a, skipping
[INFO]: Prebuilding kivy for armeabi-v7a
[INFO]: kivy has no prebuild_armeabi_v7a, skipping
[INFO]: # Building recipes
[INFO]: Building harfbuzz for armeabi-v7a
[INFO]: -> directory context /home/kivydev/.local/share/python-for-android/build/other_builds/harfbuzz/armeabi-v7a/harfbuzz
[INFO]: -> running configure --without-icu --h...(and 135 more)
[INFO]: -> running make -j5
[INFO]: -> running cp -L src/.libs/libharfbuzz...(and 80 more)
[INFO]: <- directory context /home/kivydev/test/py2apk
[INFO]: Building hostpython2 for armeabi-v7a
[INFO]: -> directory context /home/kivydev/.local/share/python-for-android/build/other_builds/hostpython2/desktop/hostpython2
[INFO]: -> running configure
[INFO]: -> running make -j5
[INFO]: -> running mv Parser/pgen hostpgen
[INFO]: -> running mv python hostpython
[INFO]: <- directory context /home/kivydev/test/py2apk
[INFO]: Building jpeg for armeabi-v7a
[INFO]: -> directory context /home/kivydev/.local/share/python-for-android/build/other_builds/jpeg/armeabi-v7a/jpeg
[INFO]: -> running ndk-build V=1 APP_ABI=armeabi-v7a
working: /home/kivydev/andr/android-ndk-r16b/bu...(and 73 more) Exception in thread background thread for pid 9708:
Traceback (most recent call last):
File "/usr/lib/python2.7/threading.py", line 801, in __bootstrap_inner
self.run()
File "/usr/lib/python2.7/threading.py", line 754, in run
self.__target(*self.__args, **self.__kwargs)
File "/home/kivydev/.local/lib/python2.7/site-packages/sh.py", line 1540, in wrap
fn(*args, **kwargs)
File "/home/kivydev/.local/lib/python2.7/site-packages/sh.py", line 2459, in background_thread
handle_exit_code(exit_code)
File "/home/kivydev/.local/lib/python2.7/site-packages/sh.py", line 2157, in fn
return self.command.handle_command_exit_code(exit_code)
File "/home/kivydev/.local/lib/python2.7/site-packages/sh.py", line 815, in handle_command_exit_code
raise exc
ErrorReturnCode_2:

RAN: /home/kivydev/andr/android-ndk-r16b/ndk-build V=1 APP_ABI=armeabi-v7a

STDOUT:
Android NDK: APP_PLATFORM not set. Defaulting to minimum supported version android-14.
Android NDK: APP_PLATFORM set to unknown platform: android-14.
/home/kivydev/andr/android-ndk-r16b/build/core/setup-app-platform.mk:135: *** Android NDK: Aborting . Stop.

STDERR:

Traceback (most recent call last):
File "/home/kivydev/.local/bin/p4a", line 11, in
sys.exit(main())
File "/home/kivydev/.local/lib/python2.7/site-packages/pythonforandroid/toolchain.py", line 973, in main
ToolchainCL()
File "/home/kivydev/.local/lib/python2.7/site-packages/pythonforandroid/toolchain.py", line 512, in init
getattr(self, args.subparser_name.replace('-', '_'))(args)
File "/home/kivydev/.local/lib/python2.7/site-packages/pythonforandroid/toolchain.py", line 147, in wrapper_func
build_dist_from_args(ctx, dist, args)
File "/home/kivydev/.local/lib/python2.7/site-packages/pythonforandroid/toolchain.py", line 191, in build_dist_from_args
build_recipes(build_order, python_modules, ctx)
File "/home/kivydev/.local/lib/python2.7/site-packages/pythonforandroid/build.py", line 573, in build_recipes
recipe.build_arch(arch)
File "/home/kivydev/.local/lib/python2.7/site-packages/pythonforandroid/recipes/jpeg/init.py", line 29, in build_arch
super(JpegRecipe, self).build_arch(arch)
File "/home/kivydev/.local/lib/python2.7/site-packages/pythonforandroid/recipe.py", line 685, in build_arch
shprint(sh.ndk_build, 'V=1', 'APP_ABI=' + arch.arch, *extra_args, _env=env)
File "/home/kivydev/.local/lib/python2.7/site-packages/pythonforandroid/logger.py", line 176, in shprint
for line in output:
File "/home/kivydev/.local/lib/python2.7/site-packages/sh.py", line 863, in next
self.wait()
File "/home/kivydev/.local/lib/python2.7/site-packages/sh.py", line 792, in wait
self.handle_command_exit_code(exit_code)
File "/home/kivydev/.local/lib/python2.7/site-packages/sh.py", line 815, in handle_command_exit_code
raise exc
sh.ErrorReturnCode_2:

RAN: /home/kivydev/andr/android-ndk-r16b/ndk-build V=1 APP_ABI=armeabi-v7a

STDOUT:
Android NDK: APP_PLATFORM not set. Defaulting to minimum supported version android-14.
Android NDK: APP_PLATFORM set to unknown platform: android-14.
/home/kivydev/andr/android-ndk-r16b/build/core/setup-app-platform.mk:135: *** Android NDK: Aborting . Stop.

STDERR:

[Subdue0]

--dist_name py2dist
--android_api 19
--minsdk 19
--sdk_dir /home/kivydev/andr/android-sdk-linux
--ndk_dir /home/kivydev/andr/android-ndk-r16b
--ndk_version 16

--arch armeabi-v7a
--requirements python2,kivy,requests,Pillow
--private .
--package com.myapp.test
--name pytest
--version 1.0
--bootstrap sdl2

It appears as if p4a doesn't pass the correct the NDK version along to the NDK makefile to me. I don't think this is related to pil/Pillow in any way, it looks like a generic p4a bug. Can you file a new ticket with this backtrace so we can see what needs fixing?

(My spontaneous guess is that the ndk-build call quoted in RAN would need to add the APP_PLATFORM option, but I'm sure other devs also have some ideas)

Also what p4a version is this? Could you possibly try git master if this is the pypi or some other stable/older version to see if that changes anything?

Just to clarify: with new ticket I mean create one here: https://github.com/kivy/python-for-android/issues/new you could title it ndk-build error during build or something