Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

ctypes.util.find_library() doesn't work on arch arm64-v8a #1770

Closed
tcdude opened this issue Mar 24, 2019 · 7 comments · Fixed by #1772
Closed

ctypes.util.find_library() doesn't work on arch arm64-v8a #1770

tcdude opened this issue Mar 24, 2019 · 7 comments · Fixed by #1772
Labels
bug WIP

Comments

@tcdude
Copy link
Contributor

tcdude commented Mar 24, 2019

Versions

  • Python: 3.6.7
  • OS: Ubuntu 18.10
  • Kivy: N/A
  • Cython: 0.29.6

Description

Build for arch arm64-v8a succeeds but raises a FileNotFoundError during execution (see logcat below)

According to @inclement, this might be related to #1701

buildozer.spec

Not using buildozer.

setup.py

from distutils.core import setup
from setuptools import find_packages


options = {'apk': {'debug': None,
                   'bootstrap': 'sdl2',
                   'requirements': 'libffi,sdl2,sdl2_image,python3,pysdl2,Pillow',
                   'package': 'org.some.put',
                   'android-api': 28,
                   'arch': 'arm64-v8a',
                   'dist-name': 'bdisttest',
                   }}

setup(
    name='foo',
    version='0.1',
    description='bar',
    author='me',
    author_email='foo@bar.com',
    packages=find_packages(),
    options=options,
    package_data={'foobar': ['*.py', 'gui/*.py', 'assets/images/*.png']}
)

Logs

adb logcat | grep python

03-24 20:29:10.692  2554  2554 V pythonutil: Checking pattern libcrystax\.so against libpython3.7m.so
03-24 20:29:10.692  2554  2554 V pythonutil: Checking pattern libcrystax\.so against libssl1.1.so
03-24 20:29:10.692  2554  2554 V pythonutil: Checking pattern libcrystax\.so against libSDL2_mixer.so
03-24 20:29:10.692  2554  2554 V pythonutil: Checking pattern libcrystax\.so against libSDL2_ttf.so
03-24 20:29:10.692  2554  2554 V pythonutil: Checking pattern libcrystax\.so against libffi.so
03-24 20:29:10.692  2554  2554 V pythonutil: Checking pattern libcrystax\.so against libmain.so
03-24 20:29:10.692  2554  2554 V pythonutil: Checking pattern libcrystax\.so against libSDL2.so
03-24 20:29:10.692  2554  2554 V pythonutil: Checking pattern libcrystax\.so against libhidapi.so
03-24 20:29:10.693  2554  2554 V pythonutil: Checking pattern libcrystax\.so against libsqlite3.so
03-24 20:29:10.693  2554  2554 V pythonutil: Checking pattern libcrystax\.so against libcrypto1.1.so
03-24 20:29:10.693  2554  2554 V pythonutil: Checking pattern libcrystax\.so against libSDL2_image.so
03-24 20:29:10.693  2554  2554 V pythonutil: Checking pattern libsqlite3\.so against libpython3.7m.so
03-24 20:29:10.693  2554  2554 V pythonutil: Checking pattern libsqlite3\.so against libssl1.1.so
03-24 20:29:10.694  2554  2554 V pythonutil: Checking pattern libsqlite3\.so against libSDL2_mixer.so
03-24 20:29:10.694  2554  2554 V pythonutil: Checking pattern libsqlite3\.so against libSDL2_ttf.so
03-24 20:29:10.694  2554  2554 V pythonutil: Checking pattern libsqlite3\.so against libffi.so
03-24 20:29:10.694  2554  2554 V pythonutil: Checking pattern libsqlite3\.so against libmain.so
03-24 20:29:10.694  2554  2554 V pythonutil: Checking pattern libsqlite3\.so against libSDL2.so
03-24 20:29:10.694  2554  2554 V pythonutil: Checking pattern libsqlite3\.so against libhidapi.so
03-24 20:29:10.694  2554  2554 V pythonutil: Checking pattern libsqlite3\.so against libsqlite3.so
03-24 20:29:10.694  2554  2554 V pythonutil: Pattern libsqlite3\.so matched file libsqlite3.so
03-24 20:29:10.694  2554  2554 V pythonutil: Checking pattern libsqlite3\.so against libcrypto1.1.so
03-24 20:29:10.695  2554  2554 V pythonutil: Checking pattern libsqlite3\.so against libSDL2_image.so
03-24 20:29:10.695  2554  2554 V pythonutil: Checking pattern libffi\.so against libpython3.7m.so
03-24 20:29:10.695  2554  2554 V pythonutil: Checking pattern libffi\.so against libssl1.1.so
03-24 20:29:10.696  2554  2554 V pythonutil: Checking pattern libffi\.so against libSDL2_mixer.so
03-24 20:29:10.696  2554  2554 V pythonutil: Checking pattern libffi\.so against libSDL2_ttf.so
03-24 20:29:10.696  2554  2554 V pythonutil: Checking pattern libffi\.so against libffi.so
03-24 20:29:10.696  2554  2554 V pythonutil: Pattern libffi\.so matched file libffi.so
03-24 20:29:10.696  2554  2554 V pythonutil: Checking pattern libffi\.so against libmain.so
03-24 20:29:10.696  2554  2554 V pythonutil: Checking pattern libffi\.so against libSDL2.so
03-24 20:29:10.696  2554  2554 V pythonutil: Checking pattern libffi\.so against libhidapi.so
03-24 20:29:10.696  2554  2554 V pythonutil: Checking pattern libffi\.so against libsqlite3.so
03-24 20:29:10.696  2554  2554 V pythonutil: Checking pattern libffi\.so against libcrypto1.1.so
03-24 20:29:10.696  2554  2554 V pythonutil: Checking pattern libffi\.so against libSDL2_image.so
03-24 20:29:10.697  2554  2554 V pythonutil: Checking pattern libssl.*\.so against libpython3.7m.so
03-24 20:29:10.697  2554  2554 V pythonutil: Checking pattern libssl.*\.so against libssl1.1.so
03-24 20:29:10.697  2554  2554 V pythonutil: Pattern libssl.*\.so matched file libssl1.1.so
03-24 20:29:10.697  2554  2554 V pythonutil: Checking pattern libssl.*\.so against libSDL2_mixer.so
03-24 20:29:10.697  2554  2554 V pythonutil: Checking pattern libssl.*\.so against libSDL2_ttf.so
03-24 20:29:10.697  2554  2554 V pythonutil: Checking pattern libssl.*\.so against libffi.so
03-24 20:29:10.698  2554  2554 V pythonutil: Checking pattern libssl.*\.so against libmain.so
03-24 20:29:10.698  2554  2554 V pythonutil: Checking pattern libssl.*\.so against libSDL2.so
03-24 20:29:10.698  2554  2554 V pythonutil: Checking pattern libssl.*\.so against libhidapi.so
03-24 20:29:10.698  2554  2554 V pythonutil: Checking pattern libssl.*\.so against libsqlite3.so
03-24 20:29:10.698  2554  2554 V pythonutil: Checking pattern libssl.*\.so against libcrypto1.1.so
03-24 20:29:10.698  2554  2554 V pythonutil: Checking pattern libssl.*\.so against libSDL2_image.so
03-24 20:29:10.698  2554  2554 V pythonutil: Checking pattern libcrypto.*\.so against libpython3.7m.so
03-24 20:29:10.699  2554  2554 V pythonutil: Checking pattern libcrypto.*\.so against libssl1.1.so
03-24 20:29:10.699  2554  2554 V pythonutil: Checking pattern libcrypto.*\.so against libSDL2_mixer.so
03-24 20:29:10.699  2554  2554 V pythonutil: Checking pattern libcrypto.*\.so against libSDL2_ttf.so
03-24 20:29:10.699  2554  2554 V pythonutil: Checking pattern libcrypto.*\.so against libffi.so
03-24 20:29:10.699  2554  2554 V pythonutil: Checking pattern libcrypto.*\.so against libmain.so
03-24 20:29:10.699  2554  2554 V pythonutil: Checking pattern libcrypto.*\.so against libSDL2.so
03-24 20:29:10.699  2554  2554 V pythonutil: Checking pattern libcrypto.*\.so against libhidapi.so
03-24 20:29:10.699  2554  2554 V pythonutil: Checking pattern libcrypto.*\.so against libsqlite3.so
03-24 20:29:10.700  2554  2554 V pythonutil: Checking pattern libcrypto.*\.so against libcrypto1.1.so
03-24 20:29:10.700  2554  2554 V pythonutil: Pattern libcrypto.*\.so matched file libcrypto1.1.so
03-24 20:29:10.700  2554  2554 V pythonutil: Checking pattern libcrypto.*\.so against libSDL2_image.so
03-24 20:29:10.700  2554  2554 V pythonutil: Loading library: sqlite3
03-24 20:29:10.702  2554  2554 V pythonutil: Loading library: ffi
03-24 20:29:10.703  2554  2554 V pythonutil: Loading library: SDL2
03-24 20:29:10.706  2554  2554 V pythonutil: Loading library: SDL2_image
03-24 20:29:10.708  2554  2554 V pythonutil: Loading library: SDL2_mixer
03-24 20:29:10.710  2554  2554 V pythonutil: Loading library: SDL2_ttf
03-24 20:29:10.712  2554  2554 V pythonutil: Loading library: ssl1.1
03-24 20:29:10.714  2554  2554 V pythonutil: Loading library: crypto1.1
03-24 20:29:10.715  2554  2554 V pythonutil: Loading library: python2.7
03-24 20:29:10.716  2554  2554 V pythonutil: Library loading error: dalvik.system.PathClassLoader[DexPathList[[zip file "/data/app/com.tizilogic.pyos-UUr8o1SWqdGHXQVCTO8uLQ==/base.apk"],nativeLibraryDirectories=[/data/app/com.tizilogic.pyos-UUr8o1SWqdGHXQVCTO8uLQ==/lib/arm64, /data/app/com.tizilogic.pyos-UUr8o1SWqdGHXQVCTO8uLQ==/base.apk!/lib/arm64-v8a, /system/lib64, /system/vendor/lib64]]] couldn't find "libpython2.7.so"
03-24 20:29:10.717  2554  2554 V pythonutil: Loading library: python3.5m
03-24 20:29:10.717  2554  2554 V pythonutil: Library loading error: dalvik.system.PathClassLoader[DexPathList[[zip file "/data/app/com.tizilogic.pyos-UUr8o1SWqdGHXQVCTO8uLQ==/base.apk"],nativeLibraryDirectories=[/data/app/com.tizilogic.pyos-UUr8o1SWqdGHXQVCTO8uLQ==/lib/arm64, /data/app/com.tizilogic.pyos-UUr8o1SWqdGHXQVCTO8uLQ==/base.apk!/lib/arm64-v8a, /system/lib64, /system/vendor/lib64]]] couldn't find "libpython3.5m.so"
03-24 20:29:10.717  2554  2554 V pythonutil: Loading library: python3.6m
03-24 20:29:10.718  2554  2554 V pythonutil: Library loading error: dalvik.system.PathClassLoader[DexPathList[[zip file "/data/app/com.tizilogic.pyos-UUr8o1SWqdGHXQVCTO8uLQ==/base.apk"],nativeLibraryDirectories=[/data/app/com.tizilogic.pyos-UUr8o1SWqdGHXQVCTO8uLQ==/lib/arm64, /data/app/com.tizilogic.pyos-UUr8o1SWqdGHXQVCTO8uLQ==/base.apk!/lib/arm64-v8a, /system/lib64, /system/vendor/lib64]]] couldn't find "libpython3.6m.so"
03-24 20:29:10.718  2554  2554 V pythonutil: Loading library: python3.7m
03-24 20:29:10.721  2554  2554 V pythonutil: Loading library: main
03-24 20:29:10.722  2554  2554 V pythonutil: Failed to load _io.so or unicodedata.so...but that's okay.
03-24 20:29:10.723  2554  2554 V pythonutil: Unsatisfied linker when loading ctypes
03-24 20:29:10.723  2554  2554 V pythonutil: Loaded everything!
03-24 20:29:10.810  2554  2588 I python  : Initializing Python for Android
03-24 20:29:10.810  2554  2588 I python  : Setting additional env vars from p4a_env_vars.txt
03-24 20:29:10.810  2554  2588 I python  : Changing directory to the one provided by ANDROID_ARGUMENT
03-24 20:29:10.810  2554  2588 I python  : /data/user/0/com.tizilogic.pyos/files/app
03-24 20:29:10.811  2554  2588 I python  : Preparing to initialize python
03-24 20:29:10.811  2554  2588 I python  : _python_bundle dir exists
03-24 20:29:10.811  2554  2588 I python  : calculated paths to be...
03-24 20:29:10.811  2554  2588 I python  : /data/user/0/com.tizilogic.pyos/files/app/_python_bundle/stdlib.zip:/data/user/0/com.tizilogic.pyos/files/app/_python_bundle/modules
03-24 20:29:10.811  2554  2588 I python  : set wchar paths...
03-24 20:29:10.849  2554  2588 I python  : Initialized python
03-24 20:29:10.849  2554  2588 I python  : AND: Init threads
03-24 20:29:10.849  2554  2588 I python  : testing python print redirection
03-24 20:29:10.850  2554  2588 I python  : Android path ['.', '/data/user/0/com.tizilogic.pyos/files/app/_python_bundle/stdlib.zip', '/data/user/0/com.tizilogic.pyos/files/app/_python_bundle/modules', '/data/user/0/com.tizilogic.pyos/files/app/_python_bundle/site-packages']
03-24 20:29:10.851  2554  2588 I python  : os.environ is environ({'PATH': '/sbin:/system/sbin:/system/bin:/system/xbin:/vendor/bin:/vendor/xbin', 'DOWNLOAD_CACHE': '/data/cache', 'ANDROID_BOOTLOGO': '1', 'ANDROID_ROOT': '/system', 'ANDROID_ASSETS': '/system/app', 'ANDROID_DATA': '/data', 'ANDROID_STORAGE': '/storage', 'EXTERNAL_STORAGE': '/sdcard', 'ASEC_MOUNTPOINT': '/mnt/asec', 'BOOTCLASSPATH': '/system/framework/core-oj.jar:/system/framework/core-libart.jar:/system/framework/conscrypt.jar:/system/framework/okhttp.jar:/system/framework/legacy-test.jar:/system/framework/bouncycastle.jar:/system/framework/ext.jar:/system/framework/framework.jar:/system/framework/telephony-common.jar:/system/framework/voip-common.jar:/system/framework/ims-common.jar:/system/framework/apache-xml.jar:/system/framework/org.apache.http.legacy.boot.jar:/system/framework/smartbondingservice.jar:/system/framework/sprengine.jar:/system/framework/android.hidl.base-V1.0-java.jar:/system/framework/android.hidl.manager-V1.0-java.jar:/system/framework/timakeystore.jar:/system/framework/fipstimakeystore.jar:/system/framework/ucmopensslenginehelper.jar:/system/framework/esecomm.jar:/system/framework/SemAudioThumbnail.jar:/system/framework/knoxsdk.jar:/system/framework/sec_edm.jar:/system/framework/sagearpolicymanager.jar:/system/framework/sec_sdp_sdk.jar:/system/framework/sec_sdp_hidden_sdk.jar:/system/framework/knoxvpnuidtag.jar', 'SYSTEMSERVERCLASSPATH': '/system/framework/services.jar:/system/framework/ethernet-service.jar:/system/framework/wifi-service.jar:/system/framework/samsung-services.jar:/system/framework/hqm.jar:/system/framework/dmf.jar:/system/framework/ssrm.jar', 'MC_AUTH_TOKEN_PATH': '/efs', 'KNOX_STORAGE': '/data/knox/ext_sdcard', 'ENC_EMULATED_STORAGE_TARGET': '/storage/enc_emulated', 'ANDROID_SOCKET_zygote': '8', 'ANDROID_ENTRYPOINT': 'main.pyc', 'ANDROID_ARGUMENT': '/data/user/0/com.tizilogic.pyos/files/app', 'ANDROID_APP_PATH': '/data/user/0/com.tizilogic.pyos/files/app', 'ANDROID_PRIVATE': '/data/user/0/com.tizilogic.pyos/files', 'ANDROID_UNPACK': '/data/user/0/com.tizilogic.pyos/files/app', 'PYTHONHOME': '/data/user/0/com.tizilogic.pyos/files/app', 'PYTHONPATH': '/data/user/0/com.tizilogic.pyos/files/app:/data/user/0/com.tizilogic.pyos/files/app/lib', 'PYTHONOPTIMIZE': '2', 'P4A_BOOTSTRAP': 'SDL2', 'PYTHON_NAME': 'python', 'P4A_IS_WINDOWED': 'False', 'P4A_ORIENTATION': 'portrait', 'P4A_NUMERIC_VERSION': 'None', 'P4A_MINSDK': '21', 'LC_CTYPE': 'C.UTF-8'})
03-24 20:29:10.851  2554  2588 I python  : Android kivy bootstrap done. __name__ is __main__
03-24 20:29:10.851  2554  2588 I python  : AND: Ran string
03-24 20:29:10.851  2554  2588 I python  : Run user program, change dir and execute entrypoint
03-24 20:29:10.985  2554  2588 I python  : Traceback (most recent call last):
03-24 20:29:10.985  2554  2588 I python  :   File "/home/userhome/workspace/build/bdist.android-arm64-v8a/pyos/main.py", line 6, in <module>
03-24 20:29:10.985  2554  2588 I python  :   File "/home/userhome/.local/share/python-for-android/build/python-installs/bdisttest/sdl2/__init__.py", line 2, in <module>
03-24 20:29:10.986  2554  2588 I python  :   File "/home/userhome/.local/share/python-for-android/build/python-installs/bdisttest/sdl2/dll.py", line 111, in <module>
03-24 20:29:10.986  2554  2588 I python  :   File "/home/userhome/.local/share/python-for-android/build/python-installs/bdisttest/sdl2/dll.py", line 49, in __init__
03-24 20:29:10.986  2554  2588 I python  :   File "/home/userhome/.local/share/python-for-android/build/python-installs/bdisttest/sdl2/dll.py", line 37, in _findlib
03-24 20:29:10.986  2554  2588 I python  :   File "/home/userhome/.local/share/python-for-android/build/other_builds/python3-libffi-openssl-sqlite3/arm64-v8a__ndk_target_21/python3/Lib/ctypes/util.py", line 78, in find_library
03-24 20:29:10.986  2554  2588 I python  : FileNotFoundError: [Errno 2] No such file or directory: '/data/data/com.tizilogic.pyos/lib'
03-24 20:29:10.986  2554  2588 I python  : Python for android ended.
@AndreMiras
Copy link
Member

Could you share your pip freeze please? The idea is to know at least the python-for-android version you're running

@tcdude
Copy link
Contributor Author

tcdude commented Mar 24, 2019

happy to:

appdirs==1.4.3  
asn1crypto==0.24.0
buildozer==0.39
colorama==0.4.1
cryptography==2.1.4
Cython==0.29.6
idna==2.6
Jinja2==2.10
keyring==10.6.0
keyrings.alt==3.0
MarkupSafe==1.1.1
pexpect==4.6.0
ptyprocess==0.6.0
pycrypto==2.6.1
pygobject==3.26.1
python-for-android==0.7.1
pyxdg==0.25
SecretStorage==2.3.1
sh==1.12.14
six==1.11.0
virtualenv==15.1.0

@ghost ghost self-assigned this Mar 25, 2019
@ghost
Copy link

ghost commented Mar 25, 2019

@tcdude can you send the .apk to jonas@thiem.email? (I promise to keep it confidential/only use it to examine this issue) If I can install it on my local device and see where things end up then I should hopefully be able to figure out what the problem is pretty quick

@tcdude
Copy link
Contributor Author

tcdude commented Mar 25, 2019

@Jonast I sent a link to the indicated address. Hope it helps, let me know if I can provide more detail/information.

@ghost
Copy link

ghost commented Mar 25, 2019

@tcdude thanks, got it, and I can see what the issue is. we are affected by this: https://stackoverflow.com/questions/41709129/lib-directory-missing-after-installing-64-bit-android-application

I'm preparing a fix

@ghost ghost mentioned this issue Mar 25, 2019
Merged
@ghost ghost added the bug label Mar 25, 2019
@ghost
Copy link

ghost commented Mar 28, 2019

@tcdude ok I got a fix now that appears to work. Can you try whether it solves the problem for you? To do so, install python-for-android from the pull request e.g. like this:

pip install -U https://github.com/JonasT/python-for-android/archive/ctypes_patch_fix.zip

You may also need to use p4a clean_all before rebuilding to make sure your python is rebuilt from scratch (or alternatively, wipe your distribution folder)

@tcdude
Copy link
Contributor Author

tcdude commented Mar 28, 2019

@Jonast I can confirm that this fixes the issue at hand.
Thank you for providing a fix so fast 👍

@ghost ghost added the WIP label Apr 4, 2019
AndreMiras added a commit that referenced this issue Apr 6, 2019
@AndreMiras
Merge pull request #1772 from JonasT/ctypes_patch_fix
6b12281 
Properly search native lib dir in ctypes, fixes #1770
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug WIP
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Properly search native lib dir in ctypes, fixes #1770
2 participants
@AndreMiras @tcdude