Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Crash using pylsl #862

Open
KyucraftV2 opened this issue Apr 25, 2023 · 38 comments · May be fixed by #872
Open

Crash using pylsl #862

KyucraftV2 opened this issue Apr 25, 2023 · 38 comments · May be fixed by #872
Labels

Comments

@KyucraftV2
Copy link

KyucraftV2 commented Apr 25, 2023

Chaquopy version

Version 7.0

Relevant parts of your code

# test LSL
outlet = StreamInfo(name='myStream', type='EEG', channel_count=8)
print(outlet)

Describe your issue

I use beeware, when i build the app i have no problem but when i start the app, the app crashes.

The error

--------- beginning of crash
04-25 13:53:26.294 15061 15061 E AndroidRuntime: FATAL EXCEPTION: main
04-25 13:53:26.294 15061 15061 E AndroidRuntime: Process: com.example.appvisulsl, PID: 15061
04-25 13:53:26.294 15061 15061 E AndroidRuntime: java.lang.RuntimeException: Unable to start activity ComponentInfo{com.example.appvisulsl/org.beeware.android.MainActivity}: com.chaquo.python.PyException: RuntimeError: LSL binary library file was not found. Please make sure that the binary file can be found in the package lib folder
04-25 13:53:26.294 15061 15061 E AndroidRuntime:  (/data/data/com.example.appvisulsl/files/chaquopy/AssetFinder/requirements/pylsl/lib)
04-25 13:53:26.294 15061 15061 E AndroidRuntime:  or the system search path. Alternatively, specify the PYLSL_LIB environment variable.
04-25 13:53:26.294 15061 15061 E AndroidRuntime:  You can install the LSL library with conda: `conda install -c conda-forge liblsl`
04-25 13:53:26.294 15061 15061 E AndroidRuntime: or otherwise download it from the liblsl releases page assets: https://github.com/sccn/liblsl/releases
04-25 13:53:26.294 15061 15061 E AndroidRuntime:        at android.app.ActivityThread.performLaunchActivity(ActivityThread.java:3645)
04-25 13:53:26.294 15061 15061 E AndroidRuntime:        at android.app.ActivityThread.handleLaunchActivity(ActivityThread.java:3782)
04-25 13:53:26.294 15061 15061 E AndroidRuntime:        at android.app.servertransaction.LaunchActivityItem.execute(LaunchActivityItem.java:101)
04-25 13:53:26.294 15061 15061 E AndroidRuntime:        at android.app.servertransaction.TransactionExecutor.executeCallbacks(TransactionExecutor.java:135)
04-25 13:53:26.294 15061 15061 E AndroidRuntime:        at android.app.servertransaction.TransactionExecutor.execute(TransactionExecutor.java:95)
04-25 13:53:26.294 15061 15061 E AndroidRuntime:        at android.app.ActivityThread$H.handleMessage(ActivityThread.java:2307)
04-25 13:53:26.294 15061 15061 E AndroidRuntime:        at android.os.Handler.dispatchMessage(Handler.java:106)
04-25 13:53:26.294 15061 15061 E AndroidRuntime:        at android.os.Looper.loopOnce(Looper.java:201)
04-25 13:53:26.294 15061 15061 E AndroidRuntime:        at android.os.Looper.loop(Looper.java:288)
04-25 13:53:26.294 15061 15061 E AndroidRuntime:        at android.app.ActivityThread.main(ActivityThread.java:7872)
04-25 13:53:26.294 15061 15061 E AndroidRuntime:        at java.lang.reflect.Method.invoke(Native Method)
04-25 13:53:26.294 15061 15061 E AndroidRuntime:        at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:548)
04-25 13:53:26.294 15061 15061 E AndroidRuntime:        at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:936)
04-25 13:53:26.294 15061 15061 E AndroidRuntime: Caused by: com.chaquo.python.PyException: RuntimeError: LSL binary library file was not found. Please make sure that the binary file can be found in the package lib folder
04-25 13:53:26.294 15061 15061 E AndroidRuntime:  (/data/data/com.example.appvisulsl/files/chaquopy/AssetFinder/requirements/pylsl/lib)
04-25 13:53:26.294 15061 15061 E AndroidRuntime:  or the system search path. Alternatively, specify the PYLSL_LIB environment variable.
04-25 13:53:26.294 15061 15061 E AndroidRuntime:  You can install the LSL library with conda: `conda install -c conda-forge liblsl`
04-25 13:53:26.294 15061 15061 E AndroidRuntime: or otherwise download it from the liblsl releases page assets: https://github.com/sccn/liblsl/releases
04-25 13:53:26.294 15061 15061 E AndroidRuntime:        at <python>.pylsl.pylsl.<module>(pylsl.py:1296)
04-25 13:53:26.294 15061 15061 E AndroidRuntime:        at <python>.java.chaquopy.import_override(import.pxi:26)
04-25 13:53:26.294 15061 15061 E AndroidRuntime:        at <python>.pylsl.<module>(__init__.py:2)
04-25 13:53:26.294 15061 15061 E AndroidRuntime:        at <python>.java.chaquopy.import_override(import.pxi:26)
04-25 13:53:26.294 15061 15061 E AndroidRuntime:        at <python>.appvisulsl.app.<module>(app.py:12)
04-25 13:53:26.294 15061 15061 E AndroidRuntime:        at <python>.java.chaquopy.import_override(import.pxi:26)
04-25 13:53:26.294 15061 15061 E AndroidRuntime:        at <python>.__main__.<module>(__main__.py:1)
04-25 13:53:26.294 15061 15061 E AndroidRuntime:        at <python>.runpy._run_code(runpy.py:87)
04-25 13:53:26.294 15061 15061 E AndroidRuntime:        at <python>.runpy._run_module_code(runpy.py:97)
04-25 13:53:26.294 15061 15061 E AndroidRuntime:        at <python>.runpy.run_module(runpy.py:207)
04-25 13:53:26.294 15061 15061 E AndroidRuntime:        at <python>.chaquopy_java.call(chaquopy_java.pyx:354)
04-25 13:53:26.294 15061 15061 E AndroidRuntime:        at <python>.chaquopy_java.Java_com_chaquo_python_PyObject_callAttrThrowsNative(chaquopy_java.pyx:326)
04-25 13:53:26.294 15061 15061 E AndroidRuntime:        at com.chaquo.python.PyObject.callAttrThrowsNative(Native Method)
04-25 13:53:26.294 15061 15061 E AndroidRuntime:        at com.chaquo.python.PyObject.callAttrThrows(PyObject.java:232)
04-25 13:53:26.294 15061 15061 E AndroidRuntime:        at com.chaquo.python.PyObject.callAttr(PyObject.java:221)
04-25 13:53:26.294 15061 15061 E AndroidRuntime:        at org.beeware.android.MainActivity.onCreate(MainActivity.java:85)
04-25 13:53:26.294 15061 15061 E AndroidRuntime:        at android.app.Activity.performCreate(Activity.java:8305)
04-25 13:53:26.294 15061 15061 E AndroidRuntime:        at android.app.Activity.performCreate(Activity.java:8284)
04-25 13:53:26.294 15061 15061 E AndroidRuntime:        at android.app.Instrumentation.callActivityOnCreate(Instrumentation.java:1417)
04-25 13:53:26.294 15061 15061 E AndroidRuntime:        at android.app.ActivityThread.performLaunchActivity(ActivityThread.java:3626)
04-25 13:53:26.294 15061 15061 E AndroidRuntime:        ... 12 more
04-25 13:53:26.368 15061 15061 I Process : Sending signal. PID: 15061 SIG: 9
@mhsmith
Copy link
Member

mhsmith commented Apr 25, 2023

This package contains native components, so it would have to be built into a wheel file. If you'd like to try doing this yourself, follow the instructions here. And if you're successful, please make a pull request so we can add the package to the public repository.

In this case, it looks like we'd need two wheels: one for pylsl and one for LSL itself. The relationship between these wheels would be similar to that between pyzmq and chaquopy-libzmq, as described in the instructions linked above.

If anyone else wants this package too, let us know by clicking the thumbs-up button above.

@KyucraftV2
Copy link
Author

KyucraftV2 commented Apr 26, 2023

I try to add pylsl, but when i run the command .\build-wheel.py --python 3.8 --abi arm64-v8a pylsl i have this output :

[main 2023-04-26T07:49:17.131Z] [SharedProcess] using utility process
[main 2023-04-26T07:49:17.330Z] update#setState idle
[main 2023-04-26T07:49:18.494Z] [UtilityProcess type: shared-process, pid: <none>]: creating new...
[main 2023-04-26T07:49:18.499Z] [UtilityProcess id: 1, type: fileWatcher, pid: <none>]: creating new...
[main 2023-04-26T07:49:18.506Z] [UtilityProcess id: 1, type: extensionHost, pid: <none>]: creating new...
[main 2023-04-26T07:49:18.514Z] [UtilityProcess type: shared-process, pid: 17080]: successfully created
[main 2023-04-26T07:49:18.527Z] [UtilityProcess id: 1, type: fileWatcher, pid: 1848]: successfully created
[main 2023-04-26T07:49:18.544Z] [UtilityProcess id: 1, type: extensionHost, pid: 10696]: successfully created
[main 2023-04-26T07:49:47.337Z] update#setState checking for updates
[main 2023-04-26T07:49:47.469Z] update#setState idle
[10020:0426/095119.438:ERROR:gpu_init.cc(481)] Passthrough is not supported, GL is disabled, ANGLE is 

And after nothing, so i dont know how to build and add package

@mhsmith
Copy link
Member

mhsmith commented Apr 27, 2023

Those appear to be VSCode messages (microsoft/vscode#179253). They have nothing to do with build-wheel, and I'm not even sure if they're errors.

Notes:

  • build-wheel must be run on Linux, as the instructions say.
  • If pylsl depends on LSL, then you'll have to build LSL first. See pyzmq and chaquopy-libzmq for an example of how to make a Python package depend on a non-Python package.

@KyucraftV2
Copy link
Author

KyucraftV2 commented Apr 27, 2023

Thanks for your help, I was able to make progress with this.
But now, after download correctly the SDK and Conda im running this command : ./build-wheel.py --python 3.8 --abi x86_64 pylsl
But i have this error i don't understand : build-wheel.py: Error: Found 0 x86_64 ZIPs in /home/killian/chaquopy/maven/com/chaquo/python/target/3.8

So I have in /home/killian/chaquopy/android-sdk i have my SDK, maybe it's in wrong folder

And for pylsl and LSL, i think pylsl has not dependecies on LSL, the module has all. But I'm not sure, did you know how can I be sure ?

@vmitro
Copy link

vmitro commented Apr 27, 2023

This sounds like you didn't go through the necessary steps in https://github.com/chaquo/chaquopy/blob/master/server/pypi/README.md. You need to download the appropriate targets and place them as mentioned in the Readme.

@KyucraftV2
Copy link
Author

KyucraftV2 commented Apr 27, 2023

Oh yes my folder is named 3.16.8... and its not shocked me...
But now i have a different error and in dont understand this error :

build-wheel.py: cd /home/killian/chaquopy/server/pypi/packages/pylsl/build/1.10.4
build-wheel.py: rm -rf /home/killian/chaquopy/server/pypi/packages/pylsl/build/1.10.4/cp38-cp38-android_21_x86_64
build-wheel.py: mkdir -p /home/killian/chaquopy/server/pypi/packages/pylsl/build/1.10.4/cp38-cp38-android_21_x86_64
build-wheel.py: Using cached sdist
build-wheel.py: unzip -d /tmp/build-wheel-0lc9c7yp -q pylsl-1.10.4.zip
build-wheel.py: mv /tmp/build-wheel-0lc9c7yp/pylsl-1.10.4 /home/killian/chaquopy/server/pypi/packages/pylsl/build/1.10.4/cp38-cp38-android_21_x86_64/src
build-wheel.py: rm -rf /tmp/build-wheel-0lc9c7yp
build-wheel.py: mkdir -p /home/killian/chaquopy/server/pypi/packages/pylsl/build/1.10.4/cp38-cp38-android_21_x86_64/requirements
build-wheel.py: mkdir -p /home/killian/chaquopy/server/pypi/packages/pylsl/build/1.10.4/cp38-cp38-android_21_x86_64/requirements/chaquopy/include
build-wheel.py: mkdir -p /home/killian/chaquopy/server/pypi/packages/pylsl/build/1.10.4/cp38-cp38-android_21_x86_64/requirements/chaquopy/lib
build-wheel.py: unzip -q -d /home/killian/chaquopy/server/pypi/packages/pylsl/build/1.10.4/cp38-cp38-android_21_x86_64/requirements/chaquopy /home/killian/chaquopy/maven/com/chaquo/python/target/3.8.16-0/target-3.8.16-0-x86_64.zip include/* jniLibs/*
build-wheel.py: mv /home/killian/chaquopy/server/pypi/packages/pylsl/build/1.10.4/cp38-cp38-android_21_x86_64/requirements/chaquopy/jniLibs/x86_64/* /home/killian/chaquopy/server/pypi/packages/pylsl/build/1.10.4/cp38-cp38-android_21_x86_64/requirements/chaquopy/lib
build-wheel.py: rm -r /home/killian/chaquopy/server/pypi/packages/pylsl/build/1.10.4/cp38-cp38-android_21_x86_64/requirements/chaquopy/jniLibs
build-wheel.py: ln -s libcrypto_chaquopy.so /home/killian/chaquopy/server/pypi/packages/pylsl/build/1.10.4/cp38-cp38-android_21_x86_64/requirements/chaquopy/lib/libcrypto.so
build-wheel.py: ln -s libssl_chaquopy.so /home/killian/chaquopy/server/pypi/packages/pylsl/build/1.10.4/cp38-cp38-android_21_x86_64/requirements/chaquopy/lib/libssl.so
build-wheel.py: ln -s libsqlite3_chaquopy.so /home/killian/chaquopy/server/pypi/packages/pylsl/build/1.10.4/cp38-cp38-android_21_x86_64/requirements/chaquopy/lib/libsqlite3.so
build-wheel.py: ln -s libpython3.8.so /home/killian/chaquopy/server/pypi/packages/pylsl/build/1.10.4/cp38-cp38-android_21_x86_64/requirements/chaquopy/lib/libpython3..so
build-wheel.py: abi=x86_64; api_level=21; prefix=/home/killian/chaquopy/server/pypi/packages/pylsl/build/1.10.4/cp38-cp38-android_21_x86_64/requirements/chaquopy; . /home/killian/chaquopy/server/pypi/../../target/build-common.sh; export
Installing NDK: this may take several minutes
build-wheel.py: /home/killian/chaquopy/android-sdk//ndk/22.1.7171670/toolchains/llvm/prebuilt/linux-x86_64/bin/llvm-ar rc /home/killian/chaquopy/server/pypi/packages/pylsl/build/1.10.4/cp38-cp38-android_21_x86_64/requirements/chaquopy/lib/libpthread.a
Traceback (most recent call last):
  File "./build-wheel.py", line 811, in <module>
    BuildWheel().main()
  File "./build-wheel.py", line 98, in main
    self.unpack_and_build()
  File "./build-wheel.py", line 144, in unpack_and_build
    self.create_dummy_libs()
  File "./build-wheel.py", line 377, in create_dummy_libs
    run(f"{os.environ['AR']} rc {self.reqs_dir}/chaquopy/lib/lib{name}.a")
  File "./build-wheel.py", line 766, in run
    return subprocess.run(command, **kwargs)
  File "/root/miniconda3/envs/build-wheel/lib/python3.8/subprocess.py", line 493, in run
    with Popen(*popenargs, **kwargs) as process:
  File "/root/miniconda3/envs/build-wheel/lib/python3.8/subprocess.py", line 858, in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
  File "/root/miniconda3/envs/build-wheel/lib/python3.8/subprocess.py", line 1704, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: '/home/killian/chaquopy/android-sdk//ndk/22.1.7171670/toolchains/llvm/prebuilt/linux-x86_64/bin/llvm-ar'

@vmitro
Copy link

vmitro commented Apr 27, 2023

Try with

export ANDROID_HOME=/path/to/sdk/without/slash/at/end

@mhsmith
Copy link
Member

mhsmith commented Apr 27, 2023

@KyucraftV2: Please mark multi-line logs with triple backticks as shown here, otherwise they're impossible to read.

The extra slash should probably be ignored, so I'm not sure what the problem is. How much of the path /home/killian/chaquopy/android-sdk//ndk/22.1.7171670/toolchains/llvm/prebuilt/linux-x86_64/bin/llvm-ar actually exists?

Maybe you interrupted it during a previous attempt to install the NDK, so the installation is incomplete. In that case, try deleting /home/killian/chaquopy/android-sdk//ndk, then running build-wheel again.

And for pylsl and LSL, i think pylsl has not dependecies on LSL, the module has all. But I'm not sure, did you know how can I be sure ?

As it says at https://pypi.org/project/pylsl/, the pylsl package only includes liblsl on Windows. On all other platforms, liblsl has to be installed separately.

@KyucraftV2
Copy link
Author

KyucraftV2 commented Apr 28, 2023

For the path who actually exists is /home/killian/chaquopy/android-sdk after, nothing exists (i have cmdlinetools/ inside just that).
And i watch what are you did for pyzmq and chaquopy-libzmq but i dont understand how can i make the same thing for lsl :(

And thanks for the remark on the backticks 👍

@KyucraftV2
Copy link
Author

KyucraftV2 commented May 2, 2023

Hi, I add lsl like chaquopy-libzmq, but now, I understand I need to build lsl before pylsl so when i use this command : ./build-wheel.py --python 3.8 --abi x86_64 lsl
I have this log with error at the end:

build-wheel.py: cd /home/killian/chaquopy/server/pypi/packages/lsl/build/1.16.1
build-wheel.py: rm -rf /home/killian/chaquopy/server/pypi/packages/lsl/build/1.16.1/py3-none-android_21_x86_64
build-wheel.py: mkdir -p /home/killian/chaquopy/server/pypi/packages/lsl/build/1.16.1/py3-none-android_21_x86_64
build-wheel.py: Using cached source archive
build-wheel.py: tar -C /tmp/build-wheel-u_buk64o -xf liblsl-1.16.1-OSX_amd64.tar.bz2
build-wheel.py: mv /tmp/build-wheel-u_buk64o /home/killian/chaquopy/server/pypi/packages/lsl/build/1.16.1/py3-none-android_21_x86_64/src
build-wheel.py: mkdir -p /home/killian/chaquopy/server/pypi/packages/lsl/build/1.16.1/py3-none-android_21_x86_64/requirements
build-wheel.py: mkdir -p /home/killian/chaquopy/server/pypi/packages/lsl/build/1.16.1/py3-none-android_21_x86_64/requirements/chaquopy/include
build-wheel.py: mkdir -p /home/killian/chaquopy/server/pypi/packages/lsl/build/1.16.1/py3-none-android_21_x86_64/requirements/chaquopy/lib
build-wheel.py: abi=x86_64; api_level=21; prefix=/home/killian/chaquopy/server/pypi/packages/lsl/build/1.16.1/py3-none-android_21_x86_64/requirements/chaquopy; . /home/killian/chaquopy/server/pypi/../../target/build-common.sh; export
Installing NDK: this may take several minutes
build-wheel.py: /home/killian/chaquopy/android-sdk/ndk/22.1.7171670/toolchains/llvm/prebuilt/linux-x86_64/bin/llvm-ar rc /home/killian/chaquopy/server/pypi/packages/lsl/build/1.16.1/py3-none-android_21_x86_64/requirements/chaquopy/lib/libpthread.a
Traceback (most recent call last):
  File "./build-wheel.py", line 811, in <module>
    BuildWheel().main()
  File "./build-wheel.py", line 98, in main
    self.unpack_and_build()
  File "./build-wheel.py", line 144, in unpack_and_build
    self.create_dummy_libs()
  File "./build-wheel.py", line 377, in create_dummy_libs
    run(f"{os.environ['AR']} rc {self.reqs_dir}/chaquopy/lib/lib{name}.a")
  File "./build-wheel.py", line 766, in run
    return subprocess.run(command, **kwargs)
  File "/root/miniconda3/envs/build-wheel/lib/python3.8/subprocess.py", line 493, in run
    with Popen(*popenargs, **kwargs) as process:
  File "/root/miniconda3/envs/build-wheel/lib/python3.8/subprocess.py", line 858, in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
  File "/root/miniconda3/envs/build-wheel/lib/python3.8/subprocess.py", line 1704, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: '/home/killian/chaquopy/android-sdk/ndk/22.1.7171670/toolchains/llvm/prebuilt/linux-x86_64/bin/llvm-ar'

But I dont understand why I have this. I have my ANDROID_HOME at /home/killian/chaquopy/android-sdk (with inside cmdlinetools/latest) like said on the readme.
So i dont understand why I have this error

if you want see my modifications for checking if Im not mistaken you can check commit at https://github.com/KyucraftV2/chaquopy/commits/master

@mhsmith
Copy link
Member

mhsmith commented May 10, 2023

It looks like the NDK installation isn't happening for some reason. Try running this command:

$ANDROID_HOME/cmdline-tools/latest/bin/sdkmanager "ndk;22.1.7171670"

And then run build-wheel again.

EDIT: For further discussion of this issue, see #1084.

@KyucraftV2
Copy link
Author

KyucraftV2 commented May 11, 2023

Hi, thanks for this command with this, now i can run ./build-wheel.py --python 3.8 --abi x86_64 lsl, but now I have a new error :

/home/killian/chaquopy/server/pypi/packages/lsl/build.sh: ligne 4: ./configure: No file or folder of this type
build-wheel.py: Error: Command returned exit status 127

But I dont understand because i have the same thing that chaquopy-libmzq so why on this library the build wheel works and on lsl it doesnt work ?

This is the entire log of the execution :

build-wheel.py: cd /home/killian/chaquopy/server/pypi/packages/lsl/build/1.16.1
build-wheel.py: rm -rf /home/killian/chaquopy/server/pypi/packages/lsl/build/1.16.1/py3-none-android_21_x86_64
build-wheel.py: mkdir -p /home/killian/chaquopy/server/pypi/packages/lsl/build/1.16.1/py3-none-android_21_x86_64
build-wheel.py: Using cached source archive
build-wheel.py: tar -C /tmp/build-wheel-h0_nd1xu -xf liblsl-1.16.1-OSX_amd64.tar.bz2
build-wheel.py: mv /tmp/build-wheel-h0_nd1xu /home/killian/chaquopy/server/pypi/packages/lsl/build/1.16.1/py3-none-android_21_x86_64/src
build-wheel.py: mkdir -p /home/killian/chaquopy/server/pypi/packages/lsl/build/1.16.1/py3-none-android_21_x86_64/requirements
build-wheel.py: mkdir -p /home/killian/chaquopy/server/pypi/packages/lsl/build/1.16.1/py3-none-android_21_x86_64/requirements/chaquopy/include
build-wheel.py: mkdir -p /home/killian/chaquopy/server/pypi/packages/lsl/build/1.16.1/py3-none-android_21_x86_64/requirements/chaquopy/lib
build-wheel.py: abi=x86_64; api_level=21; prefix=/home/killian/chaquopy/server/pypi/packages/lsl/build/1.16.1/py3-none-android_21_x86_64/requirements/chaquopy; . /home/killian/chaquopy/server/pypi/../../target/build-common.sh; export
build-wheel.py: /home/killian/chaquopy/android-sdk/ndk/22.1.7171670/toolchains/llvm/prebuilt/linux-x86_64/bin/llvm-ar rc /home/killian/chaquopy/server/pypi/packages/lsl/build/1.16.1/py3-none-android_21_x86_64/requirements/chaquopy/lib/libpthread.a
build-wheel.py: /home/killian/chaquopy/android-sdk/ndk/22.1.7171670/toolchains/llvm/prebuilt/linux-x86_64/bin/llvm-ar rc /home/killian/chaquopy/server/pypi/packages/lsl/build/1.16.1/py3-none-android_21_x86_64/requirements/chaquopy/lib/librt.a
build-wheel.py: cd /home/killian/chaquopy/server/pypi/packages/lsl/build/1.16.1/py3-none-android_21_x86_64/src
build-wheel.py: mkdir -p /home/killian/chaquopy/server/pypi/packages/lsl/build/1.16.1/py3-none-android_21_x86_64/prefix
build-wheel.py: mkdir -p /home/killian/chaquopy/server/pypi/packages/lsl/build/1.16.1/py3-none-android_21_x86_64/prefix/chaquopy
build-wheel.py: /home/killian/chaquopy/server/pypi/packages/lsl/build.sh
/home/killian/chaquopy/server/pypi/packages/lsl/build.sh: ligne 4: ./configure: Aucun fichier ou dossier de ce type
build-wheel.py: Error: Command returned exit status 127

@mhsmith
Copy link
Member

mhsmith commented May 11, 2023

Please post your meta.yaml and build.sh files for lsl.

@KyucraftV2
Copy link
Author

Ok no problem there are the two files :
build.sh :

#!/bin/bash
set -eu

./configure --host=$HOST
make -j $CPU_COUNT
make install prefix=$PREFIX

rm -r $PREFIX/bin
rm $PREFIX/lib/*.a
rm -r $PREFIX/share/man

And the meta.yaml

{% set version = "1.16.1" %}

package:
  name: lsl
  version: {{ version }}

build:
  number: 0

source:
  url: https://github.com/sccn/liblsl/releases/download/v{{ version }}/liblsl-{{ version }}-OSX_amd64.tar.bz2

@mhsmith
Copy link
Member

mhsmith commented May 11, 2023

You've set the source URL to a pre-compiled release. You need to use the source code instead.

The process of building the source code may also be different from libzmq. For example, these instructions indicate that LSL uses CMake rather than a configure script. Search for "cmake" in the other recipes for examples of how to use it.

@KyucraftV2
Copy link
Author

Okey so now i change my meta.yaml and my build.sh
But now I run and I have an error on CMakeLists and I don't understand this error :

CMake Error: The source directory "/home/killian/chaquopy/server/pypi/packages/lsl/build/1.16.1/py3-none-android_21_x86_64" does not appear to contain CMakeLists.txt.
Specify --help for usage, or press the help button on the CMake GUI.
build-wheel.py: Error: Command returned exit status 1

And there are my two files :
build.sh

#!/bin/bash
set -eu

git clone --depth=1 https://github.com/sccn/liblsl.git
cmake .. -DCMAKE_TOOLCHAIN_FILE="$SRC_DIR/../chaquopy.toolchain.cmake" \
      -DCMAKE_INSTALL_PREFIX="$PREFIX" \
      -DBUILD_TESTING=OFF
cmake --build . -j $CPU_COUNT
cmake --build . --target install

And my meta.yaml

{% set version = "1.16.1" %}

package:
  name: lsl
  version: {{ version }}

build:
  number: 0

source:
  url: https://github.com/sccn/liblsl/releases/download/v{{ version }}/liblsl-{{ version }}-OSX_amd64.tar.bz2

requirements:
  build:
    - cmake

@mhsmith
Copy link
Member

mhsmith commented May 14, 2023

Why have you added a git clone command to build.sh? None of the existing packages do that.

Remove the clone command and edit the url in meta.yaml to point to the source code. You can find the source code links for each release in the "Assets" section of https://github.com/sccn/liblsl/releases. The URL format is self-explanatory.

Also, notice that in all the existing build.sh scripts which contain a cmake .. command, they first create a build directory and change into it.

@KyucraftV2
Copy link
Author

Ok, I try to copy the format of torch/meta.yaml, because when I use one release and the format is .deb so it's not working.
So on meta.yaml now i have this :

{% set version = "1.16.1" %}

package:
  name: lsl
  version: {{ version }}

build:
  number: 0

source:
  git_url: https://github.com/sccn/liblsl.git
  git_rev: v{{ version }}

requirements:
  build:
    - cmake

And now, I run my build-wheel command, its building CXX object but on linking of shared library liblsl.so :

ld: error: undefined symbol: lsl::get_local_interfaces()
>>> referenced by api_config.cpp:218 (/home/killian/chaquopy/server/pypi/packages/lsl/build/1.16.1/py3-none-android_21_x86_64/src/src/api_config.cpp:218)
>>>               CMakeFiles/lslobj.dir/src/api_config.cpp.o:(lsl::api_config::load_from_file(std::__ndk1::basic_string<char, std::__ndk1::char_traits<char>, std::__ndk1::allocator<char> > const&))
clang++: error: linker command failed with exit code 1 (use -v to see invocation)
gmake[2]: *** [CMakeFiles/lsl.dir/build.make:159 : liblsl.so] Erreur 1
gmake[1]: *** [CMakeFiles/Makefile2:113 : CMakeFiles/lsl.dir/all] Erreur 2
gmake: *** [Makefile:156 : all] Erreur 2
build-wheel.py: Error: Command returned exit status 2

And I dont have no idea how to fix this

@mhsmith
Copy link
Member

mhsmith commented May 15, 2023

Please post your current build.sh file.

@KyucraftV2
Copy link
Author

Ok :

#!/bin/bash
set -eu

mkdir build
cd build
cmake .. -DCMAKE_TOOLCHAIN_FILE="$SRC_DIR/../chaquopy.toolchain.cmake" \
      -DCMAKE_INSTALL_PREFIX="$PREFIX" \
      -DBUILD_TESTING=OFF
cmake --build . -j $CPU_COUNT
cmake --build . --target install

I copy the torch/build.sh maybe its an error :(

@mhsmith
Copy link
Member

mhsmith commented May 16, 2023

It looks like this function was renamed in the past, but they missed one of the #if branches. Create a file packages/lsl/patches/chaquopy.patch with the following content:

diff --git a/src/netinterfaces.cpp b/src/netinterfaces.cpp                                                                                                                                                                           
index 860670f..7afd105 100644                                                                                                                                                                                                        
--- a/src/netinterfaces.cpp                                                                                                                                                                                                          
+++ b/src/netinterfaces.cpp                                                                                                                                                                                                          
@@ -114,7 +114,7 @@ std::vector<lsl::netif> lsl::get_local_interfaces() {
 }                                                                                                                                                                                                                                   
 #else                                                                                                                                                                                                                               
                                                                                                                                                                                                                                     
-std::vector<lsl::netif> get_interface_addresses() {                                                                                                                                                                                 
+std::vector<lsl::netif> lsl::get_local_interfaces() {                                                                                                                                                                               
        LOG_F(WARNING, "No implementation to enumerate network interfaces found.");                                                                                                                                                  
        return std::vector<lsl::netif>();                                                                                                                                                                                            
 }                                                                                                                                                                                                                                   

And edit your build.sh file as follows:

#!/bin/bash                                                                                                                                                                                                                          
set -eu

mkdir build
cd build

# LSL_OPTIMIZATIONS enables CMAKE_INTERPROCEDURAL_OPTIMIZATION, which in CMake 3.18.4                                                                                                                                                
# (/usr/share/cmake-3.18/Modules/Compiler/Clang.cmake) adds -fuse-ld=gold to the linker                                                                                                                                              
# command line, which causes "unknown option" errors.                                                                                                                                                                                
cmake .. -DCMAKE_TOOLCHAIN_FILE="$SRC_DIR/../chaquopy.toolchain.cmake" \
      -DCMAKE_INSTALL_PREFIX="$PREFIX" \
      -DLSL_OPTIMIZATIONS=OFF

cmake --build . -j $CPU_COUNT
cmake --build . --target install

For me, that was enough to fix the the x86_64 build.

@KyucraftV2
Copy link
Author

Hi, ok thanks for yout help but i copy and paste what you give. But now when I try to build wheel I have this error

Hunk #1 FAILED at 114.
1 out of 1 hunk FAILED -- saving rejects to file src/netinterfaces.cpp.rej
build-wheel.py: Error: Command returned exit status 1

@mhsmith
Copy link
Member

mhsmith commented May 18, 2023

Please create a fork of this repository containing all your current build.sh, meta.yaml and patch files, and post the link.

@KyucraftV2
Copy link
Author

There is my fork : https://github.com/KyucraftV2/chaquopy
Thanks for your help

@mhsmith mhsmith linked a pull request May 20, 2023 that will close this issue
@mhsmith
Copy link
Member

mhsmith commented May 20, 2023

Sorry, it turns out netinterfaces.cpp contained tabs, so patches are difficult to copy and paste. Please use the files from this PR: #872.

I've also renamed the package to chaquopy-liblsl, to follow the pattern of all our other non-Python packages.

@KyucraftV2
Copy link
Author

Ok thanks, now the building of CXX object works. I copy and paste your PR in my fork but now after the building I have an error type FileNotFoundError with this error :

Traceback (most recent call last):
  File "./build-wheel.py", line 811, in <module>
    BuildWheel().main()
  File "./build-wheel.py", line 98, in main
    self.unpack_and_build()
  File "./build-wheel.py", line 146, in unpack_and_build
    return self.fix_wheel(wheel_filename)
  File "./build-wheel.py", line 590, in fix_wheel
    run(f"patchelf --remove-rpath {fixed_path}")
  File "./build-wheel.py", line 766, in run
    return subprocess.run(command, **kwargs)
  File "/root/miniconda3/envs/build-wheel/lib/python3.8/subprocess.py", line 493, in run
    with Popen(*popenargs, **kwargs) as process:
  File "/root/miniconda3/envs/build-wheel/lib/python3.8/subprocess.py", line 858, in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
  File "/root/miniconda3/envs/build-wheel/lib/python3.8/subprocess.py", line 1704, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: 'patchelf'

I don't understand why the file patchelf is not found, maybe I forgot a things

@vmitro
Copy link

vmitro commented May 22, 2023

@KyucraftV2 It seem you don't have patchelf installed. sudo apt install patchelf should do the trick on Ubuntu.

@mhsmith
Copy link
Member

mhsmith commented May 22, 2023

I don't understand why the file patchelf is not found, maybe I forgot a things

This is already mentioned in the README, so you should probably review that in case you missed anything else.

@KyucraftV2
Copy link
Author

Ok thanks, now I build chaquopy-liblsl and after I build pylsl. But i have the message Successfully built pylsl and after in the clean up I have an error and I dont know if it change anything

Successfully built pylsl
Cleaning up...
Removed build tracker '/tmp/pip-req-tracker-wbb1kcod'
build-wheel.py: mkdir -p /home/killian/chaquopy/server/pypi/packages/pylsl/build/1.10.4/cp38-cp38-android_21_x86_64/fix_wheel
build-wheel.py: unzip -d /home/killian/chaquopy/server/pypi/packages/pylsl/build/1.10.4/cp38-cp38-android_21_x86_64/fix_wheel -q /home/killian/chaquopy/server/pypi/packages/pylsl/build/1.10.4/cp38-cp38-android_21_x86_64/src/pylsl-1.10.4-py2.py3-none-any.whl
build-wheel.py: Processing native binaries
build-wheel.py: chmod +w /home/killian/chaquopy/server/pypi/packages/pylsl/build/1.10.4/cp38-cp38-android_21_x86_64/fix_wheel/pylsl/liblsl32.so
build-wheel.py: /home/killian/chaquopy/android-sdk/ndk/22.1.7171670/toolchains/llvm/prebuilt/linux-x86_64/bin/llvm-strip --strip-unneeded /home/killian/chaquopy/server/pypi/packages/pylsl/build/1.10.4/cp38-cp38-android_21_x86_64/fix_wheel/pylsl/liblsl32.so
build-wheel.py: Error: /home/killian/chaquopy/server/pypi/packages/pylsl/build/1.10.4/cp38-cp38-android_21_x86_64/fix_wheel/pylsl/liblsl32.so is linked against unknown library 'librt.so.1'.

And if all is okay, can I merge request my fork ?

@mhsmith
Copy link
Member

mhsmith commented May 24, 2023

Pylsl 1.10.4's setup.py script is unconditionally bundling pre-built Linux binaries of liblsl. It looks like this is fixed in the current version, 1.16.1, but there are no sdists for that version on PyPI. You can work around this by downloading the source code from GitHub, just as we did for liblsl:

{% set version = "1.16.1" %}

package:
    name : pylsl
    version : {{ version }}

build:
  number: 1

source:
  url: https://github.com/labstreaminglayer/pylsl/archive/refs/tags/v{{ version }}.tar.gz

requirements:
  build:
    - packaging 22.0
  host:
    - python
    - chaquopy-liblsl {{ version }}

@KyucraftV2
Copy link
Author

Okey thanks a lot for your help, now im able to build pylsl.
Now i need to merge request ? And last question for use pylsl on an app build with toga, i need to make another things or now its just okey and i can build my app ?

@mhsmith
Copy link
Member

mhsmith commented May 24, 2023

See the README for how to use a package you built yourself.

When using Briefcase, instead of editing the build.gradle file directly, you should do this:

  • Add this line to the android section of your pyproject.toml file:

    build_gradle_extra_content = "android.defaultConfig.python.pip.options '--extra-index-url', 'path/to/dist'"

    ... where path/to/dist is "the location of the dist directory" in the link above.

  • Run briefcase create android.

  • Add the package name to the requires list in your pyproject.toml file.

  • Run briefcase build android -r.

Before you make a merge request, please check that the packages actually work, ideally by using the test harness.

@KyucraftV2
Copy link
Author

Hi, I trust your last message and I build four versions off .whl (one off each ABI) but i have this error :

07-05 12:14:13.999 10621 10621 E AndroidRuntime: FATAL EXCEPTION: main
07-05 12:14:13.999 10621 10621 E AndroidRuntime: Process: com.example.appvisulsl, PID: 10621
07-05 12:14:13.999 10621 10621 E AndroidRuntime: java.lang.RuntimeException: Unable to start activity ComponentInfo{com.example.appvisulsl/org.beeware.android.MainActivity}: com.chaquo.python.PyException: RuntimeError: LSL binary library file was not found. Please make sure that the binary file can be found in the package lib folder
07-05 12:14:13.999 10621 10621 E AndroidRuntime:  (/data/data/com.example.appvisulsl/files/chaquopy/AssetFinder/requirements/pylsl/lib)
07-05 12:14:13.999 10621 10621 E AndroidRuntime:  or the system search path. Alternatively, specify the PYLSL_LIB environment variable.
07-05 12:14:13.999 10621 10621 E AndroidRuntime:  You can install the LSL library with conda: `conda install -c conda-forge liblsl`
07-05 12:14:13.999 10621 10621 E AndroidRuntime: or otherwise download it from the liblsl releases page assets: https://github.com/sccn/liblsl/releases
07-05 12:14:13.999 10621 10621 E AndroidRuntime:        at android.app.ActivityThread.performLaunchActivity(ActivityThread.java:3645)
07-05 12:14:13.999 10621 10621 E AndroidRuntime:        at android.app.ActivityThread.handleLaunchActivity(ActivityThread.java:3782)
07-05 12:14:13.999 10621 10621 E AndroidRuntime:        at android.app.servertransaction.LaunchActivityItem.execute(LaunchActivityItem.java:101)
07-05 12:14:13.999 10621 10621 E AndroidRuntime:        at android.app.servertransaction.TransactionExecutor.executeCallbacks(TransactionExecutor.java:135)
07-05 12:14:13.999 10621 10621 E AndroidRuntime:        at android.app.servertransaction.TransactionExecutor.execute(TransactionExecutor.java:95)
07-05 12:14:13.999 10621 10621 E AndroidRuntime:        at android.app.ActivityThread$H.handleMessage(ActivityThread.java:2307)
07-05 12:14:13.999 10621 10621 E AndroidRuntime:        at android.os.Handler.dispatchMessage(Handler.java:106)
07-05 12:14:13.999 10621 10621 E AndroidRuntime:        at android.os.Looper.loopOnce(Looper.java:201)
07-05 12:14:13.999 10621 10621 E AndroidRuntime:        at android.os.Looper.loop(Looper.java:288)
07-05 12:14:13.999 10621 10621 E AndroidRuntime:        at android.app.ActivityThread.main(ActivityThread.java:7872)
07-05 12:14:13.999 10621 10621 E AndroidRuntime:        at java.lang.reflect.Method.invoke(Native Method)
07-05 12:14:13.999 10621 10621 E AndroidRuntime:        at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:548)
07-05 12:14:13.999 10621 10621 E AndroidRuntime:        at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:936)
07-05 12:14:13.999 10621 10621 E AndroidRuntime: Caused by: com.chaquo.python.PyException: RuntimeError: LSL binary library file was not found. Please make sure that the binary file can be found in the package lib folder
07-05 12:14:13.999 10621 10621 E AndroidRuntime:  (/data/data/com.example.appvisulsl/files/chaquopy/AssetFinder/requirements/pylsl/lib)
07-05 12:14:13.999 10621 10621 E AndroidRuntime:  or the system search path. Alternatively, specify the PYLSL_LIB environment variable.
07-05 12:14:13.999 10621 10621 E AndroidRuntime:  You can install the LSL library with conda: `conda install -c conda-forge liblsl`
07-05 12:14:13.999 10621 10621 E AndroidRuntime: or otherwise download it from the liblsl releases page assets: https://github.com/sccn/liblsl/releases
07-05 12:14:13.999 10621 10621 E AndroidRuntime:        at <python>.pylsl.pylsl.<module>(pylsl.py:1296)
07-05 12:14:13.999 10621 10621 E AndroidRuntime:        at <python>.java.chaquopy.import_override(import.pxi:26)
07-05 12:14:13.999 10621 10621 E AndroidRuntime:        at <python>.pylsl.<module>(__init__.py:2)
07-05 12:14:13.999 10621 10621 E AndroidRuntime:        at <python>.java.chaquopy.import_override(import.pxi:26)
07-05 12:14:13.999 10621 10621 E AndroidRuntime:        at <python>.appvisulsl.app.<module>(app.py:10)
07-05 12:14:13.999 10621 10621 E AndroidRuntime:        at <python>.java.chaquopy.import_override(import.pxi:26)
07-05 12:14:13.999 10621 10621 E AndroidRuntime:        at <python>.__main__.<module>(__main__.py:1)
07-05 12:14:13.999 10621 10621 E AndroidRuntime:        at <python>.runpy._run_code(runpy.py:87)
07-05 12:14:13.999 10621 10621 E AndroidRuntime:        at <python>.runpy._run_module_code(runpy.py:97)
07-05 12:14:13.999 10621 10621 E AndroidRuntime:        at <python>.runpy.run_module(runpy.py:207)
07-05 12:14:13.999 10621 10621 E AndroidRuntime:        at <python>.chaquopy_java.call(chaquopy_java.pyx:354)
07-05 12:14:13.999 10621 10621 E AndroidRuntime:        at <python>.chaquopy_java.Java_com_chaquo_python_PyObject_callAttrThrowsNative(chaquopy_java.pyx:326)
07-05 12:14:13.999 10621 10621 E AndroidRuntime:        at com.chaquo.python.PyObject.callAttrThrowsNative(Native Method)
07-05 12:14:13.999 10621 10621 E AndroidRuntime:        at com.chaquo.python.PyObject.callAttrThrows(PyObject.java:232)
07-05 12:14:13.999 10621 10621 E AndroidRuntime:        at com.chaquo.python.PyObject.callAttr(PyObject.java:221)
07-05 12:14:13.999 10621 10621 E AndroidRuntime:        at org.beeware.android.MainActivity.onCreate(MainActivity.java:85)
07-05 12:14:13.999 10621 10621 E AndroidRuntime:        at android.app.Activity.performCreate(Activity.java:8305)
07-05 12:14:13.999 10621 10621 E AndroidRuntime:        at android.app.Activity.performCreate(Activity.java:8284)
07-05 12:14:13.999 10621 10621 E AndroidRuntime:        at android.app.Instrumentation.callActivityOnCreate(Instrumentation.java:1417)
07-05 12:14:13.999 10621 10621 E AndroidRuntime:        at android.app.ActivityThread.performLaunchActivity(ActivityThread.java:3626)
07-05 12:14:13.999 10621 10621 E AndroidRuntime:        ... 12 more
07-05 12:14:14.051 10621 10621 I Process : Sending signal. PID: 10621 SIG: 9

Maybe I forgot something but i dont think so

@mhsmith
Copy link
Member

mhsmith commented Jul 6, 2023

I've merged your fork into #872, built the packages myself, and uploaded them to https://chaquo.com/pypi-test/. I then built the pkgtest app with the arm64 Python 3.8 package, and I can successfully import pylsl in the console.

Please give these builds a try, and if you still have problems, let me know:

  • Your app's Python version
  • Your app's Chaquopy version
  • Which device or emulator you're testing on, including its ABI

@KyucraftV2
Copy link
Author

KyucraftV2 commented Jul 7, 2023

Ok thanks for the merge.
So I use Python 3.8 and Chaquopy 7. But now i use adb for install the APK, i use this command : .\adb.exe install --abi x86_64 app.apk
The emulator is a Google Pixel 5 with ABI is x86_64 (I check on Android Studio)
The crash is always present and with logcat | grep appvisulsl i found there lines :

07-07 09:06:16.522  4915  4915 E AndroidRuntime: Process: com.example.appvisulsl, PID: 4915
07-07 09:06:16.522  4915  4915 E AndroidRuntime: java.lang.RuntimeException: Unable to start activity ComponentInfo{com.example.appvisulsl/org.beeware.android.MainActivity}: com.chaquo.python.PyException: RuntimeError: LSL binary library file was not found. Please make sure that the binary file can be found in the package lib folder
07-07 09:06:16.522  4915  4915 E AndroidRuntime:  (/data/data/com.example.appvisulsl/files/chaquopy/AssetFinder/requirements/pylsl/lib)
07-07 09:06:16.522  4915  4915 E AndroidRuntime:  (/data/data/com.example.appvisulsl/files/chaquopy/AssetFinder/requirements/pylsl/lib)
07-07 09:06:16.522  4915  4915 E AndroidRuntime:        at <python>.appvisulsl.app.<module>(app.py:10)
07-07 09:06:16.580   615   973 W ActivityTaskManager:   Force finishing activity 

@KyucraftV2
Copy link
Author

Well, I search why its not work, and I have an idea, when I build with command briefcase build android, I have this two lines :

Chaquopy: Installing for arm64-v8a
Looking in indexes: https://pypi.org/simple, https://chaquo.com/pypi-7.0, https://chaquo.com/pypi-13.1

So I think its not going to search in https://chaquo.com/pypi-test/
So maybe i need to specify this link ?

@KyucraftV2
Copy link
Author

Okey, i found the problem, i need to specify the address in my pyproject.toml with this line build_gradle_extra_content = "android.defaultConfig.python.pip.options '--extra-index-url', 'https://chaquo.com/pypi-test/'"
Thanks for your help, now i close because its work, thanks a lot

@mhsmith
Copy link
Member

mhsmith commented Jul 7, 2023

Let's leave the issue open until I've added the packages to the main repository.

@mhsmith mhsmith reopened this Jul 7, 2023
@mhsmith mhsmith mentioned this issue Dec 18, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants