Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fail to install CVXPY (or SCS) #1082

Open
kmykman opened this issue Jan 24, 2024 · 8 comments
Open

Fail to install CVXPY (or SCS) #1082

kmykman opened this issue Jan 24, 2024 · 8 comments
Labels

Comments

@kmykman
Copy link

kmykman commented Jan 24, 2024

I have tried to install CVXPY by pip { install("cvxpy")} but it gives error ERROR: Command errored out with exit status 1: and Failed to install scs==3.2.0 . Then I tried pip { install("scs")} to install scs alone, but it gives the same error. I saw from bodono/scs-python#32 (comment) said install scs in conda or new environment. I tried to install in Colab and it is fine. So it is not able to install in chaquopy. How can I do this in chaquopy? Or is there a way to install cvxpy?

@mhsmith
Copy link
Member

mhsmith commented Jan 24, 2024

This package contains native components, so it would have to be built into a wheel file. If you'd like to try doing this yourself, follow the instructions here. And if you're successful, please make a pull request so we can add the package to the public repository.

If anyone else wants this package too, let us know by clicking the thumbs-up button above.

@kmykman
Copy link
Author

kmykman commented Jan 25, 2024

I tried to add wheel file of cvxpy. As cvxpy need numpy and scipy, I add pip { install("numpy")}, I has error

FATAL EXCEPTION: main
                                                                                                    Process: com.example.python, PID: 22918
                                                                                                    java.lang.RuntimeException: Unable to start activity ComponentInfo{com.example.python/com.example.python.MainActivity}: com.chaquo.python.PyException: AttributeError: module 'os' has no attribute 'add_dll_directory'
                                                                                                    	at android.app.ActivityThread.performLaunchActivity(ActivityThread.java:3114)
                                                                                                    	at android.app.ActivityThread.handleLaunchActivity(ActivityThread.java:3257)
                                                                                                    	at android.app.servertransaction.LaunchActivityItem.execute(LaunchActivityItem.java:78)
                                                                                                    	at android.app.servertransaction.TransactionExecutor.executeCallbacks(TransactionExecutor.java:108)
                                                                                                    	at android.app.servertransaction.TransactionExecutor.execute(TransactionExecutor.java:68)
                                                                                                    	at android.app.ActivityThread$H.handleMessage(ActivityThread.java:1948)
                                                                                                    	at android.os.Handler.dispatchMessage(Handler.java:106)
                                                                                                    	at android.os.Looper.loop(Looper.java:214)
                                                                                                    	at android.app.ActivityThread.main(ActivityThread.java:7050)
                                                                                                    	at java.lang.reflect.Method.invoke(Native Method)
                                                                                                    	at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:494)
                                                                                                    	at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:965)
                                                                                                    Caused by: com.chaquo.python.PyException: AttributeError: module 'os' has no attribute 'add_dll_directory'
                                                                                                    	at <python>.scipy._delvewheel_init_patch_1_3_1(__init__.py:69)
                                                                                                    	at <python>.scipy.<module>(__init__.py:72)
                                                                                                    	at <python>.java.chaquopy.import_override(import.pxi:26)
                                                                                                    	at <python>.cvxpy.interface.matrix_utilities.<module>(matrix_utilities.py:20)
                                                                                                    	at <python>.java.chaquopy.import_override(import.pxi:26)
                                                                                                    	at <python>.cvxpy.interface.<module>(__init__.py:17)
                                                                                                    	at <python>.java.chaquopy.import_override(import.pxi:26)
                                                                                                    	at <python>.cvxpy.<module>(__init__.py:19)
                                                                                                    	at <python>.java.chaquopy.import_override(import.pxi:26)
                                                                                                    	at <python>.hello_python.<module>(hello_python.py:2)
                                                                                                    	at <python>.importlib._bootstrap._call_with_frames_removed(<frozen importlib._bootstrap>:241)
                                                                                                    	at <python>.importlib._bootstrap_external.exec_module(<frozen importlib._bootstrap_external>:940)
                                                                                                    	at <python>.java.android.importer.exec_module(importer.py:634)
                                                                                                    	at <python>.java.android.importer.exec_module(importer.py:721)
                                                                                                    	at <python>.importlib._bootstrap._load_unlocked(<frozen importlib._bootstrap>:690)
                                                                                                    	at <python>.importlib._bootstrap._find_and_load_unlocked(<frozen importlib._bootstrap>:1147)
                                                                                                    	at <python>.importlib._bootstrap._find_and_load(<frozen importlib._bootstrap>:1176)
                                                                                                    	at <python>.importlib._bootstrap._gcd_import(<frozen importlib._bootstrap>:1204)
                                                                                                    	at <python>.importlib.import_module(__init__.py:126)
                                                                                                    	at <python>.chaquopy_java.Java_com_chaquo_python_Python_getModuleNative(chaquopy_java.pyx:129)
                                                                                                    	at com.chaquo.python.Python.getModuleNative(Native Method)
                                                                                                    	at com.chaquo.python.Python.getModule(Python.java:84)
                                                                                                    	at com.example.python.MainActivity.onCreate(MainActivity.java:27)
                                                                                                    	at android.app.Activity.performCreate(Activity.java:7327)
                                                                                                    	at android.app.Activity.performCreate(Activity.java:7318)
                                                                                                    	at android.app.Instrumentation.callActivityOnCreate(Instrumentation.java:1271)
                                                                                                    	at android.app.ActivityThread.performLaunchActivity(ActivityThread.java:3094)
                                                                                                    	at android.app.ActivityThread.handleLaunchActivity(ActivityThread.java:3257) 
                                                                                                    	at android.app.servertransaction.LaunchActivityItem.execute(LaunchActivityItem.java:78) 
                                                                                                    	at android.app.servertransaction.TransactionExecutor.executeCallbacks(TransactionExecutor.java:108) 
                                                                                                    	at android.app.servertransaction.TransactionExecutor.execute(TransactionExecutor.java:68) 
                                                                                                    	at android.app.ActivityThread$H.handleMessage(ActivityThread.java:1948) 
                                                                                                    	at android.os.Handler.dispatchMessage(Handler.java:106) 
                                                                                                    	at android.os.Looper.loop(Looper.java:214) 
                                                                                                    	at android.app.ActivityThread.main(ActivityThread.java:7050) 
                                                                                                    	at java.lang.reflect.Method.invoke(Native Method) 
                                                                                                    	at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:494) 
                                                                                                    	at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:965) 

. I think the problem is in scipy. scipy is installed by wheel file of scipy-1.10.1-cp311-cp311-win_amd64.whl from https://pypi.org/project/scipy/1.10.1/#files. If I use pip { install("scipy")}, it said Failed to install scipy from https://files.pythonhosted.org/packages/30/85/cdbf2c3c460fe5aae812917866392068a88d02f07de0fe31ce738734c477/scipy-1.12.0.tar.gz#sha256=4bf5abab8a36d20193c698b0f1fc282c1d083c94723902c447e5d2f1780936a3.

If I install numpy by numpy wheel file of numpy-1.25.2-cp311-cp311-win_amd64.whl from https://pypi.org/project/numpy/1.25.2/#files, I has error

java.lang.RuntimeException: Unable to start activity ComponentInfo{com.example.python/com.example.python.MainActivity}: com.chaquo.python.PyException: ImportError: 
                                                                                                    
IMPORTANT: PLEASE READ THIS FOR ADVICE ON HOW TO SOLVE THIS ISSUE!
                                                                                                    
Importing the numpy C-extensions failed. This error can happen for
                                                                                                    many reasons, often due to issues with your setup or how NumPy was
                                                                                                    installed.
                                                                                                    
                                                                                                    We have compiled some common reasons and troubleshooting tips at:
                                                                                                    
                                                                                                        https://numpy.org/devdocs/user/troubleshooting-importerror.html
                                                                                                    
                                                                                                    Please note and check the following:
                                                                                                    
                                                                                                      * The Python version is: Python3.11 from "/system/bin/app_process64"
                                                                                                      * The NumPy version is: "1.25.2"
                                                                                                    
                                                                                                    and make sure that they are the versions you expect.
                                                                                                    Please carefully study the documentation linked above for further help.
                                                                                                    
                                                                                                    Original error was: No module named 'numpy.core._multiarray_umath'
                                                                                                    
                                                                                                    	at android.app.ActivityThread.performLaunchActivity(ActivityThread.java:3114)
                                                                                                    	at android.app.ActivityThread.handleLaunchActivity(ActivityThread.java:3257)
                                                                                                    	at android.app.servertransaction.LaunchActivityItem.execute(LaunchActivityItem.java:78)
                                                                                                    	at android.app.servertransaction.TransactionExecutor.executeCallbacks(TransactionExecutor.java:108)
                                                                                                    	at android.app.servertransaction.TransactionExecutor.execute(TransactionExecutor.java:68)
                                                                                                    	at android.app.ActivityThread$H.handleMessage(ActivityThread.java:1948)
                                                                                                    	at android.os.Handler.dispatchMessage(Handler.java:106)
                                                                                                    	at android.os.Looper.loop(Looper.java:214)
                                                                                                    	at android.app.ActivityThread.main(ActivityThread.java:7050)
                                                                                                    	at java.lang.reflect.Method.invoke(Native Method)
                                                                                                    	at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:494)
                                                                                                    	at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:965)
                                                                                                    Caused by: com.chaquo.python.PyException: ImportError: 
                                                                                                    
                                                                                                    IMPORTANT: PLEASE READ THIS FOR ADVICE ON HOW TO SOLVE THIS ISSUE!
                                                                                                    
                                                                                                    Importing the numpy C-extensions failed. This error can happen for
                                                                                                    many reasons, often due to issues with your setup or how NumPy was
                                                                                                    installed.
                                                                                                    
                                                                                                    We have compiled some common reasons and troubleshooting tips at:
                                                                                                    
                                                                                                        https://numpy.org/devdocs/user/troubleshooting-importerror.html
                                                                                                    
                                                                                                    Please note and check the following:
                                                                                                    
                                                                                                      * The Python version is: Python3.11 from "/system/bin/app_process64"
                                                                                                      * The NumPy version is: "1.24.2"
                                                                                                    
                                                                                                    and make sure that they are the versions you expect.
                                                                                                    Please carefully study the documentation linked above for further help.
                                                                                                    
                                                                                                    Original error was: No module named 'numpy.core._multiarray_umath'
                                                                                                    
                                                                                                    	at <python>.numpy.core.<module>(__init__.py:49)
                                                                                                    	at <python>.java.chaquopy.import_override(import.pxi:60)
                                                                                                    	at <python>.numpy.<module>(__init__.py:141)
                                                                                                    	at <python>.java.chaquopy.import_override(import.pxi:60)
                                                                                                    	at <python>.cvxpy.interface.matrix_utilities.<module>(matrix_utilities.py:19)
                                                                                                    	at <python>.java.chaquopy.import_override(import.pxi:60)
                                                                                                    	at <python>.cvxpy.interface.<module>(__init__.py:17)
                                                                                                    	at <python>.java.chaquopy.import_override(import.pxi:60)
                                                                                                    	at <python>.cvxpy.<module>(__init__.py:19)
                                                                                                    	at <python>.java.chaquopy.import_override(import.pxi:60)
                                                                                                    	at <python>.hello_python.<module>(hello_python.py:3)
                                                                                                    	at <python>.importlib._bootstrap._call_with_frames_removed(<frozen importlib._bootstrap>:241)
                                                                                                    	at <python>.importlib._bootstrap_external.exec_module(<frozen importlib._bootstrap_external>:940)
                                                                                                    	at <python>.java.android.importer.exec_module(importer.py:634)
                                                                                                    	at <python>.importlib._bootstrap._load_unlocked(<frozen importlib._bootstrap>:690)
                                                                                                    	at <python>.importlib._bootstrap._find_and_load_unlocked(<frozen importlib._bootstrap>:1147)
                                                                                                    	at <python>.importlib._bootstrap._find_and_load(<frozen importlib._bootstrap>:1176)
                                                                                                    	at <python>.importlib._bootstrap._gcd_import(<frozen importlib._bootstrap>:1204)
                                                                                                    	at <python>.importlib.import_module(__init__.py:126)
                                                                                                    	at <python>.chaquopy_java.Java_com_chaquo_python_Python_getModuleNative(chaquopy_java.pyx:129)
                                                                                                    	at com.chaquo.python.Python.getModuleNative(Native Method)
2024-01-25 10:40:30.739  5191-5191  AndroidRuntime          com.example.python                   E  	at com.chaquo.python.Python.getModule(Python.java:84)
                                                                                                    	at com.example.python.MainActivity.onCreate(MainActivity.java:27)
                                                                                                    	at android.app.Activity.performCreate(Activity.java:7327)
                                                                                                    	at android.app.Activity.performCreate(Activity.java:7318)
                                                                                                    	at android.app.Instrumentation.callActivityOnCreate(Instrumentation.java:1271)
                                                                                                    	at android.app.ActivityThread.performLaunchActivity(ActivityThread.java:3094)
                                                                                                    	... 11 more

How can I solve it?

@mhsmith
Copy link
Member

mhsmith commented Jan 25, 2024

Windows, Linux and macOS wheels will not work on Android. The only wheels that will work are Android wheels built with the instructions l I linked in my previous comment, or pure-Python wheels tagged with none-any.

The Android wheel repository, which Chaquopy uses automatically, contains NumPy for Python 3.8-3.12, and SciPy for Python 3.8-3.10. So as long as your app's Python version is between 3.8 and 3.10, you'll be able to install both of those packages. But you'll still have to make your own builds of cvxopt, scs, and maybe other libraries they depend on.

@kmykman
Copy link
Author

kmykman commented Jan 25, 2024

Thanks for your reply. I am trying to build it. I saw it needs .yaml file so I am looking for .yaml file for cvxpy. In the example, it said multidict .yaml file is downloaded from Pypl but I can't see any .yaml file in Pypl multidict download page. Where can I find the .yaml file?

I tried to make a meta.yaml file for cvxpy following that in multidict and run ./build-wheel.py --python 3.8 --abi arm64-v8a ./packages/cvxpy.

build-wheel: abi=arm64-v8a; api_level=21; prefix=/home/yolopoi/chaquopy/server/pypi/packages/cvxpy/build/1.4.2/cp38-cp38-android_21_arm64_v8a/requirements/chaquopy; . /home/yolopoi/chaquopy/server/pypi/../../target/build-common.sh; export
Installing NDK: this may take several minutes
/home/yolopoi/android-sdk/ndk/22.1.7171670/toolchains/llvm/prebuilt/*/bin/llvm-ar does not exist
build-wheel: Error: Command 'abi=arm64-v8a; api_level=21; prefix=/home/yolopoi/chaquopy/server/pypi/packages/cvxpy/build/1.4.2/cp38-cp38-android_21_arm64_v8a/requirements/chaquopy; . /home/yolopoi/chaquopy/server/pypi/../../target/build-common.sh; export' returned non-zero exit status 1.

I checked the ANDROID_HOME path is correct. I look into android-sdk folder and there is only cmdline-tools folder which is created according to README. How can I solve it?

Also I tried ./build-wheel.py --python 3.8 --abi arm64-v8a ./packages/numpy, it said

build-wheel: Error: Couldn't find compatible wheel for chaquopy-openblas 0.2.20. Try downloading it from https://chaquo.com/pypi-13.1/chaquopy-openblas/ into /home/yolopoi/chaquopy/server/pypi/dist/chaquopy-openblas.

and no file in in dist. Why would be like that?

Thanks.

@mhsmith
Copy link
Member

mhsmith commented Jan 26, 2024

Installing NDK: this may take several minutes
/home/yolopoi/android-sdk/ndk/22.1.7171670/toolchains/llvm/prebuilt/*/bin/llvm-ar does not exist

I've had a few reports of this, but I don't know the cause, Please see #1084 (comment), and post a response in that issue.

Where can I find the .yaml file?

You'll have to create it yourself by following the examples of the other recipes.

Also I tried ./build-wheel.py --python 3.8 --abi arm64-v8a ./packages/numpy

There's no need to build NumPy yourself. If your package needs it during the build, you can download it from the repository as the message says. But I suggest you start with a minimal meta.yaml file, and add requirements only when necessary.

@kmykman
Copy link
Author

kmykman commented Jan 26, 2024

Thanks. I just download ndk from web and unzip it in android-sdk.

@kmykman kmykman closed this as completed Jan 26, 2024
@jxx123
Copy link

jxx123 commented May 13, 2024

@kmykman Hi, I am also interested in using cvxpy in chaquopy. Have you successfully built it? Do you mind sharing more details on how you built it? Thanks!

@mhsmith mhsmith reopened this May 13, 2024
@kmykman
Copy link
Author

kmykman commented May 18, 2024

@jxx123 I have tried to build it but I didn't test it because my target is cvxopt but it failed (#1085 (comment)). You can try if it works. I have attached it.

cvxpy.zip

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

3 participants