[CORE UPDATE - PART IV] Fix cryptography+cffi #1545

opacam · 2018-12-22T10:29:48Z

This is one more part for the pr #1537 (discussed previously in #1460 and #1534)

Here I fix recipes cryptography and cffi (and touch some others which those depend on) to make it work with the core-update pull request, which also contains a new version of the openssl libs created by @tito. I tried to update the most of the packages to latest version and grants compatibility for both versions of python.

Note: this has to be merged only when the core-update has been merged because it depends on the new python2 recipe and the openssl libs (1.1)

Since the recent openssl upgrade this recipe stopped to work. Here we adapt the recipe to the new build circumstances and we also fix the cffi recipe because it is a direct dependency of cryptography and had the wrong flags for the new python2 build system.

In order to not have troubles with python3 we update most of the dependant recipes to the latest available versions.

ghost · 2019-01-02T15:21:36Z

This is another one that is quite fundamentally relevant for everyone, isn't it? Doesn't ctypes require cffi? Once this is in I can do the ctypes.util.find_library() patch on top, and finally all the core things will work properly out of the box! 😍

ghost · 2019-01-02T15:24:07Z

pythonforandroid/recipes/cffi/__init__.py

-            # https://github.com/kivy/python-for-android/pull/1250/files#diff-569e13021e33ced8b54385f55b49cbe6
-            env['CFLAGS'] += ' -I{}/include/python/'.format(ndk_dir_python)
+        env['LDFLAGS'] += ' -L{}'.format(self.ctx.python_recipe.link_root(arch.arch))
+        env['LDFLAGS'] += ' -lpython{}'.format(self.ctx.python_recipe.major_minor_version_string)


Hm does libffi actually depend on python? The old recipe doesn't seem to have done that other than for crystax, it just leaves me wondering why it is required? Or is libffi a python-specific lib

(I'm just curious, not saying it's necessarily wrong or anything!)

@opacam got any input on this particular question?

That's a pretty valid concern @Jonast and definitely deserves some investigations. In the meantime this PR is already improving the current state while not bringing any regression to me.
I think we could merge and eventually address that later

ghost · 2019-01-02T15:25:12Z

pythonforandroid/recipes/cryptography/__init__.py

-                          ' -lcrypto' + r.version
+
+        openssl_recipe = Recipe.get_recipe('openssl', self.ctx)
+        env['CFLAGS'] += openssl_recipe.include_flags(arch)


That seems clever, I like it! 👍

ghost · 2019-01-10T18:36:32Z

Edit: I previously wrote here that it doesn't work for me but my env is probably wrong and it works for others, please ignore

ghost · 2019-01-10T20:20:58Z

I tried adding libffi as depends for both python3 and setuptools, and bumping setuptools to 40.6.3, but nothing helped, sadly. Still stuck with the same error

misl6 · 2019-01-10T20:37:53Z

Worked for me. (on top of the current master).

Just added libffi as first requirement in buildozer.
requirements = libffi, openssl, python3, kivy==master, android, cython, pyjnius, pycryptodome

…nto python-core-cryptography-cffi

ghost · 2019-01-10T21:14:59Z

Ok, I spotted this output: [DEBUG]: checking for --with-system-ffi... yes - but still no _ctypes module. I will try enabling openssl now and see if that is connected (which I think it shouldn't be)

ghost · 2019-01-10T21:49:16Z

--requirements I used: libffi,openssl,python3,cffi,sdl2,pyjnius,android,Pillow,/my-app/.android-build/code-lib-copy/

In the hostpython configure output I'm spotting: [DEBUG]: checking for --with-system-ffi... yes

However, I still get above error and poked around in the build folder afterwards:

# /root/.local/share/python-for-android/build/other_builds/hostpython3/desktop/hostpython3/native-build/python
Python 3.7.1 (default, Jan 10 2019, 21:28:13) 
[GCC 7.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import ctypes
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/root/.local/share/python-for-android/build/other_builds/hostpython3/desktop/hostpython3/Lib/ctypes/__init__.py", line 7, in <module>
    from _ctypes import Union, Structure, Array
ModuleNotFoundError: No module named '_ctypes'
>>> import ssl
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/root/.local/share/python-for-android/build/other_builds/hostpython3/desktop/hostpython3/Lib/ssl.py", line 98, in <module>
    import _ssl             # if we can't import it, let the error propagate
ModuleNotFoundError: No module named '_ssl'
>>>

I don't know the exact folder layout very well, but I'd also expect _ctypes.so in here:

# ls /root/.local/share/python-for-android/build/other_builds/hostpython3/desktop/hostpython3/native-build/Modules/
Setup            _collectionsmodule.o  _math.o      _threadmodule.o  bufferedio.o  errnomodule.o   getbuildinfo.o  itertoolsmodule.o  pwdmodule.o       textio.o
Setup.local      _functoolsmodule.o    _operator.o  _tracemalloc.o   bytesio.o     faulthandler.o  getpath.o       ld_so_aix          signalmodule.o    timemodule.o
_abc.o           _iomodule.o           _sre.o       _weakref.o       config.c      fileio.o        hashtable.o     main.o             stringio.o        xxsubtype.o
_codecsmodule.o  _localemodule.o       _stat.o      atexitmodule.o   config.o      gcmodule.o      iobase.o        posixmodule.o      symtablemodule.o  zipimport.o

... and there's nothing like that in the entire folder structure:

# find /root/.local/share/python-for-android/build/other_builds/hostpython3/ | grep 'ctypes.so'
#

ghost · 2019-01-10T21:51:53Z

Ah lol. I don't have libffi-dev on the system itself, and given this is hostpython that's probably it, right? I'll install it and try again. 🤦‍♀️

opacam · 2019-01-10T22:24:50Z

Yes, you need libffi-dev for hostpython3

ghost · 2019-01-10T22:25:54Z

@opacam installed it, still not working 🤷‍♀️ I think I'll give up for today, something is missing but I don't know how to find out what it is... (probably another dependency on the host) anyway, shouldn't be a fault of this pull request since it works for @misl6

opacam · 2019-01-10T22:28:22Z

Can you give me your list of requeriments, the one you pass to p4a to build your app?

opacam · 2019-01-10T22:30:14Z

Aaa...I see, it's above...

ghost · 2019-01-10T22:31:02Z

I just realized I installed libffi-dev in the wrong environment 😂 hang on. I am testing too many things today I'm confusing myself 🤦‍♀️ lol

opacam · 2019-01-10T22:41:54Z

No problem, let me know if you need help, today I'm going to bed 😴
...when those things happen it's a signal 😂
🌛

ghost · 2019-01-10T23:46:01Z

👍 things also built for me now (with python3/non-crystax recipe) so that's good!

Because this is how you are supposed to do it, you must use LDFLAGS for linker flags and LDLIBS (or the equivalent LOADLIBES) for the libraries

opacam added 2 commits December 22, 2018 11:08

Update cryptography recipe and grants python3 support

e1bb75f

In order to not have troubles with python3 we update most of the dependant recipes to the latest available versions.

ghost reviewed Jan 6, 2019

View reviewed changes

opacam mentioned this pull request Jan 8, 2019

[CORE UPDATE - PART I] Refactor python recipes + openssl + sqlite3 #1537

Merged

6 tasks

opacam added 2 commits January 10, 2019 21:45

Merge branch 'master' of https://github.com/kivy/python-for-android i…

05fea3b

…nto python-core-cryptography-cffi

Fix dependency graph error

ee9c61c

opacam changed the title ~~[WIP][CORE UPDATE - PART IV] Fix cryptography+cffi~~ [CORE UPDATE - PART IV] Fix cryptography+cffi Jan 11, 2019

Move libraries from LDFLAGS to LIBS for cryptography recipe

84b01f0

Because this is how you are supposed to do it, you must use LDFLAGS for linker flags and LDLIBS (or the equivalent LOADLIBES) for the libraries

AndreMiras approved these changes Jan 12, 2019

View reviewed changes

AndreMiras merged commit ac76dee into kivy:master Jan 12, 2019

AndreMiras mentioned this pull request Feb 10, 2019

lxml and requests recipe: build interrupted, "_ctype" error #1700

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[CORE UPDATE - PART IV] Fix cryptography+cffi #1545

[CORE UPDATE - PART IV] Fix cryptography+cffi #1545

opacam commented Dec 22, 2018

ghost commented Jan 2, 2019

ghost Jan 2, 2019

ghost Jan 10, 2019

AndreMiras Jan 12, 2019

ghost Jan 12, 2019

ghost Jan 2, 2019

ghost commented Jan 10, 2019 •

edited by ghost

Loading

ghost commented Jan 10, 2019

misl6 commented Jan 10, 2019

ghost commented Jan 10, 2019

ghost commented Jan 10, 2019

ghost commented Jan 10, 2019

opacam commented Jan 10, 2019

ghost commented Jan 10, 2019 •

edited by ghost

Loading

opacam commented Jan 10, 2019

opacam commented Jan 10, 2019

ghost commented Jan 10, 2019

opacam commented Jan 10, 2019

ghost commented Jan 10, 2019 •

edited by ghost

Loading

[CORE UPDATE - PART IV] Fix cryptography+cffi #1545

[CORE UPDATE - PART IV] Fix cryptography+cffi #1545

Conversation

opacam commented Dec 22, 2018

ghost commented Jan 2, 2019

ghost Jan 2, 2019

Choose a reason for hiding this comment

ghost Jan 10, 2019

Choose a reason for hiding this comment

AndreMiras Jan 12, 2019

Choose a reason for hiding this comment

ghost Jan 12, 2019

Choose a reason for hiding this comment

ghost Jan 2, 2019

Choose a reason for hiding this comment

ghost commented Jan 10, 2019 • edited by ghost Loading

ghost commented Jan 10, 2019

misl6 commented Jan 10, 2019

ghost commented Jan 10, 2019

ghost commented Jan 10, 2019

ghost commented Jan 10, 2019

opacam commented Jan 10, 2019

ghost commented Jan 10, 2019 • edited by ghost Loading

opacam commented Jan 10, 2019

opacam commented Jan 10, 2019

ghost commented Jan 10, 2019

opacam commented Jan 10, 2019

ghost commented Jan 10, 2019 • edited by ghost Loading

ghost commented Jan 10, 2019 •

edited by ghost

Loading

ghost commented Jan 10, 2019 •

edited by ghost

Loading

ghost commented Jan 10, 2019 •

edited by ghost

Loading