apk with sqlite3 python3 kivy No module named '_sqlite3'

[allaboutmikey]

Ran into problems with making an APK, mostly due to incorrect software and api versions. The last of those (unable to find a valid dependency graph) was solved over in #1222

Now my APK compiles fine: and crashes on load. The problem is it can't find the _sqlite3 module. The latest pythonforandroid has a recipe for sqlite3 which seems to compile just fine. It produces a libsqlite3.so file which is included in my APK (checked), but this doesn't seem to be what python is looking for when the app loads on the device.
I am able to run the rest of the app if I remove the import and use of the sqlite3 module, but it is critical to most of the functionality in the final version that sqlite3 work.

I have tried removing from blacklist.txt (wherever I could find one, there were several) any mention of sqlite3 files: no change.
I have tried whitelisting sqlite3 files in buildozer.spec: no change.
I have tried completely removing the .buildozer directory and allowing it all to re-downlooad/install/compile: no change.
Love some help if anyone has ideas.

Edit:
Just found in MyProject/.buildozer/android/platform/build/build/other_builds/python3/armeabi-v71__ndk_target_21/python3/android_build/config.log a mention of sqlite3 that looks like:

configure:10267: checking for --enable-loadable-sqlite-extensions
configure:10277: result: no

Does this mean sqlite is not being built into python3? Not sure how I would change this as buildozer is doing all the p4a stuff for me.

Edit 2:
Further research tells me that option is not required to get sqlite3 working.

[allaboutmikey]

Trying again: I was looking at my logcat output...

01-03 10:02:44.047 15644 15644 V pythonutil: Loading library: sqlite3
01-03 10:02:44.049 15644 15644 V pythonutil: Loading library: SDL2
01-03 10:02:44.053 15644 15644 V pythonutil: Loading library: SDL2_image
01-03 10:02:44.055 15644 15644 V pythonutil: Loading library: SDL2_mixer
01-03 10:02:44.056 15644 15644 V pythonutil: Loading library: SDL2_ttf
01-03 10:02:44.059 15644 15644 V pythonutil: Loading library: python2.7
01-03 10:02:44.062 15644 15644 V pythonutil: Loading library: python3.5m
01-03 10:02:44.064 15644 15644 V pythonutil: Library loading error: dalvik.system.PathClassLoader[DexPathList[[zip file "/data/app/com.howefarms.stocktake-2/base.apk"],nativeLibraryDirectories=[/data/app/com.howefarms.stocktake-2/lib/arm, /data/app/com.howefarms.stocktake-2/base.apk!/lib/armeabi-v7a, /system/lib, /vendor/lib]]] couldn't find "libpython3.5m.so"
01-03 10:02:44.064 15644 15644 V pythonutil: Loading library: python3.6m
01-03 10:02:44.065 15644 15644 V pythonutil: Library loading error: dalvik.system.PathClassLoader[DexPathList[[zip file "/data/app/com.howefarms.stocktake-2/base.apk"],nativeLibraryDirectories=[/data/app/com.howefarms.stocktake-2/lib/arm, /data/app/com.howefarms.stocktake-2/base.apk!/lib/armeabi-v7a, /system/lib, /vendor/lib]]] couldn't find "libpython3.6m.so"
01-03 10:02:44.065 15644 15644 V pythonutil: Loading library: python3.7m
01-03 10:02:44.070 15644 15644 V pythonutil: Loading library: main
01-03 10:02:44.072 15644 15644 V pythonutil: Failed to load _io.so or unicodedata.so...but that's okay.
01-03 10:02:44.072 15644 15644 V pythonutil: Unsatisfied linker when loading ctypes
01-03 10:02:44.072 15644 15644 V pythonutil: Loaded everything!

Then, a bit further down...

01-03 10:02:44.510 15644 15685 I python  : Run user program, change dir and execute entrypoint
01-03 10:02:44.510 15644 15685 I python  : main.py
01-03 10:02:44.910 15644 15685 I python  : Traceback (most recent call last):
01-03 10:02:44.910 15644 15685 I python  :   File "main.py", line 2, in <module>
01-03 10:02:44.910 15644 15685 I python  :     import sqlite3
01-03 10:02:44.910 15644 15685 I python  :   File "/data/user/0/com.howefarms.stocktake/files/app/_python_bundle/stdlib.zip/sqlite3/__init__.py", line 23, in <module>
01-03 10:02:44.911 15644 15685 I python  :   File "/data/user/0/com.howefarms.stocktake/files/app/_python_bundle/stdlib.zip/sqlite3/dbapi2.py", line 27, in <module>
01-03 10:02:44.911 15644 15685 I python  : ModuleNotFoundError: No module named '_sqlite3'
01-03 10:02:44.924 15644 15685 I python  : Python for android ended.

I'm very confused now. Seems like it thinks sqlite3 is loaded, then won't import the module.
This seems like a bug in p4a to me, but I am happy if someone can tell me that I am just doing something wrong here?

[opacam]

The sqlite3 and openssl libs still are not python3's compatible in the current master branch. If you want to use them, you may wanna try the branch of the pr #1537, which enables python3 support for the mentioned libs.

[homdx]

Please check again. Now sqlite3 are supported with latest p4a, branch (master)

[ghost]

Note: to use sqlite put "sqlite3" into --requirements and it needs to go before "python3" in your requirements list (otherwise the core module for it won't be built)

[homdx]

Thank you, @Jonast for reccomendation.

Note: to use sqlite put "sqlite3" into --requirements and it needs to go before "python3" in your requirements list (otherwise the core module for it won't be built)

I checked today, and build apk with python3 with py3-api28ndk21 you tools with sqlite3 module https://github.com/JonasT/p4a-build-spaces
and build with p4a

[ApplySci]

Note: to use sqlite put "sqlite3" into --requirements and it needs to go before "python3" in your requirements list (otherwise the core module for it won't be built)

Oh, I didn't know that the order mattered at all. Is there some documentation on required order? Should python3 always come last, or does it need to be before kivy, or what?

[ghost]

@ApplySci everything needs to be before python3 which is needed to build the respective optional standard library modules (like sqlite3 for the standard library module for sqlite). Everything else like kivy, other dependencies, ... should go after such that python is available for it, although if the deps are set up right that might not actually break anything but it could if we ever messed that up

[hackalog]

That's odd. Doesn't p4a build a dependency graph out of that information, reordering things anyway?

[ghost]

@hackalog I don't think so, because currently sqlite3 is not a dependency. IMHO it should be (see #1576 ) but right now some of the non-vital core modules are only built when they happen to be around when the recipe is processed, and otherwise simply aren't

[inclement]

Closing as hopefully fixed by #1617, please feel free to reopen if it does still happen.