How to manually build native packages

[andreddosantos]

Hello guys i'm using nodejs-mobile-cordova and i'm facing an issue related to native packages,

When installing the sqlite3 package i want to install as well the SQLCipher extension.

Installing the SQLCipher extension

To install the SQLCipher extension you need to use node-pre-gyp in order to build the native package. And you can install it for node with following command

npm install sqlite3 --build-from-source --sqlite_libname=sqlcipher --sqlite=`brew --prefix`

If you don't provide extra flags this command generates the binding:

nodejs-project/node_modules/sqlite3/lib/binding/node-v67-darwin-x64/node_sqlite3.node

This should be it, to install the extension, but when building the cordova platform something goes wrong.

Building cordova android platform

When building the cordova platform

cordova run android --device

it realizes that sqlite native package needs to be build and calls the install script from package.json

node-pre-gyp install --fallback-to-build

this --fallback-to-build flag according to the documentation should:

--fallback-to-build: fallback to building from source if pre-built binary is not available

Since the binaries are already there from the first step this command should ignore the build, but it doesn't, so it realizes that the actual binding does not exist and compiles again generating this time a new binding:

"nodejs-project/node_modules/sqlite3/lib/binding/node_abi-platform-arch/node_sqlite3.node"

This new binding does not include the SQLCipher extension...

My question is how can i generate manually the proper binding for this platform??
I can use the "NODEJS_MOBILE_BUILD_NATIVE_MODULES_VALUE" file as 0 and generate it manually with the extension.

Can someone help?
Regards André dos Santos

[jaimecbernardo]

Hi @andreddosantos,

The build process for the native modules runs npm rebuild --build-from-source , with different environment variables for each architecture: https://github.com/janeasystems/nodejs-mobile-cordova/blob/4a79e96b8f313296fb050ecc413f053d1ac7f5c8/src/android/build.gradle#L211

You could try adding your specific flags for sqlite3 in that line.

Then it copies the resulting binaries to a native assets folder for each architecture: https://github.com/janeasystems/nodejs-mobile-cordova/blob/4a79e96b8f313296fb050ecc413f053d1ac7f5c8/src/android/build.gradle#L246

It then generates lists of what to copy at runtime for each architecture
https://github.com/janeasystems/nodejs-mobile-cordova/blob/4a79e96b8f313296fb050ecc413f053d1ac7f5c8/src/android/build.gradle#L251

It should be possible to prebuild the modules and then add a similar assets folder to your application so they can be used. The ones added from the current method should be found in platforms/android/build/nodejs-native-assets/
https://github.com/janeasystems/nodejs-mobile-cordova/blob/4a79e96b8f313296fb050ecc413f053d1ac7f5c8/src/android/build.gradle#L283

Looking at the instructions you've given for the SQLCipher extension, it seems to depend on something installed by brew on your local machine, possibly built for MacOS, so it may require additional steps in order to make it work.

[andreddosantos]

Hello @jaimecbernardo thanks again for your support, you've pointed me into the right direction, now the builds are definitely considering the flags.

commandLine npmCommandName, '--verbose', 'rebuild', '--build-from-source', '--sqlite_libname=sqlcipher', '--sqlite=/usr/local/'  

When running the command manually:

npm rebuild --build-from-source --sqlite_libname=sqlcipher --sqlite=/usr/local

it all works well generating the "binding/node-v67-darwin-x64/node_sqlite3.node" binding, however when called by the build it breaks with the following error log.

This seems to be something related to environment settings, but its strange that it manages to compile when running the command manually.

I've spent some considerable hours trying to fix the new error still remais, but at least i know the compilation of the package is now properly invoked.

[andreddosantos]

Manage to solve the "'sqlite3.h' file not found" by adding these 2 flags in the build file as well:

environment ('CPPFLAGS',"-I/usr/local/include -I/usr/local/include/sqlcipher")
environment ('CXXFLAGS',"-I/usr/local/include -I/usr/local/include/sqlcipher")

Now i have a new error :

23 warnings generated.
.../platforms/android/build/standalone-toolchains/arm-linux-androideabi/bin/arm-linux-androideabi-clang++ -shared -rdynamic -fPIC -Wl,-soname=node_sqlite3.node -o Release/obj.target/node_sqlite3.node -Wl,--start-group Release/obj.target/node_sqlite3/src/database.o Release/obj.target/node_sqlite3/src/node_sqlite3.o Release/obj.target/node_sqlite3/src/statement.o -Wl,--end-group -L/usr/local/include/lib -lsqlcipher -llog .../platforms/android/src/com/janeasystems/cdvnodejsmobile/jni/libnode//bin/armeabi-v7a/libnode.so
.../platforms/android/build/standalone-toolchains/arm-linux-androideabi/bin/../lib/gcc/arm-linux-androideabi/4.9.x/../../../../arm-linux-androideabi/bin/ld: error: cannot find -lsqlcipher
clang50++: error: linker command failed with exit code 1 (use -v to see invocation)
make: *** [Release/obj.target/node_sqlite3.node] Error 1

[andreddosantos]

Managed to fix the previous one by providing the proper directory 'usr/local'

And now i have a new error ...

platforms/android/build/standalone-toolchains/arm-linux-androideabi/bin/../lib/gcc/arm-linux-androideabi/4.9.x/../../../../arm-linux-androideabi/bin/ld: error: sqlcipher: no archive symbol table (run ranlib)

[jaimecbernardo]

Hi @andreddosantos ,

The most likely reason for the error you're getting now is that the sqlarchive symbols file the build system is trying to use from the /usr/local location only has symbols for your host machine's architecture (Darwin x86_64), while it's trying to link to Android arm. That's why it's saying it has no archive symbol table.

You'd need a sqlcipher file that's built for Android as well.

[andreddosantos]

Hello @jaimecbernardo, thank you very much for your support again ;)

Do you have any ideia of how to do it?

You'd need a sqlcipher file that's built for Android as well.

I've found this "SQLCipher for Android Application Integration", but it wouldn't be available to the nodejs layer... that would change all my architecture and i really like it as is.

IONIC_app(FE) <-consults local api-> nodejs_cordova_plugin(BE) running express(using a local api that can access the sqlite3 package and by using sqlite3 through the nodejs layer allows me to have full blob support.

[jaimecbernardo]

No idea on how to do it. This is very specific to this module.
The best place to start would be to see if it's possible and how to build it for Android in the project's main repo: https://github.com/sqlcipher/sqlcipher

[andreddosantos]

Using a local API structure based on the nodejs-mobile cordova plugin, i haven't been able to find a solution for performant database encryption with node-packages. Not being able to build the node-sqlite3 with the sqlcipher extension has forced me to use sqlcipher cordova plugins leaving the local api architecture. :(