NodeJS 0.11 Port to Android

[joshmarinacci]

Opening a new issue to discuss build problems with the new 0.11 port.

Following the instructions here:

https://github.com/paddybyers/node/wiki/Building-v0.11-for-Android

I get a compile error.

➜ v8 git:(remotes/origin/v0.11-android) make -j8 GYPFLAGS="-Dcomponent=shared_library" android_arm.debug
Makefile.android:70: *** Cannot find Android toolchain in "/Users/josh/projects/android-sdk-macosx/ndk/toolchains/arm-linux-androideabi-4.4.3/prebuilt/darwin-x86//toolchains/arm-linux-androideabi-4.6/prebuilt/darwin-x86". Stop.
make: *** [android_arm.debug] Error 2

My install has 4.4.3 instead of 4.6. This implies a newer NDK is required.

[paddybyers]

Yes, I think you need r8b or newer, and build with the 4.6 abi.

[joshmarinacci]

I get an error that it can't find the right ndk arch for macos x86_64. This fixes it.

diff --git a/deps/v8/Makefile.android b/deps/v8/Makefile.android
index aeff01c..4df09eb 100644
--- a/deps/v8/Makefile.android
+++ b/deps/v8/Makefile.android
@@ -39,7 +39,7 @@ ifeq ($(HOST_OS), linux)
   TOOLCHAIN_DIR = linux-x86
 else
   ifeq ($(HOST_OS), mac)
-    TOOLCHAIN_DIR = darwin-x86
+    TOOLCHAIN_DIR = darwin-x86_64
   else
     $(error Host platform "${HOST_OS}" is not supported)
   endif

Of course that would break the non-64bit build then. I'm not sure the correct makefileish way to fix it.

After that the build succeeds but the node executable won't run on my rooted device. It says:

/data/node # ./node
link_image[1891]:  2109 could not load needed library 'libv8.so' for './node' (load_library[1093]: Library 'libv8.so' not found)CANNOT LINK EXECUTABLE

I've put libv8.so in the same dir as node and also in /system/usr/lib. 'lib' didn't exist BTW so I had to create it. Also the fs is readonly so I had to do 'adb remount' first to copy the files over.

Any ideas how to fix the linking?

[joshmarinacci]

Looks like the libs should go into /system/lib instead of /system/usr/lib

[joshmarinacci]

Whoo hoo!:

/data/node # cat test.js
console.log("Greetings Earthling!\n");
/data/node # ./node test.js
Greetings Earthling!

/data/node #

[paddybyers]

Fixed the system/lib reference in the wiki, thanks.

[sequoiar]

very cool.

[gentijo]

Help, I am getting an error compiling with NDK 8e, V8 compiles fine but when I go back to the node directory, it complains about not finding the correct module in the V8 directory.. This is on a CentOS 64 bit with the X86 NDK installed.

[root@localhost node]# cd deps/v8/
[root@localhost v8]# make -j8 GYPFLAGS="-Dcomponent=shared_library" android_arm.debug
make[1]: Entering directory /usr/local/src/ARM/node/deps/v8' make[2]: Entering directory/usr/local/src/ARM/node/deps/v8/out'
make[2]: Nothing to be done for all'. make[2]: Leaving directory/usr/local/src/ARM/node/deps/v8/out'
make[1]: Leaving directory `/usr/local/src/ARM/node/deps/v8'
[root@localhost v8]# cd ../..
[root@localhost node]# NDK_MODULE_PATH=.:..
[root@localhost node]# export NDK_MODULE_PATH
[root@localhost node]# ndk-build NDK_PROJECT_PATH=. NDK_APPLICATION_MK=Application.mk V=1
Android NDK: deps/v8/Android.mk: Cannot find module with tag 'pty' in import path
Android NDK: Are you sure your NDK_MODULE_PATH variable is properly defined ?
Android NDK: The following directories were searched:
Android NDK:
Android.exe.mk:41: *** Android NDK: Aborting. . Stop.

[paddybyers]

OK, you're missing a module; I forgot to add that to the instructions. I've updated them - let me know how you get on.

[joshmarinacci]

Now that I have node itself working I want to create a native addon. After reading the wiki I complied the anode hello addon per the instructions and copied it to my device but I can't get it to run. It says the module can't be found. A few questions:

• how do I call the module from js? require('hello') and require('/data/data/org.meshpoint.anode/app/module/hello') don't work.
• is the node name of the module really 'hello'? While the build script does produce a hello.node file the hello.cc file doesn't have a NODE_MODULE(addonname, Init) line that the official NodeJS docs describe for how to create a native addon.
• are the jni and bridge.node files require for all native addons, or only if I want to call them from Java code?
• is /data/data/org... the right path or should there be only one 'data' in there?

Thanks. This is an awesome project.

[paddybyers]

After reading the wiki I complied the anode hello addon per the instructions and copied it to my device but I can't get it to run. It says the module can't be found.

Yes, because that addon makefile expects to be building for the anode implementation, which would be linking against libjninode.so. In this case we just have node, so it needs to link against that.

Briefly, you should be able to get it to work by making two changes.

Remove the reference to libjninode.so, and add explicit reference to v8:

    LOCAL_LDLIBS := \
        $(NODE_ROOT)/libs/armeabi/libcrypto.so \
        $(NODE_ROOT)/libs/armeabi/libssl.so \
        -lv8 \
        -llog

Allow the addon to link despite not having the required external symbols available:

    LOCAL_ALLOW_UNDEFINED_SYMBOLS := true

The environment variable NODE_ROOT points to the node directory. You should be able to define this in the Application.mk instead if you prefer.

I haven't tried with this node version yet, but this should work. There's potentially an issue with the external visibility of the node symbols but lets see how far we get with this.

is the node name of the module really 'hello'?

Yes. You will do require('hello'), or require('/path/to/hello'), depending on whether the module is in your NODE_PATH or not. node adds the various extensions it wants to try for itself.

While the build script does produce a hello.node file the hello.cc file doesn't have a NODE_MODULE(addonname, Init) line that the official NodeJS docs describe for how to create a native addon.

Yes, sometime around the 0.4 to 0.6 transition, they introduced a new way of declaring the init entrypoint. Node still, to my knowledge, supports both behaviours; you can either just use Init, or you can declare that (or any other) name as the entrypoint. With no NODE_MODULE it will still work but the recommended approach I guess would be to use NODE_MODULE.

are the jni and bridge.node files require for all native addons, or only if I want to call them from Java code?

bridge.node is only needed if you want to write addons in Java. Note of this will work yet on 0.11 because I've not migrated any of that stuff across yet.

The jni stuff serves two purposes - it is involved in getting bridge.node to work, but also provides the jni binding that allows the anode app to load node as a library. Again, this piece isn't migrated to 0.11 yet. For now just ignore both of these.

is /data/data/org... the right path or should there be only one 'data' in there?

Yes, it really is /data/data/....

[joshmarinacci]

Wonderful. I had to add the full path to libv8.so and now it works. Interestingly didn't need to turn off UNDEFINED_SYMBOLS actually.

Yes, it appears FunctionTemplate was added sometime in the 0.6x timeframe, which was causing the issues. Now I'm building only against the 0.11 node branch and I can get a commandline app to work with native modules. Next up, linking to OpenGL. :)

[paddybyers]

Next up, linking to OpenGL

You've seen this?

https://github.com/paddybyers/node-gl/commits/nodegl

I've not tried it on Android. It currently uses GLUT/GLU but I guess not a lot of work to get it to go to EGL. I heard someone got that working but I don't think their code is on github.

[gentijo]

I checked out a copy of openssl, but when I try to run ndk-build
I get the following error

root@localhost openssl-android]# ndk-build
/opt/android-ndk-r8e/build/gmsl/__gmsl:512: *** non-numeric second argument to `wordlist' function: ''. Stop.

below is the code at line 512 of _gmsl.
I am running android-ndk-r8e-linux-x86 of the NDK

Thank you for the quick response.
-John Gentilin

_gmsl @ line 512

---

Function: int_encode
Arguments: 1: A number in human-readable integer form
Returns: Returns the integer encoded as a string of x's

---

int_encode = $(__gmsl_tr1)$(wordlist 1,$1,$(__gmsl_input_int))

[gentijo]

Is there a reason that the AndroidManifest.xml is empty in the openssl project ?
I did some searching around and found this thread
https://groups.google.com/forum/?fromgroups=#!topic/android-ndk/b4DSxE1NAS0
most people suggest that a mod to the __gmsl file but one person suggested that the AndroidManafest.xml
file needs the minSdkVersion so I grabbed a version from hello-jni in the samples directory, then removed
the hello-jni specific info. This seemed to work, the compile ran and only gave me one warning in the beginning.
Here is the content of the manifest file I used

<?xml version="1.0" encoding="utf-8"?>
<manifest xmlns:android="http://schemas.android.com/apk/res/android"
      android:versionCode="1"
      android:versionName="1.0">
    <uses-sdk android:minSdkVersion="8" />
    <application android:label="@string/app_name"
                 android:debuggable="true">
    </application>
</manifest>

The warning I got

Compile thumb  : crypto <= cryptlib.c                                                                 
 /usr/local/src/ARM/openssl-android/crypto/cryptlib.c: In function 'CRYPTO_THREADID_current':          
 /usr/local/src/ARM/openssl-android/crypto/cryptlib.c:503:2: warning: passing argument 2 of  
 'CRYPTO_THREADID_set_pointer' discards 'volatile' qualifier from pointer target type 
 [enabled by  default]                                                
 /usr/local/src/ARM/openssl-android/crypto/cryptlib.c:431:6: note: expected 'void *' but 
 argument is of type 'int volatile *'                                                                                                                        

[paddybyers]

Is there a reason that the AndroidManifest.xml is empty in the openssl project ?

Simply that, historically, it wasn't needed to build a pure native library with the NDK.

I've rebased my changes to the latest version of the original project (https://github.com/guardianproject/openssl-android). Someone fixed the same bug there just a week or so ago.

[gentijo]

Paddy, thank you.. I will update my project.

Great projec BTW, t I am having fun hacking node on my cubieboard.

Will anode run on a non rooted device since it runs as an APK ?

[paddybyers]

Will anode run on a non rooted device since it runs as an APK ?

Yes.

[Mithgol]

• Is it planned to start releasing pre-built versions of such APKs eventually, so that the requirement to build them is lifted from the end users?
• Could that anode APK be automatically registered as a handler for .js files, in order for such files to run (when they're tapped by a finger) in NodeJS console?

[paddybyers]

Is it planned to start releasing pre-built versions of such APKs eventually, so that the requirement to build them is lifted from the end users?

Yes - this used to happen with the 0.6 version but github removed the downloads feature. Someone else has also talked about investing some effort to get the runtime onto Play.

Could that anode APK be automatically registered as a handler for .js files, in order for such files to run (when they're tapped by a finger) in NodeJS console?

Yes, on Android you could register the app to handle filetypes or MIME types by default - if there is more than one handler registered for a given type, then the platform asks the user which one to use.

[Mithgol]

this used to happen with the 0.6 version but github removed the downloads feature. Someone else has also talked about investing some effort to get the runtime onto Play.

There's yet another interesting workaround: you could have also started making .torrent for the most recent APK, pushing the torrent to the repository each time after an APK is built, and then leaving your favourite torrent client running online for about a day or two (in order to seed the APK initially; and after a couple of days, I guess, the first interested end users could become seeders for the rest of them).

Actually I dunno if GitHub accepts torrents in repositories, but that seems worth trying. Might be less effort than getting the runtime onto Google Play.

[asalah]

Hi Paddy,
I followed all the steps of compiling the addons, and finally got the hello.node file. I pushed it to the directory: /data/data/org.meshpoint.anode/app/module
and tried requiring it from the js code like: require('hello'), require('/data/data/org.meshpoint.anode/app/module/hello') and require('/data/data/org.meshpoint.anode/app/module/hello.node'), and it unfortunately didn't work.

I also tried putting the file hello.node beside the js file and requiring it like: require('./hello.node') and require('./hello') .

to be more clear all the attempts above gave me an error of can't load the module except the last one, it didn't give any error but on the same time it didn't execute the function hello() (that exists in hello.cc).

Any help will be highly appreciated.
Best Regards
-Abdulhafeth Salah

[feichh]

have you tried putting it into /data/data/org.meshpoint.anode/node_modules/ ?
maybe try putting hello.node into the assets dir of the anode app.

cheers

[feichh]

@paddybyers did you see this: nodejs#5514 ?

[paddybyers]

@feichh no, I hadn't seen that, thanks. I'll have a look at how it compares; I suspect not much different. It's good if we now have the changes landed upstream.

[linusmartensson]

@paddybyers Since I noticed the reference was added and looked over this, I can give a short rundown on my implementation strategy:

Rather than setting up Android.mk files - which have to be maintained separately - I went for the standalone toolchain approach. Documentation is available under docs/standalone-toolchain.html in the NDK. (Random version on the net at https://github.com/flyskywhy/android-ndk-r7b/blob/master/docs/STANDALONE-TOOLCHAIN.html)

In short, it provides you with a build environment for android cross-compilation using other toolchains than ndk-build. Based on that, I setup what you normally use for cross-compilation, i.e. exports for CC, LD, AR, etc. and ensure gyp is aware that OS=="android" rather than "linux".

To supplement this, I included a configuration script which given an NDK path can setup the cross-compilation environment and configure gyp automatically.

From there onwards, the gyp build scripts are used almost as-is, with some configuration details and compatibility issue resolutions added in uv, cares & node to resolve build errors. You see these referenced in the PR.

Unless especially build-breaking features are added (such as advanced pthread functionalities that aren't available in android) these patches should be forward-compatible with little maintenance as Node develops.

On the flipside, I haven't integrated existing shared objects from android, instead opting to push a statically linked node executable sized at 10MB. Note that this will most likely provide for better binary compatibility over multiple devices, since e.g. libssl and friends are in my experience fragile when it comes to binary compatibility between versions (which will likely cause issues when you want an apk running on G-J releases of android.)

This also means some differences when including the project in an apk - I'd probably build it separately and include it in a project as a prebuilt binary - but this is of course heavily project dependent.