Data Marshalling: Support for typed arrays

[brianchirls]

It would be helpful to be able to pass typed arrays (e.g. Float32Array, etc.) between JavaScript and Java. It seems it would be necessary for things like OpenGL or any other APIs that use blocks of binary data.

If possible, it's important that these arrays be passed by reference, not copied, since typed arrays are very slow to allocate in JS, and frequent GC on them could be a problem (see related #33).

[slavchev]

We probably can use something similar to (or reuse) DirectBuffer.cpp. For example to call the following Java method

void method1(int[] arr)

we have to use NewDirectByteBuffer then call asIntBuffer and then array() and then pass the array object to CallVoidMethodA.

[blagoev]

Thats more or less how I plan to do it. Same goes for methods returning primitive arrays. Are there any concerns we can't handle certain scenarios

[blagoev]

Unfortunately there is no viable way of transferring typed arrays without copying the data.

The problems that exists are:

• There is no way to create an array to a direct byte buffer. Java simply wants to be able to move the position of the array during GC. Getting the backing array (when and if it exists) of the direct bytebuffer is not supported
• There is no way to ping an array and get a pointer to it without the risk of coping the array to a new one. GetArrayElements simply do not guarantee this.
• There is no way to leverage GetPrimitiveArrayCritical since it holds a critical section, blocks the GC, disables JNI calls and implies a quick ReleasePrimitiveArrayCritical.

What we can do is given the required copy of the data to allow the argument conversion, but clear and lock the js typed array from being used from again. (Similar to how web workers do). This is a huge limitation which will hurt both performance and user code.

We will update this issue with more details later on.

blagoev

[slavchev]

I see this issue as a related to the one about providing better syntax for Java arrays (see #70). In particular for Java arrays of primitive type (boolean, char, byte, short, int, long, float and double). For example, let's have a Java method that has int[] parameter.

void method1(int[] arr)

At present we can call method1 as follows

var length = 5;
var arr = java.lang.reflect.Array.newInstance(java.lang.Integer.class.getField("TYPE").get(null), length);
o.method1(arr);

It makes sense to provide better syntax in NativeScript for this kind of scenarios. We already convert most native JavaScript types to the corresponding Java ones (e.g. number to int). So, in this case it is a matter of extension to convert JavaScript typed arrays to the corresponding Java ones.

var length = 5;
var arr = new Int32Array(length);
o.method1(arr);

The opposite type conversion, Java-to-JavaScript, cannot be handled that easily. Currently, NativeScript converts Java arrays (of primitive type) to JavaScript proxies instead to typed arrays. So far it is a safe approach though the performance can be sub-optimal. I think we should be careful with this feature as it creates danger for OOM and heap fragmentation. I guess, we can keep the current approach and provide a JavaScript function, say convertToTypedArray, that does the job.

[blagoev]

On the JS-to-Java convert:
The issues is that the moment we allow passing types arrays to Java we are creating a copy of the TypedArray on the Java side. One can imagine a scenario where you want to pass a typed array to a method, on the JS side change some values and then rely that the Java side can successfully read the same values.

//JavaScript code
var length = 5;
var arr = new Int32Array(length);
o.method1(arr);
arr[0] = 5;

//Java code o.getElement(int[] arr, index)
int value = o.getElement(arr, 0);

value does not equal 5

The same can be the expectation for passing a Typed Array to java (for sorting for example) and trying reading the sorted values from JavaScript.

Again the issue comes from the fact that we can not "successfully" share the same array between Java and JavaScript. We can try but it could be broken on certain devices or even different moments in time on the same device, depending on the memory state.

[blagoev]

The better syntax for arrays has its own issue here #70 and it's better to differentiate them. Arrays can contain any values not just primitive types, hence the different issues logged.

[slavchev]

One can imagine a scenario where you want to pass a typed array to a method, on the JS side change some values and then rely that the Java side can successfully read the same values.

That's exactly how ObjectManager.cpp and Platform.java communicate. They use shared memory (ByteBuffer which is created in DirectBuffer.cpp). We can easily set m_data to point to the typed array memory.

[blagoev]

The issue is not the JavaScript side. The problem is on the Java side. We can not create an array to an exact native memory location. Java does not allow this. If it allows it (*) we could use arrays in DirectBuffer.cpp and not a direct buffer itself.

*It actually allows it through GetPrimitiveArrayCritical. But its use is so limited as I have stated above.

So let me try to describe the issue again: One can not create an Java primitive array from a native pointer. The other approach for taking a pointer to Java array and creating a JS Typed Array (during the JS Type array creation) around the Java array, is not possible either again due to limitations of Java VM. This is the reason why the DirectBuffer->asIntArray()->array() method throws NotSupportoedExecption.
It boils down to this http://androidxref.com/6.0.0_r1/xref/libcore/luni/src/main/java/java/nio/ByteBufferAsIntBuffer.java#127.

So we need to copy the data hence the performance and the issues above.

In the case of our current approach we are safe since we create a JS proxy representation of the Java array which is proxying get/set calls to the real instance and that's legitimate.

[slavchev]

Indeed. Here is an excerpt from the Android documentation from my previous comment #65 (comment)

Returns the array that backs this buffer (optional operation). The returned value is the actual array, not a copy, so modifications to the array write through to the buffer.

Since this is an optional operation there will always be a copy.

[blagoev]

It is optional because there might not be a backing array for this buffer. It actually has a method to test if there is an array hasArray(). This optionality does not provide a array copy of the buffer but throws exceptions.

In early versions of Android the DirectBuffers were not backed by an actual byte array and hasArray() had a bug that throws exception instead of returning true/false.

Well for byte arrays we can make this work because DirectBuffer->array() does not throw exceptions but that's so limited since most APIs that we want to cover are Int and Float Typed arrays.

The coping of the array I am referring comes from the fact that the only viable option is to use GetArrayElements which fails to guarantee anything.

Java insists to want to be able to move arrays on GC and unlike .NET where you can pin the array in memory, here you can pin it but GC wont run until you unpin. Disaster!

[atanasovg]

ping @slavchev Make a pull request with the feature you've implemented.

[Tyg-g]

To the copying problem. Quite old topic but nobody mentioned this possibility.

You can create an array directly in Java with var myArray = Array.create("int", 100); In this way you get an object in JavaScript which is a reference to a Java array, but behaves like an array and it accesses the elements of the Java array. You can reuse this object many times and the data is never copied, if I understand well.

More info: Java to JavaScript: Array