Need efficient access to PoolIntArray, PoolVector3Array, etc.

[garyo]

For creating/manipulating/exporting meshes, it's important to have efficient access to the Pool*Arrays. It's not trivial, because they are locked to prevent multithreading issues. Godot-python would need to lock the array and get the raw pointer using array.read() to get the lock, reader.ptr() to get the raw data pointer (which can be turned into a python buffer, numpy array etc.), keeping the array.read() object around until done. Unlocking would have to be explicit in python since objects don't always get deleted until gc, something like a python-level reader.free() that would delete the underlying gdnative object.

[touilleMan]

I'm not sure this is something present in the current GDnative API.
Obviously if not we can propose an evolution on the API :-)

Regarding the syntax to get the lock from Python, I would recommend using the with syntax with a context manager:

with my_array.lock:
    ptr = my_array.read()
    ...

[garyo]

I like the context manager approach. Actually I believe everything needed is in GDnative. From gdnative/pool_arrays.h:

godot_pool_vector3_array_read_access GDAPI *godot_pool_vector3_array_read(const godot_pool_vector3_array *p_self);
const godot_vector3 GDAPI *godot_pool_vector3_array_read_access_ptr(const godot_pool_vector3_array_read_access *p_read);
void GDAPI godot_pool_vector3_array_read_access_destroy(godot_pool_vector3_array_read_access *p_read);

What do you think? Is that enough, or do we need more?

[touilleMan]

What do you think? Is that enough, or do we need more?

I never used those api function (is it the ones added in gdnative 1.1 ?), but I guess it should be good enough to do something like:

with my_array.raw_access() as my_array_raw_ptr:
    # Note I don't know the numpy api...
    my_array_numpy = numpy.new_array_from_pointer(my_array_raw_ptr)
    ...

[garyo]

Thomas Herzog added those functions to Godot on 2017-11-03 (in af5c6be074). That commit is in 3.0-stable but I'm not sure about which gdnative release that corresponds to.
As to your suggestion of implementing the raw_access() method using those API funcs -- I'll give that a try next time I need it.

In fact it can be useful even without numpy, using the standard python buffer interface; see https://docs.python.org/3.6/c-api/buffer.html and https://docs.python.org/3.6/c-api/memoryview.html#memoryview-objects. This could be used, for instance, to directly write or read a PoolArray to/from a file.

[touilleMan]

I had a look a this, and it appeared to be really easy to implement, so here it is:
e9347d6 😄