Non aligned buffers when using arrow flight

[eligbr]

Describe the usage question you have. Please include as many useful details as possible.

Hi everyone!

I have an application which acts as an arrow flight client.
When I consume arrow buffers of decimal type for example I see that the addresses of the buffers are not aligned to 64 bytes.
This causes inefficient processing which slows down my application significantly.

Can anyone say how can I make sure that the buffers are aligned? I have tried to manipulate the memory manager and memory pool but with no success. Appreciate any help,
Thanks!

Component(s)

C++

[lidavidm]

Duplicate of #32276

It may be possible to address but it will require some surgery. Otherwise the only solution is to copy each array into a new allocation (ConcatenateArrays with a single array can do this).

[eligbr]

Thank you very much David for the quick response.
The copy solution sounds indeed as the quickest one, but suppose we want to perform the surgery can you give some guidance/pointers as to what it will require?

Again, thanks a lot

[lidavidm]

#32276 (comment)

The general idea would be to inline the handrolled Protobuf parser into the Flight code that gets the buffers from gRPC. It can then parse enough of the FlightData to find the body buffer. Then, since gRPC hands us disjoint slices, we need to concatenate data anyways. So we can concatenate just the slices that comprise the body buffer into a new buffer, which will be aligned.

[lidavidm]

I'm going to close this as a duplicate of #32276 to keep the discussion in one place. Please use the other thread if you have further comments. Thanks!

[eligbr]

Thank you!