Need some help with okio Buffer

[balayanv]

I'm new to okio and haven't used it in anger yet, but something I'm building now can likely reuse Buffer of okio.

I'm working with a serialization protocol/codec that works directly on a buffer, essentially i will receive messages which are normally structs that i can literally wrap a segment of a buffer and treat it as the message i work with. Its a flyweight pattern which basically allows me to read from network card to buffer, directly manipulate data on the buffer and send it without creating bunch of objects and actually being faster than anything else i know out there. No time spent on actual serialization or deserialization.

There are few challenges here :

1. reuse of the underlying byte[]/ByteBuffer i use, and their variable size per message, while most of the messages are small lets say less than 128 bytes, there may in theory be messages that are megabytes in size (snapshots of some kind). Obviously i do not want to maintain a pool of byte[] with the maximum size observed...

2. as i "deserialize/wrap" a message i may randomly read/write at different offsets but always within some specific bounds, most messages are structs so i know exactly where to read/write and how many bytes.

I know that okio Buffer is something that works with segments under the hood but i couldn't see a good API to use them.

Ideally i should be able to do something like buffer.allocateSegment(numberOfBytes)... get a segment object (which ideally is not thread safe) so all interactions i do with the buffer are thread confined... once I'm done with the segment i should be able to return it and let the buffer mange memory...

Are there any examples that i could use to see how to achieve this with okio Buffer? There may be multiple segments that are allocated and actively used at a given time.

P.S. is there a way to treat okio Buffer as a standard byte buffer? it does not seem to extend/implement anything that could be used that way.
I understand that java devs treat us like children and seal ByteBuffer so now i have to go trough trouble if i want to make something that works with ByteBuffer to work with Buffer... its the same annoying story if i want to use chronicles Bytes or Aerons buffer equivalents...

How do you deal with libraries that rely on nio buffers?

[balayanv]

I've explored the api a little bit, it seems by the looks of it i will have to implement something similar, that allocates byte[]'s, reuses them (i dont really need all the fancy borrowing from one buffer to another).

The api is quite simple...

Buffer.allocateSegment(bytes : Int) : ByteBuffer
Buffer.returnSegment(segment : ByteBuffer)

the underlying bytebuffer is literally pointing to a reused byte[],
i dont really need the immutable ByteString thing either...

Maybe there is a way to do it with okio buffer but it seems to be engineered specifically for the okio usecase. Not sure if anyone else is using it in anger. Even if i manage to do something similar i still need nio.ByteBuffer (which "very smart" java core developers made sealed class so we cant escape their "professional" work)

[swankjesse]

We’ve got an API called UnsafeCursor that gives you lots of rope and gentle advice to not hang yourself with it. It lets you treat the Buffer as a linked list of segments, because that’s its internal form.

https://square.github.io/okio/3.x/okio/okio/okio/-buffer/-unsafe-cursor/index.html

[swankjesse]

Okio doesn’t have APIs to view its Buffer as a Java NIO Buffer or vice-versa. The Java NIO API doesn’t make that very easy, and they’re kind of annoying objects to accept because they could be direct, which interacts awkwardly with the GC.

But you can use these APIs:

• Buffer extends ByteChannel
• Buffer.read(java.nio.ByteBuffer)
• Buffer.write(java.nio.ByteBuffer)