Protobuf Speed

[MrThreepwood]

We were looking into using this library for our data transfer for a new project due to it supporting both JSON and Protobuf and since it's noticeably faster than Jackson, that is, it is for JSON anyway. I was surprised to find that your protobuf serialization/deserialization is much slower than your JSON (1.5-2x). With any other library I've used it's always very much the opposite. We were hoping to use Protobuf largely for performance purposes, but it's significantly slower by about the same margin regardless of data types.

I understand that this is still a fairly early get it working state. Is this something you're planning on working on once you have a stable release/api or is it perhaps something that might be looked into earlier?

[qwwdfsad]

Currently, we're working on an implementation of a new design. As soon as it's ~80% ready (so basic serialization API won't significantly change) we'll start working on performance of protobuf.
We are aware that current prorobuf implementation is really slow and designed new API in the way that protobuf can be implemented efficiently. It won't be as fast as code-generated protobuf though, but it will be significantly faster than JSON

[MrThreepwood]

Good to hear! For some reason I thought kotlinx was using code generation for serialization/deserialization as well since the library mentions that it doesn't use reflection, though I wasn't expecting that it would be as fast as googles code generation implementation.

[qwwdfsad]

kotlinx.serialization indeed uses code generation. But it produces code generally-applicable for any serialization format and such code cannot be as fast as the specialized version for one specific schemaful format. We expect performance loss around ~10% in comparison with protobuf which shouldn't be a high price for an easy-to-use protobuf serialization :)

[linux-china]

I did a benchmark between native protobuf, protostuff and kotlin serialization protobuf.
1 is native protobuf, 2 is protostuff and 3 is kotlin serialization protobuf

The kotlin serialization protobuf is not good, anybody can take a look? The code is here https://github.com/linux-china/protobuf-demo

[JakeWharton]

Be sure to notice the deceptive Y axis. It's not actually 50x slower.

…

On Sat, Jan 19, 2019, 4:04 PM Jacky Chan ***@***.*** wrote: I did a benchmark between native protobuf, protostuff and kotlin serialization protobuf. 1 is native protobuf, 2 is protostuff and 3 is kotlin serialization protobuf [image: image] <https://user-images.githubusercontent.com/46711/51432328-8938a980-1bea-11e9-88a5-4c95b1921bc1.png> The kotlin serialization protobuf is not good, anybody can take a look? The code is here https://github.com/linux-china/protobuf-demo — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#216 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAEEEZ7t7WTtrYhTvCGxaEkahsr0TxOXks5vE4hZgaJpZM4WulQt> .

[qwwdfsad]

The problem should be fixed in the next release of kotlinx.serialization.
Can't compare it to the original ProtoBuf (at least because it requires a careful study of various benchmark modes), but now it is significantly (x2-x10) faster than both JSON and previous ProtoBuf version

[qwwdfsad]

Fixed in 1.0.0