Encoding performance

[parkan]

We're running some large-scale ingestions involving CBOR encoding and we're seeing this kind of performance per 1k objects:

pure js: 1.3s
tinycbor (C/C++): 0.06s

In this case the pure js impl is node-cbor because #22 blocks us from using js-ipld directly, but I would expect similar performance here. This is a huge difference and can potentially make or break the use case for larger datasets. We're exploring a couple of different paths here:

• optional gyp/ffi dependency when running in node
• webasm

Would love to work w/upstream here to deliver something that works for everyone. wasm certainly seems like a future-looking option that would work isomorphically (in a couple of months, at least 😄 ) so I am interested in exploring this path.

Corresponding issue in mediachain/aleph#27 /cc @yusefnapora

[daviddias]

Hey @parkan have you looked into this again? Would love to see some benchmarks of tinycbor compiled with asm.js.

[parkan]

@diasdavid will hopefully have something next week!

[dignifiedquire]

@parkan I have started looking into optimizing cbor encoding and decoding. Could you share your benchmark that you ran? I would like to see how close I can get in JavaScript.

[dignifiedquire]

Current results, node-cbor is the default what we currently use and fast-cbor are my optimizations. Every run consists of encoding 83 small objects to cbor with most of the possible values being used.

Browser (Chrome)
encode vectors - node-cbor - 83 x 232 ops/sec ±5.34% (46 runs sampled)
encode vectors - fast-cbor - 83 x 4,248 ops/sec ±1.39% (53 runs sampled)

Node.js 7
encode vectors - node-cbor - 83 x 301 ops/sec ±5.38% (60 runs sampled)
encode vectors - fast-cbor - 83 x 3,584 ops/sec ±2.36% (74 runs sampled)

[dignifiedquire]

Getting faster,

Browser (Chrome)
encode vectors - fast-cbor - 83 x 5,576 ops/sec ±5.83% (33 runs sampled)

Node.js 7
encode vectors - fast-cbor - 83 x 4,061 ops/sec ±2.06% (79 runs sampled)

work is happening here: https://github.com/dignifiedquire/node-cbor/tree/perf

[dignifiedquire]

Some new numbers are in

Browser (Chrome)
encode vectors - node-cbor - 83 x 198 ops/sec ±10.09% (31 runs sampled)
encode vectors - fast-cbor - 83 x 5,744 ops/sec ±1.29% (36 runs sampled)
encode vectors - JSON.stringify - 83 x 18,944 ops/sec ±7.59% (30 runs sampled)

Node.js 7
encode vectors - node-cbor - 83 x 294 ops/sec ±4.95% (58 runs sampled)
encode vectors - fast-cbor - 83 x 10,884 ops/sec ±3.30% (76 runs sampled)
encode vectors - JSON.stringify - 83 x 18,166 ops/sec ±8.90% (66 runs sampled)

so things are starting to look pretty good :)

[parkan]

Hi @dignifiedquire, very happy to see those numbers! Don't have the benchmark around but the difference was something like ~700 ops/s node-cbor vs ~16000 ops/s with tinycbor (under node 6.x) so this is certainly in the ballpark.

Glanced over your branch, I see a bunch of savings through precalculating buffer size and allocating once -- nice. Curious, what tooling do you use for profling/benches/etc?

[dignifiedquire]

Moved code to https://github.com/dignifiedquire/fast-cbor

[dignifiedquire]

@parkan will do a write up of techniques and things I used when it's ready to ship :)

[parkan]

beautiful! will benchmark this w/our datasets after deps are updated