Tuple compression benchmark

[andreyaksenov]

This PR adds a benchmark that will be used to measure compression performance for different algorithms. The results will be used to replace current data in this documentation section:
https://www.tarantool.io/en/enterprise_doc/appendix/parameters/#tuple-compression-performance

Here are raw results for the benchmark:

none:

Insert RPS: 11402.200085316
Full scan RPS: 3943688.2334922
Select RPS: 1213900.4075561
Replace RPS: 10804.614238394
Space size: 41168548

zstd:

Insert RPS: 8339.2376595329
Full scan RPS: 345413.83089763
Select RPS: 282008.49348114
Replace RPS: 9087.9631912889
Space size: 21368548

lz4:

Insert RPS: 13818.036744271
Full scan RPS: 1224199.9403503
Select RPS: 682655.13252673
Replace RPS: 13915.025661323
Space size: 25268548

zlib:

Insert RPS: 10582.710660237
Full scan RPS: 138963.73630138
Select RPS: 127156.35077488
Replace RPS: 10350.445266105
Space size: 20768548

The strange thing is that the Select RPS / Replace RPS ratio in my benchmark is much higher than in the current docs:
https://www.tarantool.io/en/enterprise_doc/appendix/parameters/#tuple-compression-performance

For example, for lz4:

• old: 625452 / 260766 = 2.4
• new: 682655 / 13915 = 49

What reasons might cause such a difference?

[drewdzzz]

You have 10k RPS on replace without compression (it is a very small number). At the same time, you have 13k RPS on replace with lz4 enabled. It is really weird. I suspect that your CPU does not handle the load, and tarantool is being preempted frequently, which leads to very unstable results. Let's try to use CPU clock instead of monotonic one. It is used in clock.bench function, I rewrote benchmark functions for you:

function insert_bench()
    for i = 1, space_size do
        test_space:insert({i, tostring(initial_data)})
    end
end
insert_rps = space_size / clock.bench(insert_bench)[1]

function full_scan_bench()
    test_space:select(nil, {
        fullscan = true
    })
end
full_scan_rps = space_size / clock.bench(full_scan_bench)[1]

function select_bench()
    for i = 1, space_size do
        test_space:get(i)
    end
end
select_rps = space_size / clock.bench(select_bench)[1]

function replace_bench()
    for i = 1, space_size do
        test_space:replace({i, tostring(new_data)})
    end
end
replace_rps = space_size / clock.bench(replace_bench)[1]

[andreyaksenov]

Thanks a lot! I ran tests on an AArch64 virtual machine (I'm on mac m1). Will try to repeat benchmarks on a physical x64 device.

[drewdzzz]

I've ran your benchmark on my machine (i5-12400F, 5.6 CPU max GHz, 32GB RAM) using clock.bench function and with disabled WAL, here is the result (additionally, I print len of old and new data):

➜  build git:(master) tarantool bench.lua
old len: 	607
new len: 	404
Insert RPS: 1076236.4116005
Full scan RPS: 4076834.1360824
Select RPS: 4486299.8273133
Replace RPS: 1109822.3607231
Space size: 41168548ULL
➜  build git:(master) tarantool bench.lua zlib
old len: 	607
new len: 	404
Insert RPS: 125814.10169065
Full scan RPS: 368833.08636085
Select RPS: 325801.12411292
Replace RPS: 107616.23652306
Space size: 20768548ULL
➜  build git:(master) tarantool bench.lua lz4 
old len: 	607
new len: 	404
Insert RPS: 697920.83172733
Full scan RPS: 1941441.4282361
Select RPS: 1765400.889529
Replace RPS: 672213.32286473
Space size: 25268548ULL
➜  build git:(master) tarantool bench.lua zstd
old len: 	607
new len: 	404
Insert RPS: 19449.851145942
Full scan RPS: 314913.19159645
Select RPS: 308163.33582521
Replace RPS: 26773.642146091
Space size: 21368548ULL

[andreyaksenov]

Thanks!