Extremely slow multi_exec() performance

[eirikrye]

Hi,

While evaluating aioredis and aredis against a client that I am working on, I've noticed that aioredis appears to perform exponentially worse in larger transactions (i.e. tens/hundreds of thousands of commands):

[aioredis pipeline   ] 100 iterations, 21.4µs/it, total: 2.1ms
[aioredis transaction] 100 iterations, 21.1µs/it, total: 2.1ms
[aredis pipeline     ] 100 iterations, 23.9µs/it, total: 2.4ms
[aredis transaction  ] 100 iterations, 11.8µs/it, total: 1.2ms
[sider pipeline      ] 100 iterations, 5.9µs/it, total: 0.6ms
[sider transaction   ] 100 iterations, 5.4µs/it, total: 0.5ms
[aioredis pipeline   ] 1000 iterations, 20.7µs/it, total: 20.7ms
[aioredis transaction] 1000 iterations, 21.4µs/it, total: 21.4ms
[aredis pipeline     ] 1000 iterations, 9.2µs/it, total: 9.2ms
[aredis transaction  ] 1000 iterations, 8.9µs/it, total: 8.9ms
[sider pipeline      ] 1000 iterations, 3.6µs/it, total: 3.6ms
[sider transaction   ] 1000 iterations, 3.8µs/it, total: 3.8ms
[aioredis pipeline   ] 10000 iterations, 22.0µs/it, total: 219.7ms
[aioredis transaction] 10000 iterations, 65.6µs/it, total: 655.7ms
[aredis pipeline     ] 10000 iterations, 8.1µs/it, total: 81.3ms
[aredis transaction  ] 10000 iterations, 9.3µs/it, total: 93.0ms
[sider pipeline      ] 10000 iterations, 3.4µs/it, total: 33.7ms
[sider transaction   ] 10000 iterations, 4.2µs/it, total: 42.2ms
[aioredis pipeline   ] 50000 iterations, 25.2µs/it, total: 1260.4ms
[aioredis transaction] 50000 iterations, 335.5µs/it, total: 16776.9ms
[aredis pipeline     ] 50000 iterations, 8.0µs/it, total: 402.1ms
[aredis transaction  ] 50000 iterations, 7.8µs/it, total: 389.8ms
[sider pipeline      ] 50000 iterations, 3.3µs/it, total: 164.0ms
[sider transaction   ] 50000 iterations, 4.1µs/it, total: 204.4ms
[aioredis pipeline   ] 100000 iterations, 27.8µs/it, total: 2784.1ms
[aioredis transaction] 100000 iterations, 934.5µs/it, total: 93454.5ms
[aredis pipeline     ] 100000 iterations, 9.4µs/it, total: 937.0ms
[aredis transaction  ] 100000 iterations, 8.9µs/it, total: 887.1ms
[sider pipeline      ] 100000 iterations, 3.4µs/it, total: 343.8ms
[sider transaction   ] 100000 iterations, 4.4µs/it, total: 441.2ms

Benchmark code.

The benchmark is written to use uvloop, and has been tested on Linux (Ubuntu 18.04) with Python 3.8.1, as well as on macOS with Python 3.8.1 and Python 3.7.6. All runs have similar results.

The aioredis multi_exec() pipeline appears to be many orders of magnitude slower than both aredis and the custom client.

While looking at the implementation, aioredis appears to fire off a coroutine for every single command in the transaction, which likely explains most of the slowdown. What is the rationale behind this design?

[seandstewart]

@eirikrye -

This has been on my radar for some time as I also use this library at work and noticed the same issue.

I've just merged #891, which should provide pipeline & multi-exec performance about even with redis-py.