Simplifications and performance improvements

[bcardiff]

This PR performs a couple of changes that will hopefully allow further performance improvements. The additions are:

1. Add a concurrency test to manually check performance.
2. Make pool statements picks any connection instead of prioritising the ones that already had the statement prepared. This removes some Mutex
3. Thanks to 2. we can drop Pool#checkout_some (this is a breaking-change, but is an API that is not advertised to be used). Having a smaller Pool api will allow use to play easily with its implementation.
4. Thanks to 2. the PoolPreparedStatement becomes stateless, as PoolUnpreparedStatement. With that we can turn them intro structs and avoid GC allocation.
5. Drop the StringKeyCache Mutex since now they are only used inside a connection, which should already not be shared between threads or used with concurrent statement executions. Connections usually work with a socket so it's already a resource that can't be used concurrently.

For single-thread I have seen 1.7% increase on query throughput. While for multi-thread I have seen a 74% increase. Don't jump too much, the overall throughput for MT is still far from the ST one (For me MT is about 25% of ST for 4 crystal workers).

The concurrency test sometimes hangs in MT mode. This is even before the changes so it's something we are able to reproduce more and continue to investigate.

Thanks to @luislavena for bringing awareness of the issue and validating some of these changes.

[straight-shoota]

LGTM. Simplification is always great 🚀

I was wondering whether the breaking change should warrant a new major release, but realized crystal-db is still in 0.x, so we're effectively doing major pre-1.0 releases anyway.

[luislavena]

This is great! ❤️ 🚀 ❤️

Took a bit but here are some synthetic numbers of before and after this change:

Test system:

• System: AMD Ryzen 5 3550H
• CPU: 4C/8T RAM: 32GB
• OS: Ubuntu 22.04. LTS x86_64
• Kernel: 6.5.6
• Storage: Kingston 256GB PCIe NVMe M.2 2280 (KINGSTON OM8PDP3256B-A01), FS: ext4

pool_concurrency_test.cr

Builds:

• Single-thread (ST) release build, postfix -st
• Multi-thread (MT) release build, postfix -mt
• Before the change (pool-pre-st and pool-pre-mt)
• After the change (pool-master-st and pool-master-mt)

Comparing:

• Metric: Queries Per Second (QPS)
• Pool of 100 initial connections
• Increase of concurrency from 1, 5, 10 and 20 connections
• Test duration: 10 seconds
• Selected best of 3 runs

Single-thread (ST)

concurrency (c)	Before	After
1	572523.8	615910.0
5	1002612.5	1184462.1
10	1035447.6	1337213.0
20	1036745.2	1363128.0

Multi-thread (MT)

concurrency (c)	Before	After
1	451035.8	468717.2
5	1291908.3	1446543.0
10	1321535.7	1600798.6
20	1277237.3	1728099.9

I need to investigate a bit MT crash with SQLite3 but this performs much better!

Thank you again for the time investigating this @bcardiff 👏🏽 ❤️