Elastic Dispatchers.IO unbounded for limitedParallelism

[qwwdfsad]

Dispatchers.IO legacy

Prior to #2919 and upcoming release 1.6.0, kotlinx.coroutines had no API to limit the parallelism without creating excessive threads. Dispatchers.IO meant to be a dispatcher for "any IO", similar to Rx-like io and newCachedThreadPool.
But having an unlimited number of threads has brittle and dangerous pitfalls: the performance of an overloaded application degrades in a slow, but unlimited manner, eventually causing out-of-memory errors.
Because of that, it was decided to have a "good enough" upper limit (64 seemed to be a reasonable constant at that moment, especially for Android).

After #2919, along with newFixedThreadPoolContext becoming @DelicateCoroutinesApi, we would advocate for using e.g. Dispatchers.IO.limitedParallelism(myDbConnectionPoolSize) instead of newFixedThreadPoolContext(myDbConnectionPoolSize).

Such recommendation has a fatal flaw: -- nowadays the number of threads allocated for DB connections, blocking IO, and networks is measured in hundreds, so as soon as the user has enough "limited" slices of Dispatchers.IO, the system will start starving only during peak loads. It's almost impossible to catch such behavior during code-review, (regular) testing, and regular operational mode. Shortly speaking, this behavior is just a timebomb that will explode at the least expected moment.

Proposed change

To avoid degradations during peak loads, while still providing fluent and hard-to-misuse API, the following changes will be implemented:

• Dispatchers.IO will still be limited to 64 threads with the limit being a fail-safe mechanism
• Dispatchers.IO.limitedParallelism(n) won't be limited by the hard limit of the original Dispatchers.IO, but instead will be limited by the given parallelism
• Multiple slices of Dispatchers.IO will still be conceptually backed by the same thread-pool, being able to leverage both reduced number of threads and reduced number of context-switches due to its built-in integration with Dispatchers.Defailt

Basically, the mental model (but not the implementation!) is the following:

private val unlimitedThreadsDispatcherIO = ...

Dispatchers.IO = unlimitedThreadsDispatcherIO.limitParallelism(64)
Dispatchers.IO.limitParallelism(x) = unlimitedThreadsDispatcherIO.limitParallelism(x)