Allow to control the degree of parallelism for specific actions

[konste]

Description of the problem / feature request:

Feature request: provide a way to advise Bazel scheduler on how many instances of particular action it may schedule to run in parallel.

Feature requests: what underlying problem are you trying to solve with this feature?

Scenario #1:
One of our custom rules runs the action which is very much network I/O bound and not CPU bound at all.
For that particular action there is no point to limit parallelism to the number of available cores.
Instead it makes sense to start as many of those actions as there are ready for execution.

Scenario #2
A particular test in the test suite is memory hungry. Attempt to run more than a few of those in parallel causes intermittent OOM conditions and test failures.

What's the output of bazel info release?

3.5.0

Have you found anything relevant by searching the web?

Discussion on the user group is here.

This issue is created per request from @aiuto.

[aiuto]

A few of us were talking about this today. We think that Excution Groups would be the right place to hold this information, but it will still require scheduler changes for something like parallelism. https://docs.bazel.build/versions/master/exec-groups.html

Within Google, we are using execution groups to specify that some action types (e.g. C++ linking) require much more memory than default. The link actions get sent to executors provisioned with more memory. But this can happen in the action execution back end, without requiring scheduler changes.

[ulfjack]

I think the request is primarily for local execution, for which the ResourceManager controls parallelism. This could be added there. In order to work properly, the ResourceManager would have to be refactored (or rewritten) to support async operation. This used to be on my todo list for async execution.

[jmmv]

This is another form of #10443 so closing as a dup of that.