make scheduler consider action concurrency >1

[bdoyle0182]

Description

Have the scheduling decision maker make container creation decisions using a throughput calculation that considers how many concurrent requests a container can take. This should exhibit the same throughput guarantees as is, but without unnecessary potential fan out of containers when an action has concurrency greater than one since the scheduler didn't consider it up until this point. There is no semantic change to behavior from this commit if action concurrency == 1.

Example: consider you have 100 incoming activations in the queue for an action with a concurrency of 10 with 1 existing container up currently processing 10 activations when the scheduling decision is made. The average duration is 10ms so the container throughput for this action due to the concurrency is really 100. So needed containers should already be met with just 1 container. However, currently the decision maker since it doesn't consider concurrency would create an additional 9 containers far exceeding the true needed number of containers.

Related issue and scope

• I opened an issue to propose and discuss this change (#????)

My changes affect the following components

• API
• Controller
• Message Bus (e.g., Kafka)
• Loadbalancer
• Scheduler
• Invoker
• Intrinsic actions (e.g., sequences, conductors)
• Data stores (e.g., CouchDB)
• Tests
• Deployment
• CLI
• General tooling
• Documentation

Types of changes

• Bug fix (generally a non-breaking change which closes an issue).
• Enhancement or new feature (adds new functionality).
• Breaking change (a bug fix or enhancement which changes existing behavior).

Checklist:

• I signed an Apache CLA.
• I reviewed the style guides and followed the recommendations (Travis CI will check :).
• I added tests to cover my changes.
• My changes require further changes to the documentation.
• I updated the documentation where necessary.