Add SimultaneousTransfers limitation for incoming data transfers from Storage Deals

[jennijuju]

A miner node got OOM , due to too many transfers (had around 30+ incoming and not stalled, and 2 outgoing) coinciding with the miner doing wdPoSt, resulting in the miner losing all it's power.

As a Storage Service provider, instead of rejecting the deals, ideally, they can have the ability to open the services to the clients, accept the deals and queue them up for processing in order.

Related: Problem #2 in #6861 (comment)

[jennijuju]

@magik6k suspected this is a graphsync bug @hannahhoward

We should make sure that test TestSimultanenousTransferLimit passes and not flaky after fixing this issue.

[jennijuju]

Retitled the issue.
Related to the correction in #7066 SimultaneousTransfers is only applying to the retrieval deals currently - However, at the moment, storage provider is more impacted by the incoming storage deal transfers resource wise, therefore, some configuration to control the amount of incoming data tranfer is needed.

[jennijuju]

from #7031 (comment)

Storage Providers can get a lot of deals from one client (i.e Esturay) simultaneously, that can/will stay in a transferring state for a long time. If the Storage Provider then has a low/normal MaxSimultaneousTransfers=20, and the amount of storage-deals from that client are equal to this, another client (i.e web3.storage) could potentially be blocked from retrieving a file they have previously stored with the storage provider earlier.

a better storage deal data transfer incoming queue up control for storage providers to be able to receive large amount of deal transfer without crashing the node would be essential when the demand increases in the newtork

[hannahhoward]

filecoin-project/go-data-transfer#243 is a stop gap solution for this problem. It's not the ideal design but it may be what we can ship for 1.11.3.

[jennijuju]

filecoin-project/go-data-transfer#243 is a stop gap solution for this problem. It's not the ideal design but it may be what we can ship for 1.11.3.

Let's design and implement an ideal solution after gathering community input - postpone to v1.11.4.

[jennijuju]

gathering community input in #7259