bitswap should dht.PutProvider + send to other peers concurrently

[jbenet]

6a5b1e8#commitcomment-8854272

[jbenet]

This is sort of done, it's now async-rate-limited, but these should be totally separate. an ipfs add -r <hash> should never be block by the network.

[btc]

Wanna get this full-async for the alpha?

edit: yes

[whyrusleeping]

This is kinda tricky, since we run the risk of exploding the number of goroutines running if our network is slow. One option would be to wait until the add has completed to start sending provides messages, but that would prevent streaming from ever happening... One idea would be to buffer up provides that need to happen, and just let them pile up in a slice in a separate 'ProvideBuffer' goroutine, and have that goroutine pull them off and provide as the network allows.

[jbenet]

Yeah the goroutine count of all these operations should be rate-limited, but that doesnt mean they cant be fully async. Yep, as you describe, buffering the provides and sends, and letting those processes handle them as needed. This is similar to the add->provide need for async.

[btc]

buffer up provides that need to happen, and just let them pile up in a slice in a separate 'ProvideBuffer' goroutine, and have that goroutine pull them off and provide as the network allows.

buffering into slice'll definitely save us on RAM. I think we're on the same page. I can whip up something this evening and ping you to get your advice on some of the tricky details.