dht stream hang clogs up queries

[whyrusleeping]

Occasionally we're seeing a stream hang (i think its related with #3651 ) in the dht code that causes a locking issue and leads to a bunch of queries getting stuck. The goroutine in question that is stuck looks like goroutine 400009791 in https://ipfs.io/ipfs/Qmax6PQz7n1zxd6RxbFPdqK3MC89q93JXtc5r6e7Nuswuw/ipfs.stacks

The root of the problem is these random yamux stream hangs, but we should be able to alleviate the issue by adding timeouts to the dht, and not blocking queries on a single peer.

[ghost]

One more from pluto.i.ipfs.io: https://ipfs.io/ipfs/QmeTtVLvKb97QCwhzmJG6YKhyiSSybwCeWBhH2CmENy68y

[whyrusleeping]

I think these have been resolved by a recent fix in stream handling timeouts. Maybe keep it open until the Reset changes have been merged through?

cc @Stebalien

[Stebalien]

Which recent fix? Unfortunately, I can't download those stack traces (grumble permanent web...). I don't recall making or seeing any timeout changes related to yamux.