flaky TestResourceManagerConnInbound test

[marten-seemann]

=== RUN   TestResourceManagerConnInbound
      rcmgr_test.go:54: failed to dial Qmep35iYXharavwS9hRvZnZmGcGqsSQZSeiVZeaBHFfgXh:
            * [/ip6/::1/tcp/37[91](https://github.com/libp2p/go-libp2p/runs/5082801862?check_suite_focus=true#step:8:91)1] failed to negotiate security protocol: read tcp6 [::1]:35363->[::1]:37911: read: connection reset by peer
            * [/ip4/127.0.0.1/tcp/45447] failed to negotiate security protocol: read tcp4 127.0.0.1:35987->127.0.0.1:45447: read: connection reset by peer
            * [/ip4/10.1.0.146/tcp/45447] failed to negotiate security protocol: read tcp4 10.1.0.146:35987->10.1.0.146:45447: read: connection reset by peer
  --- FAIL: TestResourceManagerConnInbound (4.86s)