Another deadlock (bis)

[bboreham]

This time LocalPeer is blocked trying to acquire its lock:

goroutine 21 [semacquire]:
sync.runtime_Semacquire(0xc2080042a8)
    /tmp/makerelease197226928/go/src/pkg/runtime/sema.goc:199 +0x30
sync.(*RWMutex).Lock(0xc2080042a0)
    /usr/local/go/src/pkg/sync/rwmutex.go:83 +0xa3
github.com/zettio/weave/router.(*Peer).addConnection(0xc2080042a0, 0x7ff4088d0178, 0xc209230780)
    /home/bryan/golang/src/github.com/zettio/weave/router/peer.go:90 +0x32
github.com/zettio/weave/router.(*LocalPeer).handleAddConnection(0xc2090871a0, 0x7ff4088d0178, 0xc209230780, 0xc209230780)
    /home/bryan/golang/src/github.com/zettio/weave/router/local_peer.go:235 +0x4ab
github.com/zettio/weave/router.(*LocalPeer).queryLoop(0xc2090871a0, 0xc2080600e0)
    /home/bryan/golang/src/github.com/zettio/weave/router/local_peer.go:178 +0x146
created by github.com/zettio/weave/router.(*LocalPeer).Start
    /home/bryan/golang/src/github.com/zettio/weave/router/local_peer.go:35 +0x53

while that lock is held on another goroutine during a gossip-forward operation:

goroutine 34 [select]:
github.com/zettio/weave/router.(*LocalConnection).sendQuery(0xc20921a000, 0x0, 0x82dfa0, 0xc209a5c220)
    /home/bryan/golang/src/github.com/zettio/weave/router/connection.go:201 +0x16e
github.com/zettio/weave/router.(*LocalConnection).SendProtocolMsg(0xc20921a000, 0x5, 0xc209a4c900, 0x10c, 0x17a)
    /home/bryan/golang/src/github.com/zettio/weave/router/connection.go:255 +0x70
github.com/zettio/weave/router.func·008(0x6f0b79c336d, 0x7ff4088d0178, 0xc20921a000)
    /home/bryan/golang/src/github.com/zettio/weave/router/gossip.go:121 +0x73
github.com/zettio/weave/router.(*Peer).ForEachConnection(0xc2080042a0, 0x7ff408663b50)
    /home/bryan/golang/src/github.com/zettio/weave/router/peer.go:78 +0xf6
github.com/zettio/weave/router.(*GossipChannel).SendGossipMsg(0xc209083590, 0xc209a1da20, 0xf4, 0x15a)
    /home/bryan/golang/src/github.com/zettio/weave/router/gossip.go:122 +0xb8
github.com/zettio/weave/router.deliverGossip(0xc209083590, 0x6f0b79c336d, 0xc209a3b681, 0x10c, 0x10c, 0xc209a39c20, 0x0, 0x0)
    /home/bryan/golang/src/github.com/zettio/weave/router/gossip.go:113 +0x177
github.com/zettio/weave/router.(*Router).handleGossip(0xc2080f8a80, 0xc209a3b681, 0x10c, 0x10c, 0x9904d0, 0x0, 0x0)

that is in turn blocked because that connection is trying to tell LocalPeer it is established:

goroutine 33 [chan send]:
github.com/zettio/weave/router.(*LocalConnection).handleSetEstablished(0xc20921a000, 0x0, 0x0)
    /home/bryan/golang/src/github.com/zettio/weave/router/connection.go:390 +0x13c
github.com/zettio/weave/router.(*LocalConnection).queryLoop(0xc20921a000, 0xc20921c000, 0x0, 0x0)
    /home/bryan/golang/src/github.com/zettio/weave/router/connection.go:336 +0x19a
github.com/zettio/weave/router.(*LocalConnection).run(0xc20921a000, 0xc20921c000, 0xc2090d02a0, 0x84bf01)
    /home/bryan/golang/src/github.com/zettio/weave/router/connection.go:302 +0x684
created by github.com/zettio/weave/router.(*LocalConnection).Start
    /home/bryan/golang/src/github.com/zettio/weave/router/connection.go:133 +0x9c

It would be simple enough to copy the list of connections before sending in SendGossipMsg(), thus avoiding holding the lock during that operation.