Panic: unlock of unlocked mutex in resolveHook

[lthibault]

The following stack-trace was generated from a unit test in CASM. I was not able to reproduce the error when re-running the test.

goroutine 22 [running]:
sync.fatal({0x10132a746?, 0x1ffffffff?})
        /usr/local/go/src/runtime/panic.go:1031 +0x1e
sync.(*Mutex).unlockSlow(0xc0000a2f30, 0xffffffff)
        /usr/local/go/src/sync/mutex.go:229 +0x49
sync.(*Mutex).Unlock(0xc0000a2f30)
        /usr/local/go/src/sync/mutex.go:223 +0x55
capnproto.org/go/capnp/v3.resolveHook(0xc0000a2f00)
        /Users/lthibault/Go/pkg/mod/capnproto.org/go/capnp/v3@v3.0.0-alpha.9/capability.go:264 +0x96
capnproto.org/go/capnp/v3.Client.State({0xc00072a238?})
        /Users/lthibault/Go/pkg/mod/capnproto.org/go/capnp/v3@v3.0.0-alpha.9/capability.go:495 +0xcd
github.com/wetware/casm/pkg/util/stream.promise.Failed({{0xc00072a238?}, 0xc000013140?})
        /Users/lthibault/Go/src/github.com/wetware/casm/pkg/util/stream/stream.go:138 +0x3c
github.com/wetware/casm/pkg/util/stream.(*Stream[...]).Call(0xc000a0e000, {0x101625c50, 0xc000a04000}, 0x1000777bb)
        /Users/lthibault/Go/src/github.com/wetware/casm/pkg/util/stream/stream.go:63 +0x334
github.com/wetware/casm/pkg/cluster.iterator.func1({0x1032c53a8, 0x101f86020})
        /Users/lthibault/Go/src/github.com/wetware/casm/pkg/cluster/server.go:125 +0x85
github.com/wetware/casm/pkg/cluster.Server.bind({{0x10161c8a0?, 0xc00010e3c0?}}, 0xc000a10000, 0xc000a12000)
        /Users/lthibault/Go/src/github.com/wetware/casm/pkg/cluster/server.go:102 +0x139
github.com/wetware/casm/pkg/cluster.Server.Iter({{0x10161c8a0?, 0xc00010e3c0?}}, {0x101625c50, 0xc000a04000}, {0x90?})
        /Users/lthibault/Go/src/github.com/wetware/casm/pkg/cluster/server.go:55 +0x38d
github.com/wetware/casm/internal/api/routing.View_Methods.func2({0x101625c50, 0xc000a04000}, 0x102eff940?)
        /Users/lthibault/Go/src/github.com/wetware/casm/internal/api/routing/routing.capnp.go:306 +0x62
capnproto.org/go/capnp/v3/server.(*Server).handleCall(0xc00012a3f0, {0x101625c50, 0xc000a04000}, 0xc00011ed00)
        /Users/lthibault/Go/pkg/mod/capnproto.org/go/capnp/v3@v3.0.0-alpha.9/server/server.go:206 +0x111
capnproto.org/go/capnp/v3/server.(*Server).handleCalls.func2(0xc000a0a000, 0xc000110100?, {0x101625c50, 0xc000a04000}, 0xc00009a130?)
        /Users/lthibault/Go/pkg/mod/capnproto.org/go/capnp/v3@v3.0.0-alpha.9/server/server.go:182 +0x85
capnproto.org/go/capnp/v3/server.(*Server).handleCalls(0xc00012a3f0, {0x101625c50?, 0xc0001100c0})
        /Users/lthibault/Go/pkg/mod/capnproto.org/go/capnp/v3@v3.0.0-alpha.9/server/server.go:183 +0x1c5
created by capnproto.org/go/capnp/v3/server.New
        /Users/lthibault/Go/pkg/mod/capnproto.org/go/capnp/v3@v3.0.0-alpha.9/server/server.go:122 +0x5ae

[lthibault]

@zenhack This just popped again, this time in prod.