Jun 09 14:19:30 perflab-us01prfcpclu3-zkp2 nomad[125585]: 2020-06-09T14:19:30.049Z [INFO] client: node registration complete Jun 09 22:07:21 perflab-us01prfcpclu3-zkp2 nomad[125585]: SIGABRT: abort Jun 09 22:07:21 perflab-us01prfcpclu3-zkp2 nomad[125585]: PC=0x46ed57 m=0 sigcode=0 Jun 09 22:07:21 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 0 [idle]: Jun 09 22:07:21 perflab-us01prfcpclu3-zkp2 nomad[125585]: runtime.osyield(0xc00005e000, 0xc, 0x7ffdcbfe8780, 0x3, 0x57d9600, 0xc0008a6180, 0x7ffdcbfe8798, 0x46e855, 0x7ffd00000000, 0x0, ...) Jun 09 22:07:21 perflab-us01prfcpclu3-zkp2 nomad[125585]: runtime/sys_linux_amd64.s:657 +0x7 Jun 09 22:07:21 perflab-us01prfcpclu3-zkp2 nomad[125585]: runtime.adjusttimers(0xc00005e000) Jun 09 22:07:21 perflab-us01prfcpclu3-zkp2 nomad[125585]: runtime/time.go:641 +0x306 Jun 09 22:07:21 perflab-us01prfcpclu3-zkp2 nomad[125585]: runtime.checkTimers(0xc00005e000, 0x1709191ee4cbaf, 0x1709191ee4cb01, 0x0, 0x100000000) Jun 09 22:07:21 perflab-us01prfcpclu3-zkp2 nomad[125585]: runtime/proc.go:2611 +0xab Jun 09 22:07:21 perflab-us01prfcpclu3-zkp2 nomad[125585]: runtime.findrunnable(0xc00005e000, 0x0) Jun 09 22:07:21 perflab-us01prfcpclu3-zkp2 nomad[125585]: runtime/proc.go:2172 +0x18a Jun 09 22:07:21 perflab-us01prfcpclu3-zkp2 nomad[125585]: runtime.schedule() Jun 09 22:07:21 perflab-us01prfcpclu3-zkp2 nomad[125585]: runtime/proc.go:2520 +0x2fc Jun 09 22:07:21 perflab-us01prfcpclu3-zkp2 nomad[125585]: runtime.park_m(0xc0008e3c80) Jun 09 22:07:21 perflab-us01prfcpclu3-zkp2 nomad[125585]: runtime/proc.go:2690 +0x9d Jun 09 22:07:21 perflab-us01prfcpclu3-zkp2 nomad[125585]: runtime.mcall(0x2557e10) Jun 09 22:07:21 perflab-us01prfcpclu3-zkp2 nomad[125585]: runtime/asm_amd64.s:318 +0x5b Jun 09 22:07:21 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 1 [select, 467 minutes]: Jun 09 22:07:21 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/command/agent.(*Command).handleSignals(0xc0001237a0, 0xc0005f5800) Jun 09 22:07:21 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/command/agent/command.go:791 +0x1e4 Jun 09 22:07:21 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/command/agent.(*Command).Run(0xc0001237a0, 0xc0000d8020, 0x2, 0x2, 0x0) Jun 09 22:07:21 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/command/agent/command.go:706 +0xed9 Jun 09 22:07:21 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/mitchellh/cli.(*CLI).Run(0xc00056ba40, 0xc00056ba40, 0xc0003759c0, 0x38) Jun 09 22:07:21 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/mitchellh/cli/cli.go:255 +0x1da Jun 09 22:07:21 perflab-us01prfcpclu3-zkp2 nomad[125585]: main.RunCustom(0xc0000d8010, 0x3, 0x3, 0xc0000aa058) Jun 09 22:07:21 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/main.go:138 +0x482 Jun 09 22:07:21 perflab-us01prfcpclu3-zkp2 nomad[125585]: main.Run(...) Jun 09 22:07:21 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/main.go:83 Jun 09 22:07:21 perflab-us01prfcpclu3-zkp2 nomad[125585]: main.main() Jun 09 22:07:21 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/main.go:79 +0x64 Jun 09 22:07:21 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 7 [select]: Jun 09 22:07:21 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/go.opencensus.io/stats/view.(*worker).start(0xc000186a00) Jun 09 22:07:21 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/go.opencensus.io/stats/view/worker.go:154 +0x100 Jun 09 22:07:21 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/go.opencensus.io/stats/view.init.0 Jun 09 22:07:21 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/go.opencensus.io/stats/view/worker.go:32 +0x57 Jun 09 22:07:21 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 56 [IO wait]: Jun 09 22:07:21 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll.runtime_pollWait(0x7f2c51e96e08, 0x72, 0xffffffffffffffff) Jun 09 22:07:21 perflab-us01prfcpclu3-zkp2 nomad[125585]: runtime/netpoll.go:203 +0x55 Jun 09 22:07:21 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll.(*pollDesc).wait(0xc0006afe98, 0x72, 0x1000, 0x1000, 0xffffffffffffffff) Jun 09 22:07:21 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll/fd_poll_runtime.go:87 +0x45 Jun 09 22:07:21 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll.(*pollDesc).waitRead(...) Jun 09 22:07:21 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll/fd_poll_runtime.go:92 Jun 09 22:07:21 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll.(*FD).Read(0xc0006afe80, 0xc00073d000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Jun 09 22:07:21 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll/fd_unix.go:169 +0x19b Jun 09 22:07:21 perflab-us01prfcpclu3-zkp2 nomad[125585]: net.(*netFD).Read(0xc0006afe80, 0xc00073d000, 0x1000, 0x1000, 0x43c34c, 0xc000082b70, 0x469400) Jun 09 22:07:21 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/fd_unix.go:202 +0x4f Jun 09 22:07:21 perflab-us01prfcpclu3-zkp2 nomad[125585]: net.(*conn).Read(0xc000010008, 0xc00073d000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Jun 09 22:07:21 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/net.go:184 +0x8e Jun 09 22:07:21 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http.(*persistConn).Read(0xc0006f1560, 0xc00073d000, 0x1000, 0x1000, 0xc0000aa840, 0xc000082c70, 0x409c65) Jun 09 22:07:21 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/transport.go:1825 +0x75 Jun 09 22:07:21 perflab-us01prfcpclu3-zkp2 nomad[125585]: bufio.(*Reader).fill(0xc000741920) Jun 09 22:07:21 perflab-us01prfcpclu3-zkp2 nomad[125585]: bufio/bufio.go:100 +0x103 Jun 09 22:07:21 perflab-us01prfcpclu3-zkp2 nomad[125585]: bufio.(*Reader).Peek(0xc000741920, 0x1, 0x0, 0x0, 0x1, 0xc000f70f00, 0x0) Jun 09 22:07:21 perflab-us01prfcpclu3-zkp2 nomad[125585]: bufio/bufio.go:138 +0x4f Jun 09 22:07:21 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http.(*persistConn).readLoop(0xc0006f1560) Jun 09 22:07:21 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/transport.go:1978 +0x1a8 Jun 09 22:07:21 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by net/http.(*Transport).dialConn Jun 09 22:07:21 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/transport.go:1647 +0xc56 Jun 09 22:07:21 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 101 [select, 468 minutes]: Jun 09 22:07:21 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/pluginmanager/drivermanager.(*instanceManager).handleEvents(0xc000196fc0) Jun 09 22:07:21 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/pluginmanager/drivermanager/instance.go:435 +0x1fb Jun 09 22:07:21 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/pluginmanager/drivermanager.(*instanceManager).run.func2(0xc000196fc0, 0xc0005e5230) Jun 09 22:07:21 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/pluginmanager/drivermanager/instance.go:169 +0x2b Jun 09 22:07:21 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/client/pluginmanager/drivermanager.(*instanceManager).run Jun 09 22:07:21 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/pluginmanager/drivermanager/instance.go:168 +0x194 Jun 09 22:07:21 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 26 [select, 468 minutes]: Jun 09 22:07:21 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/command/agent/consul.(*checkWatcher).Run(0xc0006a81c0, 0x3814440, 0xc0006a9680) Jun 09 22:07:21 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/command/agent/consul/check_watcher.go:205 +0x253 Jun 09 22:07:21 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/command/agent/consul.(*ServiceClient).Run Jun 09 22:07:21 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/command/agent/consul/client.go:459 +0x31b Jun 09 22:07:21 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 16 [syscall, 468 minutes]: Jun 09 22:07:21 perflab-us01prfcpclu3-zkp2 nomad[125585]: os/signal.signal_recv(0x0) Jun 09 22:07:21 perflab-us01prfcpclu3-zkp2 nomad[125585]: runtime/sigqueue.go:147 +0x9c Jun 09 22:07:21 perflab-us01prfcpclu3-zkp2 nomad[125585]: os/signal.loop() Jun 09 22:07:21 perflab-us01prfcpclu3-zkp2 nomad[125585]: os/signal/signal_unix.go:23 +0x22 Jun 09 22:07:21 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by os/signal.Notify.func1 Jun 09 22:07:21 perflab-us01prfcpclu3-zkp2 nomad[125585]: os/signal/signal.go:127 +0x44 Jun 09 22:07:21 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 35 [select, 468 minutes]: Jun 09 22:07:21 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics.(*InmemSignal).run(0xc0006a8080) Jun 09 22:07:21 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics/inmem_signal.go:64 +0xb3 Jun 09 22:07:21 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics.NewInmemSignal Jun 09 22:07:21 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics/inmem_signal.go:38 +0x13e Jun 09 22:07:21 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 36 [sleep]: Jun 09 22:07:21 perflab-us01prfcpclu3-zkp2 nomad[125585]: time.Sleep(0x3b9aca00) Jun 09 22:07:21 perflab-us01prfcpclu3-zkp2 nomad[125585]: runtime/time.go:188 +0xba Jun 09 22:07:21 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics.(*Metrics).collectStats(0xc0006b0000) Jun 09 22:07:21 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics/metrics.go:230 +0x2f Jun 09 22:07:21 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics.New Jun 09 22:07:21 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics/start.go:79 +0x16d Jun 09 22:07:21 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 37 [select, 24 minutes]: Jun 09 22:07:21 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/command/agent/consul.(*ServiceClient).Run(0xc0001761c0) Jun 09 22:07:21 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/command/agent/consul/client.go:470 +0x475 Jun 09 22:07:21 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/command/agent.(*Agent).setupConsul Jun 09 22:07:21 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/command/agent/agent.go:1107 +0x1e9 Jun 09 22:07:21 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 38 [select, 25 minutes]: Jun 09 22:07:21 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/drivers/shared/eventer.(*Eventer).eventLoop(0xc0006b4240) Jun 09 22:07:21 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/drivers/shared/eventer/eventer.go:66 +0x126 Jun 09 22:07:21 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/drivers/shared/eventer.NewEventer Jun 09 22:07:21 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/drivers/shared/eventer/eventer.go:58 +0xc6 Jun 09 22:07:21 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 39 [select, 25 minutes]: Jun 09 22:07:21 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/drivers/shared/eventer.(*Eventer).eventLoop(0xc0006b4300) Jun 09 22:07:21 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/drivers/shared/eventer/eventer.go:66 +0x126 Jun 09 22:07:21 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/drivers/shared/eventer.NewEventer Jun 09 22:07:21 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/drivers/shared/eventer/eventer.go:58 +0xc6 Jun 09 22:07:21 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 40 [select, 25 minutes]: Jun 09 22:07:21 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/drivers/shared/eventer.(*Eventer).eventLoop(0xc0006b4420) Jun 09 22:07:21 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/drivers/shared/eventer/eventer.go:66 +0x126 Jun 09 22:07:21 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/drivers/shared/eventer.NewEventer Jun 09 22:07:21 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/drivers/shared/eventer/eventer.go:58 +0xc6 Jun 09 22:07:21 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 41 [select, 25 minutes]: Jun 09 22:07:21 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/drivers/shared/eventer.(*Eventer).eventLoop(0xc0006b44e0) Jun 09 22:07:21 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/drivers/shared/eventer/eventer.go:66 +0x126 Jun 09 22:07:21 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/drivers/shared/eventer.NewEventer Jun 09 22:07:21 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/drivers/shared/eventer/eventer.go:58 +0xc6 Jun 09 22:07:21 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 42 [select, 25 minutes]: Jun 09 22:07:21 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/drivers/shared/eventer.(*Eventer).eventLoop(0xc0006b45a0) Jun 09 22:07:21 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/drivers/shared/eventer/eventer.go:66 +0x126 Jun 09 22:07:21 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/drivers/shared/eventer.NewEventer Jun 09 22:07:21 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/drivers/shared/eventer/eventer.go:58 +0xc6 Jun 09 22:07:21 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 43 [select, 25 minutes]: Jun 09 22:07:21 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/drivers/shared/eventer.(*Eventer).eventLoop(0xc0006b4900) Jun 09 22:07:21 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/drivers/shared/eventer/eventer.go:66 +0x126 Jun 09 22:07:21 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/drivers/shared/eventer.NewEventer Jun 09 22:07:21 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/drivers/shared/eventer/eventer.go:58 +0xc6 Jun 09 22:07:21 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 44 [select, 25 minutes]: Jun 09 22:07:21 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/drivers/shared/eventer.(*Eventer).eventLoop(0xc0006e1560) Jun 09 22:07:21 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/drivers/shared/eventer/eventer.go:66 +0x126 Jun 09 22:07:21 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/drivers/shared/eventer.NewEventer Jun 09 22:07:21 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/drivers/shared/eventer/eventer.go:58 +0xc6 Jun 09 22:07:21 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 45 [select, 25 minutes]: Jun 09 22:07:21 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/drivers/shared/eventer.(*Eventer).eventLoop(0xc0006eb9e0) Jun 09 22:07:21 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/drivers/shared/eventer/eventer.go:66 +0x126 Jun 09 22:07:21 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/drivers/shared/eventer.NewEventer Jun 09 22:07:21 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/drivers/shared/eventer/eventer.go:58 +0xc6 Jun 09 22:07:21 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 46 [select, 25 minutes]: Jun 09 22:07:21 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/drivers/shared/eventer.(*Eventer).eventLoop(0xc0006f4120) Jun 09 22:07:21 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/drivers/shared/eventer/eventer.go:66 +0x126 Jun 09 22:07:21 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/drivers/shared/eventer.NewEventer Jun 09 22:07:21 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/drivers/shared/eventer/eventer.go:58 +0xc6 Jun 09 22:07:21 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 47 [select, 25 minutes]: Jun 09 22:07:21 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/drivers/shared/eventer.(*Eventer).eventLoop(0xc0006f4420) Jun 09 22:07:21 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/drivers/shared/eventer/eventer.go:66 +0x126 Jun 09 22:07:21 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/drivers/shared/eventer.NewEventer Jun 09 22:07:21 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/drivers/shared/eventer/eventer.go:58 +0xc6 Jun 09 22:07:21 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 48 [select]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/helper/pool.(*ConnPool).reap(0xc0006cf720) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/helper/pool/pool.go:486 +0x2bb Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/helper/pool.NewPool Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/helper/pool/pool.go:179 +0x13e Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 49 [select, 26 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/servers.(*Manager).Start(0xc0006f4cc0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/servers/manager.go:177 +0xc7 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/client.NewClient Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/client.go:356 +0x6b8 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 50 [select, 26 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client.(*Client).rpcConnListener(0xc00019e780) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/rpc.go:254 +0xf4 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/client.(*Client).setupClientRpc Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/rpc.go:232 +0x158 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 51 [select]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client.(*FingerprintManager).runFingerprint(0xc0006cf9f0, 0x37e7c80, 0xc0006fd8f0, 0x37e11d600, 0x3097b59, 0x6) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/fingerprint_manager.go:147 +0x1f3 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/client.(*FingerprintManager).setupFingerprinters Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/fingerprint_manager.go:131 +0x326 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 58 [select]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client.(*FingerprintManager).runFingerprint(0xc0006cf9f0, 0x37e7d00, 0xc0006fdbf0, 0x37e11d600, 0x3097ba7, 0x6) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/fingerprint_manager.go:147 +0x1f3 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/client.(*FingerprintManager).setupFingerprinters Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/fingerprint_manager.go:131 +0x326 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 57 [select]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http.(*persistConn).writeLoop(0xc0006f1560) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/transport.go:2277 +0x11c Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by net/http.(*Transport).dialConn Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/transport.go:1648 +0xc7b Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 32 [select, 468 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/dynamicplugins.(*pluginEventBroadcaster).run(0xc000514980) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/dynamicplugins/registry.go:369 +0x11c Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/client/dynamicplugins.newPluginEventBroadcaster Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/dynamicplugins/registry.go:363 +0xfb Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 27 [IO wait]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll.runtime_pollWait(0x7f2c51e96a88, 0x72, 0xffffffffffffffff) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: runtime/netpoll.go:203 +0x55 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll.(*pollDesc).wait(0xc00074d998, 0x72, 0x1000, 0x1000, 0xffffffffffffffff) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll/fd_poll_runtime.go:87 +0x45 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll.(*pollDesc).waitRead(...) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll/fd_poll_runtime.go:92 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll.(*FD).Read(0xc00074d980, 0xc0002cb000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll/fd_unix.go:169 +0x19b Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net.(*netFD).Read(0xc00074d980, 0xc0002cb000, 0x1000, 0x1000, 0x43c34c, 0xc000080b70, 0x469400) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/fd_unix.go:202 +0x4f Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net.(*conn).Read(0xc0000d4ad8, 0xc0002cb000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/net.go:184 +0x8e Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http.(*persistConn).Read(0xc000523680, 0xc0002cb000, 0x1000, 0x1000, 0xc00063a780, 0xc000080c70, 0x409c65) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/transport.go:1825 +0x75 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: bufio.(*Reader).fill(0xc00034eb40) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: bufio/bufio.go:100 +0x103 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: bufio.(*Reader).Peek(0xc00034eb40, 0x1, 0x0, 0x0, 0x1, 0xc001258400, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: bufio/bufio.go:138 +0x4f Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http.(*persistConn).readLoop(0xc000523680) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/transport.go:1978 +0x1a8 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by net/http.(*Transport).dialConn Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/transport.go:1647 +0xc56 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 69 [select]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client.(*FingerprintManager).runFingerprint(0xc0006cf9f0, 0x37e7f40, 0xc0006321e0, 0x37e11d600, 0x3096c80, 0x5) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/fingerprint_manager.go:147 +0x1f3 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/client.(*FingerprintManager).setupFingerprinters Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/fingerprint_manager.go:131 +0x326 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 28 [select]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http.(*persistConn).writeLoop(0xc000523680) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/transport.go:2277 +0x11c Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by net/http.(*Transport).dialConn Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/transport.go:1648 +0xc7b Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 100 [select, 24 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/pluginmanager/drivermanager.(*instanceManager).fingerprint(0xc000196fc0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/pluginmanager/drivermanager/instance.go:305 +0x1f5 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/pluginmanager/drivermanager.(*instanceManager).run.func1(0xc000196fc0, 0xc0005e5230) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/pluginmanager/drivermanager/instance.go:162 +0x2b Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/client/pluginmanager/drivermanager.(*instanceManager).run Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/pluginmanager/drivermanager/instance.go:161 +0x151 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 99 [select, 25 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/drivers/shared/eventer.(*Eventer).eventLoop(0xc00034fb60) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/drivers/shared/eventer/eventer.go:66 +0x126 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/drivers/shared/eventer.NewEventer Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/drivers/shared/eventer/eventer.go:58 +0xc6 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 33 [select, 468 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/dynamicplugins.(*dynamicRegistry).PluginsUpdatedCh.func1(0xc000514980, 0x3814440, 0xc00004f600, 0xc00034f920) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/dynamicplugins/registry.go:322 +0xc5 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/client/dynamicplugins.(*dynamicRegistry).PluginsUpdatedCh Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/dynamicplugins/registry.go:321 +0x9b Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 30 [select, 468 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/dynamicplugins.(*pluginEventBroadcaster).run(0xc000514940) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/dynamicplugins/registry.go:369 +0x11c Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/client/dynamicplugins.newPluginEventBroadcaster Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/dynamicplugins/registry.go:363 +0xfb Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 31 [select, 468 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/dynamicplugins.(*dynamicRegistry).PluginsUpdatedCh.func1(0xc000514940, 0x3814440, 0xc00004f600, 0xc00034f860) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/dynamicplugins/registry.go:322 +0xc5 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/client/dynamicplugins.(*dynamicRegistry).PluginsUpdatedCh Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/dynamicplugins/registry.go:321 +0x9b Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 78 [select, 24 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/pluginmanager/csimanager.(*csiManager).runLoop(0xc0006098c0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/pluginmanager/csimanager/manager.go:101 +0x1d0 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/client/pluginmanager/csimanager.(*csiManager).Run Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/pluginmanager/csimanager/manager.go:93 +0x3f Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 79 [semacquire, 468 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: sync.runtime_Semacquire(0xc0005e5258) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: runtime/sema.go:56 +0x42 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: sync.(*WaitGroup).Wait(0xc0005e5250) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: sync/waitgroup.go:130 +0x64 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/pluginmanager/drivermanager.(*instanceManager).run(0xc000196d80) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/pluginmanager/drivermanager/instance.go:174 +0x1a2 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/client/pluginmanager/drivermanager.newInstanceManager Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/pluginmanager/drivermanager/instance.go:132 +0x264 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 80 [semacquire, 468 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: sync.runtime_Semacquire(0xc0005e5298) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: runtime/sema.go:56 +0x42 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: sync.(*WaitGroup).Wait(0xc0005e5290) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: sync/waitgroup.go:130 +0x64 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/pluginmanager/drivermanager.(*instanceManager).run(0xc000196f00) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/pluginmanager/drivermanager/instance.go:174 +0x1a2 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/client/pluginmanager/drivermanager.newInstanceManager Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/pluginmanager/drivermanager/instance.go:132 +0x264 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 81 [semacquire, 468 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: sync.runtime_Semacquire(0xc0005e5238) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: runtime/sema.go:56 +0x42 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: sync.(*WaitGroup).Wait(0xc0005e5230) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: sync/waitgroup.go:130 +0x64 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/pluginmanager/drivermanager.(*instanceManager).run(0xc000196fc0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/pluginmanager/drivermanager/instance.go:174 +0x1a2 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/client/pluginmanager/drivermanager.newInstanceManager Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/pluginmanager/drivermanager/instance.go:132 +0x264 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 82 [semacquire, 468 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: sync.runtime_Semacquire(0xc00067f9e8) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: runtime/sema.go:56 +0x42 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: sync.(*WaitGroup).Wait(0xc00067f9e0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: sync/waitgroup.go:130 +0x64 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/pluginmanager/drivermanager.(*instanceManager).run(0xc000197080) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/pluginmanager/drivermanager/instance.go:174 +0x1a2 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/client/pluginmanager/drivermanager.newInstanceManager Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/pluginmanager/drivermanager/instance.go:132 +0x264 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 83 [semacquire, 468 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: sync.runtime_Semacquire(0xc00067faa8) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: runtime/sema.go:56 +0x42 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: sync.(*WaitGroup).Wait(0xc00067faa0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: sync/waitgroup.go:130 +0x64 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/pluginmanager/drivermanager.(*instanceManager).run(0xc000197140) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/pluginmanager/drivermanager/instance.go:174 +0x1a2 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/client/pluginmanager/drivermanager.newInstanceManager Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/pluginmanager/drivermanager/instance.go:132 +0x264 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 85 [select, 468 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/devicemanager.(*manager).fingerprint(0xc00011cd00) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/devicemanager/manager.go:172 +0x2c9 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/client/devicemanager.(*manager).Run Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/devicemanager/manager.go:166 +0x311 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 157 [select, 24 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client.(*Client).registerAndHeartbeat(0xc00019e780) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/client.go:1524 +0x180 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client.(*group).Go.func1(0xc00019e8e0, 0xc0007dc6f0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/client.go:3066 +0x51 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/client.(*group).Go Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/client.go:3064 +0x62 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 87 [select]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client.(*heartbeatStop).watch(0xc000122230) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/heartbeatstop.go:91 +0x1c5 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/client.NewClient Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/client.go:457 +0x157d Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 88 [select, 25 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client.(*AllocGarbageCollector).Run(0xc000609d40) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/gc.go:92 +0xf6 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/client.NewClient Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/client.go:473 +0x17b1 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 89 [select, 468 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client.(*Client).consulDiscovery(0xc00019e780) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/client.go:2611 +0xa9 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client.(*group).Go.func1(0xc00019e8e0, 0xc0002678d0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/client.go:3066 +0x51 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/client.(*group).Go Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/client.go:3064 +0x62 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 104 [select, 24 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/pluginmanager/drivermanager.(*instanceManager).fingerprint(0xc000196d80) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/pluginmanager/drivermanager/instance.go:305 +0x1f5 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/pluginmanager/drivermanager.(*instanceManager).run.func1(0xc000196d80, 0xc0005e5250) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/pluginmanager/drivermanager/instance.go:162 +0x2b Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/client/pluginmanager/drivermanager.(*instanceManager).run Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/pluginmanager/drivermanager/instance.go:161 +0x151 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 103 [select, 25 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/drivers/shared/eventer.(*Eventer).eventLoop(0xc00034fd40) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/drivers/shared/eventer/eventer.go:66 +0x126 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/drivers/shared/eventer.NewEventer Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/drivers/shared/eventer/eventer.go:58 +0xc6 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 105 [select, 468 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/pluginmanager/drivermanager.(*instanceManager).handleEvents(0xc000196d80) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/pluginmanager/drivermanager/instance.go:435 +0x1fb Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/pluginmanager/drivermanager.(*instanceManager).run.func2(0xc000196d80, 0xc0005e5250) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/pluginmanager/drivermanager/instance.go:169 +0x2b Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/client/pluginmanager/drivermanager.(*instanceManager).run Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/pluginmanager/drivermanager/instance.go:168 +0x194 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 92 [select, 24 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/pluginmanager/drivermanager.(*instanceManager).fingerprint(0xc000197080) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/pluginmanager/drivermanager/instance.go:305 +0x1f5 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/pluginmanager/drivermanager.(*instanceManager).run.func1(0xc000197080, 0xc00067f9e0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/pluginmanager/drivermanager/instance.go:162 +0x2b Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/client/pluginmanager/drivermanager.(*instanceManager).run Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/pluginmanager/drivermanager/instance.go:161 +0x151 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 108 [select, 24 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/pluginmanager/drivermanager.(*instanceManager).fingerprint(0xc000196f00) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/pluginmanager/drivermanager/instance.go:305 +0x1f5 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/pluginmanager/drivermanager.(*instanceManager).run.func1(0xc000196f00, 0xc0005e5290) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/pluginmanager/drivermanager/instance.go:162 +0x2b Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/client/pluginmanager/drivermanager.(*instanceManager).run Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/pluginmanager/drivermanager/instance.go:161 +0x151 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 91 [select, 25 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/drivers/shared/eventer.(*Eventer).eventLoop(0xc0005c00c0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/drivers/shared/eventer/eventer.go:66 +0x126 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/drivers/shared/eventer.NewEventer Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/drivers/shared/eventer/eventer.go:58 +0xc6 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 93 [select, 468 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/pluginmanager/drivermanager.(*instanceManager).handleEvents(0xc000197080) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/pluginmanager/drivermanager/instance.go:435 +0x1fb Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/pluginmanager/drivermanager.(*instanceManager).run.func2(0xc000197080, 0xc00067f9e0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/pluginmanager/drivermanager/instance.go:169 +0x2b Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/client/pluginmanager/drivermanager.(*instanceManager).run Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/pluginmanager/drivermanager/instance.go:168 +0x194 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 107 [select, 25 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/drivers/shared/eventer.(*Eventer).eventLoop(0xc00034ff20) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/drivers/shared/eventer/eventer.go:66 +0x126 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/drivers/shared/eventer.NewEventer Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/drivers/shared/eventer/eventer.go:58 +0xc6 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 96 [select, 24 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/pluginmanager/drivermanager.(*instanceManager).fingerprint(0xc000197140) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/pluginmanager/drivermanager/instance.go:305 +0x1f5 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/pluginmanager/drivermanager.(*instanceManager).run.func1(0xc000197140, 0xc00067faa0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/pluginmanager/drivermanager/instance.go:162 +0x2b Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/client/pluginmanager/drivermanager.(*instanceManager).run Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/pluginmanager/drivermanager/instance.go:161 +0x151 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 95 [select, 25 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/drivers/shared/eventer.(*Eventer).eventLoop(0xc0005c02a0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/drivers/shared/eventer/eventer.go:66 +0x126 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/drivers/shared/eventer.NewEventer Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/drivers/shared/eventer/eventer.go:58 +0xc6 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 109 [select, 468 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/pluginmanager/drivermanager.(*instanceManager).handleEvents(0xc000196f00) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/pluginmanager/drivermanager/instance.go:435 +0x1fb Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/pluginmanager/drivermanager.(*instanceManager).run.func2(0xc000196f00, 0xc0005e5290) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/pluginmanager/drivermanager/instance.go:169 +0x2b Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/client/pluginmanager/drivermanager.(*instanceManager).run Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/pluginmanager/drivermanager/instance.go:168 +0x194 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 97 [select, 468 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/pluginmanager/drivermanager.(*instanceManager).handleEvents(0xc000197140) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/pluginmanager/drivermanager/instance.go:435 +0x1fb Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/pluginmanager/drivermanager.(*instanceManager).run.func2(0xc000197140, 0xc00067faa0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/pluginmanager/drivermanager/instance.go:169 +0x2b Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/client/pluginmanager/drivermanager.(*instanceManager).run Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/pluginmanager/drivermanager/instance.go:168 +0x194 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 122 [select, 468 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport.(*Stream).waitOnHeader(0xc000113f00) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport/transport.go:318 +0xcc Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport.(*Stream).RecvCompress(...) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport/transport.go:333 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc.(*csAttempt).recvMsg(0xc00011db00, 0x2ea2d00, 0xc000916140, 0x0, 0x8, 0x413d7e) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/stream.go:871 +0x753 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc.(*clientStream).RecvMsg.func1(0xc00011db00, 0x203000, 0x203000) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/stream.go:736 +0x46 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc.(*clientStream).withRetry(0xc00083fd40, 0xc0007a9e30, 0xc0007a9e00, 0x382dbe0, 0x30) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/stream.go:590 +0x360 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc.(*clientStream).RecvMsg(0xc00083fd40, 0x2ea2d00, 0xc000916140, 0x20, 0xc0005c2600) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/stream.go:735 +0x103 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/internal/plugin.(*gRPCBrokerStartStreamClient).Recv(0xc00081b850, 0x31840c8, 0xc00085e780, 0xc000840780) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/internal/plugin/grpc_broker.pb.go:149 +0x62 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin.(*gRPCBrokerClientImpl).StartStream(0xc000840780, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/grpc_broker.go:194 +0x187 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin.newGRPCClient Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/grpc_client.go:63 +0x261 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 110 [select, 24 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/drivers/qemu.(*Driver).handleFingerprint(0xc000514b40, 0x3814440, 0xc000514e80, 0xc0000abe00) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/drivers/qemu/driver.go:196 +0x173 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/drivers/qemu.(*Driver).Fingerprint Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/drivers/qemu/driver.go:189 +0x7f Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 111 [select, 24 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/drivers/rawexec.(*Driver).handleFingerprint(0xc0005dd8b0, 0x3814440, 0xc000514f80, 0xc0000abf20) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/drivers/rawexec/driver.go:221 +0x195 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/drivers/rawexec.(*Driver).Fingerprint Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/drivers/rawexec/driver.go:213 +0x7f Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 112 [select, 24 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/drivers/exec.(*Driver).handleFingerprint(0xc00034ff80, 0x3814440, 0xc0005152c0, 0xc000614060) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/drivers/exec/driver.go:225 +0x1a7 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/drivers/exec.(*Driver).Fingerprint Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/drivers/exec/driver.go:217 +0x7f Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 123 [IO wait, 468 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll.runtime_pollWait(0x7f2c51e968c8, 0x72, 0xffffffffffffffff) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: runtime/netpoll.go:203 +0x55 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll.(*pollDesc).wait(0xc00011dd18, 0x72, 0x1000, 0x1000, 0xffffffffffffffff) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll/fd_poll_runtime.go:87 +0x45 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll.(*pollDesc).waitRead(...) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll/fd_poll_runtime.go:92 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll.(*FD).Read(0xc00011dd00, 0xc000872000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll/fd_unix.go:169 +0x19b Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net.(*netFD).Read(0xc00011dd00, 0xc000872000, 0x1000, 0x1000, 0x30, 0x28, 0x2def8c0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/fd_unix.go:202 +0x4f Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net.(*conn).Read(0xc0000d5210, 0xc000872000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/net.go:184 +0x8e Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http.(*persistConn).Read(0xc00083fe60, 0xc000872000, 0x1000, 0x1000, 0x734fea, 0xc000113900, 0x37bba20) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/transport.go:1825 +0x75 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: bufio.(*Reader).fill(0xc0005c1200) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: bufio/bufio.go:100 +0x103 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: bufio.(*Reader).ReadSlice(0xc0005c1200, 0x736d0a, 0x3094cf6, 0x4, 0xc0007def30, 0xc000113900, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: bufio/bufio.go:359 +0x3d Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/internal.readChunkLine(0xc0005c1200, 0x7f2c51ea4260, 0x7f2c7bcc9108, 0x203000, 0x17, 0x58058d8) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/internal/chunked.go:122 +0x34 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/internal.(*chunkedReader).beginChunk(0xc0008fe000) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/internal/chunked.go:48 +0x32 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/internal.(*chunkedReader).Read(0xc0008fe000, 0xc000645000, 0x200, 0x200, 0x78, 0x80, 0xc00011dc00) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/internal/chunked.go:93 +0x132 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http.(*body).readLocked(0xc000539e80, 0xc000645000, 0x200, 0x200, 0xc0003ebcb0, 0xc0003ebb98, 0xc0003ebc20) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/transfer.go:847 +0x5f Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http.(*body).Read(0xc000539e80, 0xc000645000, 0x200, 0x200, 0x0, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/transfer.go:839 +0xf2 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http.(*bodyEOFSignal).Read(0xc000539ec0, 0xc000645000, 0x200, 0x200, 0x0, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/transport.go:2649 +0xde Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: encoding/json.(*Decoder).refill(0xc0007c78c0, 0xc0007c78c0, 0x40e2cb) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: encoding/json/stream.go:165 +0xeb Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: encoding/json.(*Decoder).readValue(0xc0007c78c0, 0x0, 0x0, 0x2f40da0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: encoding/json/stream.go:140 +0x1e8 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: encoding/json.(*Decoder).Decode(0xc0007c78c0, 0x27b1c00, 0xc0008c2f28, 0x37bba60, 0xc000539ec0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: encoding/json/stream.go:63 +0x79 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient.(*Client).waitContainer(0xc0002f4510, 0xc0008365c0, 0x40, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient/container.go:1396 +0x2cf Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient.(*Client).WaitContainer(...) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient/container.go:1373 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/drivers/docker.(*taskHandle).run(0xc000823dd0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/drivers/docker/handle.go:191 +0xa6 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/drivers/docker.(*Driver).RecoverTask Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/drivers/docker/driver.go:238 +0x622 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 113 [select, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/vaultclient.(*vaultClient).run(0xc000606d20) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/vaultclient/vaultclient.go:494 +0x18e Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/client/vaultclient.(*vaultClient).Start Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/vaultclient/vaultclient.go:207 +0xbc Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 119 [select, 468 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc.(*ccBalancerWrapper).watcher(0xc000840700) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/balancer_conn_wrappers.go:69 +0xc2 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/google.golang.org/grpc.newCCBalancerWrapper Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/balancer_conn_wrappers.go:60 +0x16d Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 161 [IO wait, 108 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll.runtime_pollWait(0x7f2c51e96d28, 0x72, 0xffffffffffffffff) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: runtime/netpoll.go:203 +0x55 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll.(*pollDesc).wait(0xc0008a8918, 0x72, 0x8000, 0x8000, 0xffffffffffffffff) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll/fd_poll_runtime.go:87 +0x45 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll.(*pollDesc).waitRead(...) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll/fd_poll_runtime.go:92 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll.(*FD).Read(0xc0008a8900, 0xc0008f4000, 0x8000, 0x8000, 0x0, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll/fd_unix.go:169 +0x19b Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net.(*netFD).Read(0xc0008a8900, 0xc0008f4000, 0x8000, 0x8000, 0x800000601, 0x0, 0x60100000000) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/fd_unix.go:202 +0x4f Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net.(*conn).Read(0xc000011a60, 0xc0008f4000, 0x8000, 0x8000, 0x0, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/net.go:184 +0x8e Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: bufio.(*Reader).Read(0xc0008ebf20, 0xc0001a2e38, 0x9, 0x9, 0xc0007a4e08, 0x40a183, 0xc0008f2360) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: bufio/bufio.go:226 +0x24f Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: io.ReadAtLeast(0x37b4d20, 0xc0008ebf20, 0xc0001a2e38, 0x9, 0x9, 0x9, 0x466970, 0xc0008f2360, 0xc0007a4e40) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: io/io.go:310 +0x87 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: io.ReadFull(...) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: io/io.go:329 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/golang.org/x/net/http2.readFrameHeader(0xc0001a2e38, 0x9, 0x9, 0x37b4d20, 0xc0008ebf20, 0x0, 0xc000000000, 0x100000000411ce8, 0xc0008e8880) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/golang.org/x/net/http2/frame.go:237 +0x87 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/golang.org/x/net/http2.(*Framer).ReadFrame(0xc0001a2e00, 0xc001210600, 0xc001210600, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/golang.org/x/net/http2/frame.go:492 +0xa1 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport.(*http2Client).reader(0xc00032d340) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport/http2_client.go:1265 +0x16f Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport.newHTTP2Client Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport/http2_client.go:300 +0xd23 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 118 [chan receive]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin.pidWait(0x1ea02, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/process.go:17 +0x91 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin.(*Client).reattach.func1(0xc00019a460, 0x1ea02) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/client.go:820 +0x7b Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin.(*Client).reattach Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/client.go:813 +0x17d Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 142 [select, 28 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/drivers/docker.(*containerReconciler).removeDanglingContainersGoroutine(0xc0005c0300) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/drivers/docker/reconciler.go:74 +0x128 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/drivers/docker.(*containerReconciler).Start.func1 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/drivers/docker/reconciler.go:53 +0x3e Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 162 [select, 108 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport.(*controlBuffer).get(0xc0008e8870, 0x1, 0x0, 0x0, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport/controlbuf.go:395 +0x122 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport.(*loopyWriter).run(0xc0008fc000, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport/controlbuf.go:513 +0x1cd Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport.newHTTP2Client.func3(0xc00032d340) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport/http2_client.go:346 +0x7b Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport.newHTTP2Client Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport/http2_client.go:344 +0xedb Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 143 [select, 24 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/drivers/docker.(*Driver).handleFingerprint(0xc00031bc20, 0x3814440, 0xc0005158c0, 0xc0006145a0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/drivers/docker/fingerprint.go:68 +0x1cb Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/drivers/docker.(*Driver).Fingerprint Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/drivers/docker/fingerprint.go:21 +0x94 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 141 [select, 24 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/drivers/java.(*Driver).handleFingerprint(0xc000332900, 0x3814440, 0xc000515600, 0xc0006143c0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/drivers/java/driver.go:188 +0x173 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/drivers/java.(*Driver).Fingerprint Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/drivers/java/driver.go:181 +0x7f Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 121 [select, 468 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin.(*gRPCBrokerClientImpl).Recv(0xc000840780, 0x0, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/grpc_broker.go:231 +0xb3 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin.(*GRPCBroker).Run(0xc000187d60) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/grpc_broker.go:411 +0x7a Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin.newGRPCClient Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/grpc_client.go:62 +0x23f Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 120 [chan receive, 468 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc.(*addrConn).resetTransport(0xc0003d6dc0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/clientconn.go:1179 +0x6ea Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/google.golang.org/grpc.(*addrConn).connect Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/clientconn.go:815 +0x128 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 180 [chan receive, 468 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc.(*addrConn).resetTransport(0xc0003d7340) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/clientconn.go:1179 +0x6ea Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/google.golang.org/grpc.(*addrConn).connect Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/clientconn.go:815 +0x128 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 179 [select, 468 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc.(*ccBalancerWrapper).watcher(0xc000840f80) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/balancer_conn_wrappers.go:69 +0xc2 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/google.golang.org/grpc.newCCBalancerWrapper Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/balancer_conn_wrappers.go:60 +0x16d Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 178 [chan receive]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin.pidWait(0x1ea9e, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/process.go:17 +0x91 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin.(*Client).reattach.func1(0xc00019ab40, 0x1ea9e) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/client.go:820 +0x7b Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin.(*Client).reattach Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/client.go:813 +0x17d Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 128 [select, 468 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http.(*persistConn).readLoop(0xc00083fe60) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/transport.go:2099 +0x99e Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by net/http.(*Transport).dialConn Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/transport.go:1647 +0xc56 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 129 [select, 468 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http.(*persistConn).writeLoop(0xc00083fe60) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/transport.go:2277 +0x11c Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by net/http.(*Transport).dialConn Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/transport.go:1648 +0xc7b Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 181 [select, 468 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin.(*gRPCBrokerClientImpl).Recv(0xc000841000, 0x0, 0x0, 0xc00085ea20) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/grpc_broker.go:231 +0xb3 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin.(*GRPCBroker).Run(0xc000187f90) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/grpc_broker.go:411 +0x7a Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin.newGRPCClient Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/grpc_client.go:62 +0x23f Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 182 [select, 468 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport.(*Stream).waitOnHeader(0xc000934200) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport/transport.go:318 +0xcc Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport.(*Stream).RecvCompress(...) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport/transport.go:333 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc.(*csAttempt).recvMsg(0xc000910d00, 0x2ea2d00, 0xc000916410, 0x0, 0x8, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/stream.go:871 +0x753 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc.(*clientStream).RecvMsg.func1(0xc000910d00, 0x203000, 0x203000) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/stream.go:736 +0x46 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc.(*clientStream).withRetry(0xc00087ea20, 0xc0007a7e30, 0xc0007a7e00, 0x382dbe0, 0x30) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/stream.go:590 +0x360 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc.(*clientStream).RecvMsg(0xc00087ea20, 0x2ea2d00, 0xc000916410, 0x20, 0xc0005c3200) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/stream.go:735 +0x103 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/internal/plugin.(*gRPCBrokerStartStreamClient).Recv(0xc00081bab0, 0x31840c8, 0xc00085fc20, 0xc000841000) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/internal/plugin/grpc_broker.pb.go:149 +0x62 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin.(*gRPCBrokerClientImpl).StartStream(0xc000841000, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/grpc_broker.go:194 +0x187 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin.newGRPCClient Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/grpc_client.go:63 +0x261 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 183 [IO wait, 468 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll.runtime_pollWait(0x7f2c51e96708, 0x72, 0xffffffffffffffff) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: runtime/netpoll.go:203 +0x55 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll.(*pollDesc).wait(0xc000910f18, 0x72, 0x1000, 0x1000, 0xffffffffffffffff) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll/fd_poll_runtime.go:87 +0x45 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll.(*pollDesc).waitRead(...) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll/fd_poll_runtime.go:92 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll.(*FD).Read(0xc000910f00, 0xc00091b000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll/fd_unix.go:169 +0x19b Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net.(*netFD).Read(0xc000910f00, 0xc00091b000, 0x1000, 0x1000, 0x30, 0x28, 0x2def8c0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/fd_unix.go:202 +0x4f Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net.(*conn).Read(0xc0000d52e8, 0xc00091b000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/net.go:184 +0x8e Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http.(*persistConn).Read(0xc00087eb40, 0xc00091b000, 0x1000, 0x1000, 0x734fea, 0xc000934100, 0x37bba20) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/transport.go:1825 +0x75 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: bufio.(*Reader).fill(0xc00093e180) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: bufio/bufio.go:100 +0x103 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: bufio.(*Reader).ReadSlice(0xc00093e180, 0x736d0a, 0x3094cf6, 0x4, 0xc000870510, 0xc000934100, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: bufio/bufio.go:359 +0x3d Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/internal.readChunkLine(0xc00093e180, 0xc00013dc00, 0x7f2c7bcc9108, 0x203000, 0xc0007a2b70, 0x411ce8) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/internal/chunked.go:122 +0x34 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/internal.(*chunkedReader).beginChunk(0xc00086efc0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/internal/chunked.go:48 +0x32 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/internal.(*chunkedReader).Read(0xc00086efc0, 0xc000757400, 0x200, 0x200, 0x78, 0x80, 0xc000910e00) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/internal/chunked.go:93 +0x132 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http.(*body).readLocked(0xc000841540, 0xc000757400, 0x200, 0x200, 0xc00093dcb0, 0xc00093db98, 0xc00093dc20) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/transfer.go:847 +0x5f Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http.(*body).Read(0xc000841540, 0xc000757400, 0x200, 0x200, 0x0, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/transfer.go:839 +0xf2 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http.(*bodyEOFSignal).Read(0xc000841580, 0xc000757400, 0x200, 0x200, 0x0, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/transport.go:2649 +0xde Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: encoding/json.(*Decoder).refill(0xc00086d600, 0xc00086d600, 0x40e2cb) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: encoding/json/stream.go:165 +0xeb Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: encoding/json.(*Decoder).readValue(0xc00086d600, 0x0, 0x0, 0x2f40da0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: encoding/json/stream.go:140 +0x1e8 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: encoding/json.(*Decoder).Decode(0xc00086d600, 0x27b1c00, 0xc000857518, 0x37bba60, 0xc000841580) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: encoding/json/stream.go:63 +0x79 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient.(*Client).waitContainer(0xc0002f4510, 0xc000836a80, 0x40, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient/container.go:1396 +0x2cf Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient.(*Client).WaitContainer(...) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient/container.go:1373 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/drivers/docker.(*taskHandle).run(0xc0008702d0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/drivers/docker/handle.go:191 +0xa6 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/drivers/docker.(*Driver).RecoverTask Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/drivers/docker/driver.go:238 +0x622 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 187 [select, 468 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc.newClientStream.func5(0xc0003fdc00, 0xc00083fd40, 0x3814500, 0xc000815ef0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/stream.go:319 +0xd7 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/google.golang.org/grpc.newClientStream Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/stream.go:318 +0xbbf Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 199 [select, 468 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc.(*ccBalancerWrapper).watcher(0xc0008419c0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/balancer_conn_wrappers.go:69 +0xc2 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/google.golang.org/grpc.newCCBalancerWrapper Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/balancer_conn_wrappers.go:60 +0x16d Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 198 [chan receive]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin.pidWait(0x1e9d8, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/process.go:17 +0x91 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin.(*Client).reattach.func1(0xc0000e28c0, 0x1e9d8) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/client.go:820 +0x7b Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin.(*Client).reattach Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/client.go:813 +0x17d Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 188 [select, 468 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin.(*gRPCBrokerClientImpl).StartStream.func1(0xc00085e780, 0xc000840780, 0x3838980, 0xc00081b850) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/grpc_broker.go:181 +0x146 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin.(*gRPCBrokerClientImpl).StartStream Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/grpc_broker.go:179 +0x165 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 189 [IO wait, 108 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll.runtime_pollWait(0x7f2c51e967e8, 0x72, 0xffffffffffffffff) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: runtime/netpoll.go:203 +0x55 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll.(*pollDesc).wait(0xc000910a98, 0x72, 0x8000, 0x8000, 0xffffffffffffffff) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll/fd_poll_runtime.go:87 +0x45 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll.(*pollDesc).waitRead(...) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll/fd_poll_runtime.go:92 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll.(*FD).Read(0xc000910a80, 0xc00091c000, 0x8000, 0x8000, 0x0, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll/fd_unix.go:169 +0x19b Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net.(*netFD).Read(0xc000910a80, 0xc00091c000, 0x8000, 0x8000, 0x800000601, 0x0, 0x60100000000) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/fd_unix.go:202 +0x4f Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net.(*conn).Read(0xc0000d52d0, 0xc00091c000, 0x8000, 0x8000, 0x0, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/net.go:184 +0x8e Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: bufio.(*Reader).Read(0xc0005c1b60, 0xc000177618, 0x9, 0x9, 0xc0007a0e08, 0x40a183, 0xc00085fb60) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: bufio/bufio.go:226 +0x24f Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: io.ReadAtLeast(0x37b4d20, 0xc0005c1b60, 0xc000177618, 0x9, 0x9, 0x9, 0x466970, 0xc00085fb60, 0xc0007a0e40) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: io/io.go:310 +0x87 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: io.ReadFull(...) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: io/io.go:329 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/golang.org/x/net/http2.readFrameHeader(0xc000177618, 0x9, 0x9, 0x37b4d20, 0xc0005c1b60, 0x0, 0xc000000000, 0x100000000411ce8, 0xc000916290) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/golang.org/x/net/http2/frame.go:237 +0x87 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/golang.org/x/net/http2.(*Framer).ReadFrame(0xc0001775e0, 0xc001210620, 0xc001210620, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/golang.org/x/net/http2/frame.go:492 +0xa1 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport.(*http2Client).reader(0xc0004f4700) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport/http2_client.go:1265 +0x16f Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport.newHTTP2Client Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport/http2_client.go:300 +0xd23 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 190 [select, 108 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport.(*controlBuffer).get(0xc000916280, 0x1, 0x0, 0x0, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport/controlbuf.go:395 +0x122 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport.(*loopyWriter).run(0xc0005c1c20, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport/controlbuf.go:513 +0x1cd Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport.newHTTP2Client.func3(0xc0004f4700) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport/http2_client.go:346 +0x7b Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport.newHTTP2Client Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport/http2_client.go:344 +0xedb Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 194 [select, 468 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc.newClientStream.func5(0xc000914000, 0xc00087ea20, 0x3814500, 0xc00086ec60) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/stream.go:319 +0xd7 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/google.golang.org/grpc.newClientStream Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/stream.go:318 +0xbbf Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 192 [select, 468 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http.(*persistConn).readLoop(0xc00087eb40) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/transport.go:2099 +0x99e Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by net/http.(*Transport).dialConn Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/transport.go:1647 +0xc56 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 193 [select, 468 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http.(*persistConn).writeLoop(0xc00087eb40) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/transport.go:2277 +0x11c Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by net/http.(*Transport).dialConn Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/transport.go:1648 +0xc7b Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 195 [select, 468 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin.(*gRPCBrokerClientImpl).StartStream.func1(0xc00085fc20, 0xc000841000, 0x3838980, 0xc00081bab0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/grpc_broker.go:181 +0x146 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin.(*gRPCBrokerClientImpl).StartStream Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/grpc_broker.go:179 +0x165 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 196 [select, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client.(*Client).watchNodeUpdates(0xc00019e780) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/client.go:2152 +0x119 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/client.(*Client).registerAndHeartbeat Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/client.go:1508 +0x68 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 197 [select, 468 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client.(*Client).watchNodeEvents(0xc00019e780) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/client.go:1684 +0x160 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/client.(*Client).registerAndHeartbeat Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/client.go:1511 +0x8d Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 200 [chan receive, 468 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc.(*addrConn).resetTransport(0xc0003d78c0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/clientconn.go:1179 +0x6ea Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/google.golang.org/grpc.(*addrConn).connect Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/clientconn.go:815 +0x128 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 201 [select, 468 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin.(*gRPCBrokerClientImpl).Recv(0xc000841a40, 0x26a6d63, 0x28, 0xc00085fec0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/grpc_broker.go:231 +0xb3 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin.(*GRPCBroker).Run(0xc0009166e0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/grpc_broker.go:411 +0x7a Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin.newGRPCClient Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/grpc_client.go:62 +0x23f Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 202 [select, 468 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport.(*Stream).waitOnHeader(0xc0008ec700) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport/transport.go:318 +0xcc Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport.(*Stream).RecvCompress(...) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport/transport.go:333 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc.(*csAttempt).recvMsg(0xc0008a8f00, 0x2ea2d00, 0xc0008e8c30, 0x0, 0x8, 0x7f2c53048b48) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/stream.go:871 +0x753 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc.(*clientStream).RecvMsg.func1(0xc0008a8f00, 0x203000, 0x203000) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/stream.go:736 +0x46 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc.(*clientStream).withRetry(0xc0008afc20, 0xc0003e9e30, 0xc0003e9e00, 0x382dbe0, 0x30) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/stream.go:590 +0x360 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc.(*clientStream).RecvMsg(0xc0008afc20, 0x2ea2d00, 0xc0008e8c30, 0x20, 0xc000942a80) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/stream.go:735 +0x103 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/internal/plugin.(*gRPCBrokerStartStreamClient).Recv(0xc0008a5da0, 0x31840c8, 0xc0008f2a20, 0xc000841a40) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/internal/plugin/grpc_broker.pb.go:149 +0x62 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin.(*gRPCBrokerClientImpl).StartStream(0xc000841a40, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/grpc_broker.go:194 +0x187 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin.newGRPCClient Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/grpc_client.go:63 +0x261 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 203 [IO wait, 468 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll.runtime_pollWait(0x7f2c51e96548, 0x72, 0xffffffffffffffff) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: runtime/netpoll.go:203 +0x55 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll.(*pollDesc).wait(0xc000911f98, 0x72, 0x1000, 0x1000, 0xffffffffffffffff) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll/fd_poll_runtime.go:87 +0x45 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll.(*pollDesc).waitRead(...) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll/fd_poll_runtime.go:92 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll.(*FD).Read(0xc000911f80, 0xc000990000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll/fd_unix.go:169 +0x19b Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net.(*netFD).Read(0xc000911f80, 0xc000990000, 0x1000, 0x1000, 0x30, 0x28, 0x2def8c0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/fd_unix.go:202 +0x4f Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net.(*conn).Read(0xc0000d5458, 0xc000990000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/net.go:184 +0x8e Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http.(*persistConn).Read(0xc000970a20, 0xc000990000, 0x1000, 0x1000, 0x734fea, 0xc000934b00, 0x37bba20) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/transport.go:1825 +0x75 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: bufio.(*Reader).fill(0xc000988300) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: bufio/bufio.go:100 +0x103 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: bufio.(*Reader).ReadSlice(0xc000988300, 0x736d0a, 0x3094cf6, 0x4, 0xc0007df0e0, 0xc000934b00, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: bufio/bufio.go:359 +0x3d Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/internal.readChunkLine(0xc000988300, 0x7f2c51ea6a38, 0x7f2c7bcc9108, 0x203000, 0x17, 0x58058d8) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/internal/chunked.go:122 +0x34 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/internal.(*chunkedReader).beginChunk(0xc0008fe4e0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/internal/chunked.go:48 +0x32 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/internal.(*chunkedReader).Read(0xc0008fe4e0, 0xc000645200, 0x200, 0x200, 0x78, 0x80, 0xc000911e80) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/internal/chunked.go:93 +0x132 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http.(*body).readLocked(0xc000a18100, 0xc000645200, 0x200, 0x200, 0xc000939cb0, 0xc000939b98, 0xc000939c20) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/transfer.go:847 +0x5f Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http.(*body).Read(0xc000a18100, 0xc000645200, 0x200, 0x200, 0x0, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/transfer.go:839 +0xf2 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http.(*bodyEOFSignal).Read(0xc000a18140, 0xc000645200, 0x200, 0x200, 0x0, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/transport.go:2649 +0xde Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: encoding/json.(*Decoder).refill(0xc0007c7ce0, 0xc0007c7ce0, 0x40e2cb) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: encoding/json/stream.go:165 +0xeb Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: encoding/json.(*Decoder).readValue(0xc0007c7ce0, 0x0, 0x0, 0x2f40da0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: encoding/json/stream.go:140 +0x1e8 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: encoding/json.(*Decoder).Decode(0xc0007c7ce0, 0x27b1c00, 0xc0008c3018, 0x37bba60, 0xc000a18140) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: encoding/json/stream.go:63 +0x79 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient.(*Client).waitContainer(0xc0002f4510, 0xc000836e80, 0x40, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient/container.go:1396 +0x2cf Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient.(*Client).WaitContainer(...) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient/container.go:1373 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/drivers/docker.(*taskHandle).run(0xc000870750) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/drivers/docker/handle.go:191 +0xa6 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/drivers/docker.(*Driver).RecoverTask Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/drivers/docker/driver.go:238 +0x622 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 212 [chan receive, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc.(*addrConn).resetTransport(0xc00099c000) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/clientconn.go:1179 +0x6ea Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/google.golang.org/grpc.(*addrConn).connect Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/clientconn.go:815 +0x128 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 163 [IO wait, 108 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll.runtime_pollWait(0x7f2c51e96b68, 0x72, 0xffffffffffffffff) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: runtime/netpoll.go:203 +0x55 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll.(*pollDesc).wait(0xc0008a8c98, 0x72, 0x8000, 0x8000, 0xffffffffffffffff) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll/fd_poll_runtime.go:87 +0x45 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll.(*pollDesc).waitRead(...) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll/fd_poll_runtime.go:92 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll.(*FD).Read(0xc0008a8c80, 0xc000a00000, 0x8000, 0x8000, 0x0, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll/fd_unix.go:169 +0x19b Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net.(*netFD).Read(0xc0008a8c80, 0xc000a00000, 0x8000, 0x8000, 0x800000601, 0x0, 0x60100000000) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/fd_unix.go:202 +0x4f Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net.(*conn).Read(0xc000011a68, 0xc000a00000, 0x8000, 0x8000, 0x0, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/net.go:184 +0x8e Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: bufio.(*Reader).Read(0xc0008fc360, 0xc0001a2f18, 0x9, 0x9, 0xc0007a5e08, 0x40a183, 0xc0008f2780) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: bufio/bufio.go:226 +0x24f Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: io.ReadAtLeast(0x37b4d20, 0xc0008fc360, 0xc0001a2f18, 0x9, 0x9, 0x9, 0x466970, 0xc0008f2780, 0xc0007a5e40) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: io/io.go:310 +0x87 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: io.ReadFull(...) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: io/io.go:329 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/golang.org/x/net/http2.readFrameHeader(0xc0001a2f18, 0x9, 0x9, 0x37b4d20, 0xc0008fc360, 0x0, 0xc000000000, 0x100000000411ce8, 0xc0008e8a60) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/golang.org/x/net/http2/frame.go:237 +0x87 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/golang.org/x/net/http2.(*Framer).ReadFrame(0xc0001a2ee0, 0xc0012b1e20, 0xc0012b1e20, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/golang.org/x/net/http2/frame.go:492 +0xa1 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport.(*http2Client).reader(0xc00032d500) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport/http2_client.go:1265 +0x16f Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport.newHTTP2Client Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport/http2_client.go:300 +0xd23 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 164 [select, 108 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport.(*controlBuffer).get(0xc0008e8a50, 0x1, 0x0, 0x0, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport/controlbuf.go:395 +0x122 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport.(*loopyWriter).run(0xc0008fc420, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport/controlbuf.go:513 +0x1cd Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport.newHTTP2Client.func3(0xc00032d500) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport/http2_client.go:346 +0x7b Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport.newHTTP2Client Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport/http2_client.go:344 +0xedb Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 211 [select, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc.(*ccBalancerWrapper).watcher(0xc000984740) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/balancer_conn_wrappers.go:69 +0xc2 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/google.golang.org/grpc.newCCBalancerWrapper Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/balancer_conn_wrappers.go:60 +0x16d Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 210 [chan receive]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin.pidWait(0x1e3f, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/process.go:17 +0x91 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin.(*Client).reattach.func1(0xc0000e3860, 0x1e3f) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/client.go:820 +0x7b Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin.(*Client).reattach Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/client.go:813 +0x17d Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 165 [select, 468 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc.newClientStream.func5(0xc000914380, 0xc0008afc20, 0x3814500, 0xc0008fe3c0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/stream.go:319 +0xd7 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/google.golang.org/grpc.newClientStream Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/stream.go:318 +0xbbf Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 166 [select, 468 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin.(*gRPCBrokerClientImpl).StartStream.func1(0xc0008f2a20, 0xc000841a40, 0x3838980, 0xc0008a5da0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/grpc_broker.go:181 +0x146 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin.(*gRPCBrokerClientImpl).StartStream Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/grpc_broker.go:179 +0x165 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 208 [select, 468 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http.(*persistConn).readLoop(0xc000970a20) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/transport.go:2099 +0x99e Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by net/http.(*Transport).dialConn Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/transport.go:1647 +0xc56 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 209 [select, 468 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http.(*persistConn).writeLoop(0xc000970a20) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/transport.go:2277 +0x11c Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by net/http.(*Transport).dialConn Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/transport.go:1648 +0xc7b Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 213 [select, 468 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin.(*gRPCBrokerClientImpl).Recv(0xc0009847c0, 0x0, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/grpc_broker.go:231 +0xb3 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin.(*GRPCBroker).Run(0xc00097d270) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/grpc_broker.go:411 +0x7a Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin.newGRPCClient Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/grpc_client.go:62 +0x23f Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 214 [select, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport.(*Stream).waitOnHeader(0xc0008ecf00) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport/transport.go:318 +0xcc Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport.(*Stream).RecvCompress(...) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport/transport.go:333 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc.(*csAttempt).recvMsg(0xc0008a9300, 0x2ea2d00, 0xc000a56730, 0x0, 0xc000a48650, 0x7f2c51e8227c) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/stream.go:871 +0x753 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc.(*clientStream).RecvMsg.func1(0xc0008a9300, 0x203000, 0x203000) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/stream.go:736 +0x46 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc.(*clientStream).withRetry(0xc0008afe60, 0xc0007ade30, 0xc0007ade00, 0x382dbe0, 0x30) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/stream.go:590 +0x360 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc.(*clientStream).RecvMsg(0xc0008afe60, 0x2ea2d00, 0xc000a56730, 0x20, 0xc00097b380) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/stream.go:735 +0x103 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/internal/plugin.(*gRPCBrokerStartStreamClient).Recv(0xc000a48650, 0x31840c8, 0xc0008f2fc0, 0xc0009847c0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/internal/plugin/grpc_broker.pb.go:149 +0x62 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin.(*gRPCBrokerClientImpl).StartStream(0xc0009847c0, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/grpc_broker.go:194 +0x187 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin.newGRPCClient Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/grpc_client.go:63 +0x261 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 215 [IO wait, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll.runtime_pollWait(0x7f2c51e96468, 0x72, 0xffffffffffffffff) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: runtime/netpoll.go:203 +0x55 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll.(*pollDesc).wait(0xc0008a9518, 0x72, 0x1000, 0x1000, 0xffffffffffffffff) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll/fd_poll_runtime.go:87 +0x45 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll.(*pollDesc).waitRead(...) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll/fd_poll_runtime.go:92 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll.(*FD).Read(0xc0008a9500, 0xc000a46000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll/fd_unix.go:169 +0x19b Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net.(*netFD).Read(0xc0008a9500, 0xc000a46000, 0x1000, 0x1000, 0x30, 0x28, 0x2def8c0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/fd_unix.go:202 +0x4f Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net.(*conn).Read(0xc000011aa0, 0xc000a46000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/net.go:184 +0x8e Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http.(*persistConn).Read(0xc000a44000, 0xc000a46000, 0x1000, 0x1000, 0x734fea, 0xc0008ec900, 0x37bba20) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/transport.go:1825 +0x75 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: bufio.(*Reader).fill(0xc0008fd380) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: bufio/bufio.go:100 +0x103 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: bufio.(*Reader).ReadSlice(0xc0008fd380, 0x736d0a, 0x3094cf6, 0x4, 0xc0007df290, 0xc0008ec900, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: bufio/bufio.go:359 +0x3d Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/internal.readChunkLine(0xc0008fd380, 0x57da1a0, 0x7f2c7bcc97d0, 0x203000, 0xc000a3fb70, 0x411ce8) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/internal/chunked.go:122 +0x34 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/internal.(*chunkedReader).beginChunk(0xc0008ffb00) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/internal/chunked.go:48 +0x32 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/internal.(*chunkedReader).Read(0xc0008ffb00, 0xc000645400, 0x200, 0x200, 0x78, 0x80, 0xc0008a9400) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/internal/chunked.go:93 +0x132 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http.(*body).readLocked(0xc000a18640, 0xc000645400, 0x200, 0x200, 0x0, 0xc000937b98, 0xc000937c20) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/transfer.go:847 +0x5f Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http.(*body).Read(0xc000a18640, 0xc000645400, 0x200, 0x200, 0x0, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/transfer.go:839 +0xf2 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http.(*bodyEOFSignal).Read(0xc000a18680, 0xc000645400, 0x200, 0x200, 0x0, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/transport.go:2649 +0xde Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: encoding/json.(*Decoder).refill(0xc000a42840, 0xc000a42840, 0x40e2cb) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: encoding/json/stream.go:165 +0xeb Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: encoding/json.(*Decoder).readValue(0xc000a42840, 0x0, 0x0, 0x2f40da0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: encoding/json/stream.go:140 +0x1e8 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: encoding/json.(*Decoder).Decode(0xc000a42840, 0x27b1c00, 0xc0008c3878, 0x37bba60, 0xc000a18680) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: encoding/json/stream.go:63 +0x79 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient.(*Client).waitContainer(0xc0002f4510, 0xc000837840, 0x40, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient/container.go:1396 +0x2cf Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient.(*Client).WaitContainer(...) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient/container.go:1373 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/drivers/docker.(*taskHandle).run(0xc000870b40) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/drivers/docker/handle.go:191 +0xa6 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/drivers/docker.(*Driver).RecoverTask Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/drivers/docker/driver.go:238 +0x622 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 167 [IO wait, 108 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll.runtime_pollWait(0x7f2c51e96628, 0x72, 0xffffffffffffffff) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: runtime/netpoll.go:203 +0x55 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll.(*pollDesc).wait(0xc0008a9098, 0x72, 0x8000, 0x8000, 0xffffffffffffffff) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll/fd_poll_runtime.go:87 +0x45 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll.(*pollDesc).waitRead(...) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll/fd_poll_runtime.go:92 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll.(*FD).Read(0xc0008a9080, 0xc000a22000, 0x8000, 0x8000, 0x0, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll/fd_unix.go:169 +0x19b Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net.(*netFD).Read(0xc0008a9080, 0xc000a22000, 0x8000, 0x8000, 0x800000601, 0x0, 0x60100000000) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/fd_unix.go:202 +0x4f Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net.(*conn).Read(0xc000011a88, 0xc000a22000, 0x8000, 0x8000, 0x0, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/net.go:184 +0x8e Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: bufio.(*Reader).Read(0xc0008fcc60, 0xc0001a2ff8, 0x9, 0x9, 0xc000947e08, 0x40a183, 0xc0008f2ea0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: bufio/bufio.go:226 +0x24f Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: io.ReadAtLeast(0x37b4d20, 0xc0008fcc60, 0xc0001a2ff8, 0x9, 0x9, 0x9, 0x466970, 0xc0008f2ea0, 0xc000947e40) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: io/io.go:310 +0x87 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: io.ReadFull(...) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: io/io.go:329 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/golang.org/x/net/http2.readFrameHeader(0xc0001a2ff8, 0x9, 0x9, 0x37b4d20, 0xc0008fcc60, 0x0, 0xc000000000, 0x100000000411ce8, 0xc0008e8dd0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/golang.org/x/net/http2/frame.go:237 +0x87 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/golang.org/x/net/http2.(*Framer).ReadFrame(0xc0001a2fc0, 0xc0012b1e40, 0xc0012b1e40, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/golang.org/x/net/http2/frame.go:492 +0xa1 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport.(*http2Client).reader(0xc00032d6c0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport/http2_client.go:1265 +0x16f Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport.newHTTP2Client Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport/http2_client.go:300 +0xd23 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 168 [select, 108 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport.(*controlBuffer).get(0xc0008e8dc0, 0x1, 0x0, 0x0, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport/controlbuf.go:395 +0x122 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport.(*loopyWriter).run(0xc0008fcd80, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport/controlbuf.go:513 +0x1cd Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport.newHTTP2Client.func3(0xc00032d6c0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport/http2_client.go:346 +0x7b Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport.newHTTP2Client Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport/http2_client.go:344 +0xedb Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 175 [select, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc.newClientStream.func5(0xc000914700, 0xc0008afe60, 0x3814500, 0xc0008fe7e0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/stream.go:319 +0xd7 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/google.golang.org/grpc.newClientStream Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/stream.go:318 +0xbbf Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 170 [select, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http.(*persistConn).readLoop(0xc000a44000) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/transport.go:2099 +0x99e Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by net/http.(*Transport).dialConn Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/transport.go:1647 +0xc56 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 171 [select, 468 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http.(*persistConn).writeLoop(0xc000a44000) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/transport.go:2277 +0x11c Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by net/http.(*Transport).dialConn Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/transport.go:1648 +0xc7b Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 217 [select, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc.(*ccBalancerWrapper).watcher(0xc0009852c0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/balancer_conn_wrappers.go:69 +0xc2 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/google.golang.org/grpc.newCCBalancerWrapper Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/balancer_conn_wrappers.go:60 +0x16d Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 216 [chan receive]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin.pidWait(0xff1d, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/process.go:17 +0x91 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin.(*Client).reattach.func1(0xc00002a3c0, 0xff1d) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/client.go:820 +0x7b Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin.(*Client).reattach Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/client.go:813 +0x17d Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 176 [select, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin.(*gRPCBrokerClientImpl).StartStream.func1(0xc0008f2fc0, 0xc0009847c0, 0x3838980, 0xc000a48650) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/grpc_broker.go:181 +0x146 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin.(*gRPCBrokerClientImpl).StartStream Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/grpc_broker.go:179 +0x165 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 218 [chan receive, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc.(*addrConn).resetTransport(0xc00099c840) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/clientconn.go:1179 +0x6ea Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/google.golang.org/grpc.(*addrConn).connect Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/clientconn.go:815 +0x128 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 219 [select, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin.(*gRPCBrokerClientImpl).Recv(0xc000985340, 0x0, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/grpc_broker.go:231 +0xb3 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin.(*GRPCBroker).Run(0xc0009dcf00) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/grpc_broker.go:411 +0x7a Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin.newGRPCClient Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/grpc_client.go:62 +0x23f Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 220 [select, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport.(*Stream).waitOnHeader(0xc00071e000) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport/transport.go:318 +0xcc Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport.(*Stream).RecvCompress(...) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport/transport.go:333 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc.(*csAttempt).recvMsg(0xc000a5e180, 0x2ea2d00, 0xc000a56aa0, 0x0, 0x8, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/stream.go:871 +0x753 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc.(*clientStream).RecvMsg.func1(0xc000a5e180, 0x203000, 0x203000) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/stream.go:736 +0x46 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc.(*clientStream).withRetry(0xc00053c000, 0xc0003ede30, 0xc0003ede00, 0x382dbe0, 0x30) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/stream.go:590 +0x360 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc.(*clientStream).RecvMsg(0xc00053c000, 0x2ea2d00, 0xc000a56aa0, 0x20, 0xc0009e4480) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/stream.go:735 +0x103 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/internal/plugin.(*gRPCBrokerStartStreamClient).Recv(0xc0009c90e0, 0x31840c8, 0xc0008f3080, 0xc000985340) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/internal/plugin/grpc_broker.pb.go:149 +0x62 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin.(*gRPCBrokerClientImpl).StartStream(0xc000985340, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/grpc_broker.go:194 +0x187 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin.newGRPCClient Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/grpc_client.go:63 +0x261 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 221 [IO wait, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll.runtime_pollWait(0x7f2c51e962a8, 0x72, 0xffffffffffffffff) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: runtime/netpoll.go:203 +0x55 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll.(*pollDesc).wait(0xc000a5e398, 0x72, 0x1000, 0x1000, 0xffffffffffffffff) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll/fd_poll_runtime.go:87 +0x45 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll.(*pollDesc).waitRead(...) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll/fd_poll_runtime.go:92 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll.(*FD).Read(0xc000a5e380, 0xc000634000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll/fd_unix.go:169 +0x19b Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net.(*netFD).Read(0xc000a5e380, 0xc000634000, 0x1000, 0x1000, 0x30, 0xc000a3d9b8, 0x410f7c) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/fd_unix.go:202 +0x4f Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net.(*conn).Read(0xc0000103b8, 0xc000634000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/net.go:184 +0x8e Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http.(*persistConn).Read(0xc00053c120, 0xc000634000, 0x1000, 0x1000, 0x734fea, 0xc000112300, 0x37bba20) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/transport.go:1825 +0x75 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: bufio.(*Reader).fill(0xc000a59260) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: bufio/bufio.go:100 +0x103 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: bufio.(*Reader).ReadSlice(0xc000a59260, 0x736d0a, 0x3094cf6, 0x4, 0xc0007de090, 0xc000112300, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: bufio/bufio.go:359 +0x3d Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/internal.readChunkLine(0xc000a59260, 0xc000618000, 0x7f2c7bcc97d0, 0x203000, 0xc000a3db70, 0x411ce8) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/internal/chunked.go:122 +0x34 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/internal.(*chunkedReader).beginChunk(0xc0009d72f0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/internal/chunked.go:48 +0x32 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/internal.(*chunkedReader).Read(0xc0009d72f0, 0xc000756000, 0x200, 0x200, 0x7f2c51e924d0, 0xc000a1dc48, 0x410fd4) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/internal/chunked.go:93 +0x132 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http.(*body).readLocked(0xc000984200, 0xc000756000, 0x200, 0x200, 0xc000756000, 0x7f2c51e924d0, 0xc000a1dc01) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/transfer.go:847 +0x5f Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http.(*body).Read(0xc000984200, 0xc000756000, 0x200, 0x200, 0x0, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/transfer.go:839 +0xf2 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http.(*bodyEOFSignal).Read(0xc000984240, 0xc000756000, 0x200, 0x200, 0x0, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/transport.go:2649 +0xde Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: encoding/json.(*Decoder).refill(0xc0000ec160, 0xc0000ec160, 0x40e2cb) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: encoding/json/stream.go:165 +0xeb Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: encoding/json.(*Decoder).readValue(0xc0000ec160, 0x0, 0x0, 0x2f40da0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: encoding/json/stream.go:140 +0x1e8 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: encoding/json.(*Decoder).Decode(0xc0000ec160, 0x27b1c00, 0xc0008c20b8, 0x37bba60, 0xc000984240) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: encoding/json/stream.go:63 +0x79 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient.(*Client).waitContainer(0xc0002f4510, 0xc0009ec000, 0x40, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient/container.go:1396 +0x2cf Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient.(*Client).WaitContainer(...) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient/container.go:1373 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/drivers/docker.(*taskHandle).run(0xc000870f30) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/drivers/docker/handle.go:191 +0xa6 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/drivers/docker.(*Driver).RecoverTask Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/drivers/docker/driver.go:238 +0x622 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 233 [chan receive, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc.(*addrConn).resetTransport(0xc0001b1080) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/clientconn.go:1179 +0x6ea Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/google.golang.org/grpc.(*addrConn).connect Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/clientconn.go:815 +0x128 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 232 [select, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc.(*ccBalancerWrapper).watcher(0xc000a18d00) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/balancer_conn_wrappers.go:69 +0xc2 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/google.golang.org/grpc.newCCBalancerWrapper Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/balancer_conn_wrappers.go:60 +0x16d Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 231 [chan receive]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin.pidWait(0xff3f, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/process.go:17 +0x91 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin.(*Client).reattach.func1(0xc000575900, 0xff3f) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/client.go:820 +0x7b Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin.(*Client).reattach Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/client.go:813 +0x17d Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 229 [select, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http.(*persistConn).readLoop(0xc00053c120) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/transport.go:2099 +0x99e Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by net/http.(*Transport).dialConn Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/transport.go:1647 +0xc56 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 230 [select, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http.(*persistConn).writeLoop(0xc00053c120) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/transport.go:2277 +0x11c Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by net/http.(*Transport).dialConn Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/transport.go:1648 +0xc7b Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 234 [select, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin.(*gRPCBrokerClientImpl).Recv(0xc000a18d80, 0x0, 0x0, 0xc0008f2a80) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/grpc_broker.go:231 +0xb3 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin.(*GRPCBroker).Run(0xc0005dc1e0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/grpc_broker.go:411 +0x7a Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin.newGRPCClient Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/grpc_client.go:62 +0x23f Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 235 [select, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport.(*Stream).waitOnHeader(0xc00071e100) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport/transport.go:318 +0xcc Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport.(*Stream).RecvCompress(...) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport/transport.go:333 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc.(*csAttempt).recvMsg(0xc0008a8880, 0x2ea2d00, 0xc000a56dc0, 0x0, 0x8, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/stream.go:871 +0x753 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc.(*clientStream).RecvMsg.func1(0xc0008a8880, 0x203000, 0x203000) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/stream.go:736 +0x46 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc.(*clientStream).withRetry(0xc0006f0120, 0xc0002c9e30, 0xc0002c9e00, 0x382dbe0, 0x30) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/stream.go:590 +0x360 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc.(*clientStream).RecvMsg(0xc0006f0120, 0x2ea2d00, 0xc000a56dc0, 0x20, 0xc000a51380) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/stream.go:735 +0x103 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/internal/plugin.(*gRPCBrokerStartStreamClient).Recv(0xc0009c9230, 0x31840c8, 0xc0009ad6e0, 0xc000a18d80) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/internal/plugin/grpc_broker.pb.go:149 +0x62 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin.(*gRPCBrokerClientImpl).StartStream(0xc000a18d80, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/grpc_broker.go:194 +0x187 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin.newGRPCClient Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/grpc_client.go:63 +0x261 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 236 [IO wait, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll.runtime_pollWait(0x7f2c51e960e8, 0x72, 0xffffffffffffffff) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: runtime/netpoll.go:203 +0x55 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll.(*pollDesc).wait(0xc0008a8c18, 0x72, 0x1000, 0x1000, 0xffffffffffffffff) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll/fd_poll_runtime.go:87 +0x45 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll.(*pollDesc).waitRead(...) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll/fd_poll_runtime.go:92 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll.(*FD).Read(0xc0008a8c00, 0xc000679000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll/fd_unix.go:169 +0x19b Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net.(*netFD).Read(0xc0008a8c00, 0xc000679000, 0x1000, 0x1000, 0x30, 0x28, 0x2def8c0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/fd_unix.go:202 +0x4f Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net.(*conn).Read(0xc0000d41b0, 0xc000679000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/net.go:184 +0x8e Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http.(*persistConn).Read(0xc0006f0360, 0xc000679000, 0x1000, 0x1000, 0x734fea, 0xc00071e300, 0x37bba20) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/transport.go:1825 +0x75 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: bufio.(*Reader).fill(0xc0009fe2a0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: bufio/bufio.go:100 +0x103 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: bufio.(*Reader).ReadSlice(0xc0009fe2a0, 0x736d0a, 0x3094cf6, 0x4, 0xc000378510, 0xc00071e300, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: bufio/bufio.go:359 +0x3d Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/internal.readChunkLine(0xc0009fe2a0, 0xc00013dc00, 0x7f2c7bcc9108, 0x203000, 0xc0009f5b70, 0x411ce8) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/internal/chunked.go:122 +0x34 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/internal.(*chunkedReader).beginChunk(0xc0006fcb10) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/internal/chunked.go:48 +0x32 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/internal.(*chunkedReader).Read(0xc0006fcb10, 0xc000644200, 0x200, 0x200, 0x78, 0x80, 0xc0008a8a00) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/internal/chunked.go:93 +0x132 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http.(*body).readLocked(0xc000a196c0, 0xc000644200, 0x200, 0x200, 0xc0002c3cb0, 0xc0002c3b98, 0xc0002c3c20) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/transfer.go:847 +0x5f Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http.(*body).Read(0xc000a196c0, 0xc000644200, 0x200, 0x200, 0x0, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/transfer.go:839 +0xf2 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http.(*bodyEOFSignal).Read(0xc000a19700, 0xc000644200, 0x200, 0x200, 0x0, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/transport.go:2649 +0xde Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: encoding/json.(*Decoder).refill(0xc000a43b80, 0xc000a43b80, 0x40e2cb) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: encoding/json/stream.go:165 +0xeb Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: encoding/json.(*Decoder).readValue(0xc000a43b80, 0x0, 0x0, 0x2f40da0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: encoding/json/stream.go:140 +0x1e8 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: encoding/json.(*Decoder).Decode(0xc000a43b80, 0x27b1c00, 0xc000519ef8, 0x37bba60, 0xc000a19700) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: encoding/json/stream.go:63 +0x79 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient.(*Client).waitContainer(0xc0002f4510, 0xc0005e6300, 0x40, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient/container.go:1396 +0x2cf Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient.(*Client).WaitContainer(...) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient/container.go:1373 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/drivers/docker.(*taskHandle).run(0xc000378360) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/drivers/docker/handle.go:191 +0xa6 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/drivers/docker.(*Driver).RecoverTask Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/drivers/docker/driver.go:238 +0x622 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 222 [IO wait, 108 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll.runtime_pollWait(0x7f2c51e96388, 0x72, 0xffffffffffffffff) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: runtime/netpoll.go:203 +0x55 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll.(*pollDesc).wait(0xc0008a8118, 0x72, 0x8000, 0x8000, 0xffffffffffffffff) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll/fd_poll_runtime.go:87 +0x45 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll.(*pollDesc).waitRead(...) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll/fd_poll_runtime.go:92 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll.(*FD).Read(0xc0008a8100, 0xc0009f6000, 0x8000, 0x8000, 0x0, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll/fd_unix.go:169 +0x19b Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net.(*netFD).Read(0xc0008a8100, 0xc0009f6000, 0x8000, 0x8000, 0x800000601, 0x0, 0x60100000000) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/fd_unix.go:202 +0x4f Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net.(*conn).Read(0xc0000d4018, 0xc0009f6000, 0x8000, 0x8000, 0x0, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/net.go:184 +0x8e Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: bufio.(*Reader).Read(0xc0009e37a0, 0xc0001762d8, 0x9, 0x9, 0xc0009f2e08, 0x40a183, 0xc0009acf00) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: bufio/bufio.go:226 +0x24f Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: io.ReadAtLeast(0x37b4d20, 0xc0009e37a0, 0xc0001762d8, 0x9, 0x9, 0x9, 0x466970, 0xc0009acf00, 0xc0009f2e40) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: io/io.go:310 +0x87 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: io.ReadFull(...) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: io/io.go:329 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/golang.org/x/net/http2.readFrameHeader(0xc0001762d8, 0x9, 0x9, 0x37b4d20, 0xc0009e37a0, 0x0, 0xc000000000, 0x100000000411ce8, 0xc000a56920) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/golang.org/x/net/http2/frame.go:237 +0x87 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/golang.org/x/net/http2.(*Framer).ReadFrame(0xc0001762a0, 0xc0012b1e60, 0xc0012b1e60, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/golang.org/x/net/http2/frame.go:492 +0xa1 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport.(*http2Client).reader(0xc0004f4000) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport/http2_client.go:1265 +0x16f Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport.newHTTP2Client Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport/http2_client.go:300 +0xd23 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 223 [select, 108 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport.(*controlBuffer).get(0xc000a56910, 0x1, 0x0, 0x0, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport/controlbuf.go:395 +0x122 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport.(*loopyWriter).run(0xc0009e3860, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport/controlbuf.go:513 +0x1cd Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport.newHTTP2Client.func3(0xc0004f4000) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport/http2_client.go:346 +0x7b Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport.newHTTP2Client Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport/http2_client.go:344 +0xedb Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 224 [select, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc.newClientStream.func5(0xc000914e00, 0xc00053c000, 0x3814500, 0xc0008fe480) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/stream.go:319 +0xd7 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/google.golang.org/grpc.newClientStream Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/stream.go:318 +0xbbf Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 225 [select, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin.(*gRPCBrokerClientImpl).StartStream.func1(0xc0008f3080, 0xc000985340, 0x3838980, 0xc0009c90e0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/grpc_broker.go:181 +0x146 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin.(*gRPCBrokerClientImpl).StartStream Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/grpc_broker.go:179 +0x165 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 242 [IO wait, 108 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll.runtime_pollWait(0x7f2c51e961c8, 0x72, 0xffffffffffffffff) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: runtime/netpoll.go:203 +0x55 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll.(*pollDesc).wait(0xc0008a8618, 0x72, 0x8000, 0x8000, 0xffffffffffffffff) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll/fd_poll_runtime.go:87 +0x45 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll.(*pollDesc).waitRead(...) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll/fd_poll_runtime.go:92 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll.(*FD).Read(0xc0008a8600, 0xc0002f8000, 0x8000, 0x8000, 0x0, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll/fd_unix.go:169 +0x19b Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net.(*netFD).Read(0xc0008a8600, 0xc0002f8000, 0x8000, 0x8000, 0x800000601, 0x0, 0x60100000000) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/fd_unix.go:202 +0x4f Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net.(*conn).Read(0xc0000d4140, 0xc0002f8000, 0x8000, 0x8000, 0x0, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/net.go:184 +0x8e Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: bufio.(*Reader).Read(0xc0009e3980, 0xc0001763b8, 0x9, 0x9, 0xc0009f3e08, 0x40a183, 0xc0009ad440) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: bufio/bufio.go:226 +0x24f Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: io.ReadAtLeast(0x37b4d20, 0xc0009e3980, 0xc0001763b8, 0x9, 0x9, 0x9, 0x466970, 0xc0009ad440, 0xc0009f3e40) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: io/io.go:310 +0x87 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: io.ReadFull(...) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: io/io.go:329 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/golang.org/x/net/http2.readFrameHeader(0xc0001763b8, 0x9, 0x9, 0x37b4d20, 0xc0009e3980, 0x0, 0xc000000000, 0x100000000411ce8, 0xc000a56bf0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/golang.org/x/net/http2/frame.go:237 +0x87 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/golang.org/x/net/http2.(*Framer).ReadFrame(0xc000176380, 0xc0012b1e80, 0xc0012b1e80, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/golang.org/x/net/http2/frame.go:492 +0xa1 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport.(*http2Client).reader(0xc0004f4380) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport/http2_client.go:1265 +0x16f Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport.newHTTP2Client Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport/http2_client.go:300 +0xd23 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 243 [select, 108 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport.(*controlBuffer).get(0xc000a56be0, 0x1, 0x0, 0x0, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport/controlbuf.go:395 +0x122 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport.(*loopyWriter).run(0xc0009e3a40, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport/controlbuf.go:513 +0x1cd Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport.newHTTP2Client.func3(0xc0004f4380) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport/http2_client.go:346 +0x7b Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport.newHTTP2Client Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport/http2_client.go:344 +0xedb Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 244 [select, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc.newClientStream.func5(0xc0006c4000, 0xc0006f0120, 0x3814500, 0xc0009d76b0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/stream.go:319 +0xd7 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/google.golang.org/grpc.newClientStream Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/stream.go:318 +0xbbf Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 245 [select, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin.(*gRPCBrokerClientImpl).StartStream.func1(0xc0009ad6e0, 0xc000a18d80, 0x3838980, 0xc0009c9230) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/grpc_broker.go:181 +0x146 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin.(*gRPCBrokerClientImpl).StartStream Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/grpc_broker.go:179 +0x165 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 249 [chan receive]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin.pidWait(0xfce8, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/process.go:17 +0x91 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin.(*Client).reattach.func1(0xc000418dc0, 0xfce8) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/client.go:820 +0x7b Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin.(*Client).reattach Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/client.go:813 +0x17d Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 247 [select, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http.(*persistConn).readLoop(0xc0006f0360) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/transport.go:2099 +0x99e Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by net/http.(*Transport).dialConn Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/transport.go:1647 +0xc56 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 248 [select, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http.(*persistConn).writeLoop(0xc0006f0360) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/transport.go:2277 +0x11c Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by net/http.(*Transport).dialConn Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/transport.go:1648 +0xc7b Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 258 [select, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc.newClientStream.func5(0xc0004eea80, 0xc000522360, 0x3814500, 0xc0006fcf00) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/stream.go:319 +0xd7 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/google.golang.org/grpc.newClientStream Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/stream.go:318 +0xbbf Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 241 [select, 108 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport.(*controlBuffer).get(0xc0007c5270, 0x1, 0x0, 0x0, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport/controlbuf.go:395 +0x122 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport.(*loopyWriter).run(0xc000a6fa40, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport/controlbuf.go:513 +0x1cd Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport.newHTTP2Client.func3(0xc00032c540) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport/http2_client.go:346 +0x7b Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport.newHTTP2Client Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport/http2_client.go:344 +0xedb Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 240 [IO wait, 108 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll.runtime_pollWait(0x7f2c51e96008, 0x72, 0xffffffffffffffff) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: runtime/netpoll.go:203 +0x55 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll.(*pollDesc).wait(0xc000a5f418, 0x72, 0x8000, 0x8000, 0xffffffffffffffff) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll/fd_poll_runtime.go:87 +0x45 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll.(*pollDesc).waitRead(...) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll/fd_poll_runtime.go:92 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll.(*FD).Read(0xc000a5f400, 0xc000a76000, 0x8000, 0x8000, 0x0, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll/fd_unix.go:169 +0x19b Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net.(*netFD).Read(0xc000a5f400, 0xc000a76000, 0x8000, 0x8000, 0x800000601, 0x0, 0x60100000000) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/fd_unix.go:202 +0x4f Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net.(*conn).Read(0xc000010db0, 0xc000a76000, 0x8000, 0x8000, 0x0, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/net.go:184 +0x8e Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: bufio.(*Reader).Read(0xc000a6f980, 0xc0001a2738, 0x9, 0x9, 0xc00094be08, 0x40a183, 0xc0001ad5c0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: bufio/bufio.go:226 +0x24f Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: io.ReadAtLeast(0x37b4d20, 0xc000a6f980, 0xc0001a2738, 0x9, 0x9, 0x9, 0x466970, 0xc0001ad5c0, 0xc00094be40) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: io/io.go:310 +0x87 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: io.ReadFull(...) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: io/io.go:329 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/golang.org/x/net/http2.readFrameHeader(0xc0001a2738, 0x9, 0x9, 0x37b4d20, 0xc000a6f980, 0x0, 0xc000000000, 0x100000000411ce8, 0xc0007c5280) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/golang.org/x/net/http2/frame.go:237 +0x87 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/golang.org/x/net/http2.(*Framer).ReadFrame(0xc0001a2700, 0xc0012b1ea0, 0xc0012b1ea0, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/golang.org/x/net/http2/frame.go:492 +0xa1 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport.(*http2Client).reader(0xc00032c540) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport/http2_client.go:1265 +0x16f Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport.newHTTP2Client Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport/http2_client.go:300 +0xd23 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 250 [select, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc.(*ccBalancerWrapper).watcher(0xc0009855c0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/balancer_conn_wrappers.go:69 +0xc2 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/google.golang.org/grpc.newCCBalancerWrapper Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/balancer_conn_wrappers.go:60 +0x16d Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 251 [chan receive, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc.(*addrConn).resetTransport(0xc0000e42c0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/clientconn.go:1179 +0x6ea Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/google.golang.org/grpc.(*addrConn).connect Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/clientconn.go:815 +0x128 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 252 [select, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin.(*gRPCBrokerClientImpl).Recv(0xc000985640, 0x0, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/grpc_broker.go:231 +0xb3 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin.(*GRPCBroker).Run(0xc000a56f50) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/grpc_broker.go:411 +0x7a Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin.newGRPCClient Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/grpc_client.go:62 +0x23f Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 253 [select, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport.(*Stream).waitOnHeader(0xc000113500) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport/transport.go:318 +0xcc Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport.(*Stream).RecvCompress(...) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport/transport.go:333 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc.(*csAttempt).recvMsg(0xc000a5f680, 0x2ea2d00, 0xc0007c5450, 0x0, 0x8, 0x419567) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/stream.go:871 +0x753 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc.(*clientStream).RecvMsg.func1(0xc000a5f680, 0x203000, 0x203000) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/stream.go:736 +0x46 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc.(*clientStream).withRetry(0xc000522360, 0xc000a1fe30, 0xc000a1fe00, 0x382dbe0, 0x30) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/stream.go:590 +0x360 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc.(*clientStream).RecvMsg(0xc000522360, 0x2ea2d00, 0xc0007c5450, 0x20, 0xc000943080) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/stream.go:735 +0x103 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/internal/plugin.(*gRPCBrokerStartStreamClient).Recv(0xc0006dcc50, 0x31840c8, 0xc0001ad860, 0xc000985640) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/internal/plugin/grpc_broker.pb.go:149 +0x62 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin.(*gRPCBrokerClientImpl).StartStream(0xc000985640, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/grpc_broker.go:194 +0x187 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin.newGRPCClient Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/grpc_client.go:63 +0x261 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 254 [IO wait, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll.runtime_pollWait(0x7f2c5304ba28, 0x72, 0xffffffffffffffff) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: runtime/netpoll.go:203 +0x55 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll.(*pollDesc).wait(0xc000a5f898, 0x72, 0x1000, 0x1000, 0xffffffffffffffff) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll/fd_poll_runtime.go:87 +0x45 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll.(*pollDesc).waitRead(...) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll/fd_poll_runtime.go:92 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll.(*FD).Read(0xc000a5f880, 0xc000941000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll/fd_unix.go:169 +0x19b Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net.(*netFD).Read(0xc000a5f880, 0xc000941000, 0x1000, 0x1000, 0x30, 0x28, 0x2def8c0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/fd_unix.go:202 +0x4f Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net.(*conn).Read(0xc000010de8, 0xc000941000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/net.go:184 +0x8e Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http.(*persistConn).Read(0xc0005225a0, 0xc000941000, 0x1000, 0x1000, 0x734fea, 0xc000113700, 0x37bba20) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/transport.go:1825 +0x75 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: bufio.(*Reader).fill(0xc000a7e060) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: bufio/bufio.go:100 +0x103 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: bufio.(*Reader).ReadSlice(0xc000a7e060, 0x736d0a, 0x3094cf6, 0x4, 0xc0003787e0, 0xc000113700, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: bufio/bufio.go:359 +0x3d Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/internal.readChunkLine(0xc000a7e060, 0xc000618000, 0x7f2c7bcc97d0, 0x203000, 0xc0009efb70, 0x411ce8) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/internal/chunked.go:122 +0x34 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/internal.(*chunkedReader).beginChunk(0xc0006fd140) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/internal/chunked.go:48 +0x32 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/internal.(*chunkedReader).Read(0xc0006fd140, 0xc000644600, 0x200, 0x200, 0x78, 0x80, 0xc000a5f780) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/internal/chunked.go:93 +0x132 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http.(*body).readLocked(0xc000a19a00, 0xc000644600, 0x200, 0x200, 0xc000509cb0, 0xc000509b98, 0xc000509c20) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/transfer.go:847 +0x5f Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http.(*body).Read(0xc000a19a00, 0xc000644600, 0x200, 0x200, 0x0, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/transfer.go:839 +0xf2 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http.(*bodyEOFSignal).Read(0xc000a19a40, 0xc000644600, 0x200, 0x200, 0x0, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/transport.go:2649 +0xde Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: encoding/json.(*Decoder).refill(0xc0006f6840, 0xc0006f6840, 0x40e2cb) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: encoding/json/stream.go:165 +0xeb Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: encoding/json.(*Decoder).readValue(0xc0006f6840, 0x0, 0x0, 0x2f40da0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: encoding/json/stream.go:140 +0x1e8 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: encoding/json.(*Decoder).Decode(0xc0006f6840, 0x27b1c00, 0xc0006e8078, 0x37bba60, 0xc000a19a40) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: encoding/json/stream.go:63 +0x79 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient.(*Client).waitContainer(0xc0002f4510, 0xc000716080, 0x40, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient/container.go:1396 +0x2cf Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient.(*Client).WaitContainer(...) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient/container.go:1373 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/drivers/docker.(*taskHandle).run(0xc0007de360) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/drivers/docker/handle.go:191 +0xa6 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/drivers/docker.(*Driver).RecoverTask Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/drivers/docker/driver.go:238 +0x622 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 292 [select, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/allocrunner/taskrunner.(*TaskRunner).Run(0xc000886000) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/allocrunner/taskrunner/task_runner.go:545 +0xfe7 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/client/allocrunner.(*allocRunner).runTasks Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/allocrunner/alloc_runner.go:351 +0x95 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 259 [select, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin.(*gRPCBrokerClientImpl).StartStream.func1(0xc0001ad860, 0xc000985640, 0x3838980, 0xc0006dcc50) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/grpc_broker.go:181 +0x146 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin.(*gRPCBrokerClientImpl).StartStream Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/grpc_broker.go:179 +0x165 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 263 [chan receive]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin.pidWait(0xfd4f, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/process.go:17 +0x91 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin.(*Client).reattach.func1(0xc00031bae0, 0xfd4f) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/client.go:820 +0x7b Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin.(*Client).reattach Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/client.go:813 +0x17d Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 261 [select, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http.(*persistConn).readLoop(0xc0005225a0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/transport.go:2099 +0x99e Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by net/http.(*Transport).dialConn Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/transport.go:1647 +0xc56 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 262 [select, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http.(*persistConn).writeLoop(0xc0005225a0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/transport.go:2277 +0x11c Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by net/http.(*Transport).dialConn Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/transport.go:1648 +0xc7b Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 291 [select, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/allocrunner.(*allocRunner).handleAllocUpdates(0xc000627a00) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/allocrunner/alloc_runner.go:760 +0xd3 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/client/allocrunner.(*allocRunner).Run Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/allocrunner/alloc_runner.go:284 +0xac Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 290 [select, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/allocrunner.(*allocRunner).handleTaskStateUpdates(0xc000627a00) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/allocrunner/alloc_runner.go:447 +0x75c Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/client/allocrunner.(*allocRunner).Run Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/allocrunner/alloc_runner.go:281 +0x87 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 264 [select, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc.(*ccBalancerWrapper).watcher(0xc000a19f00) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/balancer_conn_wrappers.go:69 +0xc2 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/google.golang.org/grpc.newCCBalancerWrapper Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/balancer_conn_wrappers.go:60 +0x16d Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 265 [chan receive, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc.(*addrConn).resetTransport(0xc0001b1b80) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/clientconn.go:1179 +0x6ea Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/google.golang.org/grpc.(*addrConn).connect Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/clientconn.go:815 +0x128 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 266 [select, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin.(*gRPCBrokerClientImpl).Recv(0xc000a19f80, 0x0, 0x0, 0xc0001acfc0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/grpc_broker.go:231 +0xb3 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin.(*GRPCBroker).Run(0xc0007c55e0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/grpc_broker.go:411 +0x7a Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin.newGRPCClient Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/grpc_client.go:62 +0x23f Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 267 [select, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport.(*Stream).waitOnHeader(0xc00071ee00) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport/transport.go:318 +0xcc Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport.(*Stream).RecvCompress(...) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport/transport.go:333 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc.(*csAttempt).recvMsg(0xc00074cc80, 0x2ea2d00, 0xc0008b60f0, 0x0, 0x8, 0x27f6bd8) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/stream.go:871 +0x753 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc.(*clientStream).RecvMsg.func1(0xc00074cc80, 0x203000, 0x203000) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/stream.go:736 +0x46 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc.(*clientStream).withRetry(0xc0006b2ea0, 0xc000b19e30, 0xc000b19e00, 0x382dbe0, 0x30) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/stream.go:590 +0x360 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc.(*clientStream).RecvMsg(0xc0006b2ea0, 0x2ea2d00, 0xc0008b60f0, 0x20, 0xc00097a900) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/stream.go:735 +0x103 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/internal/plugin.(*gRPCBrokerStartStreamClient).Recv(0xc00052c930, 0x31840c8, 0xc00063bc80, 0xc000a19f80) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/internal/plugin/grpc_broker.pb.go:149 +0x62 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin.(*gRPCBrokerClientImpl).StartStream(0xc000a19f80, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/grpc_broker.go:194 +0x187 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin.newGRPCClient Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/grpc_client.go:63 +0x261 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 268 [IO wait, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll.runtime_pollWait(0x7f2c5304b788, 0x72, 0xffffffffffffffff) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: runtime/netpoll.go:203 +0x55 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll.(*pollDesc).wait(0xc00074ce98, 0x72, 0x1000, 0x1000, 0xffffffffffffffff) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll/fd_poll_runtime.go:87 +0x45 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll.(*pollDesc).waitRead(...) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll/fd_poll_runtime.go:92 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll.(*FD).Read(0xc00074ce80, 0xc0005d2000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll/fd_unix.go:169 +0x19b Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net.(*netFD).Read(0xc00074ce80, 0xc0005d2000, 0x1000, 0x1000, 0x30, 0x28, 0x2def8c0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/fd_unix.go:202 +0x4f Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net.(*conn).Read(0xc0000d4a90, 0xc0005d2000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/net.go:184 +0x8e Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http.(*persistConn).Read(0xc0006b2fc0, 0xc0005d2000, 0x1000, 0x1000, 0x734fea, 0xc00071ec00, 0x37bba20) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/transport.go:1825 +0x75 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: bufio.(*Reader).fill(0xc0009ff8c0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: bufio/bufio.go:100 +0x103 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: bufio.(*Reader).ReadSlice(0xc0009ff8c0, 0x736d0a, 0x3094cf6, 0x4, 0xc000379b00, 0xc00071ec00, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: bufio/bufio.go:359 +0x3d Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/internal.readChunkLine(0xc0009ff8c0, 0x57da1a0, 0x7f2c7bcc9108, 0x203000, 0xc0003d5b70, 0x411ce8) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/internal/chunked.go:122 +0x34 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/internal.(*chunkedReader).beginChunk(0xc0003f05a0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/internal/chunked.go:48 +0x32 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/internal.(*chunkedReader).Read(0xc0003f05a0, 0xc000644a00, 0x200, 0x200, 0x78, 0x80, 0xc00074cd80) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/internal/chunked.go:93 +0x132 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http.(*body).readLocked(0xc000820680, 0xc000644a00, 0x200, 0x200, 0xc000505cb0, 0xc000505b98, 0xc000505c20) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/transfer.go:847 +0x5f Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http.(*body).Read(0xc000820680, 0xc000644a00, 0x200, 0x200, 0x0, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/transfer.go:839 +0xf2 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http.(*bodyEOFSignal).Read(0xc0008206c0, 0xc000644a00, 0x200, 0x200, 0x0, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/transport.go:2649 +0xde Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: encoding/json.(*Decoder).refill(0xc0006f7340, 0xc0006f7340, 0x40e2cb) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: encoding/json/stream.go:165 +0xeb Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: encoding/json.(*Decoder).readValue(0xc0006f7340, 0x0, 0x0, 0x2f40da0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: encoding/json/stream.go:140 +0x1e8 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: encoding/json.(*Decoder).Decode(0xc0006f7340, 0x27b1c00, 0xc0000d2558, 0x37bba60, 0xc0008206c0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: encoding/json/stream.go:63 +0x79 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient.(*Client).waitContainer(0xc0002f4510, 0xc0005e73c0, 0x40, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient/container.go:1396 +0x2cf Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient.(*Client).WaitContainer(...) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient/container.go:1373 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/drivers/docker.(*taskHandle).run(0xc000378a20) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/drivers/docker/handle.go:191 +0xa6 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/drivers/docker.(*Driver).RecoverTask Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/drivers/docker/driver.go:238 +0x622 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 269 [chan receive, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/allocrunner.(*allocRunner).runTasks(0xc000627a00) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/allocrunner/alloc_runner.go:355 +0x11b Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/allocrunner.(*allocRunner).Run(0xc000627a00) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/allocrunner/alloc_runner.go:310 +0x42d Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/client.(*Client).restoreState Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/client.go:1144 +0xc45 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 270 [chan receive, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/allocrunner.(*allocRunner).runTasks(0xc0001f8800) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/allocrunner/alloc_runner.go:355 +0x11b Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/allocrunner.(*allocRunner).Run(0xc0001f8800) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/allocrunner/alloc_runner.go:310 +0x42d Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/client.(*Client).restoreState Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/client.go:1144 +0xc45 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 271 [chan receive, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/allocrunner.(*allocRunner).runTasks(0xc0001f8a00) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/allocrunner/alloc_runner.go:355 +0x11b Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/allocrunner.(*allocRunner).Run(0xc0001f8a00) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/allocrunner/alloc_runner.go:310 +0x42d Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/client.(*Client).restoreState Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/client.go:1144 +0xc45 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 272 [chan receive, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/allocrunner.(*allocRunner).runTasks(0xc000626000) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/allocrunner/alloc_runner.go:355 +0x11b Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/allocrunner.(*allocRunner).Run(0xc000626000) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/allocrunner/alloc_runner.go:310 +0x42d Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/client.(*Client).restoreState Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/client.go:1144 +0xc45 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 273 [select, 25 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client.(*Client).periodicSnapshot(0xc00019e780) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/client.go:1615 +0xc4 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client.(*group).Go.func1(0xc00019e8e0, 0xc0006dd450) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/client.go:3066 +0x51 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/client.(*group).Go Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/client.go:3064 +0x62 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 274 [select]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client.(*Client).allocSync(0xc00019e780) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/client.go:1886 +0x1a2 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client.(*group).Go.func1(0xc00019e8e0, 0xc0006dd460) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/client.go:3066 +0x51 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/client.(*group).Go Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/client.go:3064 +0x62 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 275 [select, 27 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client.(*Client).run(0xc00019e780) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/client.go:1635 +0x107 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/client.NewClient Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/client.go:535 +0x1d91 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 276 [select]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client.(*Client).emitStats(0xc00019e780) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/client.go:2743 +0x26b Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client.(*group).Go.func1(0xc00019e8e0, 0xc0006dd4a0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/client.go:3066 +0x51 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/client.(*group).Go Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/client.go:3064 +0x62 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 277 [IO wait]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll.runtime_pollWait(0x7f2c5304b948, 0x72, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: runtime/netpoll.go:203 +0x55 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll.(*pollDesc).wait(0xc000a5ff98, 0x72, 0x0, 0x0, 0x3099921) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll/fd_poll_runtime.go:87 +0x45 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll.(*pollDesc).waitRead(...) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll/fd_poll_runtime.go:92 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll.(*FD).Accept(0xc000a5ff80, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll/fd_unix.go:384 +0x1d4 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net.(*netFD).accept(0xc000a5ff80, 0x2b234a0, 0xc0006e8550, 0x3838e00) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/fd_unix.go:238 +0x42 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net.(*TCPListener).accept(0xc0005ea260, 0xc0000c8520, 0x3838e00, 0xc0000d48d0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/tcpsock_posix.go:139 +0x32 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net.(*TCPListener).Accept(0xc0005ea260, 0xc000735e70, 0x18, 0xc0009e4c00, 0x7835bc) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/tcpsock.go:261 +0x64 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http.(*Server).Serve(0xc0001a28c0, 0x37fcfc0, 0xc0005ea260, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/server.go:2901 +0x25d Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/command/agent.NewHTTPServer.func1(0xc0007c5630, 0xc0001a28c0, 0x37fcfc0, 0xc0005ea260) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/command/agent/http.go:154 +0x6f Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/command/agent.NewHTTPServer Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/command/agent/http.go:152 +0x6ec Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 293 [select, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/allocrunner/taskrunner.(*TaskRunner).Run(0xc0008862c0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/allocrunner/taskrunner/task_runner.go:545 +0xfe7 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/client/allocrunner.(*allocRunner).runTasks Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/allocrunner/alloc_runner.go:351 +0x95 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 294 [select, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/allocrunner/taskrunner.(*TaskRunner).Run(0xc000886580) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/allocrunner/taskrunner/task_runner.go:545 +0xfe7 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/client/allocrunner.(*allocRunner).runTasks Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/allocrunner/alloc_runner.go:351 +0x95 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 295 [select, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/allocrunner/taskrunner.(*TaskRunner).handleUpdates(0xc000886580) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/allocrunner/taskrunner/task_runner.go:676 +0xb9 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/client/allocrunner/taskrunner.(*TaskRunner).Run Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/allocrunner/taskrunner/task_runner.go:471 +0x20c Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 296 [select, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/LK4D4/joincontext.(*joinContext).run(0xc000985900) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/LK4D4/joincontext/context.go:73 +0x10b Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/github.com/LK4D4/joincontext.Join Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/LK4D4/joincontext/context.go:33 +0xc2 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 297 [select, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/LK4D4/joincontext.(*joinContext).run(0xc000985980) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/LK4D4/joincontext/context.go:73 +0x10b Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/github.com/LK4D4/joincontext.Join Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/LK4D4/joincontext/context.go:33 +0xc2 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 298 [chan receive]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin.pidWait(0x1e95a, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/process.go:17 +0x91 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin.(*Client).reattach.func1(0xc000419040, 0x1e95a) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/client.go:820 +0x7b Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin.(*Client).reattach Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/client.go:813 +0x17d Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 299 [select, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc.(*ccBalancerWrapper).watcher(0xc000985b40) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/balancer_conn_wrappers.go:69 +0xc2 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/google.golang.org/grpc.newCCBalancerWrapper Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/balancer_conn_wrappers.go:60 +0x16d Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 300 [chan receive, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc.(*addrConn).resetTransport(0xc0000e4840) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/clientconn.go:1179 +0x6ea Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/google.golang.org/grpc.(*addrConn).connect Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/clientconn.go:815 +0x128 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 301 [select, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin.(*gRPCBrokerClientImpl).Recv(0xc000985bc0, 0x0, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/grpc_broker.go:231 +0xb3 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin.(*GRPCBroker).Run(0xc000a57450) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/grpc_broker.go:411 +0x7a Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin.newGRPCClient Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/grpc_client.go:62 +0x23f Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 302 [select, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport.(*Stream).waitOnHeader(0xc00071f200) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport/transport.go:318 +0xcc Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport.(*Stream).RecvCompress(...) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport/transport.go:333 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc.(*csAttempt).recvMsg(0xc00074c880, 0x2ea2d00, 0xc0008b6500, 0x0, 0x8, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/stream.go:871 +0x753 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc.(*clientStream).RecvMsg.func1(0xc00074c880, 0x203000, 0x203000) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/stream.go:736 +0x46 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc.(*clientStream).withRetry(0xc0006b2d80, 0xc000b1be30, 0xc000b1be00, 0x382dbe0, 0x30) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/stream.go:590 +0x360 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc.(*clientStream).RecvMsg(0xc0006b2d80, 0x2ea2d00, 0xc0008b6500, 0x20, 0xc0004fe600) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/stream.go:735 +0x103 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/internal/plugin.(*gRPCBrokerStartStreamClient).Recv(0xc00052cd10, 0x31840c8, 0xc00063b3e0, 0xc000985bc0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/internal/plugin/grpc_broker.pb.go:149 +0x62 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin.(*gRPCBrokerClientImpl).StartStream(0xc000985bc0, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/grpc_broker.go:194 +0x187 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin.newGRPCClient Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/grpc_client.go:63 +0x261 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 303 [IO wait, 108 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll.runtime_pollWait(0x7f2c5304b868, 0x72, 0xffffffffffffffff) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: runtime/netpoll.go:203 +0x55 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll.(*pollDesc).wait(0xc00074c998, 0x72, 0x8000, 0x8000, 0xffffffffffffffff) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll/fd_poll_runtime.go:87 +0x45 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll.(*pollDesc).waitRead(...) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll/fd_poll_runtime.go:92 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll.(*FD).Read(0xc00074c980, 0xc000692000, 0x8000, 0x8000, 0x0, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll/fd_unix.go:169 +0x19b Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net.(*netFD).Read(0xc00074c980, 0xc000692000, 0x8000, 0x8000, 0x800000601, 0x0, 0x60100000000) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/fd_unix.go:202 +0x4f Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net.(*conn).Read(0xc0000d4a68, 0xc000692000, 0x8000, 0x8000, 0x0, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/net.go:184 +0x8e Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: bufio.(*Reader).Read(0xc0009ff1a0, 0xc000176738, 0x9, 0x9, 0xc0003d0e08, 0x40a183, 0xc00063bbc0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: bufio/bufio.go:226 +0x24f Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: io.ReadAtLeast(0x37b4d20, 0xc0009ff1a0, 0xc000176738, 0x9, 0x9, 0x9, 0x466970, 0xc00063bbc0, 0xc0003d0e40) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: io/io.go:310 +0x87 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: io.ReadFull(...) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: io/io.go:329 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/golang.org/x/net/http2.readFrameHeader(0xc000176738, 0x9, 0x9, 0x37b4d20, 0xc0009ff1a0, 0x0, 0xc000000000, 0x100000000411ce8, 0xc000a575f0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/golang.org/x/net/http2/frame.go:237 +0x87 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/golang.org/x/net/http2.(*Framer).ReadFrame(0xc000176700, 0xc001210640, 0xc001210640, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/golang.org/x/net/http2/frame.go:492 +0xa1 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport.(*http2Client).reader(0xc0004f4e00) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport/http2_client.go:1265 +0x16f Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport.newHTTP2Client Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport/http2_client.go:300 +0xd23 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 304 [select, 108 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport.(*controlBuffer).get(0xc000a575e0, 0x1, 0x0, 0x0, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport/controlbuf.go:395 +0x122 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport.(*loopyWriter).run(0xc0009ff380, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport/controlbuf.go:513 +0x1cd Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport.newHTTP2Client.func3(0xc0004f4e00) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport/http2_client.go:346 +0x7b Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport.newHTTP2Client Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport/http2_client.go:344 +0xedb Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 308 [select, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/allocrunner/taskrunner.(*TaskRunner).handleUpdates(0xc000886000) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/allocrunner/taskrunner/task_runner.go:676 +0xb9 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/client/allocrunner/taskrunner.(*TaskRunner).Run Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/allocrunner/taskrunner/task_runner.go:471 +0x20c Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 306 [select, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http.(*persistConn).readLoop(0xc0006b2fc0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/transport.go:2099 +0x99e Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by net/http.(*Transport).dialConn Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/transport.go:1647 +0xc56 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 307 [select, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http.(*persistConn).writeLoop(0xc0006b2fc0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/transport.go:2277 +0x11c Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by net/http.(*Transport).dialConn Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/transport.go:1648 +0xc7b Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 309 [select, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/LK4D4/joincontext.(*joinContext).run(0xc000985f00) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/LK4D4/joincontext/context.go:73 +0x10b Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/github.com/LK4D4/joincontext.Join Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/LK4D4/joincontext/context.go:33 +0xc2 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 310 [select, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/LK4D4/joincontext.(*joinContext).run(0xc000985f80) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/LK4D4/joincontext/context.go:73 +0x10b Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/github.com/LK4D4/joincontext.Join Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/LK4D4/joincontext/context.go:33 +0xc2 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 311 [chan receive]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin.pidWait(0x1e95c, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/process.go:17 +0x91 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin.(*Client).reattach.func1(0xc0004192c0, 0x1e95c) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/client.go:820 +0x7b Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin.(*Client).reattach Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/client.go:813 +0x17d Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 312 [select, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc.(*ccBalancerWrapper).watcher(0xc0006a83c0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/balancer_conn_wrappers.go:69 +0xc2 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/google.golang.org/grpc.newCCBalancerWrapper Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/balancer_conn_wrappers.go:60 +0x16d Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 313 [chan receive, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc.(*addrConn).resetTransport(0xc0000e4dc0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/clientconn.go:1179 +0x6ea Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/google.golang.org/grpc.(*addrConn).connect Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/clientconn.go:815 +0x128 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 314 [select, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin.(*gRPCBrokerClientImpl).Recv(0xc0006a8480, 0x0, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/grpc_broker.go:231 +0xb3 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin.(*GRPCBroker).Run(0xc000a57810) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/grpc_broker.go:411 +0x7a Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin.newGRPCClient Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/grpc_client.go:62 +0x23f Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 315 [select, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport.(*Stream).waitOnHeader(0xc00071f300) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport/transport.go:318 +0xcc Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport.(*Stream).RecvCompress(...) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport/transport.go:333 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc.(*csAttempt).recvMsg(0xc00074d500, 0x2ea2d00, 0xc0008b65f0, 0x0, 0x8, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/stream.go:871 +0x753 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc.(*clientStream).RecvMsg.func1(0xc00074d500, 0x203000, 0x203000) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/stream.go:736 +0x46 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc.(*clientStream).withRetry(0xc0004e67e0, 0xc000b85e30, 0xc000b85e00, 0x382dbe0, 0x30) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/stream.go:590 +0x360 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc.(*clientStream).RecvMsg(0xc0004e67e0, 0x2ea2d00, 0xc0008b65f0, 0x20, 0xc0004ff800) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/stream.go:735 +0x103 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/internal/plugin.(*gRPCBrokerStartStreamClient).Recv(0xc00052cde0, 0x31840c8, 0xc00085f1a0, 0xc0006a8480) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/internal/plugin/grpc_broker.pb.go:149 +0x62 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin.(*gRPCBrokerClientImpl).StartStream(0xc0006a8480, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/grpc_broker.go:194 +0x187 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin.newGRPCClient Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/grpc_client.go:63 +0x261 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 316 [select, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/allocrunner/taskrunner.(*TaskRunner).handleUpdates(0xc0008862c0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/allocrunner/taskrunner/task_runner.go:676 +0xb9 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/client/allocrunner/taskrunner.(*TaskRunner).Run Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/allocrunner/taskrunner/task_runner.go:471 +0x20c Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 317 [select, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/LK4D4/joincontext.(*joinContext).run(0xc0006a86c0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/LK4D4/joincontext/context.go:73 +0x10b Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/github.com/LK4D4/joincontext.Join Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/LK4D4/joincontext/context.go:33 +0xc2 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 318 [select, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/LK4D4/joincontext.(*joinContext).run(0xc0006a8740) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/LK4D4/joincontext/context.go:73 +0x10b Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/github.com/LK4D4/joincontext.Join Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/LK4D4/joincontext/context.go:33 +0xc2 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 319 [chan receive]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin.pidWait(0x1e95b, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/process.go:17 +0x91 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin.(*Client).reattach.func1(0xc000419400, 0x1e95b) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/client.go:820 +0x7b Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin.(*Client).reattach Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/client.go:813 +0x17d Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 278 [select, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/allocrunner.(*allocRunner).handleTaskStateUpdates(0xc0001f8800) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/allocrunner/alloc_runner.go:447 +0x75c Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/client/allocrunner.(*allocRunner).Run Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/allocrunner/alloc_runner.go:281 +0x87 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 279 [select, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/allocrunner.(*allocRunner).handleAllocUpdates(0xc0001f8800) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/allocrunner/alloc_runner.go:760 +0xd3 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/client/allocrunner.(*allocRunner).Run Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/allocrunner/alloc_runner.go:284 +0xac Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 280 [select, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/allocrunner/taskrunner.(*TaskRunner).Run(0xc0003d7b80) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/allocrunner/taskrunner/task_runner.go:545 +0xfe7 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/client/allocrunner.(*allocRunner).runTasks Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/allocrunner/alloc_runner.go:351 +0x95 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 281 [select, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/allocrunner/taskrunner.(*TaskRunner).handleUpdates(0xc0003d7b80) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/allocrunner/taskrunner/task_runner.go:676 +0xb9 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/client/allocrunner/taskrunner.(*TaskRunner).Run Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/allocrunner/taskrunner/task_runner.go:471 +0x20c Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 282 [select, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/LK4D4/joincontext.(*joinContext).run(0xc000538480) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/LK4D4/joincontext/context.go:73 +0x10b Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/github.com/LK4D4/joincontext.Join Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/LK4D4/joincontext/context.go:33 +0xc2 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 283 [select, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/LK4D4/joincontext.(*joinContext).run(0xc000538780) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/LK4D4/joincontext/context.go:73 +0x10b Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/github.com/LK4D4/joincontext.Join Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/LK4D4/joincontext/context.go:33 +0xc2 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 284 [chan receive]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin.pidWait(0x1e02, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/process.go:17 +0x91 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin.(*Client).reattach.func1(0xc00031bb80, 0x1e02) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/client.go:820 +0x7b Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin.(*Client).reattach Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/client.go:813 +0x17d Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 285 [select, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc.(*ccBalancerWrapper).watcher(0xc000539180) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/balancer_conn_wrappers.go:69 +0xc2 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/google.golang.org/grpc.newCCBalancerWrapper Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/balancer_conn_wrappers.go:60 +0x16d Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 286 [chan receive, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc.(*addrConn).resetTransport(0xc0003d6580) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/clientconn.go:1179 +0x6ea Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/google.golang.org/grpc.(*addrConn).connect Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/clientconn.go:815 +0x128 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 287 [select, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin.(*gRPCBrokerClientImpl).Recv(0xc0005394c0, 0x0, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/grpc_broker.go:231 +0xb3 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin.(*GRPCBroker).Run(0xc0007c5a90) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/grpc_broker.go:411 +0x7a Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin.newGRPCClient Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/grpc_client.go:62 +0x23f Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 288 [select, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport.(*Stream).waitOnHeader(0xc0008ec500) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport/transport.go:318 +0xcc Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport.(*Stream).RecvCompress(...) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport/transport.go:333 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc.(*csAttempt).recvMsg(0xc0006aea00, 0x2ea2d00, 0xc000917630, 0x0, 0x8, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/stream.go:871 +0x753 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc.(*clientStream).RecvMsg.func1(0xc0006aea00, 0x203000, 0x203000) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/stream.go:736 +0x46 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc.(*clientStream).withRetry(0xc000020a20, 0xc000b91e30, 0xc000b91e00, 0x382dbe0, 0x30) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/stream.go:590 +0x360 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc.(*clientStream).RecvMsg(0xc000020a20, 0x2ea2d00, 0xc000917630, 0x20, 0xc0009e5c80) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/stream.go:735 +0x103 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/internal/plugin.(*gRPCBrokerStartStreamClient).Recv(0xc000997750, 0x31840c8, 0xc00094d2c0, 0xc0005394c0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/internal/plugin/grpc_broker.pb.go:149 +0x62 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin.(*gRPCBrokerClientImpl).StartStream(0xc0005394c0, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/grpc_broker.go:194 +0x187 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin.newGRPCClient Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/grpc_client.go:63 +0x261 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 289 [select, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/allocrunner.(*allocRunner).handleTaskStateUpdates(0xc0001f8a00) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/allocrunner/alloc_runner.go:447 +0x75c Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/client/allocrunner.(*allocRunner).Run Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/allocrunner/alloc_runner.go:281 +0x87 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 322 [select, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/allocrunner.(*allocRunner).handleAllocUpdates(0xc0001f8a00) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/allocrunner/alloc_runner.go:760 +0xd3 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/client/allocrunner.(*allocRunner).Run Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/allocrunner/alloc_runner.go:284 +0xac Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 323 [select, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/allocrunner/taskrunner.(*TaskRunner).Run(0xc00099c2c0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/allocrunner/taskrunner/task_runner.go:545 +0xfe7 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/client/allocrunner.(*allocRunner).runTasks Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/allocrunner/alloc_runner.go:351 +0x95 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 324 [select, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/allocrunner/taskrunner.(*TaskRunner).Run(0xc00099c580) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/allocrunner/taskrunner/task_runner.go:545 +0xfe7 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/client/allocrunner.(*allocRunner).runTasks Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/allocrunner/alloc_runner.go:351 +0x95 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 325 [select, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/allocrunner/taskrunner.(*TaskRunner).handleUpdates(0xc00099c580) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/allocrunner/taskrunner/task_runner.go:676 +0xb9 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/client/allocrunner/taskrunner.(*TaskRunner).Run Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/allocrunner/taskrunner/task_runner.go:471 +0x20c Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 326 [select, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/LK4D4/joincontext.(*joinContext).run(0xc000539b00) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/LK4D4/joincontext/context.go:73 +0x10b Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/github.com/LK4D4/joincontext.Join Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/LK4D4/joincontext/context.go:33 +0xc2 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 327 [select, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/LK4D4/joincontext.(*joinContext).run(0xc000539b80) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/LK4D4/joincontext/context.go:73 +0x10b Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/github.com/LK4D4/joincontext.Join Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/LK4D4/joincontext/context.go:33 +0xc2 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 320 [select, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc.(*ccBalancerWrapper).watcher(0xc0006a8b80) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/balancer_conn_wrappers.go:69 +0xc2 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/google.golang.org/grpc.newCCBalancerWrapper Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/balancer_conn_wrappers.go:60 +0x16d Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 321 [chan receive, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc.(*addrConn).resetTransport(0xc0000e5080) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/clientconn.go:1179 +0x6ea Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/google.golang.org/grpc.(*addrConn).connect Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/clientconn.go:815 +0x128 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 338 [select, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin.(*gRPCBrokerClientImpl).Recv(0xc0006a8d40, 0x11, 0xc000636cc0, 0x17) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/grpc_broker.go:231 +0xb3 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin.(*GRPCBroker).Run(0xc000a579f0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/grpc_broker.go:411 +0x7a Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin.newGRPCClient Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/grpc_client.go:62 +0x23f Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 339 [select, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport.(*Stream).waitOnHeader(0xc00071f400) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport/transport.go:318 +0xcc Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport.(*Stream).RecvCompress(...) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport/transport.go:333 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc.(*csAttempt).recvMsg(0xc00074d900, 0x2ea2d00, 0xc0008b66e0, 0x0, 0x8, 0x8) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/stream.go:871 +0x753 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc.(*clientStream).RecvMsg.func1(0xc00074d900, 0x203000, 0x203000) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/stream.go:736 +0x46 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc.(*clientStream).withRetry(0xc000401320, 0xc000b87e30, 0xc000b87e00, 0x382dbe0, 0x30) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/stream.go:590 +0x360 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc.(*clientStream).RecvMsg(0xc000401320, 0x2ea2d00, 0xc0008b66e0, 0x20, 0xc000616480) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/stream.go:735 +0x103 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/internal/plugin.(*gRPCBrokerStartStreamClient).Recv(0xc00052cee0, 0x31840c8, 0xc0006282a0, 0xc0006a8d40) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/internal/plugin/grpc_broker.pb.go:149 +0x62 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin.(*gRPCBrokerClientImpl).StartStream(0xc0006a8d40, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/grpc_broker.go:194 +0x187 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin.newGRPCClient Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/grpc_client.go:63 +0x261 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 328 [chan receive]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin.pidWait(0xfea9, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/process.go:17 +0x91 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin.(*Client).reattach.func1(0xc00031bd60, 0xfea9) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/client.go:820 +0x7b Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin.(*Client).reattach Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/client.go:813 +0x17d Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 329 [select, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc.(*ccBalancerWrapper).watcher(0xc000332bc0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/balancer_conn_wrappers.go:69 +0xc2 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/google.golang.org/grpc.newCCBalancerWrapper Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/balancer_conn_wrappers.go:60 +0x16d Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 330 [chan receive, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc.(*addrConn).resetTransport(0xc0003d6840) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/clientconn.go:1179 +0x6ea Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/google.golang.org/grpc.(*addrConn).connect Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/clientconn.go:815 +0x128 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 331 [select, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin.(*gRPCBrokerClientImpl).Recv(0xc000332c80, 0x0, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/grpc_broker.go:231 +0xb3 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin.(*GRPCBroker).Run(0xc0007c5ef0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/grpc_broker.go:411 +0x7a Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin.newGRPCClient Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/grpc_client.go:62 +0x23f Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 332 [select, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport.(*Stream).waitOnHeader(0xc000183e00) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport/transport.go:318 +0xcc Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport.(*Stream).RecvCompress(...) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport/transport.go:333 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc.(*csAttempt).recvMsg(0xc0006af700, 0x2ea2d00, 0xc000897f40, 0x0, 0x8, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/stream.go:871 +0x753 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc.(*clientStream).RecvMsg.func1(0xc0006af700, 0x203000, 0x203000) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/stream.go:736 +0x46 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc.(*clientStream).withRetry(0xc000021d40, 0xc000b1de30, 0xc000b1de00, 0x382dbe0, 0x30) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/stream.go:590 +0x360 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc.(*clientStream).RecvMsg(0xc000021d40, 0x2ea2d00, 0xc000897f40, 0x20, 0xc0002f2f00) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/stream.go:735 +0x103 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/internal/plugin.(*gRPCBrokerStartStreamClient).Recv(0xc0002674a0, 0x31840c8, 0xc00069a300, 0xc000332c80) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/internal/plugin/grpc_broker.pb.go:149 +0x62 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin.(*gRPCBrokerClientImpl).StartStream(0xc000332c80, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/grpc_broker.go:194 +0x187 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin.newGRPCClient Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/grpc_client.go:63 +0x261 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 333 [select, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/allocrunner.(*allocRunner).handleTaskStateUpdates(0xc000626000) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/allocrunner/alloc_runner.go:447 +0x75c Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/client/allocrunner.(*allocRunner).Run Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/allocrunner/alloc_runner.go:281 +0x87 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 334 [select, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/allocrunner.(*allocRunner).handleAllocUpdates(0xc000626000) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/allocrunner/alloc_runner.go:760 +0xd3 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/client/allocrunner.(*allocRunner).Run Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/allocrunner/alloc_runner.go:284 +0xac Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 335 [select, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/allocrunner/taskrunner.(*TaskRunner).Run(0xc0001b1600) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/allocrunner/taskrunner/task_runner.go:545 +0xfe7 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/client/allocrunner.(*allocRunner).runTasks Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/allocrunner/alloc_runner.go:351 +0x95 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 336 [select, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/allocrunner/taskrunner.(*TaskRunner).Run(0xc0001b18c0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/allocrunner/taskrunner/task_runner.go:545 +0xfe7 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/client/allocrunner.(*allocRunner).runTasks Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/allocrunner/alloc_runner.go:351 +0x95 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 337 [select, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/allocrunner/taskrunner.(*TaskRunner).handleUpdates(0xc0001b18c0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/allocrunner/taskrunner/task_runner.go:676 +0xb9 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/client/allocrunner/taskrunner.(*TaskRunner).Run Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/allocrunner/taskrunner/task_runner.go:471 +0x20c Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 354 [select, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/LK4D4/joincontext.(*joinContext).run(0xc000332f80) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/LK4D4/joincontext/context.go:73 +0x10b Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/github.com/LK4D4/joincontext.Join Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/LK4D4/joincontext/context.go:33 +0xc2 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 340 [IO wait, 108 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll.runtime_pollWait(0x7f2c5304b6a8, 0x72, 0xffffffffffffffff) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: runtime/netpoll.go:203 +0x55 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll.(*pollDesc).wait(0xc00074db98, 0x72, 0x8000, 0x8000, 0xffffffffffffffff) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll/fd_poll_runtime.go:87 +0x45 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll.(*pollDesc).waitRead(...) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll/fd_poll_runtime.go:92 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll.(*FD).Read(0xc00074db80, 0xc000788000, 0x8000, 0x8000, 0x0, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll/fd_unix.go:169 +0x19b Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net.(*netFD).Read(0xc00074db80, 0xc000788000, 0x8000, 0x8000, 0x800000601, 0x0, 0x60100000000) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/fd_unix.go:202 +0x4f Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net.(*conn).Read(0xc0000d4b90, 0xc000788000, 0x8000, 0x8000, 0x0, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/net.go:184 +0x8e Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: bufio.(*Reader).Read(0xc0005d48a0, 0xc0001769d8, 0x9, 0x9, 0xc0007e9e08, 0x40a183, 0xc0006286c0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: bufio/bufio.go:226 +0x24f Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: io.ReadAtLeast(0x37b4d20, 0xc0005d48a0, 0xc0001769d8, 0x9, 0x9, 0x9, 0x466970, 0xc0006286c0, 0xc0007e9e40) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: io/io.go:310 +0x87 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: io.ReadFull(...) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: io/io.go:329 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/golang.org/x/net/http2.readFrameHeader(0xc0001769d8, 0x9, 0x9, 0x37b4d20, 0xc0005d48a0, 0x0, 0xc000000000, 0x100000000411ce8, 0xc000a57b90) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/golang.org/x/net/http2/frame.go:237 +0x87 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/golang.org/x/net/http2.(*Framer).ReadFrame(0xc0001769a0, 0xc001168e40, 0xc001168e40, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/golang.org/x/net/http2/frame.go:492 +0xa1 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport.(*http2Client).reader(0xc0004f4fc0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport/http2_client.go:1265 +0x16f Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport.newHTTP2Client Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport/http2_client.go:300 +0xd23 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 355 [select, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/LK4D4/joincontext.(*joinContext).run(0xc000333040) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/LK4D4/joincontext/context.go:73 +0x10b Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/github.com/LK4D4/joincontext.Join Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/LK4D4/joincontext/context.go:33 +0xc2 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 341 [select, 108 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport.(*controlBuffer).get(0xc000a57b80, 0x1, 0x0, 0x0, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport/controlbuf.go:395 +0x122 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport.(*loopyWriter).run(0xc0005d4a80, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport/controlbuf.go:513 +0x1cd Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport.newHTTP2Client.func3(0xc0004f4fc0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport/http2_client.go:346 +0x7b Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport.newHTTP2Client Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport/http2_client.go:344 +0xedb Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 356 [chan receive]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin.pidWait(0xfc9b, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/process.go:17 +0x91 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin.(*Client).reattach.func1(0xc00031bea0, 0xfc9b) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/client.go:820 +0x7b Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin.(*Client).reattach Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/client.go:813 +0x17d Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 357 [select, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc.(*ccBalancerWrapper).watcher(0xc000333480) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/balancer_conn_wrappers.go:69 +0xc2 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/google.golang.org/grpc.newCCBalancerWrapper Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/balancer_conn_wrappers.go:60 +0x16d Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 358 [chan receive, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc.(*addrConn).resetTransport(0xc0003d6b00) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/clientconn.go:1179 +0x6ea Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/google.golang.org/grpc.(*addrConn).connect Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/clientconn.go:815 +0x128 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 359 [select, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin.(*gRPCBrokerClientImpl).Recv(0xc000333500, 0x0, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/grpc_broker.go:231 +0xb3 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin.(*GRPCBroker).Run(0xc000896370) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/grpc_broker.go:411 +0x7a Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin.newGRPCClient Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/grpc_client.go:62 +0x23f Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 360 [select, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport.(*Stream).waitOnHeader(0xc00053a600) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport/transport.go:318 +0xcc Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport.(*Stream).RecvCompress(...) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport/transport.go:333 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc.(*csAttempt).recvMsg(0xc0006afc00, 0x2ea2d00, 0xc0008e85a0, 0x0, 0x8, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/stream.go:871 +0x753 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc.(*clientStream).RecvMsg.func1(0xc0006afc00, 0x203000, 0x203000) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/stream.go:736 +0x46 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc.(*clientStream).withRetry(0xc00015ab40, 0xc000b35e30, 0xc000b35e00, 0x382dbe0, 0x30) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/stream.go:590 +0x360 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc.(*clientStream).RecvMsg(0xc00015ab40, 0x2ea2d00, 0xc0008e85a0, 0x20, 0xc0006c6180) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/stream.go:735 +0x103 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/internal/plugin.(*gRPCBrokerStartStreamClient).Recv(0xc000267ae0, 0x31840c8, 0xc00069aa20, 0xc000333500) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/internal/plugin/grpc_broker.pb.go:149 +0x62 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin.(*gRPCBrokerClientImpl).StartStream(0xc000333500, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/grpc_broker.go:194 +0x187 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin.newGRPCClient Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/grpc_client.go:63 +0x261 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 361 [select, 27 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc0009db8c0, 0xc000e52000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x295 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: bufio.(*Reader).fill(0xc0008fdce0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: bufio/bufio.go:100 +0x103 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: bufio.(*Reader).ReadByte(0xc0008fdce0, 0xc3, 0x400, 0xc00050b810) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: bufio/bufio.go:252 +0x39 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc000e3fe80, 0x203000) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:447 +0x38 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*decReaderSwitch).readn1IO(0xc000e3c2c0, 0x203000) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:2230 +0x55 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*decReaderSwitch).readn1(0xc000e3c2c0, 0xc000e53000) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:2224 +0x79 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).readNextBd(...) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:742 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).TryDecodeAsNil(0xc0012533e0, 0x411476) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:791 +0x5d Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).mustDecode(0xc000e3c280, 0x27ae480, 0xc000bdf380) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:2536 +0x3a Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc000e3c280, 0x27ae480, 0xc000bdf380, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:2519 +0xa9 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc000db3a80, 0x27ae480, 0xc000bdf380, 0xc000db3a01, 0xc000bdf380) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x55 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadResponseHeader(0xc000db3a80, 0xc000bdf380, 0x2e4ea20, 0xc0006afc80) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:74 +0x41 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.CallWithCodec(0x3815680, 0xc000db3a80, 0x30c10a7, 0x14, 0x2e4ea20, 0xc0006afc80, 0x2baaa40, 0xc0006a77a0, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/client.go:29 +0xee Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/helper/pool.(*ConnPool).RPC(0xc0006cf720, 0x3097e77, 0x6, 0x37ed380, 0xc00054df80, 0x1, 0x30c10a7, 0x14, 0x2e4ea20, 0xc0006afc80, ...) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/helper/pool/pool.go:453 +0x184 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client.(*Client).RPC(0xc00019e780, 0x30c10a7, 0x14, 0x2e4ea20, 0xc0006afc80, 0x2baaa40, 0xc0006a77a0, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/rpc.go:70 +0x13f Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client.(*Client).watchAllocations(0xc00019e780, 0xc000a7f8c0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/client.go:1978 +0x286 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/client.(*Client).run Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/client.go:1632 +0x7c Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 342 [IO wait, 108 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll.runtime_pollWait(0x7f2c5304b4e8, 0x72, 0xffffffffffffffff) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: runtime/netpoll.go:203 +0x55 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll.(*pollDesc).wait(0xc00074de98, 0x72, 0x8000, 0x8000, 0xffffffffffffffff) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll/fd_poll_runtime.go:87 +0x45 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll.(*pollDesc).waitRead(...) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll/fd_poll_runtime.go:92 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll.(*FD).Read(0xc00074de80, 0xc00080c000, 0x8000, 0x8000, 0x0, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll/fd_unix.go:169 +0x19b Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net.(*netFD).Read(0xc00074de80, 0xc00080c000, 0x8000, 0x8000, 0x800000601, 0x0, 0x60100000000) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/fd_unix.go:202 +0x4f Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net.(*conn).Read(0xc0000d4bc8, 0xc00080c000, 0x8000, 0x8000, 0x0, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/net.go:184 +0x8e Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: bufio.(*Reader).Read(0xc0005d5140, 0xc000176ab8, 0x9, 0x9, 0xc0003d4e08, 0x40a183, 0xc000628b40) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: bufio/bufio.go:226 +0x24f Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: io.ReadAtLeast(0x37b4d20, 0xc0005d5140, 0xc000176ab8, 0x9, 0x9, 0x9, 0x466970, 0xc000628b40, 0xc0003d4e40) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: io/io.go:310 +0x87 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: io.ReadFull(...) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: io/io.go:329 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/golang.org/x/net/http2.readFrameHeader(0xc000176ab8, 0x9, 0x9, 0x37b4d20, 0xc0005d5140, 0x0, 0xc000000000, 0x100000000411ce8, 0xc000a57d70) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/golang.org/x/net/http2/frame.go:237 +0x87 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/golang.org/x/net/http2.(*Framer).ReadFrame(0xc000176a80, 0xc001168e20, 0xc001168e20, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/golang.org/x/net/http2/frame.go:492 +0xa1 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport.(*http2Client).reader(0xc0004f5340) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport/http2_client.go:1265 +0x16f Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport.newHTTP2Client Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport/http2_client.go:300 +0xd23 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 343 [select, 108 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport.(*controlBuffer).get(0xc000a57d60, 0x1, 0x0, 0x0, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport/controlbuf.go:395 +0x122 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport.(*loopyWriter).run(0xc0005d5320, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport/controlbuf.go:513 +0x1cd Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport.newHTTP2Client.func3(0xc0004f5340) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport/http2_client.go:346 +0x7b Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport.newHTTP2Client Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport/http2_client.go:344 +0xedb Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 344 [IO wait, 108 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll.runtime_pollWait(0x7f2c5304b408, 0x72, 0xffffffffffffffff) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: runtime/netpoll.go:203 +0x55 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll.(*pollDesc).wait(0xc000180518, 0x72, 0x8000, 0x8000, 0xffffffffffffffff) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll/fd_poll_runtime.go:87 +0x45 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll.(*pollDesc).waitRead(...) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll/fd_poll_runtime.go:92 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll.(*FD).Read(0xc000180500, 0xc000874000, 0x8000, 0x8000, 0x0, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll/fd_unix.go:169 +0x19b Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net.(*netFD).Read(0xc000180500, 0xc000874000, 0x8000, 0x8000, 0x800000601, 0x0, 0x60100000000) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/fd_unix.go:202 +0x4f Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net.(*conn).Read(0xc0000d4be0, 0xc000874000, 0x8000, 0x8000, 0x0, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/net.go:184 +0x8e Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: bufio.(*Reader).Read(0xc0005d59e0, 0xc000176b98, 0x9, 0x9, 0xc0007eae08, 0x40a183, 0xc000628f60) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: bufio/bufio.go:226 +0x24f Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: io.ReadAtLeast(0x37b4d20, 0xc0005d59e0, 0xc000176b98, 0x9, 0x9, 0x9, 0x466970, 0xc000628f60, 0xc0007eae40) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: io/io.go:310 +0x87 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: io.ReadFull(...) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: io/io.go:329 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/golang.org/x/net/http2.readFrameHeader(0xc000176b98, 0x9, 0x9, 0x37b4d20, 0xc0005d59e0, 0x0, 0xc000000000, 0x100000000411ce8, 0xc000a57f50) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/golang.org/x/net/http2/frame.go:237 +0x87 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/golang.org/x/net/http2.(*Framer).ReadFrame(0xc000176b60, 0xc001210660, 0xc001210660, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/golang.org/x/net/http2/frame.go:492 +0xa1 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport.(*http2Client).reader(0xc0004f5500) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport/http2_client.go:1265 +0x16f Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport.newHTTP2Client Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport/http2_client.go:300 +0xd23 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 345 [select, 108 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport.(*controlBuffer).get(0xc000a57f40, 0x1, 0x0, 0x0, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport/controlbuf.go:395 +0x122 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport.(*loopyWriter).run(0xc0005d5bc0, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport/controlbuf.go:513 +0x1cd Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport.newHTTP2Client.func3(0xc0004f5500) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport/http2_client.go:346 +0x7b Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport.newHTTP2Client Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport/http2_client.go:344 +0xedb Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 362 [IO wait, 108 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll.runtime_pollWait(0x7f2c5304b5c8, 0x72, 0xffffffffffffffff) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: runtime/netpoll.go:203 +0x55 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll.(*pollDesc).wait(0xc000910b18, 0x72, 0x8000, 0x8000, 0xffffffffffffffff) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll/fd_poll_runtime.go:87 +0x45 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll.(*pollDesc).waitRead(...) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll/fd_poll_runtime.go:92 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll.(*FD).Read(0xc000910b00, 0xc00088c000, 0x8000, 0x8000, 0x0, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll/fd_unix.go:169 +0x19b Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net.(*netFD).Read(0xc000910b00, 0xc00088c000, 0x8000, 0x8000, 0x800000601, 0x0, 0x60100000000) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/fd_unix.go:202 +0x4f Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net.(*conn).Read(0xc0000111e8, 0xc00088c000, 0x8000, 0x8000, 0x0, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/net.go:184 +0x8e Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: bufio.(*Reader).Read(0xc0007131a0, 0xc0001a2c78, 0x9, 0x9, 0xc000ac5e08, 0x40a183, 0xc00069b800) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: bufio/bufio.go:226 +0x24f Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: io.ReadAtLeast(0x37b4d20, 0xc0007131a0, 0xc0001a2c78, 0x9, 0x9, 0x9, 0x466970, 0xc00069b800, 0xc000ac5e40) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: io/io.go:310 +0x87 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: io.ReadFull(...) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: io/io.go:329 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/golang.org/x/net/http2.readFrameHeader(0xc0001a2c78, 0x9, 0x9, 0x37b4d20, 0xc0007131a0, 0x0, 0xc000000000, 0x100000000411ce8, 0xc000897000) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/golang.org/x/net/http2/frame.go:237 +0x87 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/golang.org/x/net/http2.(*Framer).ReadFrame(0xc0001a2c40, 0xc001211620, 0xc001211620, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/golang.org/x/net/http2/frame.go:492 +0xa1 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport.(*http2Client).reader(0xc00032c8c0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport/http2_client.go:1265 +0x16f Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport.newHTTP2Client Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport/http2_client.go:300 +0xd23 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 363 [select, 108 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport.(*controlBuffer).get(0xc000896ff0, 0x1, 0x0, 0x0, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport/controlbuf.go:395 +0x122 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport.(*loopyWriter).run(0xc000713380, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport/controlbuf.go:513 +0x1cd Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport.newHTTP2Client.func3(0xc00032c8c0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport/http2_client.go:346 +0x7b Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport.newHTTP2Client Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport/http2_client.go:344 +0xedb Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 364 [select, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/allocrunner/taskrunner.(*TaskRunner).handleUpdates(0xc00099c2c0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/allocrunner/taskrunner/task_runner.go:676 +0xb9 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/client/allocrunner/taskrunner.(*TaskRunner).Run Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/allocrunner/taskrunner/task_runner.go:471 +0x20c Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 365 [select, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/LK4D4/joincontext.(*joinContext).run(0xc000840cc0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/LK4D4/joincontext/context.go:73 +0x10b Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/github.com/LK4D4/joincontext.Join Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/LK4D4/joincontext/context.go:33 +0xc2 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 366 [select, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/LK4D4/joincontext.(*joinContext).run(0xc000840d40) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/LK4D4/joincontext/context.go:73 +0x10b Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/github.com/LK4D4/joincontext.Join Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/LK4D4/joincontext/context.go:33 +0xc2 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 367 [chan receive]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin.pidWait(0xfea8, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/process.go:17 +0x91 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin.(*Client).reattach.func1(0xc0005641e0, 0xfea8) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/client.go:820 +0x7b Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin.(*Client).reattach Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/client.go:813 +0x17d Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 368 [select, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc.(*ccBalancerWrapper).watcher(0xc000841200) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/balancer_conn_wrappers.go:69 +0xc2 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/google.golang.org/grpc.newCCBalancerWrapper Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/balancer_conn_wrappers.go:60 +0x16d Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 369 [chan receive, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc.(*addrConn).resetTransport(0xc0003d7080) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/clientconn.go:1179 +0x6ea Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/google.golang.org/grpc.(*addrConn).connect Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/clientconn.go:815 +0x128 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 370 [select, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin.(*gRPCBrokerClientImpl).Recv(0xc000841600, 0x0, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/grpc_broker.go:231 +0xb3 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin.(*GRPCBroker).Run(0xc000897220) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/grpc_broker.go:411 +0x7a Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin.newGRPCClient Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/grpc_client.go:62 +0x23f Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 371 [select, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport.(*Stream).waitOnHeader(0xc00053a200) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport/transport.go:318 +0xcc Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport.(*Stream).RecvCompress(...) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport/transport.go:333 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc.(*csAttempt).recvMsg(0xc000911380, 0x2ea2d00, 0xc0008e8230, 0x0, 0x8, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/stream.go:871 +0x753 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 systemd[1]: nomad.service: Main process exited, code=exited, status=2/INVALIDARGUMENT Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc.(*clientStream).RecvMsg.func1(0xc000911380, 0x203000, 0x203000) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/stream.go:736 +0x46 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc.(*clientStream).withRetry(0xc000882900, 0xc000b1fe30, 0xc000b1fe00, 0x382dbe0, 0x30) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/stream.go:590 +0x360 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc.(*clientStream).RecvMsg(0xc000882900, 0x2ea2d00, 0xc0008e8230, 0x20, 0xc0006c7380) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/stream.go:735 +0x103 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/internal/plugin.(*gRPCBrokerStartStreamClient).Recv(0xc000267910, 0x31840c8, 0xc00069bf20, 0xc000841600) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/internal/plugin/grpc_broker.pb.go:149 +0x62 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin.(*gRPCBrokerClientImpl).StartStream(0xc000841600, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/grpc_broker.go:194 +0x187 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin.newGRPCClient Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/grpc_client.go:63 +0x261 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 372 [IO wait, 108 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll.runtime_pollWait(0x7f2c5304b328, 0x72, 0xffffffffffffffff) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: runtime/netpoll.go:203 +0x55 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll.(*pollDesc).wait(0xc000911498, 0x72, 0x8000, 0x8000, 0xffffffffffffffff) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll/fd_poll_runtime.go:87 +0x45 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll.(*pollDesc).waitRead(...) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll/fd_poll_runtime.go:92 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll.(*FD).Read(0xc000911480, 0xc0009ba000, 0x8000, 0x8000, 0x0, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll/fd_unix.go:169 +0x19b Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net.(*netFD).Read(0xc000911480, 0xc0009ba000, 0x8000, 0x8000, 0x800000601, 0x0, 0x60100000000) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/fd_unix.go:202 +0x4f Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net.(*conn).Read(0xc000011230, 0xc0009ba000, 0x8000, 0x8000, 0x0, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/net.go:184 +0x8e Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 systemd[1]: nomad.service: Failed with result 'exit-code'. Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: bufio.(*Reader).Read(0xc000818000, 0xc0001a3298, 0x9, 0x9, 0xc000ac6e08, 0x40a183, 0xc000816360) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: bufio/bufio.go:226 +0x24f Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: io.ReadAtLeast(0x37b4d20, 0xc000818000, 0xc0001a3298, 0x9, 0x9, 0x9, 0x466970, 0xc000816360, 0xc000ac6e40) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: io/io.go:310 +0x87 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: io.ReadFull(...) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: io/io.go:329 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/golang.org/x/net/http2.readFrameHeader(0xc0001a3298, 0x9, 0x9, 0x37b4d20, 0xc000818000, 0x0, 0xc000000000, 0x100000000411ce8, 0xc0008973c0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/golang.org/x/net/http2/frame.go:237 +0x87 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/golang.org/x/net/http2.(*Framer).ReadFrame(0xc0001a3260, 0xc001168040, 0xc001168040, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/golang.org/x/net/http2/frame.go:492 +0xa1 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport.(*http2Client).reader(0xc00032ca80) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport/http2_client.go:1265 +0x16f Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport.newHTTP2Client Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport/http2_client.go:300 +0xd23 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 373 [select, 108 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport.(*controlBuffer).get(0xc0008973b0, 0x1, 0x0, 0x0, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport/controlbuf.go:395 +0x122 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport.(*loopyWriter).run(0xc0008181e0, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport/controlbuf.go:513 +0x1cd Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport.newHTTP2Client.func3(0xc00032ca80) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport/http2_client.go:346 +0x7b Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport.newHTTP2Client Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport/http2_client.go:344 +0xedb Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 374 [select, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/allocrunner/taskrunner.(*TaskRunner).handleUpdates(0xc0001b1600) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/allocrunner/taskrunner/task_runner.go:676 +0xb9 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/client/allocrunner/taskrunner.(*TaskRunner).Run Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/allocrunner/taskrunner/task_runner.go:471 +0x20c Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 375 [select, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/LK4D4/joincontext.(*joinContext).run(0xc000841b00) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/LK4D4/joincontext/context.go:73 +0x10b Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/github.com/LK4D4/joincontext.Join Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/LK4D4/joincontext/context.go:33 +0xc2 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 376 [select, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/LK4D4/joincontext.(*joinContext).run(0xc000841b80) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/LK4D4/joincontext/context.go:73 +0x10b Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/github.com/LK4D4/joincontext.Join Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/LK4D4/joincontext/context.go:33 +0xc2 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 377 [chan receive]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin.pidWait(0xfc9a, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/process.go:17 +0x91 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin.(*Client).reattach.func1(0xc000564320, 0xfc9a) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/client.go:820 +0x7b Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin.(*Client).reattach Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/client.go:813 +0x17d Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 378 [select, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc.(*ccBalancerWrapper).watcher(0xc000820080) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/balancer_conn_wrappers.go:69 +0xc2 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/google.golang.org/grpc.newCCBalancerWrapper Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/balancer_conn_wrappers.go:60 +0x16d Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 379 [chan receive, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc.(*addrConn).resetTransport(0xc0003d7600) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/clientconn.go:1179 +0x6ea Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/google.golang.org/grpc.(*addrConn).connect Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/clientconn.go:815 +0x128 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 380 [select, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin.(*gRPCBrokerClientImpl).Recv(0xc000820100, 0x0, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/grpc_broker.go:231 +0xb3 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin.(*GRPCBroker).Run(0xc000897680) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/grpc_broker.go:411 +0x7a Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin.newGRPCClient Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/grpc_client.go:62 +0x23f Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 381 [select, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport.(*Stream).waitOnHeader(0xc00053a800) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport/transport.go:318 +0xcc Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport.(*Stream).RecvCompress(...) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport/transport.go:333 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc.(*csAttempt).recvMsg(0xc000911d00, 0x2ea2d00, 0xc0008e8be0, 0x0, 0x8, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/stream.go:871 +0x753 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc.(*clientStream).RecvMsg.func1(0xc000911d00, 0x203000, 0x203000) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/stream.go:736 +0x46 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc.(*clientStream).withRetry(0xc00083e360, 0xc000b37e30, 0xc000b37e00, 0x382dbe0, 0x30) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/stream.go:590 +0x360 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc.(*clientStream).RecvMsg(0xc00083e360, 0x2ea2d00, 0xc0008e8be0, 0x20, 0xc000846300) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/stream.go:735 +0x103 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/internal/plugin.(*gRPCBrokerStartStreamClient).Recv(0xc000267ba0, 0x31840c8, 0xc000816a80, 0xc000820100) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/internal/plugin/grpc_broker.pb.go:149 +0x62 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin.(*gRPCBrokerClientImpl).StartStream(0xc000820100, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/grpc_broker.go:194 +0x187 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin.newGRPCClient Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/grpc_client.go:63 +0x261 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 382 [IO wait, 108 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll.runtime_pollWait(0x7f2c5304b248, 0x72, 0xffffffffffffffff) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: runtime/netpoll.go:203 +0x55 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll.(*pollDesc).wait(0xc000911e18, 0x72, 0x8000, 0x8000, 0xffffffffffffffff) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll/fd_poll_runtime.go:87 +0x45 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll.(*pollDesc).waitRead(...) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll/fd_poll_runtime.go:92 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll.(*FD).Read(0xc000911e00, 0xc000ad0000, 0x8000, 0x8000, 0x0, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll/fd_unix.go:169 +0x19b Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net.(*netFD).Read(0xc000911e00, 0xc000ad0000, 0x8000, 0x8000, 0x800000601, 0x0, 0x60100000000) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/fd_unix.go:202 +0x4f Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net.(*conn).Read(0xc000011278, 0xc000ad0000, 0x8000, 0x8000, 0x0, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/net.go:184 +0x8e Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: bufio.(*Reader).Read(0xc000818e40, 0xc0001a3458, 0x9, 0x9, 0xc000ac7e08, 0x40a183, 0xc000816ea0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: bufio/bufio.go:226 +0x24f Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: io.ReadAtLeast(0x37b4d20, 0xc000818e40, 0xc0001a3458, 0x9, 0x9, 0x9, 0x466970, 0xc000816ea0, 0xc000ac7e40) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: io/io.go:310 +0x87 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: io.ReadFull(...) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: io/io.go:329 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/golang.org/x/net/http2.readFrameHeader(0xc0001a3458, 0x9, 0x9, 0x37b4d20, 0xc000818e40, 0x0, 0xc000000000, 0x100000000411ce8, 0xc000897820) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/golang.org/x/net/http2/frame.go:237 +0x87 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/golang.org/x/net/http2.(*Framer).ReadFrame(0xc0001a3420, 0xc001169020, 0xc001169020, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/golang.org/x/net/http2/frame.go:492 +0xa1 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport.(*http2Client).reader(0xc00032cc40) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport/http2_client.go:1265 +0x16f Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport.newHTTP2Client Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport/http2_client.go:300 +0xd23 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 383 [select, 108 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport.(*controlBuffer).get(0xc000897810, 0x1, 0x0, 0x0, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport/controlbuf.go:395 +0x122 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport.(*loopyWriter).run(0xc000819020, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport/controlbuf.go:513 +0x1cd Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport.newHTTP2Client.func3(0xc00032cc40) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport/http2_client.go:346 +0x7b Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport.newHTTP2Client Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport/http2_client.go:344 +0xedb Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 384 [IO wait, 108 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll.runtime_pollWait(0x7f2c5304b168, 0x72, 0xffffffffffffffff) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: runtime/netpoll.go:203 +0x55 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll.(*pollDesc).wait(0xc00098e218, 0x72, 0x8000, 0x8000, 0xffffffffffffffff) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll/fd_poll_runtime.go:87 +0x45 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll.(*pollDesc).waitRead(...) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll/fd_poll_runtime.go:92 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll.(*FD).Read(0xc00098e200, 0xc000ae8000, 0x8000, 0x8000, 0x0, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll/fd_unix.go:169 +0x19b Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net.(*netFD).Read(0xc00098e200, 0xc000ae8000, 0x8000, 0x8000, 0x800000601, 0x0, 0x60100000000) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/fd_unix.go:202 +0x4f Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net.(*conn).Read(0xc000011280, 0xc000ae8000, 0x8000, 0x8000, 0x0, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/net.go:184 +0x8e Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: bufio.(*Reader).Read(0xc0008196e0, 0xc0001a3538, 0x9, 0x9, 0xc000ac0e08, 0x40a183, 0xc0008172c0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: bufio/bufio.go:226 +0x24f Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: io.ReadAtLeast(0x37b4d20, 0xc0008196e0, 0xc0001a3538, 0x9, 0x9, 0x9, 0x466970, 0xc0008172c0, 0xc000ac0e40) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: io/io.go:310 +0x87 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: io.ReadFull(...) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: io/io.go:329 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/golang.org/x/net/http2.readFrameHeader(0xc0001a3538, 0x9, 0x9, 0x37b4d20, 0xc0008196e0, 0x0, 0xc000000000, 0x100000000411ce8, 0xc000897a00) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/golang.org/x/net/http2/frame.go:237 +0x87 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/golang.org/x/net/http2.(*Framer).ReadFrame(0xc0001a3500, 0xc001211640, 0xc001211640, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/golang.org/x/net/http2/frame.go:492 +0xa1 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport.(*http2Client).reader(0xc00032ce00) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport/http2_client.go:1265 +0x16f Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport.newHTTP2Client Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport/http2_client.go:300 +0xd23 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 385 [select, 108 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport.(*controlBuffer).get(0xc0008979f0, 0x1, 0x0, 0x0, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport/controlbuf.go:395 +0x122 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport.(*loopyWriter).run(0xc0008198c0, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport/controlbuf.go:513 +0x1cd Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport.newHTTP2Client.func3(0xc00032ce00) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport/http2_client.go:346 +0x7b Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport.newHTTP2Client Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport/http2_client.go:344 +0xedb Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 386 [IO wait, 108 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll.runtime_pollWait(0x7f2c5304b088, 0x72, 0xffffffffffffffff) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: runtime/netpoll.go:203 +0x55 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll.(*pollDesc).wait(0xc00098e518, 0x72, 0x8000, 0x8000, 0xffffffffffffffff) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll/fd_poll_runtime.go:87 +0x45 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll.(*pollDesc).waitRead(...) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll/fd_poll_runtime.go:92 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll.(*FD).Read(0xc00098e500, 0xc000b00000, 0x8000, 0x8000, 0x0, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll/fd_unix.go:169 +0x19b Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net.(*netFD).Read(0xc00098e500, 0xc000b00000, 0x8000, 0x8000, 0x800000601, 0x0, 0x60100000000) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/fd_unix.go:202 +0x4f Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net.(*conn).Read(0xc000011288, 0xc000b00000, 0x8000, 0x8000, 0x0, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/net.go:184 +0x8e Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: bufio.(*Reader).Read(0xc000819f80, 0xc0001a3618, 0x9, 0x9, 0xc000ac1e08, 0x40a183, 0xc0008176e0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: bufio/bufio.go:226 +0x24f Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: io.ReadAtLeast(0x37b4d20, 0xc000819f80, 0xc0001a3618, 0x9, 0x9, 0x9, 0x466970, 0xc0008176e0, 0xc000ac1e40) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: io/io.go:310 +0x87 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: io.ReadFull(...) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: io/io.go:329 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/golang.org/x/net/http2.readFrameHeader(0xc0001a3618, 0x9, 0x9, 0x37b4d20, 0xc000819f80, 0x0, 0xc000000000, 0x100000000411ce8, 0xc000897be0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/golang.org/x/net/http2/frame.go:237 +0x87 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/golang.org/x/net/http2.(*Framer).ReadFrame(0xc0001a35e0, 0xc001211680, 0xc001211680, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/golang.org/x/net/http2/frame.go:492 +0xa1 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport.(*http2Client).reader(0xc00032cfc0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport/http2_client.go:1265 +0x16f Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport.newHTTP2Client Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport/http2_client.go:300 +0xd23 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 387 [select, 108 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport.(*controlBuffer).get(0xc000897bd0, 0x1, 0x0, 0x0, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport/controlbuf.go:395 +0x122 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport.(*loopyWriter).run(0xc000848180, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport/controlbuf.go:513 +0x1cd Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport.newHTTP2Client.func3(0xc00032cfc0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport/http2_client.go:346 +0x7b Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport.newHTTP2Client Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/internal/transport/http2_client.go:344 +0xedb Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 388 [select, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc.newClientStream.func5(0xc0006c4e00, 0xc000021d40, 0x3814500, 0xc0006a6f30) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/stream.go:319 +0xd7 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/google.golang.org/grpc.newClientStream Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/stream.go:318 +0xbbf Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 389 [select, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin.(*gRPCBrokerClientImpl).StartStream.func1(0xc00069a300, 0xc000332c80, 0x3838980, 0xc0002674a0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/grpc_broker.go:181 +0x146 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin.(*gRPCBrokerClientImpl).StartStream Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/grpc_broker.go:179 +0x165 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 346 [select, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc.newClientStream.func5(0xc0006c4380, 0xc0006b2ea0, 0x3814500, 0xc0006e2cc0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/stream.go:319 +0xd7 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/google.golang.org/grpc.newClientStream Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/stream.go:318 +0xbbf Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 347 [select, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin.(*gRPCBrokerClientImpl).StartStream.func1(0xc00063bc80, 0xc000a19f80, 0x3838980, 0xc00052c930) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/grpc_broker.go:181 +0x146 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin.(*gRPCBrokerClientImpl).StartStream Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/grpc_broker.go:179 +0x165 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 390 [select, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc.newClientStream.func5(0xc0006c5500, 0xc000882900, 0x3814500, 0xc00013f740) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/stream.go:319 +0xd7 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/google.golang.org/grpc.newClientStream Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/stream.go:318 +0xbbf Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 391 [select, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin.(*gRPCBrokerClientImpl).StartStream.func1(0xc00069bf20, 0xc000841600, 0x3838980, 0xc000267910) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/grpc_broker.go:181 +0x146 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin.(*gRPCBrokerClientImpl).StartStream Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/grpc_broker.go:179 +0x165 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 392 [select, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc.newClientStream.func5(0xc0006c5180, 0xc00015ab40, 0x3814500, 0xc0006a7770) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/stream.go:319 +0xd7 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/google.golang.org/grpc.newClientStream Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/stream.go:318 +0xbbf Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 393 [select, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin.(*gRPCBrokerClientImpl).StartStream.func1(0xc00069aa20, 0xc000333500, 0x3838980, 0xc000267ae0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/grpc_broker.go:181 +0x146 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin.(*gRPCBrokerClientImpl).StartStream Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/grpc_broker.go:179 +0x165 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 394 [select, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc.newClientStream.func5(0xc0006c5c00, 0xc00083e360, 0x3814500, 0xc00054c720) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/stream.go:319 +0xd7 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/google.golang.org/grpc.newClientStream Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/stream.go:318 +0xbbf Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 395 [select, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin.(*gRPCBrokerClientImpl).StartStream.func1(0xc000816a80, 0xc000820100, 0x3838980, 0xc000267ba0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/grpc_broker.go:181 +0x146 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin.(*gRPCBrokerClientImpl).StartStream Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/grpc_broker.go:179 +0x165 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 396 [select, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/LK4D4/joincontext.(*joinContext).run(0xc000821140) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/LK4D4/joincontext/context.go:73 +0x10b Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/github.com/LK4D4/joincontext.Join Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/LK4D4/joincontext/context.go:33 +0xc2 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 397 [select, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/LK4D4/joincontext.(*joinContext).run(0xc000821240) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/LK4D4/joincontext/context.go:73 +0x10b Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/github.com/LK4D4/joincontext.Join Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/LK4D4/joincontext/context.go:33 +0xc2 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 398 [select]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/allocrunner/taskrunner.(*statsHook).collectResourceUsageStats(0xc000539780, 0x3814440, 0xc000821780, 0x37b5440, 0xc000821740) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/allocrunner/taskrunner/stats_hook.go:98 +0xf0 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/client/allocrunner/taskrunner.(*statsHook).Poststart Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/allocrunner/taskrunner/stats_hook.go:64 +0xf8 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 399 [select, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/drivers/docker.(*Driver).handleWait(0xc00031bc20, 0x3814480, 0xc0000d2010, 0xc0008b0ea0, 0xc000823dd0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/drivers/docker/driver.go:1256 +0xff Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/drivers/docker.(*Driver).WaitTask Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/drivers/docker/driver.go:1250 +0xc4 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 400 [select]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: io.(*pipe).Read(0xc0009b2ea0, 0xc000eb9001, 0xdff, 0xdff, 0x5805dd8, 0x203000, 0x203000) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: io/pipe.go:57 +0xe7 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: io.(*PipeReader).Read(0xc000011970, 0xc000eb9001, 0xdff, 0xdff, 0xc0003efc78, 0x411476, 0xc00110ae00) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: io/pipe.go:134 +0x4c Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: encoding/json.(*Decoder).refill(0xc0005662c0, 0xc00110ae0a, 0x9) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: encoding/json/stream.go:165 +0xeb Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: encoding/json.(*Decoder).readValue(0xc0005662c0, 0x0, 0x0, 0x2f7a240) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: encoding/json/stream.go:140 +0x1e8 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: encoding/json.(*Decoder).Decode(0xc0005662c0, 0x272ecc0, 0xc00110ae00, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: encoding/json/stream.go:63 +0x79 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient.(*Client).Stats(0xc0002f4480, 0xc0008365c0, 0x40, 0xc0008b0f60, 0x1, 0xc00063be00, 0x0, 0x0, 0x3814440, 0xc000821780, ...) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient/container.go:1197 +0x4c4 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/drivers/docker.(*taskHandle).collectStats(0xc000823dd0, 0x3814440, 0xc000821780, 0xc000773640, 0x3b9aca00) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/drivers/docker/stats.go:123 +0x227 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/drivers/docker.(*taskHandle).Stats Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/drivers/docker/stats.go:86 +0x1a6 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 401 [select]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/drivers/docker.dockerStatsCollector(0xc000773640, 0xc0008b0f60, 0x3b9aca00) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/drivers/docker/stats.go:152 +0xeb Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/drivers/docker.(*taskHandle).collectStats Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/drivers/docker/stats.go:112 +0x133 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 402 [IO wait]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll.runtime_pollWait(0x7f2c5304afa8, 0x72, 0xffffffffffffffff) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: runtime/netpoll.go:203 +0x55 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll.(*pollDesc).wait(0xc00098f918, 0x72, 0x1000, 0x1000, 0xffffffffffffffff) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll/fd_poll_runtime.go:87 +0x45 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll.(*pollDesc).waitRead(...) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll/fd_poll_runtime.go:92 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll.(*FD).Read(0xc00098f900, 0xc000b52000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll/fd_unix.go:169 +0x19b Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net.(*netFD).Read(0xc00098f900, 0xc000b52000, 0x1000, 0x1000, 0xc22, 0x0, 0xc000b31658) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/fd_unix.go:202 +0x4f Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net.(*conn).Read(0xc000011988, 0xc000b52000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/net.go:184 +0x8e Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: bufio.(*Reader).fill(0xc0009b3140) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: bufio/bufio.go:100 +0x103 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: bufio.(*Reader).ReadSlice(0xc0009b3140, 0xc00005e00a, 0xc000b31780, 0x40ab2b, 0xc0009c23c0, 0xc0008b1078, 0x170e) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: bufio/bufio.go:359 +0x3d Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/internal.readChunkLine(0xc0009b3140, 0xc000b4c600, 0xc000b4c688, 0xc0009c2418, 0xc000b4c738, 0xc0008b1078) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/internal/chunked.go:122 +0x34 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/internal.(*chunkedReader).beginChunk(0xc0003ffc50) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/internal/chunked.go:48 +0x32 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/internal.(*chunkedReader).Read(0xc0003ffc50, 0xc000c08000, 0x8000, 0x8000, 0x2, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/internal/chunked.go:93 +0x132 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http.(*body).readLocked(0xc0006a9840, 0xc000c08000, 0x8000, 0x8000, 0xc0009b2ea0, 0xc000c08000, 0xc1b) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/transfer.go:847 +0x5f Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http.(*body).Read(0xc0006a9840, 0xc000c08000, 0x8000, 0x8000, 0x0, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/transfer.go:839 +0xf2 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: io.copyBuffer(0x37bb6c0, 0xc000011980, 0x7f2c51eabd18, 0xc0006a9840, 0xc000c08000, 0x8000, 0x8000, 0x0, 0xc000b319c8, 0x40a6ce) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: io/io.go:405 +0x122 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: io.Copy(...) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: io/io.go:364 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient.handleStreamResponse(0xc0002f4090, 0xc000b31d80, 0xc0002f4090, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient/client.go:653 +0x383 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient.(*Client).streamURL(0xc0002f4480, 0x3093fb0, 0x3, 0xc0008b1200, 0x5e, 0x10100, 0x0, 0x0, 0x0, 0x37bb6c0, ...) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient/client.go:630 +0x872 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient.(*Client).stream(0xc0002f4480, 0x3093fb0, 0x3, 0xc0008b11a0, 0x5e, 0x10100, 0x0, 0x0, 0x0, 0x37bb6c0, ...) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient/client.go:531 +0xfa Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient.(*Client).Stats.func2(0xc0009b2e40, 0xc0002f4480, 0xc0008365c0, 0x40, 0xc0008b0f60, 0x1, 0xc00063be00, 0x0, 0x0, 0x3814440, ...) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient/container.go:1159 +0x221 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient.(*Client).Stats Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient/container.go:1157 +0x2f4 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 403 [select, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient.(*Client).Stats.func3(0xc0008365c0, 0x40, 0xc0008b0f60, 0x1, 0xc00063be00, 0x0, 0x0, 0x3814440, 0xc000821780, 0xc000011970, ...) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient/container.go:1186 +0xa0 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient.(*Client).Stats Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient/container.go:1184 +0x399 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 348 [select, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc.newClientStream.func5(0xc00013c000, 0xc0006b2d80, 0x3814500, 0xc0006e2990) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/stream.go:319 +0xd7 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/google.golang.org/grpc.newClientStream Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/stream.go:318 +0xbbf Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 349 [select, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin.(*gRPCBrokerClientImpl).StartStream.func1(0xc00063b3e0, 0xc000985bc0, 0x3838980, 0xc00052cd10) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/grpc_broker.go:181 +0x146 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin.(*gRPCBrokerClientImpl).StartStream Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/grpc_broker.go:179 +0x165 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 404 [chan receive, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient.(*Client).streamURL.func1(0x3814440, 0xc0008217c0, 0x3838ec0, 0xc000011988) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient/client.go:580 +0x48 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient.(*Client).streamURL Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient/client.go:579 +0x520 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 405 [select, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/LK4D4/joincontext.(*joinContext).run(0xc0008219c0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/LK4D4/joincontext/context.go:73 +0x10b Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/github.com/LK4D4/joincontext.Join Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/LK4D4/joincontext/context.go:33 +0xc2 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 406 [select, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/LK4D4/joincontext.(*joinContext).run(0xc000821ac0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/LK4D4/joincontext/context.go:73 +0x10b Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/github.com/LK4D4/joincontext.Join Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/LK4D4/joincontext/context.go:33 +0xc2 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 407 [select, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/allocrunner/taskrunner/template.(*TaskTemplateManager).handleTemplateRerenders(0xc000821b00, 0xbfb0047aad5e9be7, 0x1782a48e1, 0x57d89a0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/allocrunner/taskrunner/template/template.go:352 +0x193 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/allocrunner/taskrunner/template.(*TaskTemplateManager).run(0xc000821b00) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/allocrunner/taskrunner/template/template.go:227 +0x2fc Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/client/allocrunner/taskrunner/template.NewTaskTemplateManager Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/allocrunner/taskrunner/template/template.go:163 +0x2db Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 408 [select, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul-template/manager.(*Runner).Start(0xc000b58360) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul-template/manager/runner.go:330 +0x410 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/client/allocrunner/taskrunner/template.(*TaskTemplateManager).run Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/allocrunner/taskrunner/template/template.go:195 +0x53 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 350 [select, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc.newClientStream.func5(0xc00013c380, 0xc0004e67e0, 0x3814500, 0xc0006e3440) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/stream.go:319 +0xd7 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/google.golang.org/grpc.newClientStream Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/stream.go:318 +0xbbf Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 351 [select, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin.(*gRPCBrokerClientImpl).StartStream.func1(0xc00085f1a0, 0xc0006a8480, 0x3838980, 0xc00052cde0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/grpc_broker.go:181 +0x146 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin.(*gRPCBrokerClientImpl).StartStream Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/grpc_broker.go:179 +0x165 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 352 [select, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc.newClientStream.func5(0xc0003fc000, 0xc000401320, 0x3814500, 0xc0003fe060) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/stream.go:319 +0xd7 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/google.golang.org/grpc.newClientStream Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/stream.go:318 +0xbbf Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 410 [select, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc.newClientStream.func5(0xc0006c4a80, 0xc000020a20, 0x3814500, 0xc0006a6660) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/stream.go:319 +0xd7 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/google.golang.org/grpc.newClientStream Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/google.golang.org/grpc/stream.go:318 +0xbbf Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 353 [select, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin.(*gRPCBrokerClientImpl).StartStream.func1(0xc0006282a0, 0xc0006a8d40, 0x3838980, 0xc00052cee0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/grpc_broker.go:181 +0x146 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin.(*gRPCBrokerClientImpl).StartStream Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/grpc_broker.go:179 +0x165 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 418 [select, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/LK4D4/joincontext.(*joinContext).run(0xc0006a9a00) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/LK4D4/joincontext/context.go:73 +0x10b Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/github.com/LK4D4/joincontext.Join Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/LK4D4/joincontext/context.go:33 +0xc2 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 419 [select, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/LK4D4/joincontext.(*joinContext).run(0xc0006a9b00) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/LK4D4/joincontext/context.go:73 +0x10b Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/github.com/LK4D4/joincontext.Join Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/LK4D4/joincontext/context.go:33 +0xc2 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 420 [select, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/allocrunner/taskrunner/template.(*TaskTemplateManager).handleTemplateRerenders(0xc0006a9b40, 0xbfb0047aad4775fe, 0x178132303, 0x57d89a0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/allocrunner/taskrunner/template/template.go:352 +0x193 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/allocrunner/taskrunner/template.(*TaskTemplateManager).run(0xc0006a9b40) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/allocrunner/taskrunner/template/template.go:227 +0x2fc Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/client/allocrunner/taskrunner/template.NewTaskTemplateManager Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/allocrunner/taskrunner/template/template.go:163 +0x2db Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 421 [select, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul-template/manager.(*Runner).Start(0xc000631680) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul-template/manager/runner.go:330 +0x410 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/client/allocrunner/taskrunner/template.(*TaskTemplateManager).run Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/allocrunner/taskrunner/template/template.go:195 +0x53 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 455 [select, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/LK4D4/joincontext.(*joinContext).run(0xc000c16100) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/LK4D4/joincontext/context.go:73 +0x10b Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/github.com/LK4D4/joincontext.Join Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/LK4D4/joincontext/context.go:33 +0xc2 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 424 [select, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/LK4D4/joincontext.(*joinContext).run(0xc000c16540) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/LK4D4/joincontext/context.go:73 +0x10b Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/github.com/LK4D4/joincontext.Join Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/LK4D4/joincontext/context.go:33 +0xc2 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 411 [select, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin.(*gRPCBrokerClientImpl).StartStream.func1(0xc00094d2c0, 0xc0005394c0, 0x3838980, 0xc000997750) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/grpc_broker.go:181 +0x146 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin.(*gRPCBrokerClientImpl).StartStream Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-plugin/grpc_broker.go:179 +0x165 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 412 [select, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/LK4D4/joincontext.(*joinContext).run(0xc000b605c0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/LK4D4/joincontext/context.go:73 +0x10b Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/github.com/LK4D4/joincontext.Join Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/LK4D4/joincontext/context.go:33 +0xc2 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 413 [select, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/LK4D4/joincontext.(*joinContext).run(0xc000b606c0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/LK4D4/joincontext/context.go:73 +0x10b Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/github.com/LK4D4/joincontext.Join Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/LK4D4/joincontext/context.go:33 +0xc2 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 417 [select, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/LK4D4/joincontext.(*joinContext).run(0xc000b60f00) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/LK4D4/joincontext/context.go:73 +0x10b Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/github.com/LK4D4/joincontext.Join Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/LK4D4/joincontext/context.go:33 +0xc2 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 415 [select, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul-template/manager.(*Runner).Start(0xc000ba4a20) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul-template/manager/runner.go:330 +0x410 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/client/allocrunner/taskrunner/template.(*TaskTemplateManager).run Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/allocrunner/taskrunner/template/template.go:195 +0x53 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 453 [IO wait, 25 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll.runtime_pollWait(0x7f2c5304aec8, 0x72, 0xffffffffffffffff) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: runtime/netpoll.go:203 +0x55 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll.(*pollDesc).wait(0xc000baae98, 0x72, 0x1000, 0x1000, 0xffffffffffffffff) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll/fd_poll_runtime.go:87 +0x45 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll.(*pollDesc).waitRead(...) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll/fd_poll_runtime.go:92 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll.(*FD).Read(0xc000baae80, 0xc000d37000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll/fd_unix.go:169 +0x19b Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net.(*netFD).Read(0xc000baae80, 0xc000d37000, 0x1000, 0x1000, 0x43c34c, 0xc000946b70, 0x469400) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/fd_unix.go:202 +0x4f Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net.(*conn).Read(0xc000d38278, 0xc000d37000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/net.go:184 +0x8e Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http.(*persistConn).Read(0xc000d219e0, 0xc000d37000, 0x1000, 0x1000, 0xc000bb1680, 0xc000946c70, 0x409c65) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/transport.go:1825 +0x75 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: bufio.(*Reader).fill(0xc000d90900) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: bufio/bufio.go:100 +0x103 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: bufio.(*Reader).Peek(0xc000d90900, 0x1, 0x0, 0x0, 0x1, 0xc0012b5000, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: bufio/bufio.go:138 +0x4f Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http.(*persistConn).readLoop(0xc000d219e0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/transport.go:1978 +0x1a8 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by net/http.(*Transport).dialConn Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/transport.go:1647 +0xc56 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 425 [select, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/LK4D4/joincontext.(*joinContext).run(0xc000c16640) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/LK4D4/joincontext/context.go:73 +0x10b Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/github.com/LK4D4/joincontext.Join Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/LK4D4/joincontext/context.go:33 +0xc2 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 445 [select, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/LK4D4/joincontext.(*joinContext).run(0xc000d2c780) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/LK4D4/joincontext/context.go:73 +0x10b Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/github.com/LK4D4/joincontext.Join Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/LK4D4/joincontext/context.go:33 +0xc2 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 427 [select, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul-template/manager.(*Runner).Start(0xc000c5ed80) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul-template/manager/runner.go:330 +0x410 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/client/allocrunner/taskrunner/template.(*TaskTemplateManager).run Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/allocrunner/taskrunner/template/template.go:195 +0x53 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 467 [select]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: io.(*pipe).Read(0xc000ce1440, 0xc0008e7001, 0xdff, 0xdff, 0x5805dd8, 0x203000, 0x203000) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: io/pipe.go:57 +0xe7 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: io.(*PipeReader).Read(0xc0000d4120, 0xc0008e7001, 0xdff, 0xdff, 0xc0002bdc78, 0x411476, 0xc000f68000) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: io/pipe.go:134 +0x4c Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: encoding/json.(*Decoder).refill(0xc000602420, 0xc000f6800a, 0x9) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: encoding/json/stream.go:165 +0xeb Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: encoding/json.(*Decoder).readValue(0xc000602420, 0x0, 0x0, 0x2f7a240) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: encoding/json/stream.go:140 +0x1e8 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: encoding/json.(*Decoder).Decode(0xc000602420, 0x272ecc0, 0xc000f68000, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: encoding/json/stream.go:63 +0x79 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient.(*Client).Stats(0xc0002f4480, 0xc000836e80, 0x40, 0xc000c526c0, 0x1, 0xc00094c4e0, 0x0, 0x0, 0x3814440, 0xc000c17200, ...) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient/container.go:1197 +0x4c4 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/drivers/docker.(*taskHandle).collectStats(0xc000870750, 0x3814440, 0xc000c17200, 0xc0007f0780, 0x3b9aca00) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/drivers/docker/stats.go:123 +0x227 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/drivers/docker.(*taskHandle).Stats Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/drivers/docker/stats.go:86 +0x1a6 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 429 [select]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/allocrunner/taskrunner.(*statsHook).collectResourceUsageStats(0xc000539580, 0x3814440, 0xc000c17200, 0x37b5440, 0xc000c171c0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/allocrunner/taskrunner/stats_hook.go:98 +0xf0 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/client/allocrunner/taskrunner.(*statsHook).Poststart Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/allocrunner/taskrunner/stats_hook.go:64 +0xf8 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 434 [select, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/LK4D4/joincontext.(*joinContext).run(0xc000b61380) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/LK4D4/joincontext/context.go:73 +0x10b Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/github.com/LK4D4/joincontext.Join Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/LK4D4/joincontext/context.go:33 +0xc2 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 435 [select, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/LK4D4/joincontext.(*joinContext).run(0xc000b61480) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/LK4D4/joincontext/context.go:73 +0x10b Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/github.com/LK4D4/joincontext.Join Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/LK4D4/joincontext/context.go:33 +0xc2 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 430 [select, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/drivers/docker.(*Driver).handleWait(0xc00031bc20, 0x3814480, 0xc0000d2010, 0xc000c52540, 0xc000870750) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/drivers/docker/driver.go:1256 +0xff Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/drivers/docker.(*Driver).WaitTask Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/drivers/docker/driver.go:1250 +0xc4 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 436 [select, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/allocrunner/taskrunner/template.(*TaskTemplateManager).handleTemplateRerenders(0xc000b614c0, 0xbfb0047ab317acf0, 0x17de35a2c, 0x57d89a0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/allocrunner/taskrunner/template/template.go:352 +0x193 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/allocrunner/taskrunner/template.(*TaskTemplateManager).run(0xc000b614c0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/allocrunner/taskrunner/template/template.go:227 +0x2fc Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/client/allocrunner/taskrunner/template.NewTaskTemplateManager Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/allocrunner/taskrunner/template/template.go:163 +0x2db Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 437 [select, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul-template/manager.(*Runner).Start(0xc000bfe900) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul-template/manager/runner.go:330 +0x410 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/client/allocrunner/taskrunner/template.(*TaskTemplateManager).run Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/allocrunner/taskrunner/template/template.go:195 +0x53 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 456 [select, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/LK4D4/joincontext.(*joinContext).run(0xc000c17d00) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/LK4D4/joincontext/context.go:73 +0x10b Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/github.com/LK4D4/joincontext.Join Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/LK4D4/joincontext/context.go:33 +0xc2 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 505 [select, 25 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http.(*persistConn).roundTrip(0xc000d219e0, 0xc000e7fc50, 0x0, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/transport.go:2498 +0x756 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http.(*Transport).roundTrip(0xc0009d3b80, 0xc000f2bc00, 0xc000fedac0, 0xc0008bb760, 0x411ce8) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/transport.go:565 +0xad9 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http.(*Transport).RoundTrip(0xc0009d3b80, 0xc000f2bc00, 0xc0009d3b80, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/roundtrip.go:17 +0x35 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http.send(0xc000f2bc00, 0x37bba20, 0xc0009d3b80, 0x0, 0x0, 0x0, 0xc000d381b8, 0xc0008bb930, 0x1, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/client.go:252 +0x43e Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http.(*Client).send(0xc000bdf890, 0xc000f2bc00, 0x0, 0x0, 0x0, 0xc000d381b8, 0x0, 0x1, 0xc000f2bc00) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/client.go:176 +0xfa Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http.(*Client).do(0xc000bdf890, 0xc000f2bc00, 0x0, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/client.go:699 +0x44a Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http.(*Client).Do(...) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/client.go:567 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/api.(*Client).doRequest(0xc000bfec60, 0xc0008a6ea0, 0xc0008bbc20, 0xc0010d6b70, 0x25, 0xc0008a6ea0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/api/api.go:873 +0xb8 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/api.(*KV).getInternal(0xc0008bbdc0, 0xc0006a289a, 0x1e, 0x0, 0xc0008bbe40, 0x57d89a0, 0xc0000a8050, 0x3186398, 0xc0008bbd18) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/api/kv.go:131 +0x285 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/api.(*KV).Get(0xc0008bbdc0, 0xc0006a289a, 0x1e, 0xc0008bbe40, 0x0, 0x0, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/api/kv.go:65 +0xa0 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul-template/dependency.(*KVGetQuery).Fetch(0xc000d1f110, 0xc000bdf860, 0xc0011dce10, 0x1, 0x1, 0xc0011cfe20, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul-template/dependency/kv_get.go:60 +0x35d Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul-template/watch.(*View).fetch(0xc0005dba40, 0xc000614900, 0xc000614a20, 0xc0009894a0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul-template/watch/view.go:202 +0x164 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul-template/watch.(*View).poll Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul-template/watch/view.go:115 +0x445 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 431 [select, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/LK4D4/joincontext.(*joinContext).run(0xc000c17400) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/LK4D4/joincontext/context.go:73 +0x10b Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/github.com/LK4D4/joincontext.Join Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/LK4D4/joincontext/context.go:33 +0xc2 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 432 [select, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/LK4D4/joincontext.(*joinContext).run(0xc000c17500) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/LK4D4/joincontext/context.go:73 +0x10b Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/github.com/LK4D4/joincontext.Join Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/LK4D4/joincontext/context.go:33 +0xc2 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 440 [select, 25 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul-template/watch.(*View).poll(0xc0005dba40, 0xc000bbcb40, 0xc000bb0480) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul-template/watch/view.go:118 +0x15f Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul-template/watch.(*Watcher).Add Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul-template/watch/watcher.go:165 +0x409 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 446 [select, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/LK4D4/joincontext.(*joinContext).run(0xc000d2cd40) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/LK4D4/joincontext/context.go:73 +0x10b Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/github.com/LK4D4/joincontext.Join Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/LK4D4/joincontext/context.go:33 +0xc2 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 447 [select, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/LK4D4/joincontext.(*joinContext).run(0xc000d2ce40) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/LK4D4/joincontext/context.go:73 +0x10b Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/github.com/LK4D4/joincontext.Join Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/LK4D4/joincontext/context.go:33 +0xc2 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 448 [select, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/allocrunner/taskrunner.(*vaultHook).run(0xc0002af970, 0xc000d3c920, 0x1a) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/allocrunner/taskrunner/vault_hook.go:273 +0x35d Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/client/allocrunner/taskrunner.(*vaultHook).Prestart Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/allocrunner/taskrunner/vault_hook.go:146 +0x173 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 489 [select, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/LK4D4/joincontext.(*joinContext).run(0xc000a194c0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/LK4D4/joincontext/context.go:73 +0x10b Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/github.com/LK4D4/joincontext.Join Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/LK4D4/joincontext/context.go:33 +0xc2 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 482 [select]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/drivers/docker.dockerStatsCollector(0xc0008dc880, 0xc000bb1f20, 0x3b9aca00) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/drivers/docker/stats.go:152 +0xeb Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/drivers/docker.(*taskHandle).collectStats Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/drivers/docker/stats.go:112 +0x133 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 465 [select]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: io.(*pipe).Read(0xc0009e2000, 0xc0009af001, 0xdff, 0xdff, 0x40a425, 0x203000, 0x203000) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: io/pipe.go:57 +0xe7 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: io.(*PipeReader).Read(0xc000d386a0, 0xc0009af001, 0xdff, 0xdff, 0xc0008b9c78, 0x411476, 0xc00110b500) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: io/pipe.go:134 +0x4c Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: encoding/json.(*Decoder).refill(0xc00098d760, 0xc00110b50a, 0x9) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: encoding/json/stream.go:165 +0xeb Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: encoding/json.(*Decoder).readValue(0xc00098d760, 0x0, 0x0, 0x2f7a240) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: encoding/json/stream.go:140 +0x1e8 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: encoding/json.(*Decoder).Decode(0xc00098d760, 0x272ecc0, 0xc00110b500, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: encoding/json/stream.go:63 +0x79 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient.(*Client).Stats(0xc0002f4480, 0xc000716080, 0x40, 0xc000bb1f20, 0x1, 0xc0009adce0, 0x0, 0x0, 0x3814440, 0xc000d2d080, ...) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient/container.go:1197 +0x4c4 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/drivers/docker.(*taskHandle).collectStats(0xc0007de360, 0x3814440, 0xc000d2d080, 0xc0008dc880, 0x3b9aca00) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/drivers/docker/stats.go:123 +0x227 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/drivers/docker.(*taskHandle).Stats Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/drivers/docker/stats.go:86 +0x1a6 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 433 [select]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/allocrunner/taskrunner.(*statsHook).collectResourceUsageStats(0xc000a19400, 0x3814440, 0xc000c17a40, 0x37b5440, 0xc000c17a00) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/allocrunner/taskrunner/stats_hook.go:98 +0xf0 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/client/allocrunner/taskrunner.(*statsHook).Poststart Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/allocrunner/taskrunner/stats_hook.go:64 +0xf8 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 454 [select, 25 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http.(*persistConn).writeLoop(0xc000d219e0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/transport.go:2277 +0x11c Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by net/http.(*Transport).dialConn Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/transport.go:1648 +0xc7b Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 466 [select, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/drivers/docker.(*Driver).handleWait(0xc00031bc20, 0x3814480, 0xc0000d2010, 0xc000c52660, 0xc000378a20) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/drivers/docker/driver.go:1256 +0xff Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/drivers/docker.(*Driver).WaitTask Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/drivers/docker/driver.go:1250 +0xc4 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 468 [select]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/drivers/docker.dockerStatsCollector(0xc0007f0780, 0xc000c526c0, 0x3b9aca00) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/drivers/docker/stats.go:152 +0xeb Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/drivers/docker.(*taskHandle).collectStats Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/drivers/docker/stats.go:112 +0x133 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 469 [IO wait]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll.runtime_pollWait(0x7f2c5304ac28, 0x72, 0xffffffffffffffff) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: runtime/netpoll.go:203 +0x55 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll.(*pollDesc).wait(0xc000180698, 0x72, 0x1000, 0x1000, 0xffffffffffffffff) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll/fd_poll_runtime.go:87 +0x45 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll.(*pollDesc).waitRead(...) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll/fd_poll_runtime.go:92 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll.(*FD).Read(0xc000180680, 0xc000d99000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll/fd_unix.go:169 +0x19b Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net.(*netFD).Read(0xc000180680, 0xc000d99000, 0x1000, 0x1000, 0xb6a, 0x0, 0xc000a21658) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/fd_unix.go:202 +0x4f Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net.(*conn).Read(0xc0000d48a0, 0xc000d99000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/net.go:184 +0x8e Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: bufio.(*Reader).fill(0xc000ce1c20) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: bufio/bufio.go:100 +0x103 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: bufio.(*Reader).ReadSlice(0xc000ce1c20, 0xc00006080a, 0xc000a21780, 0x40ab2b, 0xc0009c23c0, 0xc000c527d8, 0x170e) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: bufio/bufio.go:359 +0x3d Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/internal.readChunkLine(0xc000ce1c20, 0xc000e00300, 0xc000e00388, 0xc0009c2418, 0xc000e00438, 0xc000c527d8) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/internal/chunked.go:122 +0x34 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/internal.(*chunkedReader).beginChunk(0xc000cea9c0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/internal/chunked.go:48 +0x32 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/internal.(*chunkedReader).Read(0xc000cea9c0, 0xc000e0a000, 0x8000, 0x8000, 0x2, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/internal/chunked.go:93 +0x132 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http.(*body).readLocked(0xc000d2d180, 0xc000e0a000, 0x8000, 0x8000, 0xc000ce1440, 0xc000e0a000, 0xb63) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/transfer.go:847 +0x5f Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http.(*body).Read(0xc000d2d180, 0xc000e0a000, 0x8000, 0x8000, 0x0, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/transfer.go:839 +0xf2 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: io.copyBuffer(0x37bb6c0, 0xc0000d4130, 0x7f2c51eabd18, 0xc000d2d180, 0xc000e0a000, 0x8000, 0x8000, 0x0, 0xc000a219c8, 0x40a6ce) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: io/io.go:405 +0x122 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: io.Copy(...) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: io/io.go:364 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient.handleStreamResponse(0xc0008226c0, 0xc000a21d80, 0xc0008226c0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient/client.go:653 +0x383 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient.(*Client).streamURL(0xc0002f4480, 0x3093fb0, 0x3, 0xc000c52c60, 0x5e, 0x10100, 0x0, 0x0, 0x0, 0x37bb6c0, ...) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient/client.go:630 +0x872 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient.(*Client).stream(0xc0002f4480, 0x3093fb0, 0x3, 0xc000c52c00, 0x5e, 0x10100, 0x0, 0x0, 0x0, 0x37bb6c0, ...) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient/client.go:531 +0xfa Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient.(*Client).Stats.func2(0xc000ce13e0, 0xc0002f4480, 0xc000836e80, 0x40, 0xc000c526c0, 0x1, 0xc00094c4e0, 0x0, 0x0, 0x3814440, ...) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient/container.go:1159 +0x221 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient.(*Client).Stats Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient/container.go:1157 +0x2f4 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 470 [select, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient.(*Client).Stats.func3(0xc000836e80, 0x40, 0xc000c526c0, 0x1, 0xc00094c4e0, 0x0, 0x0, 0x3814440, 0xc000c17200, 0xc0000d4120, ...) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient/container.go:1186 +0xa0 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient.(*Client).Stats Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient/container.go:1184 +0x399 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 471 [select]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/allocrunner/taskrunner.(*statsHook).collectResourceUsageStats(0xc000a19200, 0x3814440, 0xc000d2d080, 0x37b5440, 0xc000d2d040) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/allocrunner/taskrunner/stats_hook.go:98 +0xf0 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/client/allocrunner/taskrunner.(*statsHook).Poststart Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/allocrunner/taskrunner/stats_hook.go:64 +0xf8 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 472 [select, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/drivers/docker.(*Driver).handleWait(0xc00031bc20, 0x3814480, 0xc0000d2010, 0xc000c52900, 0xc0007de360) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/drivers/docker/driver.go:1256 +0xff Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/drivers/docker.(*Driver).WaitTask Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/drivers/docker/driver.go:1250 +0xc4 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 473 [select]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: io.(*pipe).Read(0xc000ce1980, 0xc000a54001, 0xdff, 0xdff, 0x40a425, 0x203000, 0x203000) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: io/pipe.go:57 +0xe7 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: io.(*PipeReader).Read(0xc0000d47f0, 0xc000a54001, 0xdff, 0xdff, 0xc000b8dc78, 0x411476, 0xc000f68380) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: io/pipe.go:134 +0x4c Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: encoding/json.(*Decoder).refill(0xc0006026e0, 0xc000f6830a, 0x9) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: encoding/json/stream.go:165 +0xeb Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: encoding/json.(*Decoder).readValue(0xc0006026e0, 0x0, 0x0, 0x2f7a240) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: encoding/json/stream.go:140 +0x1e8 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: encoding/json.(*Decoder).Decode(0xc0006026e0, 0x272ecc0, 0xc000f68380, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: encoding/json/stream.go:63 +0x79 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient.(*Client).Stats(0xc0002f4480, 0xc0005e73c0, 0x40, 0xc000c529c0, 0x1, 0xc000614660, 0x0, 0x0, 0x3814440, 0xc000c17a40, ...) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient/container.go:1197 +0x4c4 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/drivers/docker.(*taskHandle).collectStats(0xc000378a20, 0x3814440, 0xc000c17a40, 0xc0007f1100, 0x3b9aca00) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/drivers/docker/stats.go:123 +0x227 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/drivers/docker.(*taskHandle).Stats Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/drivers/docker/stats.go:86 +0x1a6 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 474 [select]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/drivers/docker.dockerStatsCollector(0xc0007f1100, 0xc000c529c0, 0x3b9aca00) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/drivers/docker/stats.go:152 +0xeb Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/drivers/docker.(*taskHandle).collectStats Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/drivers/docker/stats.go:112 +0x133 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 475 [IO wait]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll.runtime_pollWait(0x7f2c5304e658, 0x72, 0xffffffffffffffff) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: runtime/netpoll.go:203 +0x55 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll.(*pollDesc).wait(0xc00098e098, 0x72, 0x1000, 0x1000, 0xffffffffffffffff) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll/fd_poll_runtime.go:87 +0x45 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll.(*pollDesc).waitRead(...) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll/fd_poll_runtime.go:92 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll.(*FD).Read(0xc00098e080, 0xc0002ad000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll/fd_unix.go:169 +0x19b Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net.(*netFD).Read(0xc00098e080, 0xc0002ad000, 0x1000, 0x1000, 0xc49, 0x0, 0xc000507658) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/fd_unix.go:202 +0x4f Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net.(*conn).Read(0xc000d386b8, 0xc0002ad000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/net.go:184 +0x8e Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: bufio.(*Reader).fill(0xc0009e2300) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: bufio/bufio.go:100 +0x103 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: bufio.(*Reader).ReadSlice(0xc0009e2300, 0xc00005e00a, 0xc000507780, 0x40ab2b, 0xc0008a6cc0, 0xc000c52ad8, 0x170e) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: bufio/bufio.go:359 +0x3d Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/internal.readChunkLine(0xc0009e2300, 0xc000e00c00, 0xc000e00c88, 0xc0008a6d18, 0xc000e00d38, 0xc000c52ad8) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/internal/chunked.go:122 +0x34 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/internal.(*chunkedReader).beginChunk(0xc0009a6ea0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/internal/chunked.go:48 +0x32 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/internal.(*chunkedReader).Read(0xc0009a6ea0, 0xc000dbc000, 0x8000, 0x8000, 0x2, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/internal/chunked.go:93 +0x132 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http.(*body).readLocked(0xc000a18e80, 0xc000dbc000, 0x8000, 0x8000, 0xc000ce1980, 0xc000dbc000, 0xc42) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/transfer.go:847 +0x5f Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http.(*body).Read(0xc000a18e80, 0xc000dbc000, 0x8000, 0x8000, 0x0, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/transfer.go:839 +0xf2 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: io.copyBuffer(0x37bb6c0, 0xc0000d4890, 0x7f2c51eabd18, 0xc000a18e80, 0xc000dbc000, 0x8000, 0x8000, 0x0, 0xc0005079c8, 0x40a6ce) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: io/io.go:405 +0x122 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: io.Copy(...) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: io/io.go:364 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient.handleStreamResponse(0xc0008714d0, 0xc000507d80, 0xc0008714d0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient/client.go:653 +0x383 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient.(*Client).streamURL(0xc0002f4480, 0x3093fb0, 0x3, 0xc0009ac720, 0x5e, 0x10100, 0x0, 0x0, 0x0, 0x37bb6c0, ...) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient/client.go:630 +0x872 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient.(*Client).stream(0xc0002f4480, 0x3093fb0, 0x3, 0xc0009ac5a0, 0x5e, 0x10100, 0x0, 0x0, 0x0, 0x37bb6c0, ...) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient/client.go:531 +0xfa Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient.(*Client).Stats.func2(0xc000ce1920, 0xc0002f4480, 0xc0005e73c0, 0x40, 0xc000c529c0, 0x1, 0xc000614660, 0x0, 0x0, 0x3814440, ...) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient/container.go:1159 +0x221 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient.(*Client).Stats Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient/container.go:1157 +0x2f4 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 476 [select, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient.(*Client).Stats.func3(0xc0005e73c0, 0x40, 0xc000c529c0, 0x1, 0xc000614660, 0x0, 0x0, 0x3814440, 0xc000c17a40, 0xc0000d47f0, ...) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient/container.go:1186 +0xa0 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient.(*Client).Stats Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient/container.go:1184 +0x399 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 477 [chan receive, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient.(*Client).streamURL.func1(0x3814440, 0xc000d2d100, 0x3838ec0, 0xc0000d48a0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient/client.go:580 +0x48 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient.(*Client).streamURL Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient/client.go:579 +0x520 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 457 [select]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/allocrunner/taskrunner.(*statsHook).collectResourceUsageStats(0xc000a18340, 0x3814440, 0xc000a18880, 0x37b5440, 0xc000a18780) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/allocrunner/taskrunner/stats_hook.go:98 +0xf0 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/client/allocrunner/taskrunner.(*statsHook).Poststart Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/allocrunner/taskrunner/stats_hook.go:64 +0xf8 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 458 [select, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/drivers/docker.(*Driver).handleWait(0xc00031bc20, 0x3814480, 0xc0000d2010, 0xc000bb1860, 0xc000870f30) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/drivers/docker/driver.go:1256 +0xff Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/drivers/docker.(*Driver).WaitTask Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/drivers/docker/driver.go:1250 +0xc4 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 459 [select]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: io.(*pipe).Read(0xc0009c7800, 0xc0009ae001, 0xdff, 0xdff, 0x40a425, 0x203000, 0x203000) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: io/pipe.go:57 +0xe7 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: io.(*PipeReader).Read(0xc000d38658, 0xc0009ae001, 0xdff, 0xdff, 0xc0008bdc78, 0x411476, 0xc00110b180) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: io/pipe.go:134 +0x4c Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: encoding/json.(*Decoder).refill(0xc00098d080, 0xc00110b10a, 0x9) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: encoding/json/stream.go:165 +0xeb Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: encoding/json.(*Decoder).readValue(0xc00098d080, 0x0, 0x0, 0x2f7a240) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: encoding/json/stream.go:140 +0x1e8 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: encoding/json.(*Decoder).Decode(0xc00098d080, 0x272ecc0, 0xc00110b180, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: encoding/json/stream.go:63 +0x79 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient.(*Client).Stats(0xc0002f4480, 0xc0009ec000, 0x40, 0xc000bb1920, 0x1, 0xc0009ac600, 0x0, 0x0, 0x3814440, 0xc000a18880, ...) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient/container.go:1197 +0x4c4 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/drivers/docker.(*taskHandle).collectStats(0xc000870f30, 0x3814440, 0xc000a18880, 0xc0008dc320, 0x3b9aca00) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/drivers/docker/stats.go:123 +0x227 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/drivers/docker.(*taskHandle).Stats Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/drivers/docker/stats.go:86 +0x1a6 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 460 [select]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/drivers/docker.dockerStatsCollector(0xc0008dc320, 0xc000bb1920, 0x3b9aca00) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/drivers/docker/stats.go:152 +0xeb Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/drivers/docker.(*taskHandle).collectStats Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/drivers/docker/stats.go:112 +0x133 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 461 [IO wait]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll.runtime_pollWait(0x7f2c5304ab48, 0x72, 0xffffffffffffffff) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: runtime/netpoll.go:203 +0x55 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll.(*pollDesc).wait(0xc000bab798, 0x72, 0x1000, 0x1000, 0xffffffffffffffff) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll/fd_poll_runtime.go:87 +0x45 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll.(*pollDesc).waitRead(...) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll/fd_poll_runtime.go:92 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll.(*FD).Read(0xc000bab780, 0xc000995000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll/fd_unix.go:169 +0x19b Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net.(*netFD).Read(0xc000bab780, 0xc000995000, 0x1000, 0x1000, 0xb5d, 0x0, 0xc000803658) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/fd_unix.go:202 +0x4f Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net.(*conn).Read(0xc000d38668, 0xc000995000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/net.go:184 +0x8e Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: bufio.(*Reader).fill(0xc0009c7aa0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: bufio/bufio.go:100 +0x103 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: bufio.(*Reader).ReadSlice(0xc0009c7aa0, 0xc00006080a, 0xc000803780, 0x40ab2b, 0xc0009c23c0, 0xc000bb1a38, 0x170e) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: bufio/bufio.go:359 +0x3d Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/internal.readChunkLine(0xc0009c7aa0, 0xc000d97200, 0xc000d97288, 0xc0009c2418, 0xc000d97338, 0xc000bb1a38) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/internal/chunked.go:122 +0x34 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/internal.(*chunkedReader).beginChunk(0xc0009a6840) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/internal/chunked.go:48 +0x32 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/internal.(*chunkedReader).Read(0xc0009a6840, 0xc000db4000, 0x8000, 0x8000, 0x2, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/internal/chunked.go:93 +0x132 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http.(*body).readLocked(0xc000a18ac0, 0xc000db4000, 0x8000, 0x8000, 0xc0009c7800, 0xc000db4000, 0xb56) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/transfer.go:847 +0x5f Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http.(*body).Read(0xc000a18ac0, 0xc000db4000, 0x8000, 0x8000, 0x0, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/transfer.go:839 +0xf2 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: io.copyBuffer(0x37bb6c0, 0xc000d38660, 0x7f2c51eabd18, 0xc000a18ac0, 0xc000db4000, 0x8000, 0x8000, 0x0, 0xc0008039c8, 0x40a6ce) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: io/io.go:405 +0x122 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: io.Copy(...) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: io/io.go:364 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient.handleStreamResponse(0xc000870ea0, 0xc000803d80, 0xc000870ea0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient/client.go:653 +0x383 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient.(*Client).streamURL(0xc0002f4480, 0x3093fb0, 0x3, 0xc000bb1bc0, 0x5e, 0x10100, 0x0, 0x0, 0x0, 0x37bb6c0, ...) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient/client.go:630 +0x872 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient.(*Client).stream(0xc0002f4480, 0x3093fb0, 0x3, 0xc000bb1b60, 0x5e, 0x10100, 0x0, 0x0, 0x0, 0x37bb6c0, ...) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient/client.go:531 +0xfa Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient.(*Client).Stats.func2(0xc0009c77a0, 0xc0002f4480, 0xc0009ec000, 0x40, 0xc000bb1920, 0x1, 0xc0009ac600, 0x0, 0x0, 0x3814440, ...) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient/container.go:1159 +0x221 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient.(*Client).Stats Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient/container.go:1157 +0x2f4 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 462 [select, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient.(*Client).Stats.func3(0xc0009ec000, 0x40, 0xc000bb1920, 0x1, 0xc0009ac600, 0x0, 0x0, 0x3814440, 0xc000a18880, 0xc000d38658, ...) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient/container.go:1186 +0xa0 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient.(*Client).Stats Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient/container.go:1184 +0x399 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 463 [chan receive, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient.(*Client).streamURL.func1(0x3814440, 0xc000a188c0, 0x3838ec0, 0xc000d38668) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient/client.go:580 +0x48 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient.(*Client).streamURL Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient/client.go:579 +0x520 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 483 [IO wait]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll.runtime_pollWait(0x7f2c5304ad08, 0x72, 0xffffffffffffffff) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: runtime/netpoll.go:203 +0x55 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll.(*pollDesc).wait(0xc000babd98, 0x72, 0x1000, 0x1000, 0xffffffffffffffff) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll/fd_poll_runtime.go:87 +0x45 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll.(*pollDesc).waitRead(...) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll/fd_poll_runtime.go:92 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll.(*FD).Read(0xc000babd80, 0xc0005cf000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll/fd_unix.go:169 +0x19b Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net.(*netFD).Read(0xc000babd80, 0xc0005cf000, 0x1000, 0x1000, 0xb84, 0x0, 0xc0008bf658) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/fd_unix.go:202 +0x4f Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net.(*conn).Read(0xc000d386b0, 0xc0005cf000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/net.go:184 +0x8e Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: bufio.(*Reader).fill(0xc0009e21e0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: bufio/bufio.go:100 +0x103 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: bufio.(*Reader).ReadSlice(0xc0009e21e0, 0xc00006080a, 0xc0008bf780, 0x40ab2b, 0xc0009c23c0, 0xc0009ac2f8, 0x170e) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: bufio/bufio.go:359 +0x3d Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/internal.readChunkLine(0xc0009e21e0, 0xc000d97680, 0xc000d97708, 0xc0009c2418, 0xc000d977b8, 0xc0009ac2f8) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/internal/chunked.go:122 +0x34 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/internal.(*chunkedReader).beginChunk(0xc0009a6f60) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/internal/chunked.go:48 +0x32 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/internal.(*chunkedReader).Read(0xc0009a6f60, 0xc000dc4000, 0x8000, 0x8000, 0x2, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/internal/chunked.go:93 +0x132 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http.(*body).readLocked(0xc000a18ec0, 0xc000dc4000, 0x8000, 0x8000, 0xc0009e2000, 0xc000dc4000, 0xb7d) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/transfer.go:847 +0x5f Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http.(*body).Read(0xc000a18ec0, 0xc000dc4000, 0x8000, 0x8000, 0x0, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/transfer.go:839 +0xf2 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: io.copyBuffer(0x37bb6c0, 0xc000d386a8, 0x7f2c51eabd18, 0xc000a18ec0, 0xc000dc4000, 0x8000, 0x8000, 0x0, 0xc0008bf9c8, 0x40a6ce) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: io/io.go:405 +0x122 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: io.Copy(...) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: io/io.go:364 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient.handleStreamResponse(0xc0008713b0, 0xc0008bfd80, 0xc0008713b0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient/client.go:653 +0x383 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient.(*Client).streamURL(0xc0002f4480, 0x3093fb0, 0x3, 0xc0009ac480, 0x5e, 0x10100, 0x0, 0x0, 0x0, 0x37bb6c0, ...) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient/client.go:630 +0x872 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient.(*Client).stream(0xc0002f4480, 0x3093fb0, 0x3, 0xc0009ac420, 0x5e, 0x10100, 0x0, 0x0, 0x0, 0x37bb6c0, ...) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient/client.go:531 +0xfa Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient.(*Client).Stats.func2(0xc0009c7f80, 0xc0002f4480, 0xc000716080, 0x40, 0xc000bb1f20, 0x1, 0xc0009adce0, 0x0, 0x0, 0x3814440, ...) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient/container.go:1159 +0x221 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient.(*Client).Stats Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient/container.go:1157 +0x2f4 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 484 [select, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient.(*Client).Stats.func3(0xc000716080, 0x40, 0xc000bb1f20, 0x1, 0xc0009adce0, 0x0, 0x0, 0x3814440, 0xc000d2d080, 0xc000d386a0, ...) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient/container.go:1186 +0xa0 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient.(*Client).Stats Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient/container.go:1184 +0x399 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 485 [chan receive, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient.(*Client).streamURL.func1(0x3814440, 0xc000a18b00, 0x3838ec0, 0xc000d386b0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient/client.go:580 +0x48 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient.(*Client).streamURL Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient/client.go:579 +0x520 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 486 [chan receive, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient.(*Client).streamURL.func1(0x3814440, 0xc000a18e00, 0x3838ec0, 0xc000d386b8) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient/client.go:580 +0x48 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient.(*Client).streamURL Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient/client.go:579 +0x520 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 478 [select]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/allocrunner/taskrunner.(*statsHook).collectResourceUsageStats(0xc000984c40, 0x3814440, 0xc000d2d2c0, 0x37b5440, 0xc000d2d280) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/allocrunner/taskrunner/stats_hook.go:98 +0xf0 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/client/allocrunner/taskrunner.(*statsHook).Poststart Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/allocrunner/taskrunner/stats_hook.go:64 +0xf8 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 479 [select, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/drivers/docker.(*Driver).handleWait(0xc00031bc20, 0x3814480, 0xc0000d2010, 0xc000c52de0, 0xc000378360) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/drivers/docker/driver.go:1256 +0xff Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/drivers/docker.(*Driver).WaitTask Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/drivers/docker/driver.go:1250 +0xc4 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 480 [select]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: io.(*pipe).Read(0xc000a58660, 0xc000a55001, 0xdff, 0xdff, 0x40a425, 0x203000, 0x203000) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: io/pipe.go:57 +0xe7 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: io.(*PipeReader).Read(0xc0000d49a0, 0xc000a55001, 0xdff, 0xdff, 0xc000ea7c78, 0x411476, 0xc000f68700) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: io/pipe.go:134 +0x4c Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: encoding/json.(*Decoder).refill(0xc000a422c0, 0xc000f6870a, 0x9) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: encoding/json/stream.go:165 +0xeb Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: encoding/json.(*Decoder).readValue(0xc000a422c0, 0x0, 0x0, 0x2f7a240) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: encoding/json/stream.go:140 +0x1e8 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: encoding/json.(*Decoder).Decode(0xc000a422c0, 0x272ecc0, 0xc000f68700, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: encoding/json/stream.go:63 +0x79 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient.(*Client).Stats(0xc0002f4480, 0xc0005e6300, 0x40, 0xc000c52ea0, 0x1, 0xc0000aa780, 0x0, 0x0, 0x3814440, 0xc000d2d2c0, ...) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient/container.go:1197 +0x4c4 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/drivers/docker.(*taskHandle).collectStats(0xc000378360, 0x3814440, 0xc000d2d2c0, 0xc0007f1600, 0x3b9aca00) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/drivers/docker/stats.go:123 +0x227 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/drivers/docker.(*taskHandle).Stats Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/drivers/docker/stats.go:86 +0x1a6 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 481 [select]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/drivers/docker.dockerStatsCollector(0xc0007f1600, 0xc000c52ea0, 0x3b9aca00) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/drivers/docker/stats.go:152 +0xeb Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/drivers/docker.(*taskHandle).collectStats Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/drivers/docker/stats.go:112 +0x133 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 498 [IO wait]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll.runtime_pollWait(0x7f2c5304e578, 0x72, 0xffffffffffffffff) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: runtime/netpoll.go:203 +0x55 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll.(*pollDesc).wait(0xc000181c18, 0x72, 0x1000, 0x1000, 0xffffffffffffffff) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll/fd_poll_runtime.go:87 +0x45 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll.(*pollDesc).waitRead(...) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll/fd_poll_runtime.go:92 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll.(*FD).Read(0xc000181c00, 0xc000c28000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll/fd_unix.go:169 +0x19b Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net.(*netFD).Read(0xc000181c00, 0xc000c28000, 0x1000, 0x1000, 0xc22, 0x0, 0xc000b83658) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/fd_unix.go:202 +0x4f Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net.(*conn).Read(0xc0000d49c0, 0xc000c28000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/net.go:184 +0x8e Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: bufio.(*Reader).fill(0xc000a58c00) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: bufio/bufio.go:100 +0x103 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: bufio.(*Reader).ReadSlice(0xc000a58c00, 0xc00005e00a, 0xc000b83780, 0x40ab2b, 0xc0008a6cc0, 0xc000c52fb8, 0x170e) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: bufio/bufio.go:359 +0x3d Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/internal.readChunkLine(0xc000a58c00, 0xc000e01680, 0xc000e01708, 0xc0008a6d18, 0xc000e017b8, 0xc000c52fb8) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/internal/chunked.go:122 +0x34 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/internal.(*chunkedReader).beginChunk(0xc000ceb020) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/internal/chunked.go:48 +0x32 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/internal.(*chunkedReader).Read(0xc000ceb020, 0xc000e16000, 0x8000, 0x8000, 0x2, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/internal/chunked.go:93 +0x132 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http.(*body).readLocked(0xc000d2d3c0, 0xc000e16000, 0x8000, 0x8000, 0xc000a58660, 0xc000e16000, 0xc1b) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/transfer.go:847 +0x5f Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http.(*body).Read(0xc000d2d3c0, 0xc000e16000, 0x8000, 0x8000, 0x0, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/transfer.go:839 +0xf2 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: io.copyBuffer(0x37bb6c0, 0xc0000d49b0, 0x7f2c51eabd18, 0xc000d2d3c0, 0xc000e16000, 0x8000, 0x8000, 0x0, 0xc000b839c8, 0x40a6ce) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: io/io.go:405 +0x122 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: io.Copy(...) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: io/io.go:364 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient.handleStreamResponse(0xc000822b40, 0xc000b83d80, 0xc000822b40, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient/client.go:653 +0x383 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient.(*Client).streamURL(0xc0002f4480, 0x3093fb0, 0x3, 0xc000c53140, 0x5e, 0x10100, 0x0, 0x0, 0x0, 0x37bb6c0, ...) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient/client.go:630 +0x872 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient.(*Client).stream(0xc0002f4480, 0x3093fb0, 0x3, 0xc000c530e0, 0x5e, 0x10100, 0x0, 0x0, 0x0, 0x37bb6c0, ...) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient/client.go:531 +0xfa Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient.(*Client).Stats.func2(0xc000a58600, 0xc0002f4480, 0xc0005e6300, 0x40, 0xc000c52ea0, 0x1, 0xc0000aa780, 0x0, 0x0, 0x3814440, ...) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient/container.go:1159 +0x221 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient.(*Client).Stats Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient/container.go:1157 +0x2f4 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 499 [select, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient.(*Client).Stats.func3(0xc0005e6300, 0x40, 0xc000c52ea0, 0x1, 0xc0000aa780, 0x0, 0x0, 0x3814440, 0xc000d2d2c0, 0xc0000d49a0, ...) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient/container.go:1186 +0xa0 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient.(*Client).Stats Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient/container.go:1184 +0x399 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 500 [chan receive, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient.(*Client).streamURL.func1(0x3814440, 0xc000d2d340, 0x3838ec0, 0xc0000d49c0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient/client.go:580 +0x48 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient.(*Client).streamURL Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient/client.go:579 +0x520 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 490 [select, 27 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/allocrunner/taskrunner/template.(*TaskTemplateManager).handleTemplateRerenders(0xc000a19500, 0xbfb0047ab5fbdbff, 0x180c78920, 0x57d89a0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/allocrunner/taskrunner/template/template.go:352 +0x193 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/allocrunner/taskrunner/template.(*TaskTemplateManager).run(0xc000a19500) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/allocrunner/taskrunner/template/template.go:227 +0x2fc Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/client/allocrunner/taskrunner/template.NewTaskTemplateManager Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/allocrunner/taskrunner/template/template.go:163 +0x2db Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 491 [select, 27 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul-template/manager.(*Runner).Start(0xc00097f200) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul-template/manager/runner.go:330 +0x410 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/client/allocrunner/taskrunner/template.(*TaskTemplateManager).run Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/allocrunner/taskrunner/template/template.go:195 +0x53 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 71267 [sleep, 27 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: time.Sleep(0x400e87c271) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: runtime/time.go:188 +0xba Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul-template/dependency.(*VaultReadQuery).Fetch(0xc0009dc190, 0xc00096b6b0, 0xc00126ae10, 0x1, 0x1, 0xc000c9f320, 0xc00093ecc0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul-template/dependency/vault_read.go:65 +0x469 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul-template/watch.(*View).fetch(0xc0005ee930, 0xc000f4ad20, 0xc000f4ad80, 0xc0008eb2c0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul-template/watch/view.go:202 +0x164 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul-template/watch.(*View).poll Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul-template/watch/view.go:115 +0x445 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 501 [select, 27 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul-template/watch.(*View).poll(0xc0005ee930, 0xc0009e2c60, 0xc0008f23c0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul-template/watch/view.go:118 +0x15f Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul-template/watch.(*Watcher).Add Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul-template/watch/watcher.go:165 +0x409 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 507 [select]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/allocrunner/taskrunner.(*statsHook).collectResourceUsageStats(0xc000539980, 0x3814440, 0xc000515480, 0x37b5440, 0xc000515400) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/allocrunner/taskrunner/stats_hook.go:98 +0xf0 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/client/allocrunner/taskrunner.(*statsHook).Poststart Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/allocrunner/taskrunner/stats_hook.go:64 +0xf8 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 508 [select]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/allocrunner/taskrunner.(*tasklet).run.func1(0xc0001acfc0, 0x3814440, 0xc000515680, 0xc0004187b0, 0xc0006c0340) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/allocrunner/taskrunner/tasklet.go:112 +0x17d Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/client/allocrunner/taskrunner.(*tasklet).run Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/allocrunner/taskrunner/tasklet.go:105 +0x122 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 506 [select, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/LK4D4/joincontext.(*joinContext).run(0xc0000d8200) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/LK4D4/joincontext/context.go:73 +0x10b Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/github.com/LK4D4/joincontext.Join Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/LK4D4/joincontext/context.go:33 +0xc2 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 516 [select]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: io.(*pipe).Read(0xc0009ceea0, 0xc000c12001, 0xdff, 0xdff, 0x40a425, 0x203000, 0x203000) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: io/pipe.go:57 +0xe7 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: io.(*PipeReader).Read(0xc000d388f0, 0xc000c12001, 0xdff, 0xdff, 0xc000ec3c78, 0x411476, 0xc00110b880) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: io/pipe.go:134 +0x4c Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: encoding/json.(*Decoder).refill(0xc000dff340, 0xc00110b80a, 0x9) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: encoding/json/stream.go:165 +0xeb Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: encoding/json.(*Decoder).readValue(0xc000dff340, 0x0, 0x0, 0x2f7a240) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: encoding/json/stream.go:140 +0x1e8 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: encoding/json.(*Decoder).Decode(0xc000dff340, 0x272ecc0, 0xc00110b880, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: encoding/json/stream.go:63 +0x79 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient.(*Client).Stats(0xc0002f4480, 0xc000836a80, 0x40, 0xc00063a3c0, 0x1, 0xc00085ee40, 0x0, 0x0, 0x3814440, 0xc000515480, ...) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient/container.go:1197 +0x4c4 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/drivers/docker.(*taskHandle).collectStats(0xc0008702d0, 0x3814440, 0xc000515480, 0xc00099b780, 0x3b9aca00) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/drivers/docker/stats.go:123 +0x227 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/drivers/docker.(*taskHandle).Stats Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/drivers/docker/stats.go:86 +0x1a6 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 517 [select]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/drivers/docker.dockerStatsCollector(0xc00099b780, 0xc00063a3c0, 0x3b9aca00) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/drivers/docker/stats.go:152 +0xeb Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/drivers/docker.(*taskHandle).collectStats Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/drivers/docker/stats.go:112 +0x133 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 518 [IO wait]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll.runtime_pollWait(0x7f2c5304e3b8, 0x72, 0xffffffffffffffff) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: runtime/netpoll.go:203 +0x55 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll.(*pollDesc).wait(0xc00098f698, 0x72, 0x1000, 0x1000, 0xffffffffffffffff) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll/fd_poll_runtime.go:87 +0x45 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll.(*pollDesc).waitRead(...) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll/fd_poll_runtime.go:92 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll.(*FD).Read(0xc00098f680, 0xc000380000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll/fd_unix.go:169 +0x19b Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net.(*netFD).Read(0xc00098f680, 0xc000380000, 0x1000, 0x1000, 0xb9a, 0x0, 0xc000a1b658) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/fd_unix.go:202 +0x4f Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net.(*conn).Read(0xc000d38900, 0xc000380000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/net.go:184 +0x8e Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: bufio.(*Reader).fill(0xc0009cf140) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: bufio/bufio.go:100 +0x103 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: bufio.(*Reader).ReadSlice(0xc0009cf140, 0xc00006080a, 0xc000a1b780, 0x40ab2b, 0xc0009c23c0, 0xc00063a4d8, 0x170e) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: bufio/bufio.go:359 +0x3d Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/internal.readChunkLine(0xc0009cf140, 0xc0008e3680, 0xc0008e3708, 0xc0009c2418, 0xc0008e37b8, 0xc00063a4d8) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/internal/chunked.go:122 +0x34 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/internal.(*chunkedReader).beginChunk(0xc0003f0480) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/internal/chunked.go:48 +0x32 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/internal.(*chunkedReader).Read(0xc0003f0480, 0xc000384000, 0x8000, 0x8000, 0x2, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/internal/chunked.go:93 +0x132 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http.(*body).readLocked(0xc00004ec00, 0xc000384000, 0x8000, 0x8000, 0xc0009ceea0, 0xc000384000, 0xb93) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/transfer.go:847 +0x5f Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http.(*body).Read(0xc00004ec00, 0xc000384000, 0x8000, 0x8000, 0x0, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/transfer.go:839 +0xf2 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: io.copyBuffer(0x37bb6c0, 0xc000d388f8, 0x7f2c51eabd18, 0xc00004ec00, 0xc000384000, 0x8000, 0x8000, 0x0, 0xc000a1b9c8, 0x40a6ce) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: io/io.go:405 +0x122 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: io.Copy(...) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: io/io.go:364 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient.handleStreamResponse(0xc0007df5f0, 0xc000a1bd80, 0xc0007df5f0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient/client.go:653 +0x383 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient.(*Client).streamURL(0xc0002f4480, 0x3093fb0, 0x3, 0xc00063aa20, 0x5e, 0x10100, 0x0, 0x0, 0x0, 0x37bb6c0, ...) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient/client.go:630 +0x872 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient.(*Client).stream(0xc0002f4480, 0x3093fb0, 0x3, 0xc00063a7e0, 0x5e, 0x10100, 0x0, 0x0, 0x0, 0x37bb6c0, ...) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient/client.go:531 +0xfa Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient.(*Client).Stats.func2(0xc0009cee40, 0xc0002f4480, 0xc000836a80, 0x40, 0xc00063a3c0, 0x1, 0xc00085ee40, 0x0, 0x0, 0x3814440, ...) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient/container.go:1159 +0x221 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient.(*Client).Stats Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient/container.go:1157 +0x2f4 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 519 [select, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient.(*Client).Stats.func3(0xc000836a80, 0x40, 0xc00063a3c0, 0x1, 0xc00085ee40, 0x0, 0x0, 0x3814440, 0xc000515480, 0xc000d388f0, ...) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient/container.go:1186 +0xa0 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient.(*Client).Stats Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient/container.go:1184 +0x399 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 509 [select]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/allocrunner/taskrunner.(*tasklet).run.func1(0xc0001ad080, 0x3814440, 0xc000515740, 0xc0004188f0, 0xc0006c0360) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/allocrunner/taskrunner/tasklet.go:112 +0x17d Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/client/allocrunner/taskrunner.(*tasklet).run Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/allocrunner/taskrunner/tasklet.go:105 +0x122 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 510 [select, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/drivers/docker.(*Driver).handleWait(0xc00031bc20, 0x3814480, 0xc0000d2010, 0xc0001ad620, 0xc0008702d0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/drivers/docker/driver.go:1256 +0xff Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/drivers/docker.(*Driver).WaitTask Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/drivers/docker/driver.go:1250 +0xc4 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 520 [chan receive, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient.(*Client).streamURL.func1(0x3814440, 0xc00004e680, 0x3838ec0, 0xc000d38900) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient/client.go:580 +0x48 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient.(*Client).streamURL Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient/client.go:579 +0x520 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 69911 [runnable]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: runtime.resettimer(...) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: runtime/time.go:488 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: time.resetTimer(0xc00126a4b8, 0x17091b72f0c9ec) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: runtime/time.go:223 +0x62 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: time.(*Timer).Reset(0xc00126a4b0, 0x2540be400, 0xc00126a4b0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: time/sleep.go:126 +0x7d Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).waitForSendErr(0xc000533ad0, 0xc001408580, 0xc, 0xc, 0x0, 0x0, 0xc0008ca660, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:341 +0xa7 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).waitForSend(...) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:332 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Ping(0xc000533ad0, 0xc000bcbf58, 0x2, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:288 +0x1a1 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc000533ad0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:315 +0xe5 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:116 +0x314 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 71379 [IO wait, 24 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll.runtime_pollWait(0x7f2c51e969a8, 0x72, 0xffffffffffffffff) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: runtime/netpoll.go:203 +0x55 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll.(*pollDesc).wait(0xc001356f98, 0x72, 0x1000, 0x1000, 0xffffffffffffffff) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll/fd_poll_runtime.go:87 +0x45 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll.(*pollDesc).waitRead(...) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll/fd_poll_runtime.go:92 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll.(*FD).Read(0xc001356f80, 0xc0011f1000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll/fd_unix.go:169 +0x19b Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net.(*netFD).Read(0xc001356f80, 0xc0011f1000, 0x1000, 0x1000, 0x20a, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/fd_unix.go:202 +0x4f Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net.(*conn).Read(0xc000d384b8, 0xc0011f1000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/net.go:184 +0x8e Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: bufio.(*Reader).Read(0xc000606ba0, 0xc001250050, 0xc, 0xc, 0xc000532a00, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: bufio/bufio.go:226 +0x24f Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: io.ReadAtLeast(0x37b4d20, 0xc000606ba0, 0xc001250050, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: io/io.go:310 +0x87 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: io.ReadFull(...) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: io/io.go:329 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc0005329a0, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:458 +0xe3 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc0005329a0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:437 +0x2b Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:113 +0x2a5 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 521 [select]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: io.(*pipe).Read(0xc0008a6000, 0xc000eb8001, 0xdff, 0xdff, 0x40a425, 0x203000, 0x203000) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: io/pipe.go:57 +0xe7 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: io.(*PipeReader).Read(0xc000d38960, 0xc000eb8001, 0xdff, 0xdff, 0xc000ea9c78, 0x411476, 0xc000f68a80) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: io/pipe.go:134 +0x4c Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: encoding/json.(*Decoder).refill(0xc000dffe40, 0xc000f68a0a, 0x9) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: encoding/json/stream.go:165 +0xeb Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: encoding/json.(*Decoder).readValue(0xc000dffe40, 0x0, 0x0, 0x2f7a240) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: encoding/json/stream.go:140 +0x1e8 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: encoding/json.(*Decoder).Decode(0xc000dffe40, 0x272ecc0, 0xc000f68a80, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: encoding/json/stream.go:63 +0x79 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient.(*Client).Stats(0xc0002f4480, 0xc000837840, 0x40, 0xc00069aae0, 0x1, 0xc00094d740, 0x0, 0x0, 0x3814440, 0xc000333e40, ...) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient/container.go:1197 +0x4c4 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/drivers/docker.(*taskHandle).collectStats(0xc000870b40, 0x3814440, 0xc000333e40, 0xc00099be60, 0x3b9aca00) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/drivers/docker/stats.go:123 +0x227 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/drivers/docker.(*taskHandle).Stats Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/drivers/docker/stats.go:86 +0x1a6 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 15140 [IO wait]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll.runtime_pollWait(0x7f2c5304e498, 0x72, 0xffffffffffffffff) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: runtime/netpoll.go:203 +0x55 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll.(*pollDesc).wait(0xc00109e298, 0x72, 0x1000, 0x1000, 0xffffffffffffffff) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll/fd_poll_runtime.go:87 +0x45 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll.(*pollDesc).waitRead(...) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll/fd_poll_runtime.go:92 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll.(*FD).Read(0xc00109e280, 0xc000b25000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll/fd_unix.go:169 +0x19b Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net.(*netFD).Read(0xc00109e280, 0xc000b25000, 0x1000, 0x1000, 0xc0009833e0, 0xc001442da8, 0xc001442c38) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/fd_unix.go:202 +0x4f Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net.(*conn).Read(0xc0000d5170, 0xc000b25000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/net.go:184 +0x8e Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http.(*persistConn).Read(0xc0006f0fc0, 0xc000b25000, 0x1000, 0x1000, 0xc001442eb0, 0x469d60, 0xc001442eb0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/transport.go:1825 +0x75 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: bufio.(*Reader).fill(0xc000d91e00) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: bufio/bufio.go:100 +0x103 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: bufio.(*Reader).Peek(0xc000d91e00, 0x1, 0x2, 0x0, 0x0, 0xc000817e00, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: bufio/bufio.go:138 +0x4f Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http.(*persistConn).readLoop(0xc0006f0fc0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/transport.go:1978 +0x1a8 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by net/http.(*Transport).dialConn Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/transport.go:1647 +0xc56 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 543 [select, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/LK4D4/joincontext.(*joinContext).run(0xc000333980) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/LK4D4/joincontext/context.go:73 +0x10b Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/github.com/LK4D4/joincontext.Join Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/LK4D4/joincontext/context.go:33 +0xc2 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 544 [select]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/allocrunner/taskrunner.(*statsHook).collectResourceUsageStats(0xc000841f40, 0x3814440, 0xc000333e40, 0x37b5440, 0xc000333e00) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/allocrunner/taskrunner/stats_hook.go:98 +0xf0 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/client/allocrunner/taskrunner.(*statsHook).Poststart Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/allocrunner/taskrunner/stats_hook.go:64 +0xf8 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 522 [select]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/drivers/docker.dockerStatsCollector(0xc00099be60, 0xc00069aae0, 0x3b9aca00) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/drivers/docker/stats.go:152 +0xeb Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/drivers/docker.(*taskHandle).collectStats Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/drivers/docker/stats.go:112 +0x133 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 523 [IO wait]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll.runtime_pollWait(0x7f2c5304e118, 0x72, 0xffffffffffffffff) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: runtime/netpoll.go:203 +0x55 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll.(*pollDesc).wait(0xc00098fd98, 0x72, 0x1000, 0x1000, 0xffffffffffffffff) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll/fd_poll_runtime.go:87 +0x45 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll.(*pollDesc).waitRead(...) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll/fd_poll_runtime.go:92 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll.(*FD).Read(0xc00098fd80, 0xc000398000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll/fd_unix.go:169 +0x19b Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net.(*netFD).Read(0xc00098fd80, 0xc000398000, 0x1000, 0x1000, 0xb64, 0x0, 0xc0002c7658) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/fd_unix.go:202 +0x4f Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net.(*conn).Read(0xc000d38970, 0xc000398000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/net.go:184 +0x8e Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: bufio.(*Reader).fill(0xc0008a6720) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: bufio/bufio.go:100 +0x103 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: bufio.(*Reader).ReadSlice(0xc0008a6720, 0xc00005e00a, 0xc0002c7780, 0x40ab2b, 0xc0008a6cc0, 0xc00069abf8, 0x170e) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: bufio/bufio.go:359 +0x3d Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/internal.readChunkLine(0xc0008a6720, 0xc0008e3c80, 0xc0008e3d08, 0xc0008a6d18, 0xc0008e3db8, 0xc00069abf8) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/internal/chunked.go:122 +0x34 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/internal.(*chunkedReader).beginChunk(0xc0003f13b0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/internal/chunked.go:48 +0x32 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/internal.(*chunkedReader).Read(0xc0003f13b0, 0xc000c54000, 0x8000, 0x8000, 0x2, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/internal/chunked.go:93 +0x132 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http.(*body).readLocked(0xc0006a8340, 0xc000c54000, 0x8000, 0x8000, 0xc0008a6000, 0xc000c54000, 0xb5d) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/transfer.go:847 +0x5f Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http.(*body).Read(0xc0006a8340, 0xc000c54000, 0x8000, 0x8000, 0x0, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/transfer.go:839 +0xf2 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: io.copyBuffer(0x37bb6c0, 0xc000d38968, 0x7f2c51eabd18, 0xc0006a8340, 0xc000c54000, 0x8000, 0x8000, 0x0, 0xc0002c79c8, 0x40a6ce) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: io/io.go:405 +0x122 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: io.Copy(...) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: io/io.go:364 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient.handleStreamResponse(0xc0003782d0, 0xc0002c7d80, 0xc0003782d0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient/client.go:653 +0x383 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient.(*Client).streamURL(0xc0002f4480, 0x3093fb0, 0x3, 0xc00069ad80, 0x5e, 0x10100, 0x0, 0x0, 0x0, 0x37bb6c0, ...) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient/client.go:630 +0x872 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient.(*Client).stream(0xc0002f4480, 0x3093fb0, 0x3, 0xc00069ad20, 0x5e, 0x10100, 0x0, 0x0, 0x0, 0x37bb6c0, ...) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient/client.go:531 +0xfa Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient.(*Client).Stats.func2(0xc0009cff80, 0xc0002f4480, 0xc000837840, 0x40, 0xc00069aae0, 0x1, 0xc00094d740, 0x0, 0x0, 0x3814440, ...) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient/container.go:1159 +0x221 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient.(*Client).Stats Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient/container.go:1157 +0x2f4 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 524 [select, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient.(*Client).Stats.func3(0xc000837840, 0x40, 0xc00069aae0, 0x1, 0xc00094d740, 0x0, 0x0, 0x3814440, 0xc000333e40, 0xc000d38960, ...) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient/container.go:1186 +0xa0 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient.(*Client).Stats Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient/container.go:1184 +0x399 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 525 [chan receive, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient.(*Client).streamURL.func1(0x3814440, 0xc00004f5c0, 0x3838ec0, 0xc000d38970) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient/client.go:580 +0x48 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient.(*Client).streamURL Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/fsouza/go-dockerclient/client.go:579 +0x520 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 545 [select, 467 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/drivers/docker.(*Driver).handleWait(0xc00031bc20, 0x3814480, 0xc0000d2010, 0xc000817d40, 0xc000870b40) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/drivers/docker/driver.go:1256 +0xff Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/drivers/docker.(*Driver).WaitTask Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/drivers/docker/driver.go:1250 +0xc4 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 71380 [select, 24 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc0005329a0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:399 +0x276 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:114 +0x2c7 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 71382 [select, 26 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc0005329a0, 0x0, 0x0, 0xc001259e00) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:212 +0xb6 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(...) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:202 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client.(*Client).listenConn(0xc00019e780, 0xc0005329a0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/rpc.go:271 +0x2f Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/client.(*Client).rpcConnListener Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/rpc.go:262 +0x144 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 15141 [select]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http.(*persistConn).writeLoop(0xc0006f0fc0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/transport.go:2277 +0x11c Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by net/http.(*Transport).dialConn Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/http/transport.go:1648 +0xc7b Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 69912 [select, 35 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc000533ad0, 0x442ac6, 0x31860c8, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:212 +0xb6 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(...) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:202 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client.(*Client).listenConn(0xc00019e780, 0xc000533ad0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/rpc.go:271 +0x2f Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/client.(*Client).rpcConnListener Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/client/rpc.go:262 +0x144 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 69909 [IO wait]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll.runtime_pollWait(0x7f2c51e96c48, 0x72, 0xffffffffffffffff) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: runtime/netpoll.go:203 +0x55 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll.(*pollDesc).wait(0xc000ed4d98, 0x72, 0x1000, 0x1000, 0xffffffffffffffff) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll/fd_poll_runtime.go:87 +0x45 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll.(*pollDesc).waitRead(...) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll/fd_poll_runtime.go:92 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll.(*FD).Read(0xc000ed4d80, 0xc000df7000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: internal/poll/fd_unix.go:169 +0x19b Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net.(*netFD).Read(0xc000ed4d80, 0xc000df7000, 0x1000, 0x1000, 0xc, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/fd_unix.go:202 +0x4f Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net.(*conn).Read(0xc0000d53e8, 0xc000df7000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: net/net.go:184 +0x8e Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: bufio.(*Reader).Read(0xc0008fd9e0, 0xc0010a14f0, 0xc, 0xc, 0x3184500, 0x15, 0xc000533ad0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: bufio/bufio.go:226 +0x24f Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: io.ReadAtLeast(0x37b4d20, 0xc0008fd9e0, 0xc0010a14f0, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: io/io.go:310 +0x87 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: io.ReadFull(...) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: io/io.go:329 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc000533ad0, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:458 +0xe3 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc000533ad0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:437 +0x2b Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:113 +0x2a5 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 71676 [runnable]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).sendNoWait(0xc000533ad0, 0xc001129010, 0xc, 0xc, 0x0, 0x0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:386 +0x1d0 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).handlePing.func1(0x15, 0xc000533ad0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:545 +0xae Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).handlePing Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:542 +0x15f Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 71381 [select, 24 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc0005329a0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:313 +0xc6 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:116 +0x314 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: goroutine 69910 [select, 24 minutes]: Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc000533ad0) Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:399 +0x276 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:114 +0x2c7 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: rax 0x0 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: rbx 0x0 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: rcx 0x46ed57 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: rdx 0xc00005e000 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: rdi 0x2 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: rsi 0x0 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: rbp 0x7ffdcbfe87a0 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: rsp 0x7ffdcbfe8728 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: r8 0x6 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: r9 0xb Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: r10 0x4 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: r11 0x246 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: r12 0x170929612e833b Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: r13 0x17 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: r14 0x17091cc83f8cf5 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: r15 0x0 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: rip 0x46ed57 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: rflags 0x246 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: cs 0x33 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: fs 0x0 Jun 09 22:07:22 perflab-us01prfcpclu3-zkp2 nomad[125585]: gs 0x0 Jun 09 22:07:24 perflab-us01prfcpclu3-zkp2 systemd[1]: nomad.service: Service RestartSec=2s expired, scheduling restart. Jun 09 22:07:24 perflab-us01prfcpclu3-zkp2 systemd[1]: nomad.service: Scheduled restart job, restart counter is at 1. Jun 09 22:07:24 perflab-us01prfcpclu3-zkp2 systemd[1]: Stopped "HashiCorp Nomad - Workload orchestrator". Jun 09 22:07:24 perflab-us01prfcpclu3-zkp2 systemd[1]: Started "HashiCorp Nomad - Workload orchestrator". Jun 09 22:07:24 perflab-us01prfcpclu3-zkp2 nomad[66078]: ==> Loaded configuration from /etc/nomad.d/acl.hcl, /etc/nomad.d/client.json, /etc/nomad.d/nomad.hcl, /etc/nomad.d/nomad_vault_management.hcl, /etc/nomad.d/telemetry.json Jun 09 22:07:24 perflab-us01prfcpclu3-zkp2 nomad[66078]: ==> Starting Nomad agent... Jun 09 22:07:30 perflab-us01prfcpclu3-zkp2 nomad[66078]: ==> Nomad agent configuration: Jun 09 22:07:30 perflab-us01prfcpclu3-zkp2 nomad[66078]: Advertise Addrs: HTTP: 10.68.104.202:4646 Jun 09 22:07:30 perflab-us01prfcpclu3-zkp2 nomad[66078]: Bind Addrs: HTTP: 0.0.0.0:4646 Jun 09 22:07:30 perflab-us01prfcpclu3-zkp2 nomad[66078]: Client: true Jun 09 22:07:30 perflab-us01prfcpclu3-zkp2 nomad[66078]: Log Level: INFO Jun 09 22:07:30 perflab-us01prfcpclu3-zkp2 nomad[66078]: Region: global (DC: perflab-us01prfcpclu3) Jun 09 22:07:30 perflab-us01prfcpclu3-zkp2 nomad[66078]: Server: false Jun 09 22:07:30 perflab-us01prfcpclu3-zkp2 nomad[66078]: Version: 0.11.2 Jun 09 22:07:30 perflab-us01prfcpclu3-zkp2 nomad[66078]: ==> Nomad agent started! Log data will stream in below: Jun 09 22:07:30 perflab-us01prfcpclu3-zkp2 nomad[66078]: 2020-06-09T22:07:24.364Z [WARN] agent.plugin_loader: skipping external plugins since plugin_dir doesn't exist: plugin_dir=/opt/nomad/plugins Jun 09 22:07:30 perflab-us01prfcpclu3-zkp2 nomad[66078]: 2020-06-09T22:07:24.367Z [INFO] agent: detected plugin: name=java type=driver plugin_version=0.1.0 Jun 09 22:07:30 perflab-us01prfcpclu3-zkp2 nomad[66078]: 2020-06-09T22:07:24.367Z [INFO] agent: detected plugin: name=docker type=driver plugin_version=0.1.0 Jun 09 22:07:30 perflab-us01prfcpclu3-zkp2 nomad[66078]: 2020-06-09T22:07:24.367Z [INFO] agent: detected plugin: name=raw_exec type=driver plugin_version=0.1.0 Jun 09 22:07:30 perflab-us01prfcpclu3-zkp2 nomad[66078]: 2020-06-09T22:07:24.367Z [INFO] agent: detected plugin: name=exec type=driver plugin_version=0.1.0 Jun 09 22:07:30 perflab-us01prfcpclu3-zkp2 nomad[66078]: 2020-06-09T22:07:24.367Z [INFO] agent: detected plugin: name=qemu type=driver plugin_version=0.1.0 Jun 09 22:07:30 perflab-us01prfcpclu3-zkp2 nomad[66078]: 2020-06-09T22:07:24.367Z [INFO] agent: detected plugin: name=nvidia-gpu type=device plugin_version=0.1.0 Jun 09 22:07:30 perflab-us01prfcpclu3-zkp2 nomad[66078]: 2020-06-09T22:07:24.367Z [INFO] client: using state directory: state_dir=/opt/nomad/client Jun 09 22:07:30 perflab-us01prfcpclu3-zkp2 nomad[66078]: 2020-06-09T22:07:24.368Z [INFO] client: using alloc directory: alloc_dir=/opt/nomad/alloc Jun 09 22:07:30 perflab-us01prfcpclu3-zkp2 nomad[66078]: 2020-06-09T22:07:24.372Z [INFO] client.fingerprint_mgr.cgroup: cgroups are available Jun 09 22:07:30 perflab-us01prfcpclu3-zkp2 nomad[66078]: 2020-06-09T22:07:24.378Z [INFO] client.fingerprint_mgr.consul: consul agent is available Jun 09 22:07:30 perflab-us01prfcpclu3-zkp2 nomad[66078]: 2020-06-09T22:07:24.431Z [INFO] client.fingerprint_mgr.vault: Vault is available Jun 09 22:07:30 perflab-us01prfcpclu3-zkp2 nomad[66078]: 2020-06-09T22:07:30.433Z [INFO] client.plugin: starting plugin manager: plugin-type=csi Jun 09 22:07:30 perflab-us01prfcpclu3-zkp2 nomad[66078]: 2020-06-09T22:07:30.433Z [INFO] client.plugin: starting plugin manager: plugin-type=driver Jun 09 22:07:30 perflab-us01prfcpclu3-zkp2 nomad[66078]: 2020-06-09T22:07:30.434Z [INFO] client.plugin: starting plugin manager: plugin-type=device Jun 09 22:07:30 perflab-us01prfcpclu3-zkp2 nomad[66078]: 2020-06-09T22:07:30.448Z [INFO] client.consul: discovered following servers: servers=[10.68.104.185:4647, 10.68.104.183:4647, 10.68.104.184:4647] Jun 09 22:07:30 perflab-us01prfcpclu3-zkp2 nomad[66078]: 2020-06-09T22:07:30.505Z [WARN] client.driver_mgr.docker: failed to reattach to docker logger process: driver=docker error="failed to reattach to docker logger process: Reattachment process not found" Jun 09 22:07:30 perflab-us01prfcpclu3-zkp2 nomad[66078]: 2020-06-09T22:07:30.520Z [INFO] client: node registration complete Jun 09 22:07:30 perflab-us01prfcpclu3-zkp2 nomad[66078]: 2020-06-09T22:07:30.528Z [WARN] client.driver_mgr.docker: failed to reattach to docker logger process: driver=docker error="failed to reattach to docker logger process: Reattachment process not found" Jun 09 22:07:30 perflab-us01prfcpclu3-zkp2 nomad[66078]: 2020-06-09T22:07:30.573Z [WARN] client.driver_mgr.docker: failed to reattach to docker logger process: driver=docker error="failed to reattach to docker logger process: Reattachment process not found" Jun 09 22:07:30 perflab-us01prfcpclu3-zkp2 nomad[66078]: 2020-06-09T22:07:30.601Z [WARN] client.driver_mgr.docker: failed to reattach to docker logger process: driver=docker error="failed to reattach to docker logger process: Reattachment process not found" Jun 09 22:07:30 perflab-us01prfcpclu3-zkp2 nomad[66078]: 2020-06-09T22:07:30.634Z [WARN] client.driver_mgr.docker: failed to reattach to docker logger process: driver=docker error="failed to reattach to docker logger process: Reattachment process not found" Jun 09 22:07:30 perflab-us01prfcpclu3-zkp2 nomad[66078]: 2020-06-09T22:07:30.659Z [WARN] client.driver_mgr.docker: failed to reattach to docker logger process: driver=docker error="failed to reattach to docker logger process: Reattachment process not found" Jun 09 22:07:30 perflab-us01prfcpclu3-zkp2 nomad[66078]: 2020-06-09T22:07:30.691Z [WARN] client.driver_mgr.docker: failed to reattach to docker logger process: driver=docker error="failed to reattach to docker logger process: Reattachment process not found" Jun 09 22:07:30 perflab-us01prfcpclu3-zkp2 nomad[66078]: 2020-06-09T22:07:30.722Z [WARN] client.driver_mgr.docker: failed to reattach to docker logger process: driver=docker error="failed to reattach to docker logger process: Reattachment process not found" Jun 09 22:07:30 perflab-us01prfcpclu3-zkp2 nomad[66078]: 2020-06-09T22:07:30.760Z [INFO] client: started client: node_id=0588b15b-1ad0-0cc5-79ab-652ffd1b29de Jun 09 22:07:30 perflab-us01prfcpclu3-zkp2 nomad[66078]: 2020-06-09T22:07:30.762Z [WARN] client.alloc_runner.task_runner.task_hook: failed to reattach to logmon process: alloc_id=f8f112c6-4477-08d9-a841-bd941a340ba2 task=telegraf-statsd-logging-task error="Reattachment process not found" Jun 09 22:07:30 perflab-us01prfcpclu3-zkp2 nomad[66078]: 2020-06-09T22:07:30.774Z [WARN] client.alloc_runner.task_runner.task_hook: failed to reattach to logmon process: alloc_id=2a55a069-75b6-b663-323c-0cd0591661f1 task=zookeeper-task error="Reattachment process not found" Jun 09 22:07:30 perflab-us01prfcpclu3-zkp2 nomad[66078]: 2020-06-09T22:07:30.777Z [WARN] client.alloc_runner.task_runner.task_hook: failed to reattach to logmon process: alloc_id=42672937-5216-c976-296a-d7e6b1eb5df1 task=fabio error="Reattachment process not found" Jun 09 22:07:30 perflab-us01prfcpclu3-zkp2 nomad[66078]: 2020-06-09T22:07:30.794Z [WARN] client.alloc_runner.task_runner.task_hook: failed to reattach to logmon process: alloc_id=d5ff69c7-efb9-8dbf-179a-47e6ea080a48 task=journalbeat-task error="Reattachment process not found" Jun 09 22:07:30 perflab-us01prfcpclu3-zkp2 nomad[66078]: 2020-06-09T22:07:30.796Z [WARN] client.alloc_runner.task_runner.task_hook: failed to reattach to logmon process: alloc_id=2a55a069-75b6-b663-323c-0cd0591661f1 task=zookeeper-telegraf-task error="Reattachment process not found" Jun 09 22:07:30 perflab-us01prfcpclu3-zkp2 nomad[66078]: 2020-06-09T22:07:30.797Z [WARN] client.alloc_runner.task_runner.task_hook: failed to reattach to logmon process: alloc_id=f8f112c6-4477-08d9-a841-bd941a340ba2 task=run-telegraf-with-statsd error="Reattachment process not found" Jun 09 22:07:30 perflab-us01prfcpclu3-zkp2 nomad[66078]: 2020-06-09T22:07:30.798Z [WARN] client.alloc_runner.task_runner.task_hook: failed to reattach to logmon process: alloc_id=d5ff69c7-efb9-8dbf-179a-47e6ea080a48 task=filebeat-task error="Reattachment process not found" Jun 09 22:07:30 perflab-us01prfcpclu3-zkp2 nomad[66078]: 2020-06-09T22:07:30.801Z [WARN] client.alloc_runner.task_runner.task_hook: failed to reattach to logmon process: alloc_id=2a55a069-75b6-b663-323c-0cd0591661f1 task=zookeeper-logging-task error="Reattachment process not found" Jun 09 22:07:30 perflab-us01prfcpclu3-zkp2 nomad[66078]: 2020-06-09T22:07:30.848Z [INFO] client.alloc_runner.task_runner.task_hook.logmon.nomad: opening fifo: alloc_id=f8f112c6-4477-08d9-a841-bd941a340ba2 task=telegraf-statsd-logging-task @module=logmon path=/opt/nomad/alloc/f8f112c6-4477-08d9-a841-bd941a340ba2/alloc/logs/.telegraf-statsd-logging-task.stdout.fifo timestamp=2020-06-09T22:07:30.822Z Jun 09 22:07:30 perflab-us01prfcpclu3-zkp2 nomad[66078]: 2020-06-09T22:07:30.849Z [INFO] client.alloc_runner.task_runner.task_hook.logmon.nomad: opening fifo: alloc_id=f8f112c6-4477-08d9-a841-bd941a340ba2 task=telegraf-statsd-logging-task @module=logmon path=/opt/nomad/alloc/f8f112c6-4477-08d9-a841-bd941a340ba2/alloc/logs/.telegraf-statsd-logging-task.stderr.fifo timestamp=2020-06-09T22:07:30.822Z Jun 09 22:07:31 perflab-us01prfcpclu3-zkp2 nomad[66078]: 2020-06-09T22:07:31.137Z [INFO] client.alloc_runner.task_runner.task_hook.logmon.nomad: opening fifo: alloc_id=42672937-5216-c976-296a-d7e6b1eb5df1 task=fabio @module=logmon path=/opt/nomad/alloc/42672937-5216-c976-296a-d7e6b1eb5df1/alloc/logs/.fabio.stdout.fifo timestamp=2020-06-09T22:07:31.092Z Jun 09 22:07:31 perflab-us01prfcpclu3-zkp2 nomad[66078]: 2020-06-09T22:07:31.147Z [INFO] client.alloc_runner.task_runner.task_hook.logmon.nomad: opening fifo: alloc_id=2a55a069-75b6-b663-323c-0cd0591661f1 task=zookeeper-task path=/opt/nomad/alloc/2a55a069-75b6-b663-323c-0cd0591661f1/alloc/logs/.zookeeper-task.stdout.fifo @module=logmon timestamp=2020-06-09T22:07:31.145Z Jun 09 22:07:31 perflab-us01prfcpclu3-zkp2 nomad[66078]: 2020-06-09T22:07:31.147Z [INFO] client.alloc_runner.task_runner.task_hook.logmon.nomad: opening fifo: alloc_id=2a55a069-75b6-b663-323c-0cd0591661f1 task=zookeeper-task @module=logmon path=/opt/nomad/alloc/2a55a069-75b6-b663-323c-0cd0591661f1/alloc/logs/.zookeeper-task.stderr.fifo timestamp=2020-06-09T22:07:31.146Z Jun 09 22:07:31 perflab-us01prfcpclu3-zkp2 nomad[66078]: 2020-06-09T22:07:31.187Z [INFO] client.alloc_runner.task_runner.task_hook.logmon.nomad: opening fifo: alloc_id=42672937-5216-c976-296a-d7e6b1eb5df1 task=fabio @module=logmon path=/opt/nomad/alloc/42672937-5216-c976-296a-d7e6b1eb5df1/alloc/logs/.fabio.stderr.fifo timestamp=2020-06-09T22:07:31.155Z Jun 09 22:07:31 perflab-us01prfcpclu3-zkp2 nomad[66078]: 2020/06/09 22:07:31.231701 [INFO] (runner) creating new runner (dry: false, once: false) Jun 09 22:07:31 perflab-us01prfcpclu3-zkp2 nomad[66078]: 2020/06/09 22:07:31.244426 [INFO] (runner) creating watcher Jun 09 22:07:31 perflab-us01prfcpclu3-zkp2 nomad[66078]: 2020/06/09 22:07:31.319837 [INFO] (runner) starting Jun 09 22:07:31 perflab-us01prfcpclu3-zkp2 nomad[66078]: 2020/06/09 22:07:31.480726 [INFO] (runner) creating new runner (dry: false, once: false) Jun 09 22:07:31 perflab-us01prfcpclu3-zkp2 nomad[66078]: 2020/06/09 22:07:31.481274 [INFO] (runner) creating watcher Jun 09 22:07:31 perflab-us01prfcpclu3-zkp2 nomad[66078]: 2020/06/09 22:07:31.481786 [INFO] (runner) starting Jun 09 22:07:31 perflab-us01prfcpclu3-zkp2 nomad[66078]: 2020-06-09T22:07:31.510Z [INFO] client.alloc_runner.task_runner.task_hook.logmon.nomad: opening fifo: alloc_id=d5ff69c7-efb9-8dbf-179a-47e6ea080a48 task=journalbeat-task @module=logmon path=/opt/nomad/alloc/d5ff69c7-efb9-8dbf-179a-47e6ea080a48/alloc/logs/.journalbeat-task.stdout.fifo timestamp=2020-06-09T22:07:31.484Z Jun 09 22:07:31 perflab-us01prfcpclu3-zkp2 nomad[66078]: 2020/06/09 22:07:31.572393 [INFO] (runner) creating new runner (dry: false, once: false) Jun 09 22:07:31 perflab-us01prfcpclu3-zkp2 nomad[66078]: 2020-06-09T22:07:31.583Z [INFO] client.alloc_runner.task_runner.task_hook.logmon.nomad: opening fifo: alloc_id=d5ff69c7-efb9-8dbf-179a-47e6ea080a48 task=journalbeat-task @module=logmon path=/opt/nomad/alloc/d5ff69c7-efb9-8dbf-179a-47e6ea080a48/alloc/logs/.journalbeat-task.stderr.fifo timestamp=2020-06-09T22:07:31.485Z Jun 09 22:07:31 perflab-us01prfcpclu3-zkp2 nomad[66078]: 2020-06-09T22:07:31.516Z [INFO] client.alloc_runner.task_runner.task_hook.logmon.nomad: opening fifo: alloc_id=f8f112c6-4477-08d9-a841-bd941a340ba2 task=run-telegraf-with-statsd @module=logmon path=/opt/nomad/alloc/f8f112c6-4477-08d9-a841-bd941a340ba2/alloc/logs/.run-telegraf-with-statsd.stdout.fifo timestamp=2020-06-09T22:07:31.513Z Jun 09 22:07:31 perflab-us01prfcpclu3-zkp2 nomad[66078]: 2020-06-09T22:07:31.539Z [INFO] client.alloc_runner.task_runner.task_hook.logmon.nomad: opening fifo: alloc_id=2a55a069-75b6-b663-323c-0cd0591661f1 task=zookeeper-logging-task @module=logmon path=/opt/nomad/alloc/2a55a069-75b6-b663-323c-0cd0591661f1/alloc/logs/.zookeeper-logging-task.stdout.fifo timestamp=2020-06-09T22:07:31.519Z Jun 09 22:07:31 perflab-us01prfcpclu3-zkp2 nomad[66078]: 2020-06-09T22:07:31.539Z [INFO] client.alloc_runner.task_runner.task_hook.logmon.nomad: opening fifo: alloc_id=d5ff69c7-efb9-8dbf-179a-47e6ea080a48 task=filebeat-task @module=logmon path=/opt/nomad/alloc/d5ff69c7-efb9-8dbf-179a-47e6ea080a48/alloc/logs/.filebeat-task.stdout.fifo timestamp=2020-06-09T22:07:31.516Z Jun 09 22:07:31 perflab-us01prfcpclu3-zkp2 nomad[66078]: 2020-06-09T22:07:31.587Z [INFO] client.alloc_runner.task_runner.task_hook.logmon.nomad: opening fifo: alloc_id=2a55a069-75b6-b663-323c-0cd0591661f1 task=zookeeper-telegraf-task path=/opt/nomad/alloc/2a55a069-75b6-b663-323c-0cd0591661f1/alloc/logs/.zookeeper-telegraf-task.stdout.fifo @module=logmon timestamp=2020-06-09T22:07:31.554Z Jun 09 22:07:31 perflab-us01prfcpclu3-zkp2 nomad[66078]: 2020-06-09T22:07:31.587Z [INFO] client.alloc_runner.task_runner.task_hook.logmon.nomad: opening fifo: alloc_id=77f333fd-82a0-d852-0010-5a261489aa7b task=journalbeat-task @module=logmon path=/opt/nomad/alloc/77f333fd-82a0-d852-0010-5a261489aa7b/alloc/logs/.journalbeat-task.stdout.fifo timestamp=2020-06-09T22:07:31.545Z Jun 09 22:07:31 perflab-us01prfcpclu3-zkp2 nomad[66078]: 2020-06-09T22:07:31.588Z [INFO] client.alloc_runner.task_runner.task_hook.logmon.nomad: opening fifo: alloc_id=3e62a85a-7823-6892-bd03-993ff4339264 task=fabio @module=logmon path=/opt/nomad/alloc/3e62a85a-7823-6892-bd03-993ff4339264/alloc/logs/.fabio.stdout.fifo timestamp=2020-06-09T22:07:31.558Z Jun 09 22:07:31 perflab-us01prfcpclu3-zkp2 nomad[66078]: 2020-06-09T22:07:31.605Z [INFO] client.alloc_runner.task_runner.task_hook.logmon.nomad: opening fifo: alloc_id=3e62a85a-7823-6892-bd03-993ff4339264 task=fabio path=/opt/nomad/alloc/3e62a85a-7823-6892-bd03-993ff4339264/alloc/logs/.fabio.stderr.fifo @module=logmon timestamp=2020-06-09T22:07:31.579Z Jun 09 22:07:31 perflab-us01prfcpclu3-zkp2 nomad[66078]: 2020/06/09 22:07:31.588191 [INFO] (runner) creating watcher Jun 09 22:07:31 perflab-us01prfcpclu3-zkp2 nomad[66078]: 2020-06-09T22:07:31.606Z [INFO] client.alloc_runner.task_runner.task_hook.logmon.nomad: opening fifo: alloc_id=f8f112c6-4477-08d9-a841-bd941a340ba2 task=run-telegraf-with-statsd path=/opt/nomad/alloc/f8f112c6-4477-08d9-a841-bd941a340ba2/alloc/logs/.run-telegraf-with-statsd.stderr.fifo @module=logmon timestamp=2020-06-09T22:07:31.513Z Jun 09 22:07:31 perflab-us01prfcpclu3-zkp2 nomad[66078]: 2020-06-09T22:07:31.606Z [INFO] client.alloc_runner.task_runner.task_hook.logmon.nomad: opening fifo: alloc_id=2a55a069-75b6-b663-323c-0cd0591661f1 task=zookeeper-logging-task path=/opt/nomad/alloc/2a55a069-75b6-b663-323c-0cd0591661f1/alloc/logs/.zookeeper-logging-task.stderr.fifo @module=logmon timestamp=2020-06-09T22:07:31.520Z Jun 09 22:07:31 perflab-us01prfcpclu3-zkp2 nomad[66078]: 2020-06-09T22:07:31.606Z [INFO] client.alloc_runner.task_runner.task_hook.logmon.nomad: opening fifo: alloc_id=d5ff69c7-efb9-8dbf-179a-47e6ea080a48 task=filebeat-task @module=logmon path=/opt/nomad/alloc/d5ff69c7-efb9-8dbf-179a-47e6ea080a48/alloc/logs/.filebeat-task.stderr.fifo timestamp=2020-06-09T22:07:31.516Z Jun 09 22:07:31 perflab-us01prfcpclu3-zkp2 nomad[66078]: 2020-06-09T22:07:31.606Z [INFO] client.alloc_runner.task_runner.task_hook.logmon.nomad: opening fifo: alloc_id=2a55a069-75b6-b663-323c-0cd0591661f1 task=zookeeper-telegraf-task path=/opt/nomad/alloc/2a55a069-75b6-b663-323c-0cd0591661f1/alloc/logs/.zookeeper-telegraf-task.stderr.fifo @module=logmon timestamp=2020-06-09T22:07:31.554Z Jun 09 22:07:31 perflab-us01prfcpclu3-zkp2 nomad[66078]: 2020-06-09T22:07:31.606Z [INFO] client.alloc_runner.task_runner.task_hook.logmon.nomad: opening fifo: alloc_id=77f333fd-82a0-d852-0010-5a261489aa7b task=journalbeat-task @module=logmon path=/opt/nomad/alloc/77f333fd-82a0-d852-0010-5a261489aa7b/alloc/logs/.journalbeat-task.stderr.fifo timestamp=2020-06-09T22:07:31.546Z Jun 09 22:07:31 perflab-us01prfcpclu3-zkp2 nomad[66078]: 2020/06/09 22:07:31.627912 [INFO] (runner) starting Jun 09 22:07:31 perflab-us01prfcpclu3-zkp2 nomad[66078]: 2020/06/09 22:07:31.630960 [INFO] (runner) creating new runner (dry: false, once: false) Jun 09 22:07:31 perflab-us01prfcpclu3-zkp2 nomad[66078]: 2020/06/09 22:07:31.643841 [INFO] (runner) creating watcher Jun 09 22:07:31 perflab-us01prfcpclu3-zkp2 nomad[66078]: 2020-06-09T22:07:31.641Z [ERROR] client.vault: error during renewal of token: error="failed to renew the vault token: Error making API request. Jun 09 22:07:31 perflab-us01prfcpclu3-zkp2 nomad[66078]: URL: PUT http://vault.service.consul:8200/v1/auth/token/renew-self Jun 09 22:07:31 perflab-us01prfcpclu3-zkp2 nomad[66078]: Code: 403. Errors: Jun 09 22:07:31 perflab-us01prfcpclu3-zkp2 nomad[66078]: * permission denied" Jun 09 22:07:31 perflab-us01prfcpclu3-zkp2 nomad[66078]: 2020-06-09T22:07:31.680Z [ERROR] client.alloc_runner.task_runner.task_hook.vault: failed to start renewal of Vault token: alloc_id=42672937-5216-c976-296a-d7e6b1eb5df1 task=fabio error="failed to renew the vault token: Error making API request. Jun 09 22:07:31 perflab-us01prfcpclu3-zkp2 nomad[66078]: URL: PUT http://vault.service.consul:8200/v1/auth/token/renew-self Jun 09 22:07:31 perflab-us01prfcpclu3-zkp2 nomad[66078]: Code: 403. Errors: Jun 09 22:07:31 perflab-us01prfcpclu3-zkp2 nomad[66078]: * permission denied" Jun 09 22:07:31 perflab-us01prfcpclu3-zkp2 nomad[66078]: 2020-06-09T22:07:31.642Z [INFO] client.alloc_runner.task_runner.task_hook.logmon.nomad: opening fifo: alloc_id=77f333fd-82a0-d852-0010-5a261489aa7b task=filebeat-task @module=logmon path=/opt/nomad/alloc/77f333fd-82a0-d852-0010-5a261489aa7b/alloc/logs/.filebeat-task.stdout.fifo timestamp=2020-06-09T22:07:31.608Z Jun 09 22:07:31 perflab-us01prfcpclu3-zkp2 nomad[66078]: 2020-06-09T22:07:31.675Z [INFO] client.alloc_runner.task_runner.task_hook.logmon.nomad: opening fifo: alloc_id=01a73550-2830-af63-dabd-01626120be53 task=zookeeper-task @module=logmon path=/opt/nomad/alloc/01a73550-2830-af63-dabd-01626120be53/alloc/logs/.zookeeper-task.stdout.fifo timestamp=2020-06-09T22:07:31.591Z Jun 09 22:07:31 perflab-us01prfcpclu3-zkp2 nomad[66078]: 2020-06-09T22:07:31.717Z [INFO] client.alloc_runner.task_runner.task_hook.logmon.nomad: opening fifo: alloc_id=01a73550-2830-af63-dabd-01626120be53 task=zookeeper-task @module=logmon path=/opt/nomad/alloc/01a73550-2830-af63-dabd-01626120be53/alloc/logs/.zookeeper-task.stderr.fifo timestamp=2020-06-09T22:07:31.592Z Jun 09 22:07:31 perflab-us01prfcpclu3-zkp2 nomad[66078]: 2020-06-09T22:07:31.767Z [INFO] client.alloc_runner.task_runner.task_hook.logmon.nomad: opening fifo: alloc_id=77f333fd-82a0-d852-0010-5a261489aa7b task=filebeat-task @module=logmon path=/opt/nomad/alloc/77f333fd-82a0-d852-0010-5a261489aa7b/alloc/logs/.filebeat-task.stderr.fifo timestamp=2020-06-09T22:07:31.615Z Jun 09 22:07:31 perflab-us01prfcpclu3-zkp2 nomad[66078]: 2020-06-09T22:07:31.772Z [ERROR] client.vault: error deriving vault tokens: error="Can't request Vault token for terminal allocation" Jun 09 22:07:31 perflab-us01prfcpclu3-zkp2 nomad[66078]: 2020-06-09T22:07:31.773Z [ERROR] client.vault: error deriving token: error="Can't request Vault token for terminal allocation" alloc_id=42672937-5216-c976-296a-d7e6b1eb5df1 task_names=[fabio] Jun 09 22:07:31 perflab-us01prfcpclu3-zkp2 nomad[66078]: 2020-06-09T22:07:31.773Z [ERROR] client.alloc_runner.task_runner.task_hook.vault: failed to derive Vault token: alloc_id=42672937-5216-c976-296a-d7e6b1eb5df1 task=fabio error="Can't request Vault token for terminal allocation" server_side=true Jun 09 22:07:31 perflab-us01prfcpclu3-zkp2 nomad[66078]: 2020/06/09 22:07:31.672075 [INFO] (runner) creating new runner (dry: false, once: false) Jun 09 22:07:31 perflab-us01prfcpclu3-zkp2 nomad[66078]: 2020/06/09 22:07:31.774500 [INFO] (runner) creating watcher Jun 09 22:07:31 perflab-us01prfcpclu3-zkp2 nomad[66078]: 2020/06/09 22:07:31.683476 [INFO] (runner) starting Jun 09 22:07:31 perflab-us01prfcpclu3-zkp2 nomad[66078]: 2020/06/09 22:07:31.796545 [INFO] (runner) starting Jun 09 22:07:31 perflab-us01prfcpclu3-zkp2 nomad[66078]: 2020/06/09 22:07:31.821701 [INFO] (runner) creating new runner (dry: false, once: false) Jun 09 22:07:31 perflab-us01prfcpclu3-zkp2 nomad[66078]: 2020/06/09 22:07:31.821930 [INFO] (runner) creating watcher Jun 09 22:07:31 perflab-us01prfcpclu3-zkp2 nomad[66078]: 2020/06/09 22:07:31.822096 [INFO] (runner) starting Jun 09 22:07:32 perflab-us01prfcpclu3-zkp2 nomad[66078]: 2020-06-09T22:07:31.982Z [INFO] client.alloc_runner.task_runner.task_hook.logmon.nomad: opening fifo: alloc_id=82d4b9eb-3b5e-1bab-d2c2-3971ea9dbd96 task=run-telegraf-with-statsd @module=logmon path=/opt/nomad/alloc/82d4b9eb-3b5e-1bab-d2c2-3971ea9dbd96/alloc/logs/.run-telegraf-with-statsd.stdout.fifo timestamp=2020-06-09T22:07:31.982Z Jun 09 22:07:32 perflab-us01prfcpclu3-zkp2 nomad[66078]: 2020/06/09 22:07:32.002149 [INFO] (runner) creating new runner (dry: false, once: false) Jun 09 22:07:32 perflab-us01prfcpclu3-zkp2 nomad[66078]: 2020/06/09 22:07:32.009584 [INFO] (runner) creating watcher Jun 09 22:07:32 perflab-us01prfcpclu3-zkp2 nomad[66078]: 2020/06/09 22:07:32.010413 [INFO] (runner) starting Jun 09 22:07:32 perflab-us01prfcpclu3-zkp2 nomad[66078]: 2020-06-09T22:07:32.022Z [INFO] client.alloc_runner.task_runner.task_hook.logmon.nomad: opening fifo: alloc_id=01a73550-2830-af63-dabd-01626120be53 task=zookeeper-logging-task @module=logmon path=/opt/nomad/alloc/01a73550-2830-af63-dabd-01626120be53/alloc/logs/.zookeeper-logging-task.stdout.fifo timestamp=2020-06-09T22:07:32.019Z Jun 09 22:07:32 perflab-us01prfcpclu3-zkp2 nomad[66078]: 2020-06-09T22:07:32.054Z [INFO] client.alloc_runner.task_runner.task_hook.logmon.nomad: opening fifo: alloc_id=01a73550-2830-af63-dabd-01626120be53 task=zookeeper-logging-task path=/opt/nomad/alloc/01a73550-2830-af63-dabd-01626120be53/alloc/logs/.zookeeper-logging-task.stderr.fifo @module=logmon timestamp=2020-06-09T22:07:32.020Z Jun 09 22:07:32 perflab-us01prfcpclu3-zkp2 nomad[66078]: 2020/06/09 22:07:32.051002 [INFO] (runner) rendered "(dynamic)" => "/opt/nomad/alloc/77f333fd-82a0-d852-0010-5a261489aa7b/journalbeat-task/local/journalbeat.yml" Jun 09 22:07:32 perflab-us01prfcpclu3-zkp2 nomad[66078]: 2020/06/09 22:07:32.059849 [INFO] (runner) creating new runner (dry: false, once: false) Jun 09 22:07:32 perflab-us01prfcpclu3-zkp2 nomad[66078]: 2020/06/09 22:07:32.060465 [INFO] (runner) creating watcher Jun 09 22:07:32 perflab-us01prfcpclu3-zkp2 nomad[66078]: 2020/06/09 22:07:32.060588 [INFO] (runner) starting Jun 09 22:07:32 perflab-us01prfcpclu3-zkp2 nomad[66078]: 2020/06/09 22:07:32.059969 [INFO] (runner) creating new runner (dry: false, once: false) Jun 09 22:07:32 perflab-us01prfcpclu3-zkp2 nomad[66078]: 2020/06/09 22:07:32.063455 [INFO] (runner) creating watcher Jun 09 22:07:32 perflab-us01prfcpclu3-zkp2 nomad[66078]: 2020/06/09 22:07:32.063859 [INFO] (runner) starting Jun 09 22:07:32 perflab-us01prfcpclu3-zkp2 nomad[66078]: 2020/06/09 22:07:32.063241 [INFO] (runner) rendered "(dynamic)" => "/opt/nomad/alloc/77f333fd-82a0-d852-0010-5a261489aa7b/journalbeat-task/local/CA.crt" Jun 09 22:07:32 perflab-us01prfcpclu3-zkp2 nomad[66078]: 2020/06/09 22:07:32.068930 [INFO] (runner) rendered "(dynamic)" => "/opt/nomad/alloc/01a73550-2830-af63-dabd-01626120be53/zookeeper-task/local/zookeeper_health_check.sh" Jun 09 22:07:32 perflab-us01prfcpclu3-zkp2 nomad[66078]: 2020/06/09 22:07:32.069629 [INFO] (runner) rendered "(dynamic)" => "/opt/nomad/alloc/77f333fd-82a0-d852-0010-5a261489aa7b/filebeat-task/local/filebeat.yml" Jun 09 22:07:32 perflab-us01prfcpclu3-zkp2 nomad[66078]: 2020-06-09T22:07:32.087Z [INFO] client.alloc_runner.task_runner.task_hook.logmon.nomad: opening fifo: alloc_id=82d4b9eb-3b5e-1bab-d2c2-3971ea9dbd96 task=run-telegraf-with-statsd @module=logmon path=/opt/nomad/alloc/82d4b9eb-3b5e-1bab-d2c2-3971ea9dbd96/alloc/logs/.run-telegraf-with-statsd.stderr.fifo timestamp=2020-06-09T22:07:32.077Z Jun 09 22:07:32 perflab-us01prfcpclu3-zkp2 nomad[66078]: 2020/06/09 22:07:32.195974 [INFO] (runner) rendered "(dynamic)" => "/opt/nomad/alloc/01a73550-2830-af63-dabd-01626120be53/zookeeper-task/zk-env/zk.env" Jun 09 22:07:32 perflab-us01prfcpclu3-zkp2 nomad[66078]: 2020/06/09 22:07:32.208924 [INFO] (runner) creating new runner (dry: false, once: false) Jun 09 22:07:32 perflab-us01prfcpclu3-zkp2 nomad[66078]: 2020/06/09 22:07:32.217604 [INFO] (runner) creating watcher Jun 09 22:07:32 perflab-us01prfcpclu3-zkp2 nomad[66078]: 2020/06/09 22:07:32.219479 [INFO] (runner) starting Jun 09 22:07:32 perflab-us01prfcpclu3-zkp2 nomad[66078]: 2020/06/09 22:07:32.246807 [INFO] (runner) rendered "(dynamic)" => "/opt/nomad/alloc/82d4b9eb-3b5e-1bab-d2c2-3971ea9dbd96/run-telegraf-with-statsd/local/telegraf.statsd.conf" Jun 09 22:07:32 perflab-us01prfcpclu3-zkp2 nomad[66078]: 2020-06-09T22:07:32.298Z [INFO] client.alloc_runner.task_runner.task_hook.logmon.nomad: opening fifo: alloc_id=01a73550-2830-af63-dabd-01626120be53 task=zookeeper-telegraf-task @module=logmon path=/opt/nomad/alloc/01a73550-2830-af63-dabd-01626120be53/alloc/logs/.zookeeper-telegraf-task.stdout.fifo timestamp=2020-06-09T22:07:32.279Z Jun 09 22:07:32 perflab-us01prfcpclu3-zkp2 nomad[66078]: 2020-06-09T22:07:32.344Z [INFO] client.alloc_runner.task_runner.task_hook.logmon.nomad: opening fifo: alloc_id=01a73550-2830-af63-dabd-01626120be53 task=zookeeper-telegraf-task path=/opt/nomad/alloc/01a73550-2830-af63-dabd-01626120be53/alloc/logs/.zookeeper-telegraf-task.stderr.fifo @module=logmon timestamp=2020-06-09T22:07:32.280Z Jun 09 22:07:32 perflab-us01prfcpclu3-zkp2 nomad[66078]: 2020/06/09 22:07:32.345275 [INFO] (runner) rendered "(dynamic)" => "/opt/nomad/alloc/82d4b9eb-3b5e-1bab-d2c2-3971ea9dbd96/run-telegraf-with-statsd/local/hashicorp.telegraf.conf" Jun 09 22:07:32 perflab-us01prfcpclu3-zkp2 nomad[66078]: 2020-06-09T22:07:32.407Z [INFO] client.alloc_runner.task_runner.task_hook.logmon.nomad: opening fifo: alloc_id=82d4b9eb-3b5e-1bab-d2c2-3971ea9dbd96 task=telegraf-statsd-logging-task @module=logmon path=/opt/nomad/alloc/82d4b9eb-3b5e-1bab-d2c2-3971ea9dbd96/alloc/logs/.telegraf-statsd-logging-task.stdout.fifo timestamp=2020-06-09T22:07:32.406Z Jun 09 22:07:32 perflab-us01prfcpclu3-zkp2 nomad[66078]: 2020-06-09T22:07:32.407Z [INFO] client.alloc_runner.task_runner.task_hook.logmon.nomad: opening fifo: alloc_id=82d4b9eb-3b5e-1bab-d2c2-3971ea9dbd96 task=telegraf-statsd-logging-task @module=logmon path=/opt/nomad/alloc/82d4b9eb-3b5e-1bab-d2c2-3971ea9dbd96/alloc/logs/.telegraf-statsd-logging-task.stderr.fifo timestamp=2020-06-09T22:07:32.407Z Jun 09 22:07:32 perflab-us01prfcpclu3-zkp2 nomad[66078]: 2020/06/09 22:07:32.441050 [INFO] (runner) creating new runner (dry: false, once: false) Jun 09 22:07:32 perflab-us01prfcpclu3-zkp2 nomad[66078]: 2020/06/09 22:07:32.441235 [INFO] (runner) creating watcher Jun 09 22:07:32 perflab-us01prfcpclu3-zkp2 nomad[66078]: 2020/06/09 22:07:32.441369 [INFO] (runner) starting Jun 09 22:07:32 perflab-us01prfcpclu3-zkp2 nomad[66078]: 2020/06/09 22:07:32.480251 [INFO] (runner) creating new runner (dry: false, once: false) Jun 09 22:07:32 perflab-us01prfcpclu3-zkp2 nomad[66078]: 2020/06/09 22:07:32.480442 [INFO] (runner) creating watcher Jun 09 22:07:32 perflab-us01prfcpclu3-zkp2 nomad[66078]: 2020/06/09 22:07:32.480594 [INFO] (runner) starting Jun 09 22:07:32 perflab-us01prfcpclu3-zkp2 nomad[66078]: 2020/06/09 22:07:32.485759 [INFO] (runner) rendered "(dynamic)" => "/opt/nomad/alloc/01a73550-2830-af63-dabd-01626120be53/zookeeper-telegraf-task/local/zookeeper.telegraf.conf" Jun 09 22:07:32 perflab-us01prfcpclu3-zkp2 nomad[66078]: 2020-06-09T22:07:32.519Z [INFO] client.driver_mgr.docker: created container: driver=docker container_id=fa29126dafd390ae973476bc95abd5a12d8fa3a4213901228c583fb1bb90083e Jun 09 22:07:32 perflab-us01prfcpclu3-zkp2 nomad[66078]: 2020/06/09 22:07:32.555805 [INFO] (runner) rendered "(dynamic)" => "/opt/nomad/alloc/3e62a85a-7823-6892-bd03-993ff4339264/fabio/secrets/fabio.properties" Jun 09 22:07:32 perflab-us01prfcpclu3-zkp2 nomad[66078]: 2020-06-09T22:07:32.654Z [INFO] client.driver_mgr.docker: created container: driver=docker container_id=8f9a04f9a9cdf51f10de7292fccdac28dcc08ced29b923ac221afbceb4d204d6 Jun 09 22:07:32 perflab-us01prfcpclu3-zkp2 nomad[66078]: 2020-06-09T22:07:32.703Z [INFO] client.driver_mgr.docker: created container: driver=docker container_id=df8d7cf306283ad378330dfeaf6b7855f5fc283d25e78432a0d7dbf408de45df Jun 09 22:07:32 perflab-us01prfcpclu3-zkp2 nomad[66078]: 2020-06-09T22:07:32.929Z [INFO] client.driver_mgr.docker: created container: driver=docker container_id=ca32bd20f61e9655a434fd043cd3da12b94ce1b5b1847781d5380ec697d09ec7 Jun 09 22:07:33 perflab-us01prfcpclu3-zkp2 nomad[66078]: 2020-06-09T22:07:33.010Z [INFO] client.driver_mgr.docker: created container: driver=docker container_id=aab24d173804daaf3b6568a953a2b5b711db41058edf5e3794c8836f7c051543 Jun 09 22:07:33 perflab-us01prfcpclu3-zkp2 nomad[66078]: 2020-06-09T22:07:33.029Z [INFO] client.driver_mgr.docker: created container: driver=docker container_id=9f9e732fa67f4608d6561e4ca79e1644dbf74ea6d318c4197c86d8ff51e234e6 Jun 09 22:07:33 perflab-us01prfcpclu3-zkp2 nomad[66078]: 2020-06-09T22:07:33.091Z [INFO] client.driver_mgr.docker: created container: driver=docker container_id=a50d0343ae4a82a5b0731c6ccac2dd430506cd9fad06f50bc72cb93504b3e2c0 Jun 09 22:07:33 perflab-us01prfcpclu3-zkp2 nomad[66078]: 2020-06-09T22:07:33.183Z [INFO] client.driver_mgr.docker: created container: driver=docker container_id=4a370a7f5b9450c6886212f5e1f7774a175d55e7974269ac49ebccb1091c1699 Jun 09 22:07:33 perflab-us01prfcpclu3-zkp2 nomad[66078]: 2020-06-09T22:07:33.214Z [INFO] client.gc: marking allocation for GC: alloc_id=42672937-5216-c976-296a-d7e6b1eb5df1 Jun 09 22:07:33 perflab-us01prfcpclu3-zkp2 nomad[66078]: 2020/06/09 22:07:33.220675 [INFO] (runner) stopping Jun 09 22:07:33 perflab-us01prfcpclu3-zkp2 nomad[66078]: 2020/06/09 22:07:33.233666 [INFO] (runner) received finish Jun 09 22:07:33 perflab-us01prfcpclu3-zkp2 nomad[66078]: 2020/06/09 22:07:33.280772 [INFO] (runner) stopping Jun 09 22:07:33 perflab-us01prfcpclu3-zkp2 nomad[66078]: 2020/06/09 22:07:33.280868 [INFO] (runner) received finish Jun 09 22:07:33 perflab-us01prfcpclu3-zkp2 nomad[66078]: 2020/06/09 22:07:33.371994 [INFO] (runner) stopping Jun 09 22:07:33 perflab-us01prfcpclu3-zkp2 nomad[66078]: 2020/06/09 22:07:33.372100 [INFO] (runner) received finish Jun 09 22:07:33 perflab-us01prfcpclu3-zkp2 nomad[66078]: 2020-06-09T22:07:33.750Z [INFO] client.gc: marking allocation for GC: alloc_id=d5ff69c7-efb9-8dbf-179a-47e6ea080a48 Jun 09 22:07:33 perflab-us01prfcpclu3-zkp2 nomad[66078]: 2020/06/09 22:07:33.777500 [INFO] (runner) stopping Jun 09 22:07:33 perflab-us01prfcpclu3-zkp2 nomad[66078]: 2020/06/09 22:07:33.778027 [INFO] (runner) received finish Jun 09 22:07:34 perflab-us01prfcpclu3-zkp2 nomad[66078]: 2020-06-09T22:07:34.266Z [INFO] client.driver_mgr.docker: started container: driver=docker container_id=df8d7cf306283ad378330dfeaf6b7855f5fc283d25e78432a0d7dbf408de45df Jun 09 22:07:35 perflab-us01prfcpclu3-zkp2 nomad[66078]: 2020-06-09T22:07:35.944Z [INFO] client: node registration complete Jun 09 22:07:36 perflab-us01prfcpclu3-zkp2 nomad[66078]: 2020-06-09T22:07:36.223Z [INFO] client.driver_mgr.docker: started container: driver=docker container_id=a50d0343ae4a82a5b0731c6ccac2dd430506cd9fad06f50bc72cb93504b3e2c0 Jun 09 22:07:36 perflab-us01prfcpclu3-zkp2 nomad[66078]: 2020-06-09T22:07:36.707Z [INFO] client.driver_mgr.docker: started container: driver=docker container_id=fa29126dafd390ae973476bc95abd5a12d8fa3a4213901228c583fb1bb90083e Jun 09 22:07:37 perflab-us01prfcpclu3-zkp2 nomad[66078]: 2020-06-09T22:07:37.573Z [INFO] client.driver_mgr.docker: stopped container: container_id=481e7da8a5ec87b2bea5c3eae80920ef9a02f447eaaa6bdcd518e5fcced57f1e driver=docker Jun 09 22:07:37 perflab-us01prfcpclu3-zkp2 nomad[66078]: 2020-06-09T22:07:37.611Z [INFO] client.driver_mgr.docker: stopped container: container_id=c4bf82da279d2e28ae3a100b45359140b66bfd131eb07e2268deada8c1a3572a driver=docker Jun 09 22:07:37 perflab-us01prfcpclu3-zkp2 nomad[66078]: 2020-06-09T22:07:37.621Z [INFO] client.gc: marking allocation for GC: alloc_id=f8f112c6-4477-08d9-a841-bd941a340ba2 Jun 09 22:07:37 perflab-us01prfcpclu3-zkp2 nomad[66078]: 2020/06/09 22:07:37.639732 [INFO] (runner) stopping Jun 09 22:07:37 perflab-us01prfcpclu3-zkp2 nomad[66078]: 2020/06/09 22:07:37.640176 [INFO] (runner) received finish Jun 09 22:07:37 perflab-us01prfcpclu3-zkp2 nomad[66078]: 2020-06-09T22:07:37.693Z [INFO] client.gc: marking allocation for GC: alloc_id=2a55a069-75b6-b663-323c-0cd0591661f1 Jun 09 22:07:37 perflab-us01prfcpclu3-zkp2 nomad[66078]: 2020/06/09 22:07:37.721389 [INFO] (runner) stopping Jun 09 22:07:37 perflab-us01prfcpclu3-zkp2 nomad[66078]: 2020/06/09 22:07:37.722182 [INFO] (runner) received finish Jun 09 22:07:37 perflab-us01prfcpclu3-zkp2 nomad[66078]: 2020-06-09T22:07:37.941Z [INFO] client.driver_mgr.docker: started container: driver=docker container_id=8f9a04f9a9cdf51f10de7292fccdac28dcc08ced29b923ac221afbceb4d204d6 Jun 09 22:07:38 perflab-us01prfcpclu3-zkp2 nomad[66078]: 2020-06-09T22:07:38.468Z [INFO] client.driver_mgr.docker: started container: driver=docker container_id=ca32bd20f61e9655a434fd043cd3da12b94ce1b5b1847781d5380ec697d09ec7 Jun 09 22:07:39 perflab-us01prfcpclu3-zkp2 nomad[66078]: 2020-06-09T22:07:39.082Z [INFO] client.driver_mgr.docker: started container: driver=docker container_id=aab24d173804daaf3b6568a953a2b5b711db41058edf5e3794c8836f7c051543 Jun 09 22:07:39 perflab-us01prfcpclu3-zkp2 nomad[66078]: 2020-06-09T22:07:39.233Z [INFO] client.driver_mgr.docker: started container: driver=docker container_id=9f9e732fa67f4608d6561e4ca79e1644dbf74ea6d318c4197c86d8ff51e234e6 Jun 09 22:07:39 perflab-us01prfcpclu3-zkp2 nomad[66078]: 2020-06-09T22:07:39.282Z [INFO] client.driver_mgr.docker: started container: driver=docker container_id=4a370a7f5b9450c6886212f5e1f7774a175d55e7974269ac49ebccb1091c1699 Jun 09 22:08:39 perflab-us01prfcpclu3-zkp2 nomad[66078]: 2020-06-09T22:08:39.621Z [WARN] consul.sync: failed to update services in Consul: error="Unexpected response code: 500 (Unknown check "_nomad-check-4686196c7148fa01045e4a27b8706ff07aab00b3")" Jun 09 22:08:40 perflab-us01prfcpclu3-zkp2 nomad[66078]: 2020-06-09T22:08:40.625Z [INFO] consul.sync: successfully updated services in Consul