Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Kapacitor hangs on most requests after disabling script which consumes no data. #749

Closed
phemmer opened this issue Jul 25, 2016 · 3 comments

Comments

@phemmer
Copy link

phemmer commented Jul 25, 2016

When you have a script with no stream or batch, the define operation hangs.

cat > /tmp/foo <<'EOF'
var foo = 'bar'
EOF

kapacitor define foo -type stream -tick /tmp/foo -dbrp telegraf.autogen

The kapacitor command just sits there. No error.

This is kapacitor 1.0 beta 3 on CentOS/7 with InfluxDB 1.0 beta 3.

EDIT:
Actually, after trying to load the bad script, kapacitor started hanging on all operations. I get about 2 or 3 requests in after the load, then it hangs.

Here's the output of SIGQUIT:

SIGQUIT: quit
PC=0x4646c1 m=0

goroutine 0 [idle]:
    runtime.futex(0x13ac5a8, 0x0, 0x0, 0x0, 0x0, 0x13ab930, 0x0, 0x0, 0x412814, 0x13ac5a8, ...)
    /usr/local/go/src/runtime/sys_linux_amd64.s:306 +0x21
    runtime.futexsleep(0x13ac5a8, 0x0, 0xffffffffffffffff)
    /usr/local/go/src/runtime/os1_linux.go:40 +0x53
    runtime.notesleep(0x13ac5a8)
    /usr/local/go/src/runtime/lock_futex.go:145 +0xa4
    runtime.stopm()
    /usr/local/go/src/runtime/proc.go:1538 +0x10b
    runtime.findrunnable(0xc820014000, 0x0)
    /usr/local/go/src/runtime/proc.go:1976 +0x739
    runtime.schedule()
    /usr/local/go/src/runtime/proc.go:2075 +0x24f
    runtime.exitsyscall0(0xc8203c1380)
    /usr/local/go/src/runtime/proc.go:2588 +0x17b
    runtime.mcall(0x7ffd3e6a9b00)
    /usr/local/go/src/runtime/asm_amd64.s:233 +0x5b

goroutine 1 [chan receive, 3836 minutes]:
    main.(*Main).Run(0xc82025bf10, 0xc82005c0d0, 0x2, 0x2, 0x0, 0x0)
    /root/go/src/github.com/influxdata/kapacitor/cmd/kapacitord/main.go:96 +0xc02
    main.main()
    /root/go/src/github.com/influxdata/kapacitor/cmd/kapacitord/main.go:41 +0x361

goroutine 17 [syscall, 3836 minutes, locked to thread]:
    runtime.goexit()
    /usr/local/go/src/runtime/asm_amd64.s:1998 +0x1

goroutine 19 [syscall, 3836 minutes]:
    os/signal.signal_recv(0x0)
    /usr/local/go/src/runtime/sigqueue.go:116 +0x132
    os/signal.loop()
    /usr/local/go/src/os/signal/signal_unix.go:22 +0x18
    created by os/signal.init.1
    /usr/local/go/src/os/signal/signal_unix.go:28 +0x37

goroutine 84 [select, 10 minutes]:
    github.com/influxdata/kapacitor.(*Edge).CollectPoint(0xc8201c81c0, 0xc8203d0a40, 0x6, 0xc82066ee7c, 0x8, 0xc82066ee95, 0x7, 0xe56498, 0x0, 0x0, ...)
    /root/go/src/github.com/influxdata/kapacitor/edge.go:171 +0x1c6
    github.com/influxdata/kapacitor.(*TaskMaster).WritePoints(0xc820064c00, 0xc82066ee7c, 0x8, 0xc82066ee95, 0x7, 0x3, 0xc820910200, 0x12, 0x20, 0x0, ...)
    /root/go/src/github.com/influxdata/kapacitor/task_master.go:533 +0x294
    github.com/influxdata/kapacitor/services/httpd.(*Handler).serveWriteLine(0xc8201c8230, 0x7fbb72003320, 0xc8205653e0, 0xc820234d20, 0xc820912000, 0x105f, 0x1e00, 0x0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:430 +0x862
    github.com/influxdata/kapacitor/services/httpd.(*Handler).serveWrite(0xc8201c8230, 0x7fbb72003320, 0xc8205653e0, 0xc820234d20, 0x0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:393 +0x680
    github.com/influxdata/kapacitor/services/httpd.(*Handler).(github.com/influxdata/kapacitor/services/httpd.serveWrite)-fm(0x7fbb72003320, 0xc8205653e0, 0xc820234d20, 0x0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:132 +0x48
    github.com/influxdata/kapacitor/services/httpd.authenticate.func1(0x7fbb72003320, 0xc8205653e0, 0xc820234d20)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:532 +0x52
    net/http.HandlerFunc.ServeHTTP(0xc8201dde60, 0x7fbb72003320, 0xc8205653e0, 0xc820234d20)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.jsonContent.func1(0x7fbb72003320, 0xc8205653e0, 0xc820234d20)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:570 +0xaa
    net/http.HandlerFunc.ServeHTTP(0xc8201dde80, 0x7fbb72003320, 0xc8205653e0, 0xc820234d20)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.gzipFilter.func1(0x7fbb72003298, 0xc8205652e0, 0xc820234d20)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:563 +0x2af
    net/http.HandlerFunc.ServeHTTP(0xc8201ddea0, 0x7fbb72003298, 0xc8205652e0, 0xc820234d20)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.versionHeader.func1(0x7fbb72003298, 0xc8205652e0, 0xc820234d20)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:579 +0xc0
    net/http.HandlerFunc.ServeHTTP(0xc8201ddec0, 0x7fbb72003298, 0xc8205652e0, 0xc820234d20)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.cors.func1(0x7fbb72003298, 0xc8205652e0, 0xc820234d20)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:612 +0x329
    net/http.HandlerFunc.ServeHTTP(0xc8201ddee0, 0x7fbb72003298, 0xc8205652e0, 0xc820234d20)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.requestID.func1(0x7fbb72003298, 0xc8205652e0, 0xc820234d20)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:622 +0x15d
    net/http.HandlerFunc.ServeHTTP(0xc8201ddf00, 0x7fbb72003298, 0xc8205652e0, 0xc820234d20)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.recovery.func1(0x7fbb72003238, 0xc820371ba0, 0xc820234d20)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:639 +0x114
    net/http.HandlerFunc.ServeHTTP(0xc8201ddf20, 0x7fbb72003238, 0xc820371ba0, 0xc820234d20)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.(*ServeMux).ServeHTTP(0xc8201cda10, 0x7fbb72003238, 0xc820371ba0, 0xc820234d20)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/mux.go:163 +0x17d
    github.com/influxdata/kapacitor/services/httpd.(*Handler).ServeHTTP(0xc8201c8230, 0x7fbb72003238, 0xc820371ba0, 0xc820234d20)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:306 +0xe5
    net/http.serverHandler.ServeHTTP(0xc8201d6680, 0x7fbb72003238, 0xc820371ba0, 0xc820234d20)
    /usr/local/go/src/net/http/server.go:2081 +0x19e
    net/http.(*conn).serve(0xc820092580)
    /usr/local/go/src/net/http/server.go:1472 +0xf2e
    created by net/http.(*Server).Serve
    /usr/local/go/src/net/http/server.go:2137 +0x44e

goroutine 21 [semacquire, 10 minutes]:
    sync.runtime_Semacquire(0xc820064d60)
    /usr/local/go/src/runtime/sema.go:47 +0x26
    sync.(*RWMutex).RLock(0xc820064d54)
    /usr/local/go/src/sync/rwmutex.go:37 +0x58
    github.com/influxdata/kapacitor.(*TaskMaster).forkPoint(0xc820064c00, 0xc82076a100, 0x6, 0xc82066e01c, 0x8, 0xc82066e035, 0x7, 0xe56498, 0x0, 0x0, ...)
    /root/go/src/github.com/influxdata/kapacitor/task_master.go:474 +0x46
    github.com/influxdata/kapacitor.(*TaskMaster).runForking(0xc820064c00, 0xc8201c81c0)
    /root/go/src/github.com/influxdata/kapacitor/task_master.go:469 +0xbb
    created by github.com/influxdata/kapacitor.(*TaskMaster).stream
    /root/go/src/github.com/influxdata/kapacitor/task_master.go:462 +0x26b

goroutine 47098 [chan send, 250 minutes]:
    github.com/influxdata/kapacitor/tick/ast.lexToken(0xc8207cd260, 0x1024920)
    /root/go/src/github.com/influxdata/kapacitor/tick/ast/lex.go:343 +0x335
    github.com/influxdata/kapacitor/tick/ast.(*lexer).run(0xc8207cd260)
    /root/go/src/github.com/influxdata/kapacitor/tick/ast/lex.go:238 +0x32
    created by github.com/influxdata/kapacitor/tick/ast.lex
    /root/go/src/github.com/influxdata/kapacitor/tick/ast/lex.go:230 +0xc1

goroutine 55346 [semacquire, 2 minutes]:
    sync.runtime_Semacquire(0xc820064d60)
    /usr/local/go/src/runtime/sema.go:47 +0x26
    sync.(*RWMutex).RLock(0xc820064d54)
    /usr/local/go/src/sync/rwmutex.go:37 +0x58
    github.com/influxdata/kapacitor.(*TaskMaster).IsExecuting(0xc820064c00, 0xc820682670, 0xf, 0x400)
    /root/go/src/github.com/influxdata/kapacitor/task_master.go:422 +0x3d
    github.com/influxdata/kapacitor/services/task_store.(*Service).convertTask(0xc82017f0e0, 0xc820682670, 0xf, 0x0, 0xc820664860, 0x1, 0x1, 0xc8201b2480, 0x44c, 0x0, ...)
    /root/go/src/github.com/influxdata/kapacitor/services/task_store/service.go:965 +0x15a
    github.com/influxdata/kapacitor/services/task_store.(*Service).handleTask(0xc82017f0e0, 0x7fbb72003320, 0xc8210c5800, 0xc822f0ee00)
    /root/go/src/github.com/influxdata/kapacitor/services/task_store/service.go:464 +0x665
    github.com/influxdata/kapacitor/services/task_store.(*Service).(github.com/influxdata/kapacitor/services/task_store.handleTask)-fm(0x7fbb72003320, 0xc8210c5800, 0xc822f0ee00)
    /root/go/src/github.com/influxdata/kapacitor/services/task_store/service.go:95 +0x3e
    net/http.HandlerFunc.ServeHTTP(0xc82000e850, 0x7fbb72003320, 0xc8210c5800, 0xc822f0ee00)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.jsonContent.func1(0x7fbb72003320, 0xc8210c5800, 0xc822f0ee00)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:570 +0xaa
    net/http.HandlerFunc.ServeHTTP(0xc820012860, 0x7fbb72003320, 0xc8210c5800, 0xc822f0ee00)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.gzipFilter.func1(0x7fbb72003298, 0xc8210c57c0, 0xc822f0ee00)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:563 +0x2af
    net/http.HandlerFunc.ServeHTTP(0xc820012880, 0x7fbb72003298, 0xc8210c57c0, 0xc822f0ee00)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.versionHeader.func1(0x7fbb72003298, 0xc8210c57c0, 0xc822f0ee00)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:579 +0xc0
    net/http.HandlerFunc.ServeHTTP(0xc8200128a0, 0x7fbb72003298, 0xc8210c57c0, 0xc822f0ee00)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.cors.func1(0x7fbb72003298, 0xc8210c57c0, 0xc822f0ee00)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:612 +0x329
    net/http.HandlerFunc.ServeHTTP(0xc8200128c0, 0x7fbb72003298, 0xc8210c57c0, 0xc822f0ee00)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.requestID.func1(0x7fbb72003298, 0xc8210c57c0, 0xc822f0ee00)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:622 +0x15d
    net/http.HandlerFunc.ServeHTTP(0xc8200128e0, 0x7fbb72003298, 0xc8210c57c0, 0xc822f0ee00)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.recovery.func1(0x7fbb72003238, 0xc820991ad0, 0xc822f0ee00)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:639 +0x114
    net/http.HandlerFunc.ServeHTTP(0xc820012900, 0x7fbb72003238, 0xc820991ad0, 0xc822f0ee00)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.(*ServeMux).ServeHTTP(0xc8201cd9b0, 0x7fbb72003238, 0xc820991ad0, 0xc822f0ee00)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/mux.go:163 +0x17d
    github.com/influxdata/kapacitor/services/httpd.(*Handler).ServeHTTP(0xc8201c8230, 0x7fbb72003238, 0xc820991ad0, 0xc822f0ee00)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:306 +0xe5
    net/http.serverHandler.ServeHTTP(0xc8201d6680, 0x7fbb72003238, 0xc820991ad0, 0xc822f0ee00)
    /usr/local/go/src/net/http/server.go:2081 +0x19e
    net/http.(*conn).serve(0xc820670e00)
    /usr/local/go/src/net/http/server.go:1472 +0xf2e
    created by net/http.(*Server).Serve
    /usr/local/go/src/net/http/server.go:2137 +0x44e

goroutine 55222 [select, 8 minutes]:
    github.com/influxdata/kapacitor.(*Edge).CollectPoint(0xc8201c81c0, 0xc821c7cbe8, 0x4, 0xc8205a7fac, 0x8, 0xc8205a7fc5, 0x7, 0xe56498, 0x0, 0x0, ...)
    /root/go/src/github.com/influxdata/kapacitor/edge.go:171 +0x1c6
    github.com/influxdata/kapacitor.(*TaskMaster).WritePoints(0xc820064c00, 0xc8205a7fac, 0x8, 0xc8205a7fc5, 0x7, 0x3, 0xc8200ccf00, 0xd, 0x10, 0x0, ...)
    /root/go/src/github.com/influxdata/kapacitor/task_master.go:533 +0x294
    github.com/influxdata/kapacitor/services/httpd.(*Handler).serveWriteLine(0xc8201c8230, 0x7fbb72003320, 0xc82054c600, 0xc8205870a0, 0xc8206ad000, 0xb4f, 0xe00, 0x0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:430 +0x862
    github.com/influxdata/kapacitor/services/httpd.(*Handler).serveWrite(0xc8201c8230, 0x7fbb72003320, 0xc82054c600, 0xc8205870a0, 0x0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:393 +0x680
    github.com/influxdata/kapacitor/services/httpd.(*Handler).(github.com/influxdata/kapacitor/services/httpd.serveWrite)-fm(0x7fbb72003320, 0xc82054c600, 0xc8205870a0, 0x0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:132 +0x48
    github.com/influxdata/kapacitor/services/httpd.authenticate.func1(0x7fbb72003320, 0xc82054c600, 0xc8205870a0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:532 +0x52
    net/http.HandlerFunc.ServeHTTP(0xc8201dde60, 0x7fbb72003320, 0xc82054c600, 0xc8205870a0)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.jsonContent.func1(0x7fbb72003320, 0xc82054c600, 0xc8205870a0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:570 +0xaa
    net/http.HandlerFunc.ServeHTTP(0xc8201dde80, 0x7fbb72003320, 0xc82054c600, 0xc8205870a0)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.gzipFilter.func1(0x7fbb72003298, 0xc82054c5c0, 0xc8205870a0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:563 +0x2af
    net/http.HandlerFunc.ServeHTTP(0xc8201ddea0, 0x7fbb72003298, 0xc82054c5c0, 0xc8205870a0)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.versionHeader.func1(0x7fbb72003298, 0xc82054c5c0, 0xc8205870a0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:579 +0xc0
    net/http.HandlerFunc.ServeHTTP(0xc8201ddec0, 0x7fbb72003298, 0xc82054c5c0, 0xc8205870a0)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.cors.func1(0x7fbb72003298, 0xc82054c5c0, 0xc8205870a0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:612 +0x329
    net/http.HandlerFunc.ServeHTTP(0xc8201ddee0, 0x7fbb72003298, 0xc82054c5c0, 0xc8205870a0)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.requestID.func1(0x7fbb72003298, 0xc82054c5c0, 0xc8205870a0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:622 +0x15d
    net/http.HandlerFunc.ServeHTTP(0xc8201ddf00, 0x7fbb72003298, 0xc82054c5c0, 0xc8205870a0)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.recovery.func1(0x7fbb72003238, 0xc820990270, 0xc8205870a0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:639 +0x114
    net/http.HandlerFunc.ServeHTTP(0xc8201ddf20, 0x7fbb72003238, 0xc820990270, 0xc8205870a0)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.(*ServeMux).ServeHTTP(0xc8201cda10, 0x7fbb72003238, 0xc820990270, 0xc8205870a0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/mux.go:163 +0x17d
    github.com/influxdata/kapacitor/services/httpd.(*Handler).ServeHTTP(0xc8201c8230, 0x7fbb72003238, 0xc820990270, 0xc8205870a0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:306 +0xe5
    net/http.serverHandler.ServeHTTP(0xc8201d6680, 0x7fbb72003238, 0xc820990270, 0xc8205870a0)
    /usr/local/go/src/net/http/server.go:2081 +0x19e
    net/http.(*conn).serve(0xc820670700)
    /usr/local/go/src/net/http/server.go:1472 +0xf2e
    created by net/http.(*Server).Serve
    /usr/local/go/src/net/http/server.go:2137 +0x44e

goroutine 55182 [select, 10 minutes]:
    github.com/influxdata/kapacitor.(*Edge).CollectPoint(0xc8201c81c0, 0xc82024c778, 0x8, 0xc8205a692c, 0x9, 0xc8205a6946, 0x7, 0xe56498, 0x0, 0x0, ...)
    /root/go/src/github.com/influxdata/kapacitor/edge.go:171 +0x1c6
    github.com/influxdata/kapacitor.(*TaskMaster).WritePoints(0xc820064c00, 0xc8205a692c, 0x9, 0xc8205a6946, 0x7, 0x3, 0xc821250000, 0x69, 0x80, 0x0, ...)
    /root/go/src/github.com/influxdata/kapacitor/task_master.go:533 +0x294
    github.com/influxdata/kapacitor/services/httpd.(*Handler).serveWriteLine(0xc8201c8230, 0x7fbb72003320, 0xc820556860, 0xc820235960, 0xc820722000, 0x554e, 0x7e00, 0x0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:430 +0x862
    github.com/influxdata/kapacitor/services/httpd.(*Handler).serveWrite(0xc8201c8230, 0x7fbb72003320, 0xc820556860, 0xc820235960, 0x0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:393 +0x680
    github.com/influxdata/kapacitor/services/httpd.(*Handler).(github.com/influxdata/kapacitor/services/httpd.serveWrite)-fm(0x7fbb72003320, 0xc820556860, 0xc820235960, 0x0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:132 +0x48
    github.com/influxdata/kapacitor/services/httpd.authenticate.func1(0x7fbb72003320, 0xc820556860, 0xc820235960)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:532 +0x52
    net/http.HandlerFunc.ServeHTTP(0xc8201dde60, 0x7fbb72003320, 0xc820556860, 0xc820235960)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.jsonContent.func1(0x7fbb72003320, 0xc820556860, 0xc820235960)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:570 +0xaa
    net/http.HandlerFunc.ServeHTTP(0xc8201dde80, 0x7fbb72003320, 0xc820556860, 0xc820235960)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.gzipFilter.func1(0x7fbb72003298, 0xc820556800, 0xc820235960)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:563 +0x2af
    net/http.HandlerFunc.ServeHTTP(0xc8201ddea0, 0x7fbb72003298, 0xc820556800, 0xc820235960)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.versionHeader.func1(0x7fbb72003298, 0xc820556800, 0xc820235960)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:579 +0xc0
    net/http.HandlerFunc.ServeHTTP(0xc8201ddec0, 0x7fbb72003298, 0xc820556800, 0xc820235960)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.cors.func1(0x7fbb72003298, 0xc820556800, 0xc820235960)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:612 +0x329
    net/http.HandlerFunc.ServeHTTP(0xc8201ddee0, 0x7fbb72003298, 0xc820556800, 0xc820235960)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.requestID.func1(0x7fbb72003298, 0xc820556800, 0xc820235960)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:622 +0x15d
    net/http.HandlerFunc.ServeHTTP(0xc8201ddf00, 0x7fbb72003298, 0xc820556800, 0xc820235960)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.recovery.func1(0x7fbb72003238, 0xc820159930, 0xc820235960)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:639 +0x114
    net/http.HandlerFunc.ServeHTTP(0xc8201ddf20, 0x7fbb72003238, 0xc820159930, 0xc820235960)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.(*ServeMux).ServeHTTP(0xc8201cda10, 0x7fbb72003238, 0xc820159930, 0xc820235960)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/mux.go:163 +0x17d
    github.com/influxdata/kapacitor/services/httpd.(*Handler).ServeHTTP(0xc8201c8230, 0x7fbb72003238, 0xc820159930, 0xc820235960)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:306 +0xe5
    net/http.serverHandler.ServeHTTP(0xc8201d6680, 0x7fbb72003238, 0xc820159930, 0xc820235960)
    /usr/local/go/src/net/http/server.go:2081 +0x19e
    net/http.(*conn).serve(0xc82090e380)
    /usr/local/go/src/net/http/server.go:1472 +0xf2e
    created by net/http.(*Server).Serve
    /usr/local/go/src/net/http/server.go:2137 +0x44e

goroutine 9 [chan receive, 2 minutes]:
    github.com/influxdata/kapacitor/services/influxdb.(*influxdb).Open.func1(0xc8201a7e60)
    /root/go/src/github.com/influxdata/kapacitor/services/influxdb/service.go:205 +0x5b
    created by github.com/influxdata/kapacitor/services/influxdb.(*influxdb).Open
    /root/go/src/github.com/influxdata/kapacitor/services/influxdb/service.go:208 +0x11a

goroutine 55361 [select, 2 minutes]:
    github.com/influxdata/kapacitor.(*Edge).CollectPoint(0xc8201c81c0, 0xc82053a7f4, 0x6, 0xc82066fcdc, 0x8, 0xc82066fcf5, 0x7, 0xe56498, 0x0, 0x0, ...)
    /root/go/src/github.com/influxdata/kapacitor/edge.go:171 +0x1c6
    github.com/influxdata/kapacitor.(*TaskMaster).WritePoints(0xc820064c00, 0xc82066fcdc, 0x8, 0xc82066fcf5, 0x7, 0x3, 0xc8208a3d80, 0x5, 0x8, 0x0, ...)
    /root/go/src/github.com/influxdata/kapacitor/task_master.go:533 +0x294
    github.com/influxdata/kapacitor/services/httpd.(*Handler).serveWriteLine(0xc8201c8230, 0x7fbb72003320, 0xc82054cf60, 0xc822f0fa40, 0xc8202f7200, 0x3f9, 0x600, 0x0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:430 +0x862
    github.com/influxdata/kapacitor/services/httpd.(*Handler).serveWrite(0xc8201c8230, 0x7fbb72003320, 0xc82054cf60, 0xc822f0fa40, 0x0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:393 +0x680
    github.com/influxdata/kapacitor/services/httpd.(*Handler).(github.com/influxdata/kapacitor/services/httpd.serveWrite)-fm(0x7fbb72003320, 0xc82054cf60, 0xc822f0fa40, 0x0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:132 +0x48
    github.com/influxdata/kapacitor/services/httpd.authenticate.func1(0x7fbb72003320, 0xc82054cf60, 0xc822f0fa40)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:532 +0x52
    net/http.HandlerFunc.ServeHTTP(0xc8201dde60, 0x7fbb72003320, 0xc82054cf60, 0xc822f0fa40)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.jsonContent.func1(0x7fbb72003320, 0xc82054cf60, 0xc822f0fa40)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:570 +0xaa
    net/http.HandlerFunc.ServeHTTP(0xc8201dde80, 0x7fbb72003320, 0xc82054cf60, 0xc822f0fa40)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.gzipFilter.func1(0x7fbb72003298, 0xc82054cee0, 0xc822f0fa40)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:563 +0x2af
    net/http.HandlerFunc.ServeHTTP(0xc8201ddea0, 0x7fbb72003298, 0xc82054cee0, 0xc822f0fa40)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.versionHeader.func1(0x7fbb72003298, 0xc82054cee0, 0xc822f0fa40)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:579 +0xc0
    net/http.HandlerFunc.ServeHTTP(0xc8201ddec0, 0x7fbb72003298, 0xc82054cee0, 0xc822f0fa40)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.cors.func1(0x7fbb72003298, 0xc82054cee0, 0xc822f0fa40)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:612 +0x329
    net/http.HandlerFunc.ServeHTTP(0xc8201ddee0, 0x7fbb72003298, 0xc82054cee0, 0xc822f0fa40)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.requestID.func1(0x7fbb72003298, 0xc82054cee0, 0xc822f0fa40)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:622 +0x15d
    net/http.HandlerFunc.ServeHTTP(0xc8201ddf00, 0x7fbb72003298, 0xc82054cee0, 0xc822f0fa40)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.recovery.func1(0x7fbb72003238, 0xc820371a00, 0xc822f0fa40)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:639 +0x114
    net/http.HandlerFunc.ServeHTTP(0xc8201ddf20, 0x7fbb72003238, 0xc820371a00, 0xc822f0fa40)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.(*ServeMux).ServeHTTP(0xc8201cda10, 0x7fbb72003238, 0xc820371a00, 0xc822f0fa40)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/mux.go:163 +0x17d
    github.com/influxdata/kapacitor/services/httpd.(*Handler).ServeHTTP(0xc8201c8230, 0x7fbb72003238, 0xc820371a00, 0xc822f0fa40)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:306 +0xe5
    net/http.serverHandler.ServeHTTP(0xc8201d6680, 0x7fbb72003238, 0xc820371a00, 0xc822f0fa40)
    /usr/local/go/src/net/http/server.go:2081 +0x19e
    net/http.(*conn).serve(0xc820671a00)
    /usr/local/go/src/net/http/server.go:1472 +0xf2e
    created by net/http.(*Server).Serve
    /usr/local/go/src/net/http/server.go:2137 +0x44e

goroutine 55161 [select, 10 minutes]:
    github.com/influxdata/kapacitor.(*Edge).NextPoint(0xc82004c460, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
    /root/go/src/github.com/influxdata/kapacitor/edge.go:145 +0x1d5
    github.com/influxdata/kapacitor.(*InfluxQLNode).runStreamInfluxQL(0xc8200ccc00, 0x0, 0x0)
    /root/go/src/github.com/influxdata/kapacitor/influxql.go:118 +0xc6a
    github.com/influxdata/kapacitor.(*InfluxQLNode).runInfluxQLs(0xc8200ccc00, 0x0, 0x0, 0x0, 0x0, 0x0)
    /root/go/src/github.com/influxdata/kapacitor/influxql.go:40 +0x55
    github.com/influxdata/kapacitor.(*InfluxQLNode).(github.com/influxdata/kapacitor.runInfluxQLs)-fm(0x0, 0x0, 0x0, 0x0, 0x0)
    /root/go/src/github.com/influxdata/kapacitor/influxql.go:33 +0x4c
    github.com/influxdata/kapacitor.(*node).start.func1(0xc8200ccc00, 0x0, 0x0, 0x0)
    /root/go/src/github.com/influxdata/kapacitor/node.go:130 +0x7f
    created by github.com/influxdata/kapacitor.(*node).start
    /root/go/src/github.com/influxdata/kapacitor/node.go:131 +0x53

goroutine 55327 [select, 4 minutes]:
    github.com/influxdata/kapacitor.(*Edge).CollectPoint(0xc8201c81c0, 0xc8205148c0, 0x8, 0xc8205a750c, 0x9, 0xc8205a7526, 0x7, 0xe56498, 0x0, 0x0, ...)
    /root/go/src/github.com/influxdata/kapacitor/edge.go:171 +0x1c6
    github.com/influxdata/kapacitor.(*TaskMaster).WritePoints(0xc820064c00, 0xc8205a750c, 0x9, 0xc8205a7526, 0x7, 0x3, 0xc8209bb000, 0x69, 0x80, 0x0, ...)
    /root/go/src/github.com/influxdata/kapacitor/task_master.go:533 +0x294
    github.com/influxdata/kapacitor/services/httpd.(*Handler).serveWriteLine(0xc8201c8230, 0x7fbb72003320, 0xc820598cc0, 0xc820510e00, 0xc8209ca000, 0x554e, 0x7e00, 0x0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:430 +0x862
    github.com/influxdata/kapacitor/services/httpd.(*Handler).serveWrite(0xc8201c8230, 0x7fbb72003320, 0xc820598cc0, 0xc820510e00, 0x0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:393 +0x680
    github.com/influxdata/kapacitor/services/httpd.(*Handler).(github.com/influxdata/kapacitor/services/httpd.serveWrite)-fm(0x7fbb72003320, 0xc820598cc0, 0xc820510e00, 0x0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:132 +0x48
    github.com/influxdata/kapacitor/services/httpd.authenticate.func1(0x7fbb72003320, 0xc820598cc0, 0xc820510e00)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:532 +0x52
    net/http.HandlerFunc.ServeHTTP(0xc8201dde60, 0x7fbb72003320, 0xc820598cc0, 0xc820510e00)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.jsonContent.func1(0x7fbb72003320, 0xc820598cc0, 0xc820510e00)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:570 +0xaa
    net/http.HandlerFunc.ServeHTTP(0xc8201dde80, 0x7fbb72003320, 0xc820598cc0, 0xc820510e00)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.gzipFilter.func1(0x7fbb72003298, 0xc820598c80, 0xc820510e00)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:563 +0x2af
    net/http.HandlerFunc.ServeHTTP(0xc8201ddea0, 0x7fbb72003298, 0xc820598c80, 0xc820510e00)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.versionHeader.func1(0x7fbb72003298, 0xc820598c80, 0xc820510e00)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:579 +0xc0
    net/http.HandlerFunc.ServeHTTP(0xc8201ddec0, 0x7fbb72003298, 0xc820598c80, 0xc820510e00)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.cors.func1(0x7fbb72003298, 0xc820598c80, 0xc820510e00)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:612 +0x329
    net/http.HandlerFunc.ServeHTTP(0xc8201ddee0, 0x7fbb72003298, 0xc820598c80, 0xc820510e00)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.requestID.func1(0x7fbb72003298, 0xc820598c80, 0xc820510e00)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:622 +0x15d
    net/http.HandlerFunc.ServeHTTP(0xc8201ddf00, 0x7fbb72003298, 0xc820598c80, 0xc820510e00)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.recovery.func1(0x7fbb72003238, 0xc820991e10, 0xc820510e00)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:639 +0x114
    net/http.HandlerFunc.ServeHTTP(0xc8201ddf20, 0x7fbb72003238, 0xc820991e10, 0xc820510e00)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.(*ServeMux).ServeHTTP(0xc8201cda10, 0x7fbb72003238, 0xc820991e10, 0xc820510e00)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/mux.go:163 +0x17d
    github.com/influxdata/kapacitor/services/httpd.(*Handler).ServeHTTP(0xc8201c8230, 0x7fbb72003238, 0xc820991e10, 0xc820510e00)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:306 +0xe5
    net/http.serverHandler.ServeHTTP(0xc8201d6680, 0x7fbb72003238, 0xc820991e10, 0xc820510e00)
    /usr/local/go/src/net/http/server.go:2081 +0x19e
    net/http.(*conn).serve(0xc820671800)
    /usr/local/go/src/net/http/server.go:1472 +0xf2e
    created by net/http.(*Server).Serve
    /usr/local/go/src/net/http/server.go:2137 +0x44e

goroutine 55342 [select]:
    github.com/influxdata/kapacitor.(*Edge).CollectPoint(0xc8201c81c0, 0xc820522950, 0x8, 0xc82066f6ec, 0x9, 0xc82066f706, 0x7, 0xe56498, 0x0, 0x0, ...)
    /root/go/src/github.com/influxdata/kapacitor/edge.go:171 +0x1c6
    github.com/influxdata/kapacitor.(*TaskMaster).WritePoints(0xc820064c00, 0xc82066f6ec, 0x9, 0xc82066f706, 0x7, 0x3, 0xc822f0a800, 0x69, 0x80, 0x0, ...)
    /root/go/src/github.com/influxdata/kapacitor/task_master.go:533 +0x294
    github.com/influxdata/kapacitor/services/httpd.(*Handler).serveWriteLine(0xc8201c8230, 0x7fbb72003320, 0xc8206651a0, 0xc820510700, 0xc82087c000, 0x554e, 0x7e00, 0x0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:430 +0x862
    github.com/influxdata/kapacitor/services/httpd.(*Handler).serveWrite(0xc8201c8230, 0x7fbb72003320, 0xc8206651a0, 0xc820510700, 0x0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:393 +0x680
    github.com/influxdata/kapacitor/services/httpd.(*Handler).(github.com/influxdata/kapacitor/services/httpd.serveWrite)-fm(0x7fbb72003320, 0xc8206651a0, 0xc820510700, 0x0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:132 +0x48
    github.com/influxdata/kapacitor/services/httpd.authenticate.func1(0x7fbb72003320, 0xc8206651a0, 0xc820510700)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:532 +0x52
    net/http.HandlerFunc.ServeHTTP(0xc8201dde60, 0x7fbb72003320, 0xc8206651a0, 0xc820510700)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.jsonContent.func1(0x7fbb72003320, 0xc8206651a0, 0xc820510700)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:570 +0xaa
    net/http.HandlerFunc.ServeHTTP(0xc8201dde80, 0x7fbb72003320, 0xc8206651a0, 0xc820510700)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.gzipFilter.func1(0x7fbb72003298, 0xc8206650e0, 0xc820510700)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:563 +0x2af
    net/http.HandlerFunc.ServeHTTP(0xc8201ddea0, 0x7fbb72003298, 0xc8206650e0, 0xc820510700)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.versionHeader.func1(0x7fbb72003298, 0xc8206650e0, 0xc820510700)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:579 +0xc0
    net/http.HandlerFunc.ServeHTTP(0xc8201ddec0, 0x7fbb72003298, 0xc8206650e0, 0xc820510700)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.cors.func1(0x7fbb72003298, 0xc8206650e0, 0xc820510700)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:612 +0x329
    net/http.HandlerFunc.ServeHTTP(0xc8201ddee0, 0x7fbb72003298, 0xc8206650e0, 0xc820510700)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.requestID.func1(0x7fbb72003298, 0xc8206650e0, 0xc820510700)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:622 +0x15d
    net/http.HandlerFunc.ServeHTTP(0xc8201ddf00, 0x7fbb72003298, 0xc8206650e0, 0xc820510700)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.recovery.func1(0x7fbb72003238, 0xc820371e10, 0xc820510700)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:639 +0x114
    net/http.HandlerFunc.ServeHTTP(0xc8201ddf20, 0x7fbb72003238, 0xc820371e10, 0xc820510700)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.(*ServeMux).ServeHTTP(0xc8201cda10, 0x7fbb72003238, 0xc820371e10, 0xc820510700)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/mux.go:163 +0x17d
    github.com/influxdata/kapacitor/services/httpd.(*Handler).ServeHTTP(0xc8201c8230, 0x7fbb72003238, 0xc820371e10, 0xc820510700)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:306 +0xe5
    net/http.serverHandler.ServeHTTP(0xc8201d6680, 0x7fbb72003238, 0xc820371e10, 0xc820510700)
    /usr/local/go/src/net/http/server.go:2081 +0x19e
    net/http.(*conn).serve(0xc8208a2f00)
    /usr/local/go/src/net/http/server.go:1472 +0xf2e
    created by net/http.(*Server).Serve
    /usr/local/go/src/net/http/server.go:2137 +0x44e

goroutine 55270 [select, 6 minutes]:
    github.com/influxdata/kapacitor.(*Edge).CollectPoint(0xc8201c81c0, 0xc820a8cef8, 0x6, 0xc8205977dc, 0x8, 0xc8205977f5, 0x7, 0xe56498, 0x0, 0x0, ...)
    /root/go/src/github.com/influxdata/kapacitor/edge.go:171 +0x1c6
    github.com/influxdata/kapacitor.(*TaskMaster).WritePoints(0xc820064c00, 0xc8205977dc, 0x8, 0xc8205977f5, 0x7, 0x3, 0xc8201d0c00, 0x17, 0x20, 0x0, ...)
    /root/go/src/github.com/influxdata/kapacitor/task_master.go:533 +0x294
    github.com/influxdata/kapacitor/services/httpd.(*Handler).serveWriteLine(0xc8201c8230, 0x7fbb72003320, 0xc8210c5a40, 0xc822f0e7e0, 0xc8207bc000, 0x15fc, 0x1e00, 0x0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:430 +0x862
    github.com/influxdata/kapacitor/services/httpd.(*Handler).serveWrite(0xc8201c8230, 0x7fbb72003320, 0xc8210c5a40, 0xc822f0e7e0, 0x0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:393 +0x680
    github.com/influxdata/kapacitor/services/httpd.(*Handler).(github.com/influxdata/kapacitor/services/httpd.serveWrite)-fm(0x7fbb72003320, 0xc8210c5a40, 0xc822f0e7e0, 0x0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:132 +0x48
    github.com/influxdata/kapacitor/services/httpd.authenticate.func1(0x7fbb72003320, 0xc8210c5a40, 0xc822f0e7e0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:532 +0x52
    net/http.HandlerFunc.ServeHTTP(0xc8201dde60, 0x7fbb72003320, 0xc8210c5a40, 0xc822f0e7e0)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.jsonContent.func1(0x7fbb72003320, 0xc8210c5a40, 0xc822f0e7e0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:570 +0xaa
    net/http.HandlerFunc.ServeHTTP(0xc8201dde80, 0x7fbb72003320, 0xc8210c5a40, 0xc822f0e7e0)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.gzipFilter.func1(0x7fbb72003298, 0xc8210c5a00, 0xc822f0e7e0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:563 +0x2af
    net/http.HandlerFunc.ServeHTTP(0xc8201ddea0, 0x7fbb72003298, 0xc8210c5a00, 0xc822f0e7e0)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.versionHeader.func1(0x7fbb72003298, 0xc8210c5a00, 0xc822f0e7e0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:579 +0xc0
    net/http.HandlerFunc.ServeHTTP(0xc8201ddec0, 0x7fbb72003298, 0xc8210c5a00, 0xc822f0e7e0)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.cors.func1(0x7fbb72003298, 0xc8210c5a00, 0xc822f0e7e0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:612 +0x329
    net/http.HandlerFunc.ServeHTTP(0xc8201ddee0, 0x7fbb72003298, 0xc8210c5a00, 0xc822f0e7e0)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.requestID.func1(0x7fbb72003298, 0xc8210c5a00, 0xc822f0e7e0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:622 +0x15d
    net/http.HandlerFunc.ServeHTTP(0xc8201ddf00, 0x7fbb72003298, 0xc8210c5a00, 0xc822f0e7e0)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.recovery.func1(0x7fbb72003238, 0xc8209912b0, 0xc822f0e7e0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:639 +0x114
    net/http.HandlerFunc.ServeHTTP(0xc8201ddf20, 0x7fbb72003238, 0xc8209912b0, 0xc822f0e7e0)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.(*ServeMux).ServeHTTP(0xc8201cda10, 0x7fbb72003238, 0xc8209912b0, 0xc822f0e7e0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/mux.go:163 +0x17d
    github.com/influxdata/kapacitor/services/httpd.(*Handler).ServeHTTP(0xc8201c8230, 0x7fbb72003238, 0xc8209912b0, 0xc822f0e7e0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:306 +0xe5
    net/http.serverHandler.ServeHTTP(0xc8201d6680, 0x7fbb72003238, 0xc8209912b0, 0xc822f0e7e0)
    /usr/local/go/src/net/http/server.go:2081 +0x19e
    net/http.(*conn).serve(0xc8208a2780)
    /usr/local/go/src/net/http/server.go:1472 +0xf2e
    created by net/http.(*Server).Serve
    /usr/local/go/src/net/http/server.go:2137 +0x44e

goroutine 55302 [semacquire, 4 minutes]:
    sync.runtime_Semacquire(0xc820064d60)
    /usr/local/go/src/runtime/sema.go:47 +0x26
    sync.(*RWMutex).RLock(0xc820064d54)
    /usr/local/go/src/sync/rwmutex.go:37 +0x58
    github.com/influxdata/kapacitor.(*TaskMaster).IsExecuting(0xc820064c00, 0xc82076a050, 0x9, 0x600)
    /root/go/src/github.com/influxdata/kapacitor/task_master.go:422 +0x3d
    github.com/influxdata/kapacitor/services/task_store.(*Service).convertTask(0xc82017f0e0, 0xc82076a050, 0x9, 0x0, 0xc8205abb20, 0x1, 0x1, 0xc820072700, 0x67a, 0x0, ...)
    /root/go/src/github.com/influxdata/kapacitor/services/task_store/service.go:965 +0x15a
    github.com/influxdata/kapacitor/services/task_store.(*Service).handleTask(0xc82017f0e0, 0x7fbb72003320, 0xc8210c5660, 0xc822f0ed20)
    /root/go/src/github.com/influxdata/kapacitor/services/task_store/service.go:464 +0x665
    github.com/influxdata/kapacitor/services/task_store.(*Service).(github.com/influxdata/kapacitor/services/task_store.handleTask)-fm(0x7fbb72003320, 0xc8210c5660, 0xc822f0ed20)
    /root/go/src/github.com/influxdata/kapacitor/services/task_store/service.go:95 +0x3e
    net/http.HandlerFunc.ServeHTTP(0xc82000e850, 0x7fbb72003320, 0xc8210c5660, 0xc822f0ed20)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.jsonContent.func1(0x7fbb72003320, 0xc8210c5660, 0xc822f0ed20)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:570 +0xaa
    net/http.HandlerFunc.ServeHTTP(0xc820012860, 0x7fbb72003320, 0xc8210c5660, 0xc822f0ed20)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.gzipFilter.func1(0x7fbb72003298, 0xc8210c55e0, 0xc822f0ed20)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:563 +0x2af
    net/http.HandlerFunc.ServeHTTP(0xc820012880, 0x7fbb72003298, 0xc8210c55e0, 0xc822f0ed20)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.versionHeader.func1(0x7fbb72003298, 0xc8210c55e0, 0xc822f0ed20)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:579 +0xc0
    net/http.HandlerFunc.ServeHTTP(0xc8200128a0, 0x7fbb72003298, 0xc8210c55e0, 0xc822f0ed20)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.cors.func1(0x7fbb72003298, 0xc8210c55e0, 0xc822f0ed20)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:612 +0x329
    net/http.HandlerFunc.ServeHTTP(0xc8200128c0, 0x7fbb72003298, 0xc8210c55e0, 0xc822f0ed20)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.requestID.func1(0x7fbb72003298, 0xc8210c55e0, 0xc822f0ed20)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:622 +0x15d
    net/http.HandlerFunc.ServeHTTP(0xc8200128e0, 0x7fbb72003298, 0xc8210c55e0, 0xc822f0ed20)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.recovery.func1(0x7fbb72003238, 0xc820370dd0, 0xc822f0ed20)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:639 +0x114
    net/http.HandlerFunc.ServeHTTP(0xc820012900, 0x7fbb72003238, 0xc820370dd0, 0xc822f0ed20)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.(*ServeMux).ServeHTTP(0xc8201cd9b0, 0x7fbb72003238, 0xc820370dd0, 0xc822f0ed20)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/mux.go:163 +0x17d
    github.com/influxdata/kapacitor/services/httpd.(*Handler).ServeHTTP(0xc8201c8230, 0x7fbb72003238, 0xc820370dd0, 0xc822f0ed20)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:306 +0xe5
    net/http.serverHandler.ServeHTTP(0xc8201d6680, 0x7fbb72003238, 0xc820370dd0, 0xc822f0ed20)
    /usr/local/go/src/net/http/server.go:2081 +0x19e
    net/http.(*conn).serve(0xc8208a2700)
    /usr/local/go/src/net/http/server.go:1472 +0xf2e
    created by net/http.(*Server).Serve
    /usr/local/go/src/net/http/server.go:2137 +0x44e

goroutine 55329 [select, 2 minutes]:
    github.com/influxdata/kapacitor.(*Edge).CollectPoint(0xc8201c81c0, 0xc820a8c920, 0x4, 0xc8205a6d3c, 0x8, 0xc8205a6d55, 0x7, 0xe56498, 0x0, 0x0, ...)
    /root/go/src/github.com/influxdata/kapacitor/edge.go:171 +0x1c6
    github.com/influxdata/kapacitor.(*TaskMaster).WritePoints(0xc820064c00, 0xc8205a6d3c, 0x8, 0xc8205a6d55, 0x7, 0x3, 0xc8201d0600, 0x14, 0x20, 0x0, ...)
    /root/go/src/github.com/influxdata/kapacitor/task_master.go:533 +0x294
    github.com/influxdata/kapacitor/services/httpd.(*Handler).serveWriteLine(0xc8201c8230, 0x7fbb72003320, 0xc8210c5280, 0xc8205875e0, 0xc820250000, 0x1378, 0x1e00, 0x0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:430 +0x862
    github.com/influxdata/kapacitor/services/httpd.(*Handler).serveWrite(0xc8201c8230, 0x7fbb72003320, 0xc8210c5280, 0xc8205875e0, 0x0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:393 +0x680
    github.com/influxdata/kapacitor/services/httpd.(*Handler).(github.com/influxdata/kapacitor/services/httpd.serveWrite)-fm(0x7fbb72003320, 0xc8210c5280, 0xc8205875e0, 0x0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:132 +0x48
    github.com/influxdata/kapacitor/services/httpd.authenticate.func1(0x7fbb72003320, 0xc8210c5280, 0xc8205875e0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:532 +0x52
    net/http.HandlerFunc.ServeHTTP(0xc8201dde60, 0x7fbb72003320, 0xc8210c5280, 0xc8205875e0)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.jsonContent.func1(0x7fbb72003320, 0xc8210c5280, 0xc8205875e0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:570 +0xaa
    net/http.HandlerFunc.ServeHTTP(0xc8201dde80, 0x7fbb72003320, 0xc8210c5280, 0xc8205875e0)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.gzipFilter.func1(0x7fbb72003298, 0xc8210c5240, 0xc8205875e0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:563 +0x2af
    net/http.HandlerFunc.ServeHTTP(0xc8201ddea0, 0x7fbb72003298, 0xc8210c5240, 0xc8205875e0)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.versionHeader.func1(0x7fbb72003298, 0xc8210c5240, 0xc8205875e0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:579 +0xc0
    net/http.HandlerFunc.ServeHTTP(0xc8201ddec0, 0x7fbb72003298, 0xc8210c5240, 0xc8205875e0)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.cors.func1(0x7fbb72003298, 0xc8210c5240, 0xc8205875e0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:612 +0x329
    net/http.HandlerFunc.ServeHTTP(0xc8201ddee0, 0x7fbb72003298, 0xc8210c5240, 0xc8205875e0)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.requestID.func1(0x7fbb72003298, 0xc8210c5240, 0xc8205875e0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:622 +0x15d
    net/http.HandlerFunc.ServeHTTP(0xc8201ddf00, 0x7fbb72003298, 0xc8210c5240, 0xc8205875e0)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.recovery.func1(0x7fbb72003238, 0xc8209916c0, 0xc8205875e0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:639 +0x114
    net/http.HandlerFunc.ServeHTTP(0xc8201ddf20, 0x7fbb72003238, 0xc8209916c0, 0xc8205875e0)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.(*ServeMux).ServeHTTP(0xc8201cda10, 0x7fbb72003238, 0xc8209916c0, 0xc8205875e0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/mux.go:163 +0x17d
    github.com/influxdata/kapacitor/services/httpd.(*Handler).ServeHTTP(0xc8201c8230, 0x7fbb72003238, 0xc8209916c0, 0xc8205875e0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:306 +0xe5
    net/http.serverHandler.ServeHTTP(0xc8201d6680, 0x7fbb72003238, 0xc8209916c0, 0xc8205875e0)
    /usr/local/go/src/net/http/server.go:2081 +0x19e
    net/http.(*conn).serve(0xc8208a2680)
    /usr/local/go/src/net/http/server.go:1472 +0xf2e
    created by net/http.(*Server).Serve
    /usr/local/go/src/net/http/server.go:2137 +0x44e

goroutine 47248 [chan send, 247 minutes]:
    github.com/influxdata/kapacitor/tick/ast.lexToken(0xc820866630, 0x1024920)
    /root/go/src/github.com/influxdata/kapacitor/tick/ast/lex.go:343 +0x335
    github.com/influxdata/kapacitor/tick/ast.(*lexer).run(0xc820866630)
    /root/go/src/github.com/influxdata/kapacitor/tick/ast/lex.go:238 +0x32
    created by github.com/influxdata/kapacitor/tick/ast.lex
    /root/go/src/github.com/influxdata/kapacitor/tick/ast/lex.go:230 +0xc1

goroutine 72 [select, 10 minutes]:
    github.com/influxdata/kapacitor.(*Edge).CollectPoint(0xc8201c81c0, 0xc8203d0d00, 0x8, 0xc820597cdc, 0x9, 0xc820597cf6, 0x7, 0xe56498, 0x0, 0x0, ...)
    /root/go/src/github.com/influxdata/kapacitor/edge.go:171 +0x1c6
    github.com/influxdata/kapacitor.(*TaskMaster).WritePoints(0xc820064c00, 0xc820597cdc, 0x9, 0xc820597cf6, 0x7, 0x3, 0xc8201c7800, 0x69, 0x80, 0x0, ...)
    /root/go/src/github.com/influxdata/kapacitor/task_master.go:533 +0x294
    github.com/influxdata/kapacitor/services/httpd.(*Handler).serveWriteLine(0xc8201c8230, 0x7fbb72003320, 0xc8205688c0, 0xc8202717a0, 0xc8252ea000, 0x554e, 0x7e00, 0x0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:430 +0x862
    github.com/influxdata/kapacitor/services/httpd.(*Handler).serveWrite(0xc8201c8230, 0x7fbb72003320, 0xc8205688c0, 0xc8202717a0, 0x0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:393 +0x680
    github.com/influxdata/kapacitor/services/httpd.(*Handler).(github.com/influxdata/kapacitor/services/httpd.serveWrite)-fm(0x7fbb72003320, 0xc8205688c0, 0xc8202717a0, 0x0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:132 +0x48
    github.com/influxdata/kapacitor/services/httpd.authenticate.func1(0x7fbb72003320, 0xc8205688c0, 0xc8202717a0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:532 +0x52
    net/http.HandlerFunc.ServeHTTP(0xc8201dde60, 0x7fbb72003320, 0xc8205688c0, 0xc8202717a0)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.jsonContent.func1(0x7fbb72003320, 0xc8205688c0, 0xc8202717a0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:570 +0xaa
    net/http.HandlerFunc.ServeHTTP(0xc8201dde80, 0x7fbb72003320, 0xc8205688c0, 0xc8202717a0)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.gzipFilter.func1(0x7fbb72003298, 0xc820568880, 0xc8202717a0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:563 +0x2af
    net/http.HandlerFunc.ServeHTTP(0xc8201ddea0, 0x7fbb72003298, 0xc820568880, 0xc8202717a0)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.versionHeader.func1(0x7fbb72003298, 0xc820568880, 0xc8202717a0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:579 +0xc0
    net/http.HandlerFunc.ServeHTTP(0xc8201ddec0, 0x7fbb72003298, 0xc820568880, 0xc8202717a0)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.cors.func1(0x7fbb72003298, 0xc820568880, 0xc8202717a0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:612 +0x329
    net/http.HandlerFunc.ServeHTTP(0xc8201ddee0, 0x7fbb72003298, 0xc820568880, 0xc8202717a0)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.requestID.func1(0x7fbb72003298, 0xc820568880, 0xc8202717a0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:622 +0x15d
    net/http.HandlerFunc.ServeHTTP(0xc8201ddf00, 0x7fbb72003298, 0xc820568880, 0xc8202717a0)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.recovery.func1(0x7fbb72003238, 0xc820158d00, 0xc8202717a0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:639 +0x114
    net/http.HandlerFunc.ServeHTTP(0xc8201ddf20, 0x7fbb72003238, 0xc820158d00, 0xc8202717a0)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.(*ServeMux).ServeHTTP(0xc8201cda10, 0x7fbb72003238, 0xc820158d00, 0xc8202717a0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/mux.go:163 +0x17d
    github.com/influxdata/kapacitor/services/httpd.(*Handler).ServeHTTP(0xc8201c8230, 0x7fbb72003238, 0xc820158d00, 0xc8202717a0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:306 +0xe5
    net/http.serverHandler.ServeHTTP(0xc8201d6680, 0x7fbb72003238, 0xc820158d00, 0xc8202717a0)
    /usr/local/go/src/net/http/server.go:2081 +0x19e
    net/http.(*conn).serve(0xc820686200)
    /usr/local/go/src/net/http/server.go:1472 +0xf2e
    created by net/http.(*Server).Serve
    /usr/local/go/src/net/http/server.go:2137 +0x44e

goroutine 55299 [select, 6 minutes]:
    github.com/influxdata/kapacitor.(*Edge).CollectPoint(0xc8201c81c0, 0xc8206833e0, 0x8, 0xc8205a71ec, 0x9, 0xc8205a7206, 0x7, 0xe56498, 0x0, 0x0, ...)
    /root/go/src/github.com/influxdata/kapacitor/edge.go:171 +0x1c6
    github.com/influxdata/kapacitor.(*TaskMaster).WritePoints(0xc820064c00, 0xc8205a71ec, 0x9, 0xc8205a7206, 0x7, 0x3, 0xc822643000, 0x69, 0x80, 0x0, ...)
    /root/go/src/github.com/influxdata/kapacitor/task_master.go:533 +0x294
    github.com/influxdata/kapacitor/services/httpd.(*Handler).serveWriteLine(0xc8201c8230, 0x7fbb72003320, 0xc82054d920, 0xc822f0eee0, 0xc8207d6000, 0x554e, 0x7e00, 0x0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:430 +0x862
    github.com/influxdata/kapacitor/services/httpd.(*Handler).serveWrite(0xc8201c8230, 0x7fbb72003320, 0xc82054d920, 0xc822f0eee0, 0x0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:393 +0x680
    github.com/influxdata/kapacitor/services/httpd.(*Handler).(github.com/influxdata/kapacitor/services/httpd.serveWrite)-fm(0x7fbb72003320, 0xc82054d920, 0xc822f0eee0, 0x0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:132 +0x48
    github.com/influxdata/kapacitor/services/httpd.authenticate.func1(0x7fbb72003320, 0xc82054d920, 0xc822f0eee0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:532 +0x52
    net/http.HandlerFunc.ServeHTTP(0xc8201dde60, 0x7fbb72003320, 0xc82054d920, 0xc822f0eee0)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.jsonContent.func1(0x7fbb72003320, 0xc82054d920, 0xc822f0eee0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:570 +0xaa
    net/http.HandlerFunc.ServeHTTP(0xc8201dde80, 0x7fbb72003320, 0xc82054d920, 0xc822f0eee0)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.gzipFilter.func1(0x7fbb72003298, 0xc82054d8c0, 0xc822f0eee0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:563 +0x2af
    net/http.HandlerFunc.ServeHTTP(0xc8201ddea0, 0x7fbb72003298, 0xc82054d8c0, 0xc822f0eee0)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.versionHeader.func1(0x7fbb72003298, 0xc82054d8c0, 0xc822f0eee0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:579 +0xc0
    net/http.HandlerFunc.ServeHTTP(0xc8201ddec0, 0x7fbb72003298, 0xc82054d8c0, 0xc822f0eee0)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.cors.func1(0x7fbb72003298, 0xc82054d8c0, 0xc822f0eee0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:612 +0x329
    net/http.HandlerFunc.ServeHTTP(0xc8201ddee0, 0x7fbb72003298, 0xc82054d8c0, 0xc822f0eee0)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.requestID.func1(0x7fbb72003298, 0xc82054d8c0, 0xc822f0eee0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:622 +0x15d
    net/http.HandlerFunc.ServeHTTP(0xc8201ddf00, 0x7fbb72003298, 0xc82054d8c0, 0xc822f0eee0)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.recovery.func1(0x7fbb72003238, 0xc820371380, 0xc822f0eee0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:639 +0x114
    net/http.HandlerFunc.ServeHTTP(0xc8201ddf20, 0x7fbb72003238, 0xc820371380, 0xc822f0eee0)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.(*ServeMux).ServeHTTP(0xc8201cda10, 0x7fbb72003238, 0xc820371380, 0xc822f0eee0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/mux.go:163 +0x17d
    github.com/influxdata/kapacitor/services/httpd.(*Handler).ServeHTTP(0xc8201c8230, 0x7fbb72003238, 0xc820371380, 0xc822f0eee0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:306 +0xe5
    net/http.serverHandler.ServeHTTP(0xc8201d6680, 0x7fbb72003238, 0xc820371380, 0xc822f0eee0)
    /usr/local/go/src/net/http/server.go:2081 +0x19e
    net/http.(*conn).serve(0xc8208a3280)
    /usr/local/go/src/net/http/server.go:1472 +0xf2e
    created by net/http.(*Server).Serve
    /usr/local/go/src/net/http/server.go:2137 +0x44e

goroutine 55183 [semacquire, 10 minutes]:
    sync.runtime_Semacquire(0xc820064d60)
    /usr/local/go/src/runtime/sema.go:47 +0x26
    sync.(*RWMutex).RLock(0xc820064d54)
    /usr/local/go/src/sync/rwmutex.go:37 +0x58
    github.com/influxdata/kapacitor.(*TaskMaster).IsExecuting(0xc820064c00, 0xc8202115f0, 0x9, 0x600)
    /root/go/src/github.com/influxdata/kapacitor/task_master.go:422 +0x3d
    github.com/influxdata/kapacitor/services/task_store.(*Service).convertTask(0xc82017f0e0, 0xc8202115f0, 0x9, 0x0, 0xc820557ce0, 0x1, 0x1, 0xc82012c080, 0x67a, 0x0, ...)
    /root/go/src/github.com/influxdata/kapacitor/services/task_store/service.go:965 +0x15a
    github.com/influxdata/kapacitor/services/task_store.(*Service).handleTask(0xc82017f0e0, 0x7fbb72003320, 0xc820556980, 0xc822f0e000)
    /root/go/src/github.com/influxdata/kapacitor/services/task_store/service.go:464 +0x665
    github.com/influxdata/kapacitor/services/task_store.(*Service).(github.com/influxdata/kapacitor/services/task_store.handleTask)-fm(0x7fbb72003320, 0xc820556980, 0xc822f0e000)
    /root/go/src/github.com/influxdata/kapacitor/services/task_store/service.go:95 +0x3e
    net/http.HandlerFunc.ServeHTTP(0xc82000e850, 0x7fbb72003320, 0xc820556980, 0xc822f0e000)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.jsonContent.func1(0x7fbb72003320, 0xc820556980, 0xc822f0e000)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:570 +0xaa
    net/http.HandlerFunc.ServeHTTP(0xc820012860, 0x7fbb72003320, 0xc820556980, 0xc822f0e000)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.gzipFilter.func1(0x7fbb72003298, 0xc820556940, 0xc822f0e000)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:563 +0x2af
    net/http.HandlerFunc.ServeHTTP(0xc820012880, 0x7fbb72003298, 0xc820556940, 0xc822f0e000)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.versionHeader.func1(0x7fbb72003298, 0xc820556940, 0xc822f0e000)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:579 +0xc0
    net/http.HandlerFunc.ServeHTTP(0xc8200128a0, 0x7fbb72003298, 0xc820556940, 0xc822f0e000)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.cors.func1(0x7fbb72003298, 0xc820556940, 0xc822f0e000)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:612 +0x329
    net/http.HandlerFunc.ServeHTTP(0xc8200128c0, 0x7fbb72003298, 0xc820556940, 0xc822f0e000)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.requestID.func1(0x7fbb72003298, 0xc820556940, 0xc822f0e000)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:622 +0x15d
    net/http.HandlerFunc.ServeHTTP(0xc8200128e0, 0x7fbb72003298, 0xc820556940, 0xc822f0e000)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.recovery.func1(0x7fbb72003238, 0xc820159ba0, 0xc822f0e000)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:639 +0x114
    net/http.HandlerFunc.ServeHTTP(0xc820012900, 0x7fbb72003238, 0xc820159ba0, 0xc822f0e000)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.(*ServeMux).ServeHTTP(0xc8201cd9b0, 0x7fbb72003238, 0xc820159ba0, 0xc822f0e000)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/mux.go:163 +0x17d
    github.com/influxdata/kapacitor/services/httpd.(*Handler).ServeHTTP(0xc8201c8230, 0x7fbb72003238, 0xc820159ba0, 0xc822f0e000)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:306 +0xe5
    net/http.serverHandler.ServeHTTP(0xc8201d6680, 0x7fbb72003238, 0xc820159ba0, 0xc822f0e000)
    /usr/local/go/src/net/http/server.go:2081 +0x19e
    net/http.(*conn).serve(0xc820671f00)
    /usr/local/go/src/net/http/server.go:1472 +0xf2e
    created by net/http.(*Server).Serve
    /usr/local/go/src/net/http/server.go:2137 +0x44e

goroutine 55369 [select, 2 minutes]:
    github.com/influxdata/kapacitor.(*Edge).CollectPoint(0xc8201c81c0, 0xc8202d0994, 0x6, 0xc82057601c, 0x8, 0xc820576035, 0x7, 0xe56498, 0x0, 0x0, ...)
    /root/go/src/github.com/influxdata/kapacitor/edge.go:171 +0x1c6
    github.com/influxdata/kapacitor.(*TaskMaster).WritePoints(0xc820064c00, 0xc82057601c, 0x8, 0xc820576035, 0x7, 0x3, 0xc82090e000, 0x6, 0x8, 0x0, ...)
    /root/go/src/github.com/influxdata/kapacitor/task_master.go:533 +0x294
    github.com/influxdata/kapacitor/services/httpd.(*Handler).serveWriteLine(0xc8201c8230, 0x7fbb72003320, 0xc820599240, 0xc822f0fce0, 0xc820134c00, 0x590, 0x600, 0x0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:430 +0x862
    github.com/influxdata/kapacitor/services/httpd.(*Handler).serveWrite(0xc8201c8230, 0x7fbb72003320, 0xc820599240, 0xc822f0fce0, 0x0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:393 +0x680
    github.com/influxdata/kapacitor/services/httpd.(*Handler).(github.com/influxdata/kapacitor/services/httpd.serveWrite)-fm(0x7fbb72003320, 0xc820599240, 0xc822f0fce0, 0x0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:132 +0x48
    github.com/influxdata/kapacitor/services/httpd.authenticate.func1(0x7fbb72003320, 0xc820599240, 0xc822f0fce0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:532 +0x52
    net/http.HandlerFunc.ServeHTTP(0xc8201dde60, 0x7fbb72003320, 0xc820599240, 0xc822f0fce0)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.jsonContent.func1(0x7fbb72003320, 0xc820599240, 0xc822f0fce0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:570 +0xaa
    net/http.HandlerFunc.ServeHTTP(0xc8201dde80, 0x7fbb72003320, 0xc820599240, 0xc822f0fce0)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.gzipFilter.func1(0x7fbb72003298, 0xc820599200, 0xc822f0fce0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:563 +0x2af
    net/http.HandlerFunc.ServeHTTP(0xc8201ddea0, 0x7fbb72003298, 0xc820599200, 0xc822f0fce0)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.versionHeader.func1(0x7fbb72003298, 0xc820599200, 0xc822f0fce0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:579 +0xc0
    net/http.HandlerFunc.ServeHTTP(0xc8201ddec0, 0x7fbb72003298, 0xc820599200, 0xc822f0fce0)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.cors.func1(0x7fbb72003298, 0xc820599200, 0xc822f0fce0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:612 +0x329
    net/http.HandlerFunc.ServeHTTP(0xc8201ddee0, 0x7fbb72003298, 0xc820599200, 0xc822f0fce0)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.requestID.func1(0x7fbb72003298, 0xc820599200, 0xc822f0fce0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:622 +0x15d
    net/http.HandlerFunc.ServeHTTP(0xc8201ddf00, 0x7fbb72003298, 0xc820599200, 0xc822f0fce0)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.recovery.func1(0x7fbb72003238, 0xc8201581a0, 0xc822f0fce0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:639 +0x114
    net/http.HandlerFunc.ServeHTTP(0xc8201ddf20, 0x7fbb72003238, 0xc8201581a0, 0xc822f0fce0)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.(*ServeMux).ServeHTTP(0xc8201cda10, 0x7fbb72003238, 0xc8201581a0, 0xc822f0fce0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/mux.go:163 +0x17d
    github.com/influxdata/kapacitor/services/httpd.(*Handler).ServeHTTP(0xc8201c8230, 0x7fbb72003238, 0xc8201581a0, 0xc822f0fce0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:306 +0xe5
    net/http.serverHandler.ServeHTTP(0xc8201d6680, 0x7fbb72003238, 0xc8201581a0, 0xc822f0fce0)
    /usr/local/go/src/net/http/server.go:2081 +0x19e
    net/http.(*conn).serve(0xc820671d00)
    /usr/local/go/src/net/http/server.go:1472 +0xf2e
    created by net/http.(*Server).Serve
    /usr/local/go/src/net/http/server.go:2137 +0x44e

goroutine 55271 [select, 6 minutes]:
    github.com/influxdata/kapacitor.(*Edge).CollectPoint(0xc8201c81c0, 0xc821178a48, 0x8, 0xc8205a7c3c, 0x9, 0xc8205a7c56, 0x7, 0xe56498, 0x0, 0x0, ...)
    /root/go/src/github.com/influxdata/kapacitor/edge.go:171 +0x1c6
    github.com/influxdata/kapacitor.(*TaskMaster).WritePoints(0xc820064c00, 0xc8205a7c3c, 0x9, 0xc8205a7c56, 0x7, 0x3, 0xc820288000, 0x69, 0x80, 0x0, ...)
    /root/go/src/github.com/influxdata/kapacitor/task_master.go:533 +0x294
    github.com/influxdata/kapacitor/services/httpd.(*Handler).serveWriteLine(0xc8201c8230, 0x7fbb72003320, 0xc8251c3ac0, 0xc822f0e8c0, 0xc8206ae000, 0x554e, 0x7e00, 0x0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:430 +0x862
    github.com/influxdata/kapacitor/services/httpd.(*Handler).serveWrite(0xc8201c8230, 0x7fbb72003320, 0xc8251c3ac0, 0xc822f0e8c0, 0x0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:393 +0x680
    github.com/influxdata/kapacitor/services/httpd.(*Handler).(github.com/influxdata/kapacitor/services/httpd.serveWrite)-fm(0x7fbb72003320, 0xc8251c3ac0, 0xc822f0e8c0, 0x0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:132 +0x48
    github.com/influxdata/kapacitor/services/httpd.authenticate.func1(0x7fbb72003320, 0xc8251c3ac0, 0xc822f0e8c0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:532 +0x52
    net/http.HandlerFunc.ServeHTTP(0xc8201dde60, 0x7fbb72003320, 0xc8251c3ac0, 0xc822f0e8c0)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.jsonContent.func1(0x7fbb72003320, 0xc8251c3ac0, 0xc822f0e8c0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:570 +0xaa
    net/http.HandlerFunc.ServeHTTP(0xc8201dde80, 0x7fbb72003320, 0xc8251c3ac0, 0xc822f0e8c0)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.gzipFilter.func1(0x7fbb72003298, 0xc8251c3a80, 0xc822f0e8c0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:563 +0x2af
    net/http.HandlerFunc.ServeHTTP(0xc8201ddea0, 0x7fbb72003298, 0xc8251c3a80, 0xc822f0e8c0)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.versionHeader.func1(0x7fbb72003298, 0xc8251c3a80, 0xc822f0e8c0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:579 +0xc0
    net/http.HandlerFunc.ServeHTTP(0xc8201ddec0, 0x7fbb72003298, 0xc8251c3a80, 0xc822f0e8c0)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.cors.func1(0x7fbb72003298, 0xc8251c3a80, 0xc822f0e8c0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:612 +0x329
    net/http.HandlerFunc.ServeHTTP(0xc8201ddee0, 0x7fbb72003298, 0xc8251c3a80, 0xc822f0e8c0)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.requestID.func1(0x7fbb72003298, 0xc8251c3a80, 0xc822f0e8c0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:622 +0x15d
    net/http.HandlerFunc.ServeHTTP(0xc8201ddf00, 0x7fbb72003298, 0xc8251c3a80, 0xc822f0e8c0)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.recovery.func1(0x7fbb72003238, 0xc820371110, 0xc822f0e8c0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:639 +0x114
    net/http.HandlerFunc.ServeHTTP(0xc8201ddf20, 0x7fbb72003238, 0xc820371110, 0xc822f0e8c0)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.(*ServeMux).ServeHTTP(0xc8201cda10, 0x7fbb72003238, 0xc820371110, 0xc822f0e8c0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/mux.go:163 +0x17d
    github.com/influxdata/kapacitor/services/httpd.(*Handler).ServeHTTP(0xc8201c8230, 0x7fbb72003238, 0xc820371110, 0xc822f0e8c0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:306 +0xe5
    net/http.serverHandler.ServeHTTP(0xc8201d6680, 0x7fbb72003238, 0xc820371110, 0xc822f0e8c0)
    /usr/local/go/src/net/http/server.go:2081 +0x19e
    net/http.(*conn).serve(0xc8208a2900)
    /usr/local/go/src/net/http/server.go:1472 +0xf2e
    created by net/http.(*Server).Serve
    /usr/local/go/src/net/http/server.go:2137 +0x44e

goroutine 55143 [select, 10 minutes]:
    github.com/influxdata/kapacitor.(*Edge).NextPoint(0xc8201c8150, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
    /root/go/src/github.com/influxdata/kapacitor/edge.go:145 +0x1d5
    github.com/influxdata/kapacitor.(*FromNode).runStream(0xc82263e000, 0x0, 0x0, 0x0, 0x0, 0x0)
    /root/go/src/github.com/influxdata/kapacitor/stream.go:77 +0x469
    github.com/influxdata/kapacitor.(*FromNode).(github.com/influxdata/kapacitor.runStream)-fm(0x0, 0x0, 0x0, 0x0, 0x0)
    /root/go/src/github.com/influxdata/kapacitor/stream.go:60 +0x4c
    github.com/influxdata/kapacitor.(*node).start.func1(0xc82263e000, 0x0, 0x0, 0x0)
    /root/go/src/github.com/influxdata/kapacitor/node.go:130 +0x7f
    created by github.com/influxdata/kapacitor.(*node).start
    /root/go/src/github.com/influxdata/kapacitor/node.go:131 +0x53

goroutine 55147 [select, 10 minutes]:
    github.com/influxdata/kapacitor.(*Edge).NextPoint(0xc8201c82a0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
    /root/go/src/github.com/influxdata/kapacitor/edge.go:145 +0x1d5
    github.com/influxdata/kapacitor.(*AlertNode).runAlert(0xc82006d1e0, 0x0, 0x0, 0x0, 0x0, 0x0)
    /root/go/src/github.com/influxdata/kapacitor/alert.go:361 +0x5e3
    github.com/influxdata/kapacitor.(*AlertNode).(github.com/influxdata/kapacitor.runAlert)-fm(0x0, 0x0, 0x0, 0x0, 0x0)
    /root/go/src/github.com/influxdata/kapacitor/alert.go:127 +0x4c
    github.com/influxdata/kapacitor.(*node).start.func1(0xc82006d1e0, 0x0, 0x0, 0x0)
    /root/go/src/github.com/influxdata/kapacitor/node.go:130 +0x7f
    created by github.com/influxdata/kapacitor.(*node).start
    /root/go/src/github.com/influxdata/kapacitor/node.go:131 +0x53

goroutine 55242 [select, 8 minutes]:
    github.com/influxdata/kapacitor.(*Edge).CollectPoint(0xc8201c81c0, 0xc821141350, 0x8, 0xc82059674c, 0x9, 0xc820596766, 0x7, 0xe56498, 0x0, 0x0, ...)
    /root/go/src/github.com/influxdata/kapacitor/edge.go:171 +0x1c6
    github.com/influxdata/kapacitor.(*TaskMaster).WritePoints(0xc820064c00, 0xc82059674c, 0x9, 0xc820596766, 0x7, 0x3, 0xc820171800, 0x69, 0x80, 0x0, ...)
    /root/go/src/github.com/influxdata/kapacitor/task_master.go:533 +0x294
    github.com/influxdata/kapacitor/services/httpd.(*Handler).serveWriteLine(0xc8201c8230, 0x7fbb72003320, 0xc8210c48a0, 0xc822f0e460, 0xc821164000, 0x554e, 0x7e00, 0x0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:430 +0x862
    github.com/influxdata/kapacitor/services/httpd.(*Handler).serveWrite(0xc8201c8230, 0x7fbb72003320, 0xc8210c48a0, 0xc822f0e460, 0x0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:393 +0x680
    github.com/influxdata/kapacitor/services/httpd.(*Handler).(github.com/influxdata/kapacitor/services/httpd.serveWrite)-fm(0x7fbb72003320, 0xc8210c48a0, 0xc822f0e460, 0x0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:132 +0x48
    github.com/influxdata/kapacitor/services/httpd.authenticate.func1(0x7fbb72003320, 0xc8210c48a0, 0xc822f0e460)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:532 +0x52
    net/http.HandlerFunc.ServeHTTP(0xc8201dde60, 0x7fbb72003320, 0xc8210c48a0, 0xc822f0e460)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.jsonContent.func1(0x7fbb72003320, 0xc8210c48a0, 0xc822f0e460)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:570 +0xaa
    net/http.HandlerFunc.ServeHTTP(0xc8201dde80, 0x7fbb72003320, 0xc8210c48a0, 0xc822f0e460)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.gzipFilter.func1(0x7fbb72003298, 0xc8210c4860, 0xc822f0e460)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:563 +0x2af
    net/http.HandlerFunc.ServeHTTP(0xc8201ddea0, 0x7fbb72003298, 0xc8210c4860, 0xc822f0e460)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.versionHeader.func1(0x7fbb72003298, 0xc8210c4860, 0xc822f0e460)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:579 +0xc0
    net/http.HandlerFunc.ServeHTTP(0xc8201ddec0, 0x7fbb72003298, 0xc8210c4860, 0xc822f0e460)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.cors.func1(0x7fbb72003298, 0xc8210c4860, 0xc822f0e460)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:612 +0x329
    net/http.HandlerFunc.ServeHTTP(0xc8201ddee0, 0x7fbb72003298, 0xc8210c4860, 0xc822f0e460)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.requestID.func1(0x7fbb72003298, 0xc8210c4860, 0xc822f0e460)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:622 +0x15d
    net/http.HandlerFunc.ServeHTTP(0xc8201ddf00, 0x7fbb72003298, 0xc8210c4860, 0xc822f0e460)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.recovery.func1(0x7fbb72003238, 0xc8209908f0, 0xc822f0e460)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:639 +0x114
    net/http.HandlerFunc.ServeHTTP(0xc8201ddf20, 0x7fbb72003238, 0xc8209908f0, 0xc822f0e460)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.(*ServeMux).ServeHTTP(0xc8201cda10, 0x7fbb72003238, 0xc8209908f0, 0xc822f0e460)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/mux.go:163 +0x17d
    github.com/influxdata/kapacitor/services/httpd.(*Handler).ServeHTTP(0xc8201c8230, 0x7fbb72003238, 0xc8209908f0, 0xc822f0e460)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:306 +0xe5
    net/http.serverHandler.ServeHTTP(0xc8201d6680, 0x7fbb72003238, 0xc8209908f0, 0xc822f0e460)
    /usr/local/go/src/net/http/server.go:2081 +0x19e
    net/http.(*conn).serve(0xc820670e80)
    /usr/local/go/src/net/http/server.go:1472 +0xf2e
    created by net/http.(*Server).Serve
    /usr/local/go/src/net/http/server.go:2137 +0x44e

goroutine 61 [semacquire, 10 minutes]:
    sync.runtime_Semacquire(0xc820064d60)
    /usr/local/go/src/runtime/sema.go:47 +0x26
    sync.(*RWMutex).RLock(0xc820064d54)
    /usr/local/go/src/sync/rwmutex.go:37 +0x58
    github.com/influxdata/kapacitor.(*TaskMaster).forkPoint(0xc820064c00, 0xe761e0, 0x9, 0xc8201b1280, 0xa, 0xc8201b1340, 0x7, 0xe56498, 0x0, 0x0, ...)
    /root/go/src/github.com/influxdata/kapacitor/task_master.go:474 +0x46
    github.com/influxdata/kapacitor.(*TaskMaster).runForking(0xc820064c00, 0xc8201301c0)
    /root/go/src/github.com/influxdata/kapacitor/task_master.go:469 +0xbb
    created by github.com/influxdata/kapacitor.(*TaskMaster).stream
    /root/go/src/github.com/influxdata/kapacitor/task_master.go:462 +0x26b

goroutine 62 [select, 8 minutes]:
    github.com/influxdata/kapacitor.(*Edge).CollectPoint(0xc8201301c0, 0xe5d298, 0x7, 0xc8201b1280, 0xa, 0xc8201b1340, 0x7, 0xe56498, 0x0, 0x0, ...)
    /root/go/src/github.com/influxdata/kapacitor/edge.go:171 +0x1c6
    github.com/influxdata/kapacitor/services/stats.(*Service).reportStats(0xc820094870)
    /root/go/src/github.com/influxdata/kapacitor/services/stats/service.go:132 +0x405
    github.com/influxdata/kapacitor/services/stats.(*Service).sendStats(0xc820094870)
    /root/go/src/github.com/influxdata/kapacitor/services/stats/service.go:110 +0x167
    created by github.com/influxdata/kapacitor/services/stats.(*Service).Open
    /root/go/src/github.com/influxdata/kapacitor/services/stats/service.go:82 +0x1fa

goroutine 64 [select, 236 minutes]:
    github.com/influxdata/kapacitor/services/reporting.(*Service).usage(0xc820094900)
    /root/go/src/github.com/influxdata/kapacitor/services/reporting/service.go:98 +0x297
    github.com/influxdata/kapacitor/services/reporting.(*Service).Open.func2(0xc820094900)
    /root/go/src/github.com/influxdata/kapacitor/services/reporting/service.go:77 +0x59
    created by github.com/influxdata/kapacitor/services/reporting.(*Service).Open
    /root/go/src/github.com/influxdata/kapacitor/services/reporting/service.go:78 +0x3c4

goroutine 65 [select]:
    github.com/influxdata/kapacitor/services/httpd.(*Service).manage(0xc82017f040)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/service.go:174 +0x796
    created by github.com/influxdata/kapacitor/services/httpd.(*Service).Open
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/service.go:110 +0x9c0

goroutine 66 [IO wait]:
    net.runtime_pollWait(0x7fbb71ffa080, 0x72, 0x0)
    /usr/local/go/src/runtime/netpoll.go:160 +0x60
    net.(*pollDesc).Wait(0xc820130290, 0x72, 0x0, 0x0)
    /usr/local/go/src/net/fd_poll_runtime.go:73 +0x3a
    net.(*pollDesc).WaitRead(0xc820130290, 0x0, 0x0)
    /usr/local/go/src/net/fd_poll_runtime.go:78 +0x36
    net.(*netFD).accept(0xc820130230, 0x0, 0x7fbb7202d5f0, 0xc8205aa180)
    /usr/local/go/src/net/fd_unix.go:426 +0x27c
    net.(*TCPListener).AcceptTCP(0xc8200600d8, 0xc820041e08, 0x0, 0x0)
    /usr/local/go/src/net/tcpsock_posix.go:254 +0x4d
    net.(*TCPListener).Accept(0xc8200600d8, 0x0, 0x0, 0x0, 0x0)
    /usr/local/go/src/net/tcpsock_posix.go:264 +0x3d
    net/http.(*Server).Serve(0xc8201d6680, 0x7fbb72003108, 0xc8200600d8, 0x0, 0x0)
    /usr/local/go/src/net/http/server.go:2117 +0x129
    github.com/influxdata/kapacitor/services/httpd.(*Service).serve(0xc82017f040)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/service.go:228 +0x9d
    created by github.com/influxdata/kapacitor/services/httpd.(*Service).Open
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/service.go:113 +0xa0b

goroutine 67 [chan receive, 3836 minutes]:
    github.com/influxdata/kapacitor/cmd/kapacitord/run.(*Server).Open.func2(0xc8201a77a0)
    /root/go/src/github.com/influxdata/kapacitor/cmd/kapacitord/run/server.go:453 +0x69
    created by github.com/influxdata/kapacitor/cmd/kapacitord/run.(*Server).Open
    /root/go/src/github.com/influxdata/kapacitor/cmd/kapacitord/run/server.go:456 +0x8f

goroutine 68 [select, 3836 minutes]:
    github.com/influxdata/kapacitor/cmd/kapacitord/run.(*Command).monitorServerErrors(0xc820094630)
    /root/go/src/github.com/influxdata/kapacitor/cmd/kapacitord/run/command.go:152 +0x286
    created by github.com/influxdata/kapacitor/cmd/kapacitord/run.(*Command).Run
    /root/go/src/github.com/influxdata/kapacitor/cmd/kapacitord/run/command.go:132 +0x152a

goroutine 69 [select, 3836 minutes, locked to thread]:
    runtime.gopark(0x1026150, 0xc8203bf728, 0xe5faf8, 0x6, 0x18, 0x2)
    /usr/local/go/src/runtime/proc.go:262 +0x163
    runtime.selectgoImpl(0xc8203bf728, 0x0, 0x18)
    /usr/local/go/src/runtime/select.go:392 +0xa67
    runtime.selectgo(0xc8203bf728)
    /usr/local/go/src/runtime/select.go:215 +0x12
    runtime.ensureSigM.func1()
    /usr/local/go/src/runtime/signal1_unix.go:279 +0x358
    runtime.goexit()
    /usr/local/go/src/runtime/asm_amd64.s:1998 +0x1

goroutine 55328 [semacquire, 2 minutes]:
    sync.runtime_Semacquire(0xc820064d60)
    /usr/local/go/src/runtime/sema.go:47 +0x26
    sync.(*RWMutex).RLock(0xc820064d54)
    /usr/local/go/src/sync/rwmutex.go:37 +0x58
    github.com/influxdata/kapacitor.(*TaskMaster).IsExecuting(0xc820064c00, 0xc820974c10, 0x9, 0x600)
    /root/go/src/github.com/influxdata/kapacitor/task_master.go:422 +0x3d
    github.com/influxdata/kapacitor/services/task_store.(*Service).convertTask(0xc82017f0e0, 0xc820974c10, 0x9, 0x0, 0xc8210c50a0, 0x1, 0x1, 0xc82012d400, 0x67a, 0x0, ...)
    /root/go/src/github.com/influxdata/kapacitor/services/task_store/service.go:965 +0x15a
    github.com/influxdata/kapacitor/services/task_store.(*Service).handleTask(0xc82017f0e0, 0x7fbb72003320, 0xc8210c41a0, 0xc820586460)
    /root/go/src/github.com/influxdata/kapacitor/services/task_store/service.go:464 +0x665
    github.com/influxdata/kapacitor/services/task_store.(*Service).(github.com/influxdata/kapacitor/services/task_store.handleTask)-fm(0x7fbb72003320, 0xc8210c41a0, 0xc820586460)
    /root/go/src/github.com/influxdata/kapacitor/services/task_store/service.go:95 +0x3e
    net/http.HandlerFunc.ServeHTTP(0xc82000e850, 0x7fbb72003320, 0xc8210c41a0, 0xc820586460)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.jsonContent.func1(0x7fbb72003320, 0xc8210c41a0, 0xc820586460)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:570 +0xaa
    net/http.HandlerFunc.ServeHTTP(0xc820012860, 0x7fbb72003320, 0xc8210c41a0, 0xc820586460)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.gzipFilter.func1(0x7fbb72003298, 0xc8210c40e0, 0xc820586460)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:563 +0x2af
    net/http.HandlerFunc.ServeHTTP(0xc820012880, 0x7fbb72003298, 0xc8210c40e0, 0xc820586460)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.versionHeader.func1(0x7fbb72003298, 0xc8210c40e0, 0xc820586460)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:579 +0xc0
    net/http.HandlerFunc.ServeHTTP(0xc8200128a0, 0x7fbb72003298, 0xc8210c40e0, 0xc820586460)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.cors.func1(0x7fbb72003298, 0xc8210c40e0, 0xc820586460)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:612 +0x329
    net/http.HandlerFunc.ServeHTTP(0xc8200128c0, 0x7fbb72003298, 0xc8210c40e0, 0xc820586460)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.requestID.func1(0x7fbb72003298, 0xc8210c40e0, 0xc820586460)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:622 +0x15d
    net/http.HandlerFunc.ServeHTTP(0xc8200128e0, 0x7fbb72003298, 0xc8210c40e0, 0xc820586460)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.recovery.func1(0x7fbb72003238, 0xc820990ea0, 0xc820586460)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:639 +0x114
    net/http.HandlerFunc.ServeHTTP(0xc820012900, 0x7fbb72003238, 0xc820990ea0, 0xc820586460)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.(*ServeMux).ServeHTTP(0xc8201cd9b0, 0x7fbb72003238, 0xc820990ea0, 0xc820586460)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/mux.go:163 +0x17d
    github.com/influxdata/kapacitor/services/httpd.(*Handler).ServeHTTP(0xc8201c8230, 0x7fbb72003238, 0xc820990ea0, 0xc820586460)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:306 +0xe5
    net/http.serverHandler.ServeHTTP(0xc8201d6680, 0x7fbb72003238, 0xc820990ea0, 0xc820586460)
    /usr/local/go/src/net/http/server.go:2081 +0x19e
    net/http.(*conn).serve(0xc8208a2500)
    /usr/local/go/src/net/http/server.go:1472 +0xf2e
    created by net/http.(*Server).Serve
    /usr/local/go/src/net/http/server.go:2137 +0x44e

goroutine 55227 [select, 8 minutes]:
    github.com/influxdata/kapacitor.(*Edge).CollectPoint(0xc8201c81c0, 0xc8211786f4, 0x4, 0xc820596b5c, 0x8, 0xc820596b75, 0x7, 0xe56498, 0x0, 0x0, ...)
    /root/go/src/github.com/influxdata/kapacitor/edge.go:171 +0x1c6
    github.com/influxdata/kapacitor.(*TaskMaster).WritePoints(0xc820064c00, 0xc820596b5c, 0x8, 0xc820596b75, 0x7, 0x3, 0xc8208a2d80, 0x6, 0x8, 0x0, ...)
    /root/go/src/github.com/influxdata/kapacitor/task_master.go:533 +0x294
    github.com/influxdata/kapacitor/services/httpd.(*Handler).serveWriteLine(0xc8201c8230, 0x7fbb72003320, 0xc8210c59c0, 0xc822f0e540, 0xc820551200, 0x56e, 0x600, 0x0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:430 +0x862
    github.com/influxdata/kapacitor/services/httpd.(*Handler).serveWrite(0xc8201c8230, 0x7fbb72003320, 0xc8210c59c0, 0xc822f0e540, 0x0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:393 +0x680
    github.com/influxdata/kapacitor/services/httpd.(*Handler).(github.com/influxdata/kapacitor/services/httpd.serveWrite)-fm(0x7fbb72003320, 0xc8210c59c0, 0xc822f0e540, 0x0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:132 +0x48
    github.com/influxdata/kapacitor/services/httpd.authenticate.func1(0x7fbb72003320, 0xc8210c59c0, 0xc822f0e540)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:532 +0x52
    net/http.HandlerFunc.ServeHTTP(0xc8201dde60, 0x7fbb72003320, 0xc8210c59c0, 0xc822f0e540)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.jsonContent.func1(0x7fbb72003320, 0xc8210c59c0, 0xc822f0e540)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:570 +0xaa
    net/http.HandlerFunc.ServeHTTP(0xc8201dde80, 0x7fbb72003320, 0xc8210c59c0, 0xc822f0e540)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.gzipFilter.func1(0x7fbb72003298, 0xc8210c5980, 0xc822f0e540)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:563 +0x2af
    net/http.HandlerFunc.ServeHTTP(0xc8201ddea0, 0x7fbb72003298, 0xc8210c5980, 0xc822f0e540)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.versionHeader.func1(0x7fbb72003298, 0xc8210c5980, 0xc822f0e540)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:579 +0xc0
    net/http.HandlerFunc.ServeHTTP(0xc8201ddec0, 0x7fbb72003298, 0xc8210c5980, 0xc822f0e540)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.cors.func1(0x7fbb72003298, 0xc8210c5980, 0xc822f0e540)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:612 +0x329
    net/http.HandlerFunc.ServeHTTP(0xc8201ddee0, 0x7fbb72003298, 0xc8210c5980, 0xc822f0e540)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.requestID.func1(0x7fbb72003298, 0xc8210c5980, 0xc822f0e540)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:622 +0x15d
    net/http.HandlerFunc.ServeHTTP(0xc8201ddf00, 0x7fbb72003298, 0xc8210c5980, 0xc822f0e540)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.recovery.func1(0x7fbb72003238, 0xc820990b60, 0xc822f0e540)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:639 +0x114
    net/http.HandlerFunc.ServeHTTP(0xc8201ddf20, 0x7fbb72003238, 0xc820990b60, 0xc822f0e540)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.(*ServeMux).ServeHTTP(0xc8201cda10, 0x7fbb72003238, 0xc820990b60, 0xc822f0e540)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/mux.go:163 +0x17d
    github.com/influxdata/kapacitor/services/httpd.(*Handler).ServeHTTP(0xc8201c8230, 0x7fbb72003238, 0xc820990b60, 0xc822f0e540)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:306 +0xe5
    net/http.serverHandler.ServeHTTP(0xc8201d6680, 0x7fbb72003238, 0xc820990b60, 0xc822f0e540)
    /usr/local/go/src/net/http/server.go:2081 +0x19e
    net/http.(*conn).serve(0xc8208a2c80)
    /usr/local/go/src/net/http/server.go:1472 +0xf2e
    created by net/http.(*Server).Serve
    /usr/local/go/src/net/http/server.go:2137 +0x44e

goroutine 55284 [select, 6 minutes]:
    github.com/influxdata/kapacitor.(*Edge).CollectPoint(0xc8201c81c0, 0xc82024d374, 0x4, 0xc82066e65c, 0x8, 0xc82066e675, 0x7, 0xe56498, 0x0, 0x0, ...)
    /root/go/src/github.com/influxdata/kapacitor/edge.go:171 +0x1c6
    github.com/influxdata/kapacitor.(*TaskMaster).WritePoints(0xc820064c00, 0xc82066e65c, 0x8, 0xc82066e675, 0x7, 0x3, 0xc8201d1a00, 0x19, 0x20, 0x0, ...)
    /root/go/src/github.com/influxdata/kapacitor/task_master.go:533 +0x294
    github.com/influxdata/kapacitor/services/httpd.(*Handler).serveWriteLine(0xc8201c8230, 0x7fbb72003320, 0xc8205a5a00, 0xc820587b20, 0xc82124c000, 0x184e, 0x1e00, 0x0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:430 +0x862
    github.com/influxdata/kapacitor/services/httpd.(*Handler).serveWrite(0xc8201c8230, 0x7fbb72003320, 0xc8205a5a00, 0xc820587b20, 0x0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:393 +0x680
    github.com/influxdata/kapacitor/services/httpd.(*Handler).(github.com/influxdata/kapacitor/services/httpd.serveWrite)-fm(0x7fbb72003320, 0xc8205a5a00, 0xc820587b20, 0x0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:132 +0x48
    github.com/influxdata/kapacitor/services/httpd.authenticate.func1(0x7fbb72003320, 0xc8205a5a00, 0xc820587b20)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:532 +0x52
    net/http.HandlerFunc.ServeHTTP(0xc8201dde60, 0x7fbb72003320, 0xc8205a5a00, 0xc820587b20)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.jsonContent.func1(0x7fbb72003320, 0xc8205a5a00, 0xc820587b20)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:570 +0xaa
    net/http.HandlerFunc.ServeHTTP(0xc8201dde80, 0x7fbb72003320, 0xc8205a5a00, 0xc820587b20)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.gzipFilter.func1(0x7fbb72003298, 0xc8205a59c0, 0xc820587b20)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:563 +0x2af
    net/http.HandlerFunc.ServeHTTP(0xc8201ddea0, 0x7fbb72003298, 0xc8205a59c0, 0xc820587b20)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.versionHeader.func1(0x7fbb72003298, 0xc8205a59c0, 0xc820587b20)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:579 +0xc0
    net/http.HandlerFunc.ServeHTTP(0xc8201ddec0, 0x7fbb72003298, 0xc8205a59c0, 0xc820587b20)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.cors.func1(0x7fbb72003298, 0xc8205a59c0, 0xc820587b20)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:612 +0x329
    net/http.HandlerFunc.ServeHTTP(0xc8201ddee0, 0x7fbb72003298, 0xc8205a59c0, 0xc820587b20)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.requestID.func1(0x7fbb72003298, 0xc8205a59c0, 0xc820587b20)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:622 +0x15d
    net/http.HandlerFunc.ServeHTTP(0xc8201ddf00, 0x7fbb72003298, 0xc8205a59c0, 0xc820587b20)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.recovery.func1(0x7fbb72003238, 0xc820991860, 0xc820587b20)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:639 +0x114
    net/http.HandlerFunc.ServeHTTP(0xc8201ddf20, 0x7fbb72003238, 0xc820991860, 0xc820587b20)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.(*ServeMux).ServeHTTP(0xc8201cda10, 0x7fbb72003238, 0xc820991860, 0xc820587b20)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/mux.go:163 +0x17d
    github.com/influxdata/kapacitor/services/httpd.(*Handler).ServeHTTP(0xc8201c8230, 0x7fbb72003238, 0xc820991860, 0xc820587b20)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:306 +0xe5
    net/http.serverHandler.ServeHTTP(0xc8201d6680, 0x7fbb72003238, 0xc820991860, 0xc820587b20)
    /usr/local/go/src/net/http/server.go:2081 +0x19e
    net/http.(*conn).serve(0xc820671100)
    /usr/local/go/src/net/http/server.go:1472 +0xf2e
    created by net/http.(*Server).Serve
    /usr/local/go/src/net/http/server.go:2137 +0x44e

goroutine 46744 [chan send, 262 minutes]:
    github.com/influxdata/kapacitor/tick/ast.lexToken(0xc820859110, 0x1024920)
    /root/go/src/github.com/influxdata/kapacitor/tick/ast/lex.go:343 +0x335
    github.com/influxdata/kapacitor/tick/ast.(*lexer).run(0xc820859110)
    /root/go/src/github.com/influxdata/kapacitor/tick/ast/lex.go:238 +0x32
    created by github.com/influxdata/kapacitor/tick/ast.lex
    /root/go/src/github.com/influxdata/kapacitor/tick/ast/lex.go:230 +0xc1

goroutine 46603 [chan send, 268 minutes]:
    github.com/influxdata/kapacitor/tick/ast.lexToken(0xc820988450, 0x1024920)
    /root/go/src/github.com/influxdata/kapacitor/tick/ast/lex.go:343 +0x335
    github.com/influxdata/kapacitor/tick/ast.(*lexer).run(0xc820988450)
    /root/go/src/github.com/influxdata/kapacitor/tick/ast/lex.go:238 +0x32
    created by github.com/influxdata/kapacitor/tick/ast.lex
    /root/go/src/github.com/influxdata/kapacitor/tick/ast/lex.go:230 +0xc1

goroutine 55326 [select, 4 minutes]:
    github.com/influxdata/kapacitor.(*Edge).CollectPoint(0xc8201c81c0, 0xc82053a898, 0x6, 0xc82066f41c, 0x8, 0xc82066f435, 0x7, 0xe56498, 0x0, 0x0, ...)
    /root/go/src/github.com/influxdata/kapacitor/edge.go:171 +0x1c6
    github.com/influxdata/kapacitor.(*TaskMaster).WritePoints(0xc820064c00, 0xc82066f41c, 0x8, 0xc82066f435, 0x7, 0x3, 0xc820911c00, 0x15, 0x20, 0x0, ...)
    /root/go/src/github.com/influxdata/kapacitor/task_master.go:533 +0x294
    github.com/influxdata/kapacitor/services/httpd.(*Handler).serveWriteLine(0xc8201c8230, 0x7fbb72003320, 0xc82059a3e0, 0xc8205108c0, 0xc8209be000, 0xf68, 0x1e00, 0x0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:430 +0x862
    github.com/influxdata/kapacitor/services/httpd.(*Handler).serveWrite(0xc8201c8230, 0x7fbb72003320, 0xc82059a3e0, 0xc8205108c0, 0x0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:393 +0x680
    github.com/influxdata/kapacitor/services/httpd.(*Handler).(github.com/influxdata/kapacitor/services/httpd.serveWrite)-fm(0x7fbb72003320, 0xc82059a3e0, 0xc8205108c0, 0x0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:132 +0x48
    github.com/influxdata/kapacitor/services/httpd.authenticate.func1(0x7fbb72003320, 0xc82059a3e0, 0xc8205108c0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:532 +0x52
    net/http.HandlerFunc.ServeHTTP(0xc8201dde60, 0x7fbb72003320, 0xc82059a3e0, 0xc8205108c0)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.jsonContent.func1(0x7fbb72003320, 0xc82059a3e0, 0xc8205108c0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:570 +0xaa
    net/http.HandlerFunc.ServeHTTP(0xc8201dde80, 0x7fbb72003320, 0xc82059a3e0, 0xc8205108c0)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.gzipFilter.func1(0x7fbb72003298, 0xc82059a360, 0xc8205108c0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:563 +0x2af
    net/http.HandlerFunc.ServeHTTP(0xc8201ddea0, 0x7fbb72003298, 0xc82059a360, 0xc8205108c0)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.versionHeader.func1(0x7fbb72003298, 0xc82059a360, 0xc8205108c0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:579 +0xc0
    net/http.HandlerFunc.ServeHTTP(0xc8201ddec0, 0x7fbb72003298, 0xc82059a360, 0xc8205108c0)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.cors.func1(0x7fbb72003298, 0xc82059a360, 0xc8205108c0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:612 +0x329
    net/http.HandlerFunc.ServeHTTP(0xc8201ddee0, 0x7fbb72003298, 0xc82059a360, 0xc8205108c0)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.requestID.func1(0x7fbb72003298, 0xc82059a360, 0xc8205108c0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:622 +0x15d
    net/http.HandlerFunc.ServeHTTP(0xc8201ddf00, 0x7fbb72003298, 0xc82059a360, 0xc8205108c0)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.recovery.func1(0x7fbb72003238, 0xc820371ad0, 0xc8205108c0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:639 +0x114
    net/http.HandlerFunc.ServeHTTP(0xc8201ddf20, 0x7fbb72003238, 0xc820371ad0, 0xc8205108c0)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.(*ServeMux).ServeHTTP(0xc8201cda10, 0x7fbb72003238, 0xc820371ad0, 0xc8205108c0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/mux.go:163 +0x17d
    github.com/influxdata/kapacitor/services/httpd.(*Handler).ServeHTTP(0xc8201c8230, 0x7fbb72003238, 0xc820371ad0, 0xc8205108c0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:306 +0xe5
    net/http.serverHandler.ServeHTTP(0xc8201d6680, 0x7fbb72003238, 0xc820371ad0, 0xc8205108c0)
    /usr/local/go/src/net/http/server.go:2081 +0x19e
    net/http.(*conn).serve(0xc820671680)
    /usr/local/go/src/net/http/server.go:1472 +0xf2e
    created by net/http.(*Server).Serve
    /usr/local/go/src/net/http/server.go:2137 +0x44e

goroutine 55188 [semacquire, 10 minutes]:
    sync.runtime_Semacquire(0xc82084c254)
    /usr/local/go/src/runtime/sema.go:47 +0x26
    sync.(*Mutex).Lock(0xc82084c250)
    /usr/local/go/src/sync/mutex.go:83 +0x1c4
    github.com/influxdata/kapacitor.(*node).Wait(0xc82084c1e0, 0x0, 0x0)
    /root/go/src/github.com/influxdata/kapacitor/node.go:148 +0x41
    github.com/influxdata/kapacitor.(*ExecutingTask).stop.func1(0x7fbb71ffadc0, 0xc82084c1e0, 0x0, 0x0)
    /root/go/src/github.com/influxdata/kapacitor/task.go:234 +0x57
    github.com/influxdata/kapacitor.(*ExecutingTask).walk(0xc82092e6c0, 0xc82071ebb8, 0x0, 0x0)
    /root/go/src/github.com/influxdata/kapacitor/task.go:136 +0xb6
    github.com/influxdata/kapacitor.(*ExecutingTask).stop(0xc82092e6c0, 0x0, 0x0)
    /root/go/src/github.com/influxdata/kapacitor/task.go:239 +0x78
    github.com/influxdata/kapacitor.(*TaskMaster).stopTask(0xc820064c00, 0xc8211786c0, 0x9, 0x0, 0x0)
    /root/go/src/github.com/influxdata/kapacitor/task_master.go:411 +0x12e
    github.com/influxdata/kapacitor.(*TaskMaster).StopTask(0xc820064c00, 0xc8211786c0, 0x9, 0x0, 0x0)
    /root/go/src/github.com/influxdata/kapacitor/task_master.go:397 +0x9e
    github.com/influxdata/kapacitor/services/task_store.(*Service).stopTask(0xc82017f0e0, 0xc8211786c0, 0x9)
    /root/go/src/github.com/influxdata/kapacitor/services/task_store/service.go:1872 +0x62
    github.com/influxdata/kapacitor/services/task_store.(*Service).handleUpdateTask(0xc82017f0e0, 0x7fbb72003320, 0xc8206640c0, 0xc820586000)
    /root/go/src/github.com/influxdata/kapacitor/services/task_store/service.go:940 +0xb16
    github.com/influxdata/kapacitor/services/task_store.(*Service).(github.com/influxdata/kapacitor/services/task_store.handleUpdateTask)-fm(0x7fbb72003320, 0xc8206640c0, 0xc820586000)
    /root/go/src/github.com/influxdata/kapacitor/services/task_store/service.go:114 +0x3e
    net/http.HandlerFunc.ServeHTTP(0xc82000e870, 0x7fbb72003320, 0xc8206640c0, 0xc820586000)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.jsonContent.func1(0x7fbb72003320, 0xc8206640c0, 0xc820586000)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:570 +0xaa
    net/http.HandlerFunc.ServeHTTP(0xc820012b00, 0x7fbb72003320, 0xc8206640c0, 0xc820586000)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.gzipFilter.func1(0x7fbb72003298, 0xc820664080, 0xc820586000)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:563 +0x2af
    net/http.HandlerFunc.ServeHTTP(0xc820012b20, 0x7fbb72003298, 0xc820664080, 0xc820586000)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.versionHeader.func1(0x7fbb72003298, 0xc820664080, 0xc820586000)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:579 +0xc0
    net/http.HandlerFunc.ServeHTTP(0xc820012b40, 0x7fbb72003298, 0xc820664080, 0xc820586000)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.cors.func1(0x7fbb72003298, 0xc820664080, 0xc820586000)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:612 +0x329
    net/http.HandlerFunc.ServeHTTP(0xc820012b60, 0x7fbb72003298, 0xc820664080, 0xc820586000)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.requestID.func1(0x7fbb72003298, 0xc820664080, 0xc820586000)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:622 +0x15d
    net/http.HandlerFunc.ServeHTTP(0xc820012b80, 0x7fbb72003298, 0xc820664080, 0xc820586000)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.recovery.func1(0x7fbb72003238, 0xc820990000, 0xc820586000)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:639 +0x114
    net/http.HandlerFunc.ServeHTTP(0xc820012ba0, 0x7fbb72003238, 0xc820990000, 0xc820586000)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.(*ServeMux).ServeHTTP(0xc8201cda70, 0x7fbb72003238, 0xc820990000, 0xc820586000)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/mux.go:163 +0x17d
    github.com/influxdata/kapacitor/services/httpd.(*Handler).ServeHTTP(0xc8201c8230, 0x7fbb72003238, 0xc820990000, 0xc820586000)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:306 +0xe5
    net/http.serverHandler.ServeHTTP(0xc8201d6680, 0x7fbb72003238, 0xc820990000, 0xc820586000)
    /usr/local/go/src/net/http/server.go:2081 +0x19e
    net/http.(*conn).serve(0xc8208a2000)
    /usr/local/go/src/net/http/server.go:1472 +0xf2e
    created by net/http.(*Server).Serve
    /usr/local/go/src/net/http/server.go:2137 +0x44e

goroutine 47131 [chan send, 250 minutes]:
    github.com/influxdata/kapacitor/tick/ast.lexToken(0xc8207cdb60, 0x1024920)
    /root/go/src/github.com/influxdata/kapacitor/tick/ast/lex.go:343 +0x335
    github.com/influxdata/kapacitor/tick/ast.(*lexer).run(0xc8207cdb60)
    /root/go/src/github.com/influxdata/kapacitor/tick/ast/lex.go:238 +0x32
    created by github.com/influxdata/kapacitor/tick/ast.lex
    /root/go/src/github.com/influxdata/kapacitor/tick/ast/lex.go:230 +0xc1

goroutine 55181 [select, 10 minutes]:
    github.com/influxdata/kapacitor.(*Edge).CollectPoint(0xc8201c81c0, 0xc820210668, 0x6, 0xc8205a688c, 0x8, 0xc8205a68a5, 0x7, 0xe56498, 0x0, 0x0, ...)
    /root/go/src/github.com/influxdata/kapacitor/edge.go:171 +0x1c6
    github.com/influxdata/kapacitor.(*TaskMaster).WritePoints(0xc820064c00, 0xc8205a688c, 0x8, 0xc8205a68a5, 0x7, 0x3, 0xc8201aff00, 0xe, 0x10, 0x0, ...)
    /root/go/src/github.com/influxdata/kapacitor/task_master.go:533 +0x294
    github.com/influxdata/kapacitor/services/httpd.(*Handler).serveWriteLine(0xc8201c8230, 0x7fbb72003320, 0xc820564fc0, 0xc820235420, 0xc820703000, 0xc65, 0xe00, 0x0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:430 +0x862
    github.com/influxdata/kapacitor/services/httpd.(*Handler).serveWrite(0xc8201c8230, 0x7fbb72003320, 0xc820564fc0, 0xc820235420, 0x0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:393 +0x680
    github.com/influxdata/kapacitor/services/httpd.(*Handler).(github.com/influxdata/kapacitor/services/httpd.serveWrite)-fm(0x7fbb72003320, 0xc820564fc0, 0xc820235420, 0x0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:132 +0x48
    github.com/influxdata/kapacitor/services/httpd.authenticate.func1(0x7fbb72003320, 0xc820564fc0, 0xc820235420)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:532 +0x52
    net/http.HandlerFunc.ServeHTTP(0xc8201dde60, 0x7fbb72003320, 0xc820564fc0, 0xc820235420)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.jsonContent.func1(0x7fbb72003320, 0xc820564fc0, 0xc820235420)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:570 +0xaa
    net/http.HandlerFunc.ServeHTTP(0xc8201dde80, 0x7fbb72003320, 0xc820564fc0, 0xc820235420)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.gzipFilter.func1(0x7fbb72003298, 0xc820564f60, 0xc820235420)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:563 +0x2af
    net/http.HandlerFunc.ServeHTTP(0xc8201ddea0, 0x7fbb72003298, 0xc820564f60, 0xc820235420)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.versionHeader.func1(0x7fbb72003298, 0xc820564f60, 0xc820235420)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:579 +0xc0
    net/http.HandlerFunc.ServeHTTP(0xc8201ddec0, 0x7fbb72003298, 0xc820564f60, 0xc820235420)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.cors.func1(0x7fbb72003298, 0xc820564f60, 0xc820235420)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:612 +0x329
    net/http.HandlerFunc.ServeHTTP(0xc8201ddee0, 0x7fbb72003298, 0xc820564f60, 0xc820235420)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.requestID.func1(0x7fbb72003298, 0xc820564f60, 0xc820235420)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:622 +0x15d
    net/http.HandlerFunc.ServeHTTP(0xc8201ddf00, 0x7fbb72003298, 0xc820564f60, 0xc820235420)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.recovery.func1(0x7fbb72003238, 0xc820159450, 0xc820235420)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:639 +0x114
    net/http.HandlerFunc.ServeHTTP(0xc8201ddf20, 0x7fbb72003238, 0xc820159450, 0xc820235420)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.(*ServeMux).ServeHTTP(0xc8201cda10, 0x7fbb72003238, 0xc820159450, 0xc820235420)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/mux.go:163 +0x17d
    github.com/influxdata/kapacitor/services/httpd.(*Handler).ServeHTTP(0xc8201c8230, 0x7fbb72003238, 0xc820159450, 0xc820235420)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:306 +0xe5
    net/http.serverHandler.ServeHTTP(0xc8201d6680, 0x7fbb72003238, 0xc820159450, 0xc820235420)
    /usr/local/go/src/net/http/server.go:2081 +0x19e
    net/http.(*conn).serve(0xc820671d80)
    /usr/local/go/src/net/http/server.go:1472 +0xf2e
    created by net/http.(*Server).Serve
    /usr/local/go/src/net/http/server.go:2137 +0x44e

goroutine 55198 [select, 10 minutes]:
    github.com/influxdata/kapacitor.(*Edge).CollectPoint(0xc8201c81c0, 0xc82024cdc0, 0x6, 0xc82066f00c, 0x8, 0xc82066f025, 0x7, 0xe56498, 0x0, 0x0, ...)
    /root/go/src/github.com/influxdata/kapacitor/edge.go:171 +0x1c6
    github.com/influxdata/kapacitor.(*TaskMaster).WritePoints(0xc820064c00, 0xc82066f00c, 0x8, 0xc82066f025, 0x7, 0x3, 0xc820911200, 0x15, 0x20, 0x0, ...)
    /root/go/src/github.com/influxdata/kapacitor/task_master.go:533 +0x294
    github.com/influxdata/kapacitor/services/httpd.(*Handler).serveWriteLine(0xc8201c8230, 0x7fbb72003320, 0xc820544380, 0xc820090460, 0xc820754000, 0xf69, 0x1e00, 0x0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:430 +0x862
    github.com/influxdata/kapacitor/services/httpd.(*Handler).serveWrite(0xc8201c8230, 0x7fbb72003320, 0xc820544380, 0xc820090460, 0x0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:393 +0x680
    github.com/influxdata/kapacitor/services/httpd.(*Handler).(github.com/influxdata/kapacitor/services/httpd.serveWrite)-fm(0x7fbb72003320, 0xc820544380, 0xc820090460, 0x0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:132 +0x48
    github.com/influxdata/kapacitor/services/httpd.authenticate.func1(0x7fbb72003320, 0xc820544380, 0xc820090460)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:532 +0x52
    net/http.HandlerFunc.ServeHTTP(0xc8201dde60, 0x7fbb72003320, 0xc820544380, 0xc820090460)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.jsonContent.func1(0x7fbb72003320, 0xc820544380, 0xc820090460)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:570 +0xaa
    net/http.HandlerFunc.ServeHTTP(0xc8201dde80, 0x7fbb72003320, 0xc820544380, 0xc820090460)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.gzipFilter.func1(0x7fbb72003298, 0xc820544340, 0xc820090460)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:563 +0x2af
    net/http.HandlerFunc.ServeHTTP(0xc8201ddea0, 0x7fbb72003298, 0xc820544340, 0xc820090460)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.versionHeader.func1(0x7fbb72003298, 0xc820544340, 0xc820090460)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:579 +0xc0
    net/http.HandlerFunc.ServeHTTP(0xc8201ddec0, 0x7fbb72003298, 0xc820544340, 0xc820090460)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.cors.func1(0x7fbb72003298, 0xc820544340, 0xc820090460)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:612 +0x329
    net/http.HandlerFunc.ServeHTTP(0xc8201ddee0, 0x7fbb72003298, 0xc820544340, 0xc820090460)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.requestID.func1(0x7fbb72003298, 0xc820544340, 0xc820090460)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:622 +0x15d
    net/http.HandlerFunc.ServeHTTP(0xc8201ddf00, 0x7fbb72003298, 0xc820544340, 0xc820090460)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.recovery.func1(0x7fbb72003238, 0xc820371ee0, 0xc820090460)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:639 +0x114
    net/http.HandlerFunc.ServeHTTP(0xc8201ddf20, 0x7fbb72003238, 0xc820371ee0, 0xc820090460)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.(*ServeMux).ServeHTTP(0xc8201cda10, 0x7fbb72003238, 0xc820371ee0, 0xc820090460)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/mux.go:163 +0x17d
    github.com/influxdata/kapacitor/services/httpd.(*Handler).ServeHTTP(0xc8201c8230, 0x7fbb72003238, 0xc820371ee0, 0xc820090460)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:306 +0xe5
    net/http.serverHandler.ServeHTTP(0xc8201d6680, 0x7fbb72003238, 0xc820371ee0, 0xc820090460)
    /usr/local/go/src/net/http/server.go:2081 +0x19e
    net/http.(*conn).serve(0xc820686280)
    /usr/local/go/src/net/http/server.go:1472 +0xf2e
    created by net/http.(*Server).Serve
    /usr/local/go/src/net/http/server.go:2137 +0x44e

goroutine 55254 [select, 8 minutes]:
    github.com/influxdata/kapacitor.(*Edge).CollectPoint(0xc8201c81c0, 0xc82077b190, 0x8, 0xc8205a6fbc, 0x9, 0xc8205a6fd6, 0x7, 0xe56498, 0x0, 0x0, ...)
    /root/go/src/github.com/influxdata/kapacitor/edge.go:171 +0x1c6
    github.com/influxdata/kapacitor.(*TaskMaster).WritePoints(0xc820064c00, 0xc8205a6fbc, 0x9, 0xc8205a6fd6, 0x7, 0x3, 0xc822f08800, 0x69, 0x80, 0x0, ...)
    /root/go/src/github.com/influxdata/kapacitor/task_master.go:533 +0x294
    github.com/influxdata/kapacitor/services/httpd.(*Handler).serveWriteLine(0xc8201c8230, 0x7fbb72003320, 0xc8205aafc0, 0xc822f0ec40, 0xc82094e000, 0x554e, 0x7e00, 0x0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:430 +0x862
    github.com/influxdata/kapacitor/services/httpd.(*Handler).serveWrite(0xc8201c8230, 0x7fbb72003320, 0xc8205aafc0, 0xc822f0ec40, 0x0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:393 +0x680
    github.com/influxdata/kapacitor/services/httpd.(*Handler).(github.com/influxdata/kapacitor/services/httpd.serveWrite)-fm(0x7fbb72003320, 0xc8205aafc0, 0xc822f0ec40, 0x0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:132 +0x48
    github.com/influxdata/kapacitor/services/httpd.authenticate.func1(0x7fbb72003320, 0xc8205aafc0, 0xc822f0ec40)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:532 +0x52
    net/http.HandlerFunc.ServeHTTP(0xc8201dde60, 0x7fbb72003320, 0xc8205aafc0, 0xc822f0ec40)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.jsonContent.func1(0x7fbb72003320, 0xc8205aafc0, 0xc822f0ec40)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:570 +0xaa
    net/http.HandlerFunc.ServeHTTP(0xc8201dde80, 0x7fbb72003320, 0xc8205aafc0, 0xc822f0ec40)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.gzipFilter.func1(0x7fbb72003298, 0xc8205aaf80, 0xc822f0ec40)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:563 +0x2af
    net/http.HandlerFunc.ServeHTTP(0xc8201ddea0, 0x7fbb72003298, 0xc8205aaf80, 0xc822f0ec40)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.versionHeader.func1(0x7fbb72003298, 0xc8205aaf80, 0xc822f0ec40)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:579 +0xc0
    net/http.HandlerFunc.ServeHTTP(0xc8201ddec0, 0x7fbb72003298, 0xc8205aaf80, 0xc822f0ec40)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.cors.func1(0x7fbb72003298, 0xc8205aaf80, 0xc822f0ec40)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:612 +0x329
    net/http.HandlerFunc.ServeHTTP(0xc8201ddee0, 0x7fbb72003298, 0xc8205aaf80, 0xc822f0ec40)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.requestID.func1(0x7fbb72003298, 0xc8205aaf80, 0xc822f0ec40)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:622 +0x15d
    net/http.HandlerFunc.ServeHTTP(0xc8201ddf00, 0x7fbb72003298, 0xc8205aaf80, 0xc822f0ec40)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.recovery.func1(0x7fbb72003238, 0xc8203715f0, 0xc822f0ec40)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:639 +0x114
    net/http.HandlerFunc.ServeHTTP(0xc8201ddf20, 0x7fbb72003238, 0xc8203715f0, 0xc822f0ec40)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.(*ServeMux).ServeHTTP(0xc8201cda10, 0x7fbb72003238, 0xc8203715f0, 0xc822f0ec40)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/mux.go:163 +0x17d
    github.com/influxdata/kapacitor/services/httpd.(*Handler).ServeHTTP(0xc8201c8230, 0x7fbb72003238, 0xc8203715f0, 0xc822f0ec40)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:306 +0xe5
    net/http.serverHandler.ServeHTTP(0xc8201d6680, 0x7fbb72003238, 0xc8203715f0, 0xc822f0ec40)
    /usr/local/go/src/net/http/server.go:2081 +0x19e
    net/http.(*conn).serve(0xc8208a3480)
    /usr/local/go/src/net/http/server.go:1472 +0xf2e
    created by net/http.(*Server).Serve
    /usr/local/go/src/net/http/server.go:2137 +0x44e

goroutine 55360 [semacquire, 2 minutes]:
    sync.runtime_Semacquire(0xc820064d60)
    /usr/local/go/src/runtime/sema.go:47 +0x26
    sync.(*RWMutex).RLock(0xc820064d54)
    /usr/local/go/src/sync/rwmutex.go:37 +0x58
    github.com/influxdata/kapacitor.(*TaskMaster).IsExecuting(0xc820064c00, 0xc82024c3c0, 0xf, 0x400)
    /root/go/src/github.com/influxdata/kapacitor/task_master.go:422 +0x3d
    github.com/influxdata/kapacitor/services/task_store.(*Service).convertTask(0xc82017f0e0, 0xc82024c3c0, 0xf, 0x0, 0xc82054cdc0, 0x1, 0x1, 0xc820654000, 0x44c, 0x0, ...)
    /root/go/src/github.com/influxdata/kapacitor/services/task_store/service.go:965 +0x15a
    github.com/influxdata/kapacitor/services/task_store.(*Service).handleTask(0xc82017f0e0, 0x7fbb72003320, 0xc8205a5900, 0xc822f0f960)
    /root/go/src/github.com/influxdata/kapacitor/services/task_store/service.go:464 +0x665
    github.com/influxdata/kapacitor/services/task_store.(*Service).(github.com/influxdata/kapacitor/services/task_store.handleTask)-fm(0x7fbb72003320, 0xc8205a5900, 0xc822f0f960)
    /root/go/src/github.com/influxdata/kapacitor/services/task_store/service.go:95 +0x3e
    net/http.HandlerFunc.ServeHTTP(0xc82000e850, 0x7fbb72003320, 0xc8205a5900, 0xc822f0f960)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.jsonContent.func1(0x7fbb72003320, 0xc8205a5900, 0xc822f0f960)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:570 +0xaa
    net/http.HandlerFunc.ServeHTTP(0xc820012860, 0x7fbb72003320, 0xc8205a5900, 0xc822f0f960)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.gzipFilter.func1(0x7fbb72003298, 0xc8205a58a0, 0xc822f0f960)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:563 +0x2af
    net/http.HandlerFunc.ServeHTTP(0xc820012880, 0x7fbb72003298, 0xc8205a58a0, 0xc822f0f960)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.versionHeader.func1(0x7fbb72003298, 0xc8205a58a0, 0xc822f0f960)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:579 +0xc0
    net/http.HandlerFunc.ServeHTTP(0xc8200128a0, 0x7fbb72003298, 0xc8205a58a0, 0xc822f0f960)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.cors.func1(0x7fbb72003298, 0xc8205a58a0, 0xc822f0f960)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:612 +0x329
    net/http.HandlerFunc.ServeHTTP(0xc8200128c0, 0x7fbb72003298, 0xc8205a58a0, 0xc822f0f960)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.requestID.func1(0x7fbb72003298, 0xc8205a58a0, 0xc822f0f960)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:622 +0x15d
    net/http.HandlerFunc.ServeHTTP(0xc8200128e0, 0x7fbb72003298, 0xc8205a58a0, 0xc822f0f960)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.recovery.func1(0x7fbb72003238, 0xc820371860, 0xc822f0f960)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:639 +0x114
    net/http.HandlerFunc.ServeHTTP(0xc820012900, 0x7fbb72003238, 0xc820371860, 0xc822f0f960)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.(*ServeMux).ServeHTTP(0xc8201cd9b0, 0x7fbb72003238, 0xc820371860, 0xc822f0f960)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/mux.go:163 +0x17d
    github.com/influxdata/kapacitor/services/httpd.(*Handler).ServeHTTP(0xc8201c8230, 0x7fbb72003238, 0xc820371860, 0xc822f0f960)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:306 +0xe5
    net/http.serverHandler.ServeHTTP(0xc8201d6680, 0x7fbb72003238, 0xc820371860, 0xc822f0f960)
    /usr/local/go/src/net/http/server.go:2081 +0x19e
    net/http.(*conn).serve(0xc820671980)
    /usr/local/go/src/net/http/server.go:1472 +0xf2e
    created by net/http.(*Server).Serve
    /usr/local/go/src/net/http/server.go:2137 +0x44e

goroutine 55220 [semacquire, 8 minutes]:
    sync.runtime_Semacquire(0xc820064d60)
    /usr/local/go/src/runtime/sema.go:47 +0x26
    sync.(*RWMutex).RLock(0xc820064d54)
    /usr/local/go/src/sync/rwmutex.go:37 +0x58
    github.com/influxdata/kapacitor.(*TaskMaster).IsExecuting(0xc820064c00, 0xc82024c0c0, 0x3, 0x0)
    /root/go/src/github.com/influxdata/kapacitor/task_master.go:422 +0x3d
    github.com/influxdata/kapacitor/services/task_store.(*Service).convertTask(0xc82017f0e0, 0xc82024c0c0, 0x3, 0x0, 0xc8251c2160, 0x1, 0x1, 0xc82024c110, 0x10, 0x0, ...)
    /root/go/src/github.com/influxdata/kapacitor/services/task_store/service.go:965 +0x15a
    github.com/influxdata/kapacitor/services/task_store.(*Service).handleCreateTask(0xc82017f0e0, 0x7fbb72003320, 0xc8251c2080, 0xc8205860e0)
    /root/go/src/github.com/influxdata/kapacitor/services/task_store/service.go:785 +0x1188
    github.com/influxdata/kapacitor/services/task_store.(*Service).(github.com/influxdata/kapacitor/services/task_store.handleCreateTask)-fm(0x7fbb72003320, 0xc8251c2080, 0xc8205860e0)
    /root/go/src/github.com/influxdata/kapacitor/services/task_store/service.go:126 +0x3e
    net/http.HandlerFunc.ServeHTTP(0xc82000e890, 0x7fbb72003320, 0xc8251c2080, 0xc8205860e0)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.jsonContent.func1(0x7fbb72003320, 0xc8251c2080, 0xc8205860e0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:570 +0xaa
    net/http.HandlerFunc.ServeHTTP(0xc820012cc0, 0x7fbb72003320, 0xc8251c2080, 0xc8205860e0)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.gzipFilter.func1(0x7fbb72003298, 0xc8251c2040, 0xc8205860e0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:563 +0x2af
    net/http.HandlerFunc.ServeHTTP(0xc820012ce0, 0x7fbb72003298, 0xc8251c2040, 0xc8205860e0)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.versionHeader.func1(0x7fbb72003298, 0xc8251c2040, 0xc8205860e0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:579 +0xc0
    net/http.HandlerFunc.ServeHTTP(0xc820012d00, 0x7fbb72003298, 0xc8251c2040, 0xc8205860e0)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.cors.func1(0x7fbb72003298, 0xc8251c2040, 0xc8205860e0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:612 +0x329
    net/http.HandlerFunc.ServeHTTP(0xc820012d20, 0x7fbb72003298, 0xc8251c2040, 0xc8205860e0)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.requestID.func1(0x7fbb72003298, 0xc8251c2040, 0xc8205860e0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:622 +0x15d
    net/http.HandlerFunc.ServeHTTP(0xc820012d40, 0x7fbb72003298, 0xc8251c2040, 0xc8205860e0)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.recovery.func1(0x7fbb72003238, 0xc820370000, 0xc8205860e0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:639 +0x114
    net/http.HandlerFunc.ServeHTTP(0xc820012d60, 0x7fbb72003238, 0xc820370000, 0xc8205860e0)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.(*ServeMux).ServeHTTP(0xc8201cda10, 0x7fbb72003238, 0xc820370000, 0xc8205860e0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/mux.go:163 +0x17d
    github.com/influxdata/kapacitor/services/httpd.(*Handler).ServeHTTP(0xc8201c8230, 0x7fbb72003238, 0xc820370000, 0xc8205860e0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:306 +0xe5
    net/http.serverHandler.ServeHTTP(0xc8201d6680, 0x7fbb72003238, 0xc820370000, 0xc8205860e0)
    /usr/local/go/src/net/http/server.go:2081 +0x19e
    net/http.(*conn).serve(0xc820670300)
    /usr/local/go/src/net/http/server.go:1472 +0xf2e
    created by net/http.(*Server).Serve
    /usr/local/go/src/net/http/server.go:2137 +0x44e

goroutine 55123 [chan receive, 10 minutes]:
    github.com/influxdata/kapacitor.(*node).Wait(0xc82084c1e0, 0x0, 0x0)
    /root/go/src/github.com/influxdata/kapacitor/node.go:152 +0xc6
    github.com/influxdata/kapacitor.(*ExecutingTask).Wait.func1(0x7fbb71ffadc0, 0xc82084c1e0, 0x0, 0x0)
    /root/go/src/github.com/influxdata/kapacitor/task.go:317 +0x33
    github.com/influxdata/kapacitor.(*ExecutingTask).rwalk(0xc82092e6c0, 0x1024598, 0x0, 0x0)
    /root/go/src/github.com/influxdata/kapacitor/task.go:147 +0x91
    github.com/influxdata/kapacitor.(*ExecutingTask).Wait(0xc82092e6c0, 0x0, 0x0)
    /root/go/src/github.com/influxdata/kapacitor/task.go:318 +0x39
    github.com/influxdata/kapacitor/services/task_store.(*Service).startTask.func1(0xc82092e6c0, 0xc82017f0e0, 0xc820064c00, 0xc8224d91c0)
    /root/go/src/github.com/influxdata/kapacitor/services/task_store/service.go:1853 +0x30
    created by github.com/influxdata/kapacitor/services/task_store.(*Service).startTask
    /root/go/src/github.com/influxdata/kapacitor/services/task_store/service.go:1867 +0x324

goroutine 55285 [select, 4 minutes]:
    github.com/influxdata/kapacitor.(*Edge).CollectPoint(0xc8201c81c0, 0xc8205a8f64, 0x7, 0xc82066e06c, 0x8, 0xc82066e085, 0x7, 0xe56498, 0x0, 0x0, ...)
    /root/go/src/github.com/influxdata/kapacitor/edge.go:171 +0x1c6
    github.com/influxdata/kapacitor.(*TaskMaster).WritePoints(0xc820064c00, 0xc82066e06c, 0x8, 0xc82066e085, 0x7, 0x3, 0xc820910400, 0x14, 0x20, 0x0, ...)
    /root/go/src/github.com/influxdata/kapacitor/task_master.go:533 +0x294
    github.com/influxdata/kapacitor/services/httpd.(*Handler).serveWriteLine(0xc8201c8230, 0x7fbb72003320, 0xc8210c4160, 0xc820586ee0, 0xc8205b6000, 0x123a, 0x1e00, 0x0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:430 +0x862
    github.com/influxdata/kapacitor/services/httpd.(*Handler).serveWrite(0xc8201c8230, 0x7fbb72003320, 0xc8210c4160, 0xc820586ee0, 0x0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:393 +0x680
    github.com/influxdata/kapacitor/services/httpd.(*Handler).(github.com/influxdata/kapacitor/services/httpd.serveWrite)-fm(0x7fbb72003320, 0xc8210c4160, 0xc820586ee0, 0x0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:132 +0x48
    github.com/influxdata/kapacitor/services/httpd.authenticate.func1(0x7fbb72003320, 0xc8210c4160, 0xc820586ee0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:532 +0x52
    net/http.HandlerFunc.ServeHTTP(0xc8201dde60, 0x7fbb72003320, 0xc8210c4160, 0xc820586ee0)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.jsonContent.func1(0x7fbb72003320, 0xc8210c4160, 0xc820586ee0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:570 +0xaa
    net/http.HandlerFunc.ServeHTTP(0xc8201dde80, 0x7fbb72003320, 0xc8210c4160, 0xc820586ee0)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.gzipFilter.func1(0x7fbb72003298, 0xc8210c4120, 0xc820586ee0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:563 +0x2af
    net/http.HandlerFunc.ServeHTTP(0xc8201ddea0, 0x7fbb72003298, 0xc8210c4120, 0xc820586ee0)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.versionHeader.func1(0x7fbb72003298, 0xc8210c4120, 0xc820586ee0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:579 +0xc0
    net/http.HandlerFunc.ServeHTTP(0xc8201ddec0, 0x7fbb72003298, 0xc8210c4120, 0xc820586ee0)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.cors.func1(0x7fbb72003298, 0xc8210c4120, 0xc820586ee0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:612 +0x329
    net/http.HandlerFunc.ServeHTTP(0xc8201ddee0, 0x7fbb72003298, 0xc8210c4120, 0xc820586ee0)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.requestID.func1(0x7fbb72003298, 0xc8210c4120, 0xc820586ee0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:622 +0x15d
    net/http.HandlerFunc.ServeHTTP(0xc8201ddf00, 0x7fbb72003298, 0xc8210c4120, 0xc820586ee0)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.recovery.func1(0x7fbb72003238, 0xc820370c30, 0xc820586ee0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:639 +0x114
    net/http.HandlerFunc.ServeHTTP(0xc8201ddf20, 0x7fbb72003238, 0xc820370c30, 0xc820586ee0)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.(*ServeMux).ServeHTTP(0xc8201cda10, 0x7fbb72003238, 0xc820370c30, 0xc820586ee0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/mux.go:163 +0x17d
    github.com/influxdata/kapacitor/services/httpd.(*Handler).ServeHTTP(0xc8201c8230, 0x7fbb72003238, 0xc820370c30, 0xc820586ee0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:306 +0xe5
    net/http.serverHandler.ServeHTTP(0xc8201d6680, 0x7fbb72003238, 0xc820370c30, 0xc820586ee0)
    /usr/local/go/src/net/http/server.go:2081 +0x19e
    net/http.(*conn).serve(0xc820670200)
    /usr/local/go/src/net/http/server.go:1472 +0xf2e
    created by net/http.(*Server).Serve
    /usr/local/go/src/net/http/server.go:2137 +0x44e

goroutine 55149 [select]:
    github.com/influxdata/kapacitor.(*ExecutingTask).calcThroughput(0xc82079c360)
    /root/go/src/github.com/influxdata/kapacitor/task.go:422 +0x318
    created by github.com/influxdata/kapacitor.(*ExecutingTask).start
    /root/go/src/github.com/influxdata/kapacitor/task.go:226 +0x2a1

goroutine 55163 [select, 2 minutes]:
    github.com/influxdata/kapacitor.(*ExecutingTask).runSnapshotter(0xc82079d3b0)
    /root/go/src/github.com/influxdata/kapacitor/task.go:530 +0x84d
    created by github.com/influxdata/kapacitor.(*ExecutingTask).start
    /root/go/src/github.com/influxdata/kapacitor/task.go:222 +0x256

goroutine 55241 [select, 8 minutes]:
    github.com/influxdata/kapacitor.(*Edge).CollectPoint(0xc8201c81c0, 0xc8211782a8, 0x4, 0xc82059647c, 0x8, 0xc820596495, 0x7, 0xe56498, 0x0, 0x0, ...)
    /root/go/src/github.com/influxdata/kapacitor/edge.go:171 +0x1c6
    github.com/influxdata/kapacitor.(*TaskMaster).WritePoints(0xc820064c00, 0xc82059647c, 0x8, 0xc820596495, 0x7, 0x3, 0xc8200cd400, 0xd, 0x10, 0x0, ...)
    /root/go/src/github.com/influxdata/kapacitor/task_master.go:533 +0x294
    github.com/influxdata/kapacitor/services/httpd.(*Handler).serveWriteLine(0xc8201c8230, 0x7fbb72003320, 0xc8251c3960, 0xc822f0e380, 0xc8203b4000, 0xb57, 0xe00, 0x0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:430 +0x862
    github.com/influxdata/kapacitor/services/httpd.(*Handler).serveWrite(0xc8201c8230, 0x7fbb72003320, 0xc8251c3960, 0xc822f0e380, 0x0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:393 +0x680
    github.com/influxdata/kapacitor/services/httpd.(*Handler).(github.com/influxdata/kapacitor/services/httpd.serveWrite)-fm(0x7fbb72003320, 0xc8251c3960, 0xc822f0e380, 0x0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:132 +0x48
    github.com/influxdata/kapacitor/services/httpd.authenticate.func1(0x7fbb72003320, 0xc8251c3960, 0xc822f0e380)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:532 +0x52
    net/http.HandlerFunc.ServeHTTP(0xc8201dde60, 0x7fbb72003320, 0xc8251c3960, 0xc822f0e380)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.jsonContent.func1(0x7fbb72003320, 0xc8251c3960, 0xc822f0e380)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:570 +0xaa
    net/http.HandlerFunc.ServeHTTP(0xc8201dde80, 0x7fbb72003320, 0xc8251c3960, 0xc822f0e380)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.gzipFilter.func1(0x7fbb72003298, 0xc8251c3920, 0xc822f0e380)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:563 +0x2af
    net/http.HandlerFunc.ServeHTTP(0xc8201ddea0, 0x7fbb72003298, 0xc8251c3920, 0xc822f0e380)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.versionHeader.func1(0x7fbb72003298, 0xc8251c3920, 0xc822f0e380)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:579 +0xc0
    net/http.HandlerFunc.ServeHTTP(0xc8201ddec0, 0x7fbb72003298, 0xc8251c3920, 0xc822f0e380)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.cors.func1(0x7fbb72003298, 0xc8251c3920, 0xc822f0e380)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:612 +0x329
    net/http.HandlerFunc.ServeHTTP(0xc8201ddee0, 0x7fbb72003298, 0xc8251c3920, 0xc822f0e380)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.requestID.func1(0x7fbb72003298, 0xc8251c3920, 0xc822f0e380)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:622 +0x15d
    net/http.HandlerFunc.ServeHTTP(0xc8201ddf00, 0x7fbb72003298, 0xc8251c3920, 0xc822f0e380)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.recovery.func1(0x7fbb72003238, 0xc820990750, 0xc822f0e380)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:639 +0x114
    net/http.HandlerFunc.ServeHTTP(0xc8201ddf20, 0x7fbb72003238, 0xc820990750, 0xc822f0e380)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.(*ServeMux).ServeHTTP(0xc8201cda10, 0x7fbb72003238, 0xc820990750, 0xc822f0e380)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/mux.go:163 +0x17d
    github.com/influxdata/kapacitor/services/httpd.(*Handler).ServeHTTP(0xc8201c8230, 0x7fbb72003238, 0xc820990750, 0xc822f0e380)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:306 +0xe5
    net/http.serverHandler.ServeHTTP(0xc8201d6680, 0x7fbb72003238, 0xc820990750, 0xc822f0e380)
    /usr/local/go/src/net/http/server.go:2081 +0x19e
    net/http.(*conn).serve(0xc8208a2980)
    /usr/local/go/src/net/http/server.go:1472 +0xf2e
    created by net/http.(*Server).Serve
    /usr/local/go/src/net/http/server.go:2137 +0x44e

goroutine 55223 [select, 8 minutes]:
    github.com/influxdata/kapacitor.(*Edge).CollectPoint(0xc8201c81c0, 0xc8209dcbe8, 0x8, 0xc8205a6bac, 0x9, 0xc8205a6bc6, 0x7, 0xe56498, 0x0, 0x0, ...)
    /root/go/src/github.com/influxdata/kapacitor/edge.go:171 +0x1c6
    github.com/influxdata/kapacitor.(*TaskMaster).WritePoints(0xc820064c00, 0xc8205a6bac, 0x9, 0xc8205a6bc6, 0x7, 0x3, 0xc8203c8800, 0x69, 0x80, 0x0, ...)
    /root/go/src/github.com/influxdata/kapacitor/task_master.go:533 +0x294
    github.com/influxdata/kapacitor/services/httpd.(*Handler).serveWriteLine(0xc8201c8230, 0x7fbb72003320, 0xc82054c920, 0xc820587500, 0xc820678000, 0x554e, 0x7e00, 0x0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:430 +0x862
    github.com/influxdata/kapacitor/services/httpd.(*Handler).serveWrite(0xc8201c8230, 0x7fbb72003320, 0xc82054c920, 0xc820587500, 0x0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:393 +0x680
    github.com/influxdata/kapacitor/services/httpd.(*Handler).(github.com/influxdata/kapacitor/services/httpd.serveWrite)-fm(0x7fbb72003320, 0xc82054c920, 0xc820587500, 0x0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:132 +0x48
    github.com/influxdata/kapacitor/services/httpd.authenticate.func1(0x7fbb72003320, 0xc82054c920, 0xc820587500)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:532 +0x52
    net/http.HandlerFunc.ServeHTTP(0xc8201dde60, 0x7fbb72003320, 0xc82054c920, 0xc820587500)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.jsonContent.func1(0x7fbb72003320, 0xc82054c920, 0xc820587500)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:570 +0xaa
    net/http.HandlerFunc.ServeHTTP(0xc8201dde80, 0x7fbb72003320, 0xc82054c920, 0xc820587500)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.gzipFilter.func1(0x7fbb72003298, 0xc82054c8c0, 0xc820587500)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:563 +0x2af
    net/http.HandlerFunc.ServeHTTP(0xc8201ddea0, 0x7fbb72003298, 0xc82054c8c0, 0xc820587500)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.versionHeader.func1(0x7fbb72003298, 0xc82054c8c0, 0xc820587500)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:579 +0xc0
    net/http.HandlerFunc.ServeHTTP(0xc8201ddec0, 0x7fbb72003298, 0xc82054c8c0, 0xc820587500)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.cors.func1(0x7fbb72003298, 0xc82054c8c0, 0xc820587500)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:612 +0x329
    net/http.HandlerFunc.ServeHTTP(0xc8201ddee0, 0x7fbb72003298, 0xc82054c8c0, 0xc820587500)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.requestID.func1(0x7fbb72003298, 0xc82054c8c0, 0xc820587500)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:622 +0x15d
    net/http.HandlerFunc.ServeHTTP(0xc8201ddf00, 0x7fbb72003298, 0xc82054c8c0, 0xc820587500)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.recovery.func1(0x7fbb72003238, 0xc820370ea0, 0xc820587500)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:639 +0x114
    net/http.HandlerFunc.ServeHTTP(0xc8201ddf20, 0x7fbb72003238, 0xc820370ea0, 0xc820587500)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.(*ServeMux).ServeHTTP(0xc8201cda10, 0x7fbb72003238, 0xc820370ea0, 0xc820587500)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/mux.go:163 +0x17d
    github.com/influxdata/kapacitor/services/httpd.(*Handler).ServeHTTP(0xc8201c8230, 0x7fbb72003238, 0xc820370ea0, 0xc820587500)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:306 +0xe5
    net/http.serverHandler.ServeHTTP(0xc8201d6680, 0x7fbb72003238, 0xc820370ea0, 0xc820587500)
    /usr/local/go/src/net/http/server.go:2081 +0x19e
    net/http.(*conn).serve(0xc820670780)
    /usr/local/go/src/net/http/server.go:1472 +0xf2e
    created by net/http.(*Server).Serve
    /usr/local/go/src/net/http/server.go:2137 +0x44e

goroutine 55311 [semacquire, 4 minutes]:
    sync.runtime_Semacquire(0xc820064d60)
    /usr/local/go/src/runtime/sema.go:47 +0x26
    sync.(*RWMutex).RLock(0xc820064d54)
    /usr/local/go/src/sync/rwmutex.go:37 +0x58
    github.com/influxdata/kapacitor.(*TaskMaster).IsExecuting(0xc820064c00, 0xc820523410, 0x9, 0x600)
    /root/go/src/github.com/influxdata/kapacitor/task_master.go:422 +0x3d
    github.com/influxdata/kapacitor/services/task_store.(*Service).convertTask(0xc82017f0e0, 0xc820523410, 0x9, 0x0, 0xc820598b40, 0x1, 0x1, 0xc8209b4000, 0x67a, 0x0, ...)
    /root/go/src/github.com/influxdata/kapacitor/services/task_store/service.go:965 +0x15a
    github.com/influxdata/kapacitor/services/task_store.(*Service).handleTask(0xc82017f0e0, 0x7fbb72003320, 0xc820665ac0, 0xc822f0f420)
    /root/go/src/github.com/influxdata/kapacitor/services/task_store/service.go:464 +0x665
    github.com/influxdata/kapacitor/services/task_store.(*Service).(github.com/influxdata/kapacitor/services/task_store.handleTask)-fm(0x7fbb72003320, 0xc820665ac0, 0xc822f0f420)
    /root/go/src/github.com/influxdata/kapacitor/services/task_store/service.go:95 +0x3e
    net/http.HandlerFunc.ServeHTTP(0xc82000e850, 0x7fbb72003320, 0xc820665ac0, 0xc822f0f420)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.jsonContent.func1(0x7fbb72003320, 0xc820665ac0, 0xc822f0f420)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:570 +0xaa
    net/http.HandlerFunc.ServeHTTP(0xc820012860, 0x7fbb72003320, 0xc820665ac0, 0xc822f0f420)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.gzipFilter.func1(0x7fbb72003298, 0xc8206659c0, 0xc822f0f420)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:563 +0x2af
    net/http.HandlerFunc.ServeHTTP(0xc820012880, 0x7fbb72003298, 0xc8206659c0, 0xc822f0f420)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.versionHeader.func1(0x7fbb72003298, 0xc8206659c0, 0xc822f0f420)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:579 +0xc0
    net/http.HandlerFunc.ServeHTTP(0xc8200128a0, 0x7fbb72003298, 0xc8206659c0, 0xc822f0f420)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.cors.func1(0x7fbb72003298, 0xc8206659c0, 0xc822f0f420)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:612 +0x329
    net/http.HandlerFunc.ServeHTTP(0xc8200128c0, 0x7fbb72003298, 0xc8206659c0, 0xc822f0f420)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.requestID.func1(0x7fbb72003298, 0xc8206659c0, 0xc822f0f420)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:622 +0x15d
    net/http.HandlerFunc.ServeHTTP(0xc8200128e0, 0x7fbb72003298, 0xc8206659c0, 0xc822f0f420)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.recovery.func1(0x7fbb72003238, 0xc820991ba0, 0xc822f0f420)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:639 +0x114
    net/http.HandlerFunc.ServeHTTP(0xc820012900, 0x7fbb72003238, 0xc820991ba0, 0xc822f0f420)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.(*ServeMux).ServeHTTP(0xc8201cd9b0, 0x7fbb72003238, 0xc820991ba0, 0xc822f0f420)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/mux.go:163 +0x17d
    github.com/influxdata/kapacitor/services/httpd.(*Handler).ServeHTTP(0xc8201c8230, 0x7fbb72003238, 0xc820991ba0, 0xc822f0f420)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:306 +0xe5
    net/http.serverHandler.ServeHTTP(0xc8201d6680, 0x7fbb72003238, 0xc820991ba0, 0xc822f0f420)
    /usr/local/go/src/net/http/server.go:2081 +0x19e
    net/http.(*conn).serve(0xc8208a3980)
    /usr/local/go/src/net/http/server.go:1472 +0xf2e
    created by net/http.(*Server).Serve
    /usr/local/go/src/net/http/server.go:2137 +0x44e

goroutine 55213 [select, 8 minutes]:
    github.com/influxdata/kapacitor.(*Edge).CollectPoint(0xc8201c81c0, 0xc82090ce20, 0x6, 0xc8205a69cc, 0x8, 0xc8205a69e5, 0x7, 0xe56498, 0x0, 0x0, ...)
    /root/go/src/github.com/influxdata/kapacitor/edge.go:171 +0x1c6
    github.com/influxdata/kapacitor.(*TaskMaster).WritePoints(0xc820064c00, 0xc8205a69cc, 0x8, 0xc8205a69e5, 0x7, 0x3, 0xc8200cc800, 0xf, 0x10, 0x0, ...)
    /root/go/src/github.com/influxdata/kapacitor/task_master.go:533 +0x294
    github.com/influxdata/kapacitor/services/httpd.(*Handler).serveWriteLine(0xc8201c8230, 0x7fbb72003320, 0xc8251c23a0, 0xc822f0e700, 0xc8206cc000, 0xd9f, 0xe00, 0x0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:430 +0x862
    github.com/influxdata/kapacitor/services/httpd.(*Handler).serveWrite(0xc8201c8230, 0x7fbb72003320, 0xc8251c23a0, 0xc822f0e700, 0x0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:393 +0x680
    github.com/influxdata/kapacitor/services/httpd.(*Handler).(github.com/influxdata/kapacitor/services/httpd.serveWrite)-fm(0x7fbb72003320, 0xc8251c23a0, 0xc822f0e700, 0x0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:132 +0x48
    github.com/influxdata/kapacitor/services/httpd.authenticate.func1(0x7fbb72003320, 0xc8251c23a0, 0xc822f0e700)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:532 +0x52
    net/http.HandlerFunc.ServeHTTP(0xc8201dde60, 0x7fbb72003320, 0xc8251c23a0, 0xc822f0e700)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.jsonContent.func1(0x7fbb72003320, 0xc8251c23a0, 0xc822f0e700)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:570 +0xaa
    net/http.HandlerFunc.ServeHTTP(0xc8201dde80, 0x7fbb72003320, 0xc8251c23a0, 0xc822f0e700)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.gzipFilter.func1(0x7fbb72003298, 0xc8251c2360, 0xc822f0e700)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:563 +0x2af
    net/http.HandlerFunc.ServeHTTP(0xc8201ddea0, 0x7fbb72003298, 0xc8251c2360, 0xc822f0e700)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.versionHeader.func1(0x7fbb72003298, 0xc8251c2360, 0xc822f0e700)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:579 +0xc0
    net/http.HandlerFunc.ServeHTTP(0xc8201ddec0, 0x7fbb72003298, 0xc8251c2360, 0xc822f0e700)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.cors.func1(0x7fbb72003298, 0xc8251c2360, 0xc822f0e700)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:612 +0x329
    net/http.HandlerFunc.ServeHTTP(0xc8201ddee0, 0x7fbb72003298, 0xc8251c2360, 0xc822f0e700)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.requestID.func1(0x7fbb72003298, 0xc8251c2360, 0xc822f0e700)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:622 +0x15d
    net/http.HandlerFunc.ServeHTTP(0xc8201ddf00, 0x7fbb72003298, 0xc8251c2360, 0xc822f0e700)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.recovery.func1(0x7fbb72003238, 0xc8203700d0, 0xc822f0e700)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:639 +0x114
    net/http.HandlerFunc.ServeHTTP(0xc8201ddf20, 0x7fbb72003238, 0xc8203700d0, 0xc822f0e700)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.(*ServeMux).ServeHTTP(0xc8201cda10, 0x7fbb72003238, 0xc8203700d0, 0xc822f0e700)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/mux.go:163 +0x17d
    github.com/influxdata/kapacitor/services/httpd.(*Handler).ServeHTTP(0xc8201c8230, 0x7fbb72003238, 0xc8203700d0, 0xc822f0e700)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:306 +0xe5
    net/http.serverHandler.ServeHTTP(0xc8201d6680, 0x7fbb72003238, 0xc8203700d0, 0xc822f0e700)
    /usr/local/go/src/net/http/server.go:2081 +0x19e
    net/http.(*conn).serve(0xc8208a2580)
    /usr/local/go/src/net/http/server.go:1472 +0xf2e
    created by net/http.(*Server).Serve
    /usr/local/go/src/net/http/server.go:2137 +0x44e

goroutine 55160 [select, 10 minutes]:
    github.com/influxdata/kapacitor.(*Edge).NextPoint(0xc82004c3f0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
    /root/go/src/github.com/influxdata/kapacitor/edge.go:145 +0x1d5
    github.com/influxdata/kapacitor.(*FromNode).runStream(0xc8206734a0, 0x0, 0x0, 0x0, 0x0, 0x0)
    /root/go/src/github.com/influxdata/kapacitor/stream.go:77 +0x469
    github.com/influxdata/kapacitor.(*FromNode).(github.com/influxdata/kapacitor.runStream)-fm(0x0, 0x0, 0x0, 0x0, 0x0)
    /root/go/src/github.com/influxdata/kapacitor/stream.go:60 +0x4c
    github.com/influxdata/kapacitor.(*node).start.func1(0xc8206734a0, 0x0, 0x0, 0x0)
    /root/go/src/github.com/influxdata/kapacitor/node.go:130 +0x7f
    created by github.com/influxdata/kapacitor.(*node).start
    /root/go/src/github.com/influxdata/kapacitor/node.go:131 +0x53

goroutine 55335 [select, 2 minutes]:
    github.com/influxdata/kapacitor.(*Edge).CollectPoint(0xc8201c81c0, 0xc820296c48, 0x8, 0xc82066fe1c, 0x9, 0xc82066fe36, 0x7, 0xe56498, 0x0, 0x0, ...)
    /root/go/src/github.com/influxdata/kapacitor/edge.go:171 +0x1c6
    github.com/influxdata/kapacitor.(*TaskMaster).WritePoints(0xc820064c00, 0xc82066fe1c, 0x9, 0xc82066fe36, 0x7, 0x3, 0xc820454800, 0x69, 0x80, 0x0, ...)
    /root/go/src/github.com/influxdata/kapacitor/task_master.go:533 +0x294
    github.com/influxdata/kapacitor/services/httpd.(*Handler).serveWriteLine(0xc8201c8230, 0x7fbb72003320, 0xc82054d160, 0xc8205101c0, 0xc82045e000, 0x554e, 0x7e00, 0x0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:430 +0x862
    github.com/influxdata/kapacitor/services/httpd.(*Handler).serveWrite(0xc8201c8230, 0x7fbb72003320, 0xc82054d160, 0xc8205101c0, 0x0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:393 +0x680
    github.com/influxdata/kapacitor/services/httpd.(*Handler).(github.com/influxdata/kapacitor/services/httpd.serveWrite)-fm(0x7fbb72003320, 0xc82054d160, 0xc8205101c0, 0x0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:132 +0x48
    github.com/influxdata/kapacitor/services/httpd.authenticate.func1(0x7fbb72003320, 0xc82054d160, 0xc8205101c0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:532 +0x52
    net/http.HandlerFunc.ServeHTTP(0xc8201dde60, 0x7fbb72003320, 0xc82054d160, 0xc8205101c0)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.jsonContent.func1(0x7fbb72003320, 0xc82054d160, 0xc8205101c0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:570 +0xaa
    net/http.HandlerFunc.ServeHTTP(0xc8201dde80, 0x7fbb72003320, 0xc82054d160, 0xc8205101c0)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.gzipFilter.func1(0x7fbb72003298, 0xc82054d120, 0xc8205101c0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:563 +0x2af
    net/http.HandlerFunc.ServeHTTP(0xc8201ddea0, 0x7fbb72003298, 0xc82054d120, 0xc8205101c0)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.versionHeader.func1(0x7fbb72003298, 0xc82054d120, 0xc8205101c0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:579 +0xc0
    net/http.HandlerFunc.ServeHTTP(0xc8201ddec0, 0x7fbb72003298, 0xc82054d120, 0xc8205101c0)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.cors.func1(0x7fbb72003298, 0xc82054d120, 0xc8205101c0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:612 +0x329
    net/http.HandlerFunc.ServeHTTP(0xc8201ddee0, 0x7fbb72003298, 0xc82054d120, 0xc8205101c0)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.requestID.func1(0x7fbb72003298, 0xc82054d120, 0xc8205101c0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:622 +0x15d
    net/http.HandlerFunc.ServeHTTP(0xc8201ddf00, 0x7fbb72003298, 0xc82054d120, 0xc8205101c0)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.recovery.func1(0x7fbb72003238, 0xc820371d40, 0xc8205101c0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:639 +0x114
    net/http.HandlerFunc.ServeHTTP(0xc8201ddf20, 0x7fbb72003238, 0xc820371d40, 0xc8205101c0)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.(*ServeMux).ServeHTTP(0xc8201cda10, 0x7fbb72003238, 0xc820371d40, 0xc8205101c0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/mux.go:163 +0x17d
    github.com/influxdata/kapacitor/services/httpd.(*Handler).ServeHTTP(0xc8201c8230, 0x7fbb72003238, 0xc820371d40, 0xc8205101c0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:306 +0xe5
    net/http.serverHandler.ServeHTTP(0xc8201d6680, 0x7fbb72003238, 0xc820371d40, 0xc8205101c0)
    /usr/local/go/src/net/http/server.go:2081 +0x19e
    net/http.(*conn).serve(0xc8208a3e00)
    /usr/local/go/src/net/http/server.go:1472 +0xf2e
    created by net/http.(*Server).Serve
    /usr/local/go/src/net/http/server.go:2137 +0x44e

goroutine 55315 [select, 4 minutes]:
    github.com/influxdata/kapacitor.(*Edge).CollectPoint(0xc8201c81c0, 0xc820580b60, 0x8, 0xc8205a6f1c, 0x9, 0xc8205a6f36, 0x7, 0xe56498, 0x0, 0x0, ...)
    /root/go/src/github.com/influxdata/kapacitor/edge.go:171 +0x1c6
    github.com/influxdata/kapacitor.(*TaskMaster).WritePoints(0xc820064c00, 0xc8205a6f1c, 0x9, 0xc8205a6f36, 0x7, 0x3, 0xc8209e1800, 0x69, 0x80, 0x0, ...)
    /root/go/src/github.com/influxdata/kapacitor/task_master.go:533 +0x294
    github.com/influxdata/kapacitor/services/httpd.(*Handler).serveWriteLine(0xc8201c8230, 0x7fbb72003320, 0xc82054cbc0, 0xc820587ea0, 0xc8207de000, 0x554e, 0x7e00, 0x0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:430 +0x862
    github.com/influxdata/kapacitor/services/httpd.(*Handler).serveWrite(0xc8201c8230, 0x7fbb72003320, 0xc82054cbc0, 0xc820587ea0, 0x0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:393 +0x680
    github.com/influxdata/kapacitor/services/httpd.(*Handler).(github.com/influxdata/kapacitor/services/httpd.serveWrite)-fm(0x7fbb72003320, 0xc82054cbc0, 0xc820587ea0, 0x0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:132 +0x48
    github.com/influxdata/kapacitor/services/httpd.authenticate.func1(0x7fbb72003320, 0xc82054cbc0, 0xc820587ea0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:532 +0x52
    net/http.HandlerFunc.ServeHTTP(0xc8201dde60, 0x7fbb72003320, 0xc82054cbc0, 0xc820587ea0)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.jsonContent.func1(0x7fbb72003320, 0xc82054cbc0, 0xc820587ea0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:570 +0xaa
    net/http.HandlerFunc.ServeHTTP(0xc8201dde80, 0x7fbb72003320, 0xc82054cbc0, 0xc820587ea0)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.gzipFilter.func1(0x7fbb72003298, 0xc82054cb60, 0xc820587ea0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:563 +0x2af
    net/http.HandlerFunc.ServeHTTP(0xc8201ddea0, 0x7fbb72003298, 0xc82054cb60, 0xc820587ea0)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.versionHeader.func1(0x7fbb72003298, 0xc82054cb60, 0xc820587ea0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:579 +0xc0
    net/http.HandlerFunc.ServeHTTP(0xc8201ddec0, 0x7fbb72003298, 0xc82054cb60, 0xc820587ea0)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.cors.func1(0x7fbb72003298, 0xc82054cb60, 0xc820587ea0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:612 +0x329
    net/http.HandlerFunc.ServeHTTP(0xc8201ddee0, 0x7fbb72003298, 0xc82054cb60, 0xc820587ea0)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.requestID.func1(0x7fbb72003298, 0xc82054cb60, 0xc820587ea0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:622 +0x15d
    net/http.HandlerFunc.ServeHTTP(0xc8201ddf00, 0x7fbb72003298, 0xc82054cb60, 0xc820587ea0)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.recovery.func1(0x7fbb72003238, 0xc820991930, 0xc820587ea0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:639 +0x114
    net/http.HandlerFunc.ServeHTTP(0xc8201ddf20, 0x7fbb72003238, 0xc820991930, 0xc820587ea0)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.(*ServeMux).ServeHTTP(0xc8201cda10, 0x7fbb72003238, 0xc820991930, 0xc820587ea0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/mux.go:163 +0x17d
    github.com/influxdata/kapacitor/services/httpd.(*Handler).ServeHTTP(0xc8201c8230, 0x7fbb72003238, 0xc820991930, 0xc820587ea0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:306 +0xe5
    net/http.serverHandler.ServeHTTP(0xc8201d6680, 0x7fbb72003238, 0xc820991930, 0xc820587ea0)
    /usr/local/go/src/net/http/server.go:2081 +0x19e
    net/http.(*conn).serve(0xc820671300)
    /usr/local/go/src/net/http/server.go:1472 +0xf2e
    created by net/http.(*Server).Serve
    /usr/local/go/src/net/http/server.go:2137 +0x44e

goroutine 55253 [select, 8 minutes]:
    github.com/influxdata/kapacitor.(*Edge).CollectPoint(0xc8201c81c0, 0xc82076af08, 0x7, 0xc820597aac, 0x8, 0xc820597ac5, 0x7, 0xe56498, 0x0, 0x0, ...)
    /root/go/src/github.com/influxdata/kapacitor/edge.go:171 +0x1c6
    github.com/influxdata/kapacitor.(*TaskMaster).WritePoints(0xc820064c00, 0xc820597aac, 0x8, 0xc820597ac5, 0x7, 0x3, 0xc820178c00, 0x14, 0x20, 0x0, ...)
    /root/go/src/github.com/influxdata/kapacitor/task_master.go:533 +0x294
    github.com/influxdata/kapacitor/services/httpd.(*Handler).serveWriteLine(0xc8201c8230, 0x7fbb72003320, 0xc8206657a0, 0xc822f0eb60, 0xc822644000, 0x12c4, 0x1e00, 0x0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:430 +0x862
    github.com/influxdata/kapacitor/services/httpd.(*Handler).serveWrite(0xc8201c8230, 0x7fbb72003320, 0xc8206657a0, 0xc822f0eb60, 0x0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:393 +0x680
    github.com/influxdata/kapacitor/services/httpd.(*Handler).(github.com/influxdata/kapacitor/services/httpd.serveWrite)-fm(0x7fbb72003320, 0xc8206657a0, 0xc822f0eb60, 0x0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:132 +0x48
    github.com/influxdata/kapacitor/services/httpd.authenticate.func1(0x7fbb72003320, 0xc8206657a0, 0xc822f0eb60)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:532 +0x52
    net/http.HandlerFunc.ServeHTTP(0xc8201dde60, 0x7fbb72003320, 0xc8206657a0, 0xc822f0eb60)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.jsonContent.func1(0x7fbb72003320, 0xc8206657a0, 0xc822f0eb60)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:570 +0xaa
    net/http.HandlerFunc.ServeHTTP(0xc8201dde80, 0x7fbb72003320, 0xc8206657a0, 0xc822f0eb60)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.gzipFilter.func1(0x7fbb72003298, 0xc820665760, 0xc822f0eb60)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:563 +0x2af
    net/http.HandlerFunc.ServeHTTP(0xc8201ddea0, 0x7fbb72003298, 0xc820665760, 0xc822f0eb60)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.versionHeader.func1(0x7fbb72003298, 0xc820665760, 0xc822f0eb60)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:579 +0xc0
    net/http.HandlerFunc.ServeHTTP(0xc8201ddec0, 0x7fbb72003298, 0xc820665760, 0xc822f0eb60)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.cors.func1(0x7fbb72003298, 0xc820665760, 0xc822f0eb60)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:612 +0x329
    net/http.HandlerFunc.ServeHTTP(0xc8201ddee0, 0x7fbb72003298, 0xc820665760, 0xc822f0eb60)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.requestID.func1(0x7fbb72003298, 0xc820665760, 0xc822f0eb60)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:622 +0x15d
    net/http.HandlerFunc.ServeHTTP(0xc8201ddf00, 0x7fbb72003298, 0xc820665760, 0xc822f0eb60)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.recovery.func1(0x7fbb72003238, 0xc820991450, 0xc822f0eb60)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:639 +0x114
    net/http.HandlerFunc.ServeHTTP(0xc8201ddf20, 0x7fbb72003238, 0xc820991450, 0xc822f0eb60)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.(*ServeMux).ServeHTTP(0xc8201cda10, 0x7fbb72003238, 0xc820991450, 0xc822f0eb60)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/mux.go:163 +0x17d
    github.com/influxdata/kapacitor/services/httpd.(*Handler).ServeHTTP(0xc8201c8230, 0x7fbb72003238, 0xc820991450, 0xc822f0eb60)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:306 +0xe5
    net/http.serverHandler.ServeHTTP(0xc8201d6680, 0x7fbb72003238, 0xc820991450, 0xc822f0eb60)
    /usr/local/go/src/net/http/server.go:2081 +0x19e
    net/http.(*conn).serve(0xc8208a3300)
    /usr/local/go/src/net/http/server.go:1472 +0xf2e
    created by net/http.(*Server).Serve
    /usr/local/go/src/net/http/server.go:2137 +0x44e

goroutine 55370 [select, 2 minutes]:
    github.com/influxdata/kapacitor.(*Edge).CollectPoint(0xc8201c81c0, 0xc82019cb58, 0x8, 0xc82054274c, 0x9, 0xc820542766, 0x7, 0xe56498, 0x0, 0x0, ...)
    /root/go/src/github.com/influxdata/kapacitor/edge.go:171 +0x1c6
    github.com/influxdata/kapacitor.(*TaskMaster).WritePoints(0xc820064c00, 0xc82054274c, 0x9, 0xc820542766, 0x7, 0x3, 0xc820449800, 0x69, 0x80, 0x0, ...)
    /root/go/src/github.com/influxdata/kapacitor/task_master.go:533 +0x294
    github.com/influxdata/kapacitor/services/httpd.(*Handler).serveWriteLine(0xc8201c8230, 0x7fbb72003320, 0xc82059a480, 0xc822f0fdc0, 0xc82088e000, 0x554e, 0x7e00, 0x0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:430 +0x862
    github.com/influxdata/kapacitor/services/httpd.(*Handler).serveWrite(0xc8201c8230, 0x7fbb72003320, 0xc82059a480, 0xc822f0fdc0, 0x0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:393 +0x680
    github.com/influxdata/kapacitor/services/httpd.(*Handler).(github.com/influxdata/kapacitor/services/httpd.serveWrite)-fm(0x7fbb72003320, 0xc82059a480, 0xc822f0fdc0, 0x0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:132 +0x48
    github.com/influxdata/kapacitor/services/httpd.authenticate.func1(0x7fbb72003320, 0xc82059a480, 0xc822f0fdc0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:532 +0x52
    net/http.HandlerFunc.ServeHTTP(0xc8201dde60, 0x7fbb72003320, 0xc82059a480, 0xc822f0fdc0)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.jsonContent.func1(0x7fbb72003320, 0xc82059a480, 0xc822f0fdc0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:570 +0xaa
    net/http.HandlerFunc.ServeHTTP(0xc8201dde80, 0x7fbb72003320, 0xc82059a480, 0xc822f0fdc0)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.gzipFilter.func1(0x7fbb72003298, 0xc82059a420, 0xc822f0fdc0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:563 +0x2af
    net/http.HandlerFunc.ServeHTTP(0xc8201ddea0, 0x7fbb72003298, 0xc82059a420, 0xc822f0fdc0)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.versionHeader.func1(0x7fbb72003298, 0xc82059a420, 0xc822f0fdc0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:579 +0xc0
    net/http.HandlerFunc.ServeHTTP(0xc8201ddec0, 0x7fbb72003298, 0xc82059a420, 0xc822f0fdc0)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.cors.func1(0x7fbb72003298, 0xc82059a420, 0xc822f0fdc0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:612 +0x329
    net/http.HandlerFunc.ServeHTTP(0xc8201ddee0, 0x7fbb72003298, 0xc82059a420, 0xc822f0fdc0)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.requestID.func1(0x7fbb72003298, 0xc82059a420, 0xc822f0fdc0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:622 +0x15d
    net/http.HandlerFunc.ServeHTTP(0xc8201ddf00, 0x7fbb72003298, 0xc82059a420, 0xc822f0fdc0)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.recovery.func1(0x7fbb72003238, 0xc8200e2410, 0xc822f0fdc0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:639 +0x114
    net/http.HandlerFunc.ServeHTTP(0xc8201ddf20, 0x7fbb72003238, 0xc8200e2410, 0xc822f0fdc0)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.(*ServeMux).ServeHTTP(0xc8201cda10, 0x7fbb72003238, 0xc8200e2410, 0xc822f0fdc0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/mux.go:163 +0x17d
    github.com/influxdata/kapacitor/services/httpd.(*Handler).ServeHTTP(0xc8201c8230, 0x7fbb72003238, 0xc8200e2410, 0xc822f0fdc0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:306 +0xe5
    net/http.serverHandler.ServeHTTP(0xc8201d6680, 0x7fbb72003238, 0xc8200e2410, 0xc822f0fdc0)
    /usr/local/go/src/net/http/server.go:2081 +0x19e
    net/http.(*conn).serve(0xc820671e80)
    /usr/local/go/src/net/http/server.go:1472 +0xf2e
    created by net/http.(*Server).Serve
    /usr/local/go/src/net/http/server.go:2137 +0x44e

goroutine 55142 [select, 10 minutes]:
    github.com/influxdata/kapacitor.(*Edge).NextPoint(0xc8201c8310, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
    /root/go/src/github.com/influxdata/kapacitor/edge.go:145 +0x1d5
    github.com/influxdata/kapacitor.(*StreamNode).runSourceStream(0xc82084c4b0, 0x0, 0x0, 0x0, 0x0, 0x0)
    /root/go/src/github.com/influxdata/kapacitor/stream.go:28 +0x1f0
    github.com/influxdata/kapacitor.(*StreamNode).(github.com/influxdata/kapacitor.runSourceStream)-fm(0x0, 0x0, 0x0, 0x0, 0x0)
    /root/go/src/github.com/influxdata/kapacitor/stream.go:23 +0x4c
    github.com/influxdata/kapacitor.(*node).start.func1(0xc82084c4b0, 0x0, 0x0, 0x0)
    /root/go/src/github.com/influxdata/kapacitor/node.go:130 +0x7f
    created by github.com/influxdata/kapacitor.(*node).start
    /root/go/src/github.com/influxdata/kapacitor/node.go:131 +0x53

goroutine 55255 [select, 6 minutes]:
    github.com/influxdata/kapacitor.(*Edge).CollectPoint(0xc8201c81c0, 0xc8209dcc00, 0x6, 0xc8205a6b0c, 0x8, 0xc8205a6b25, 0x7, 0xe56498, 0x0, 0x0, ...)
    /root/go/src/github.com/influxdata/kapacitor/edge.go:171 +0x1c6
    github.com/influxdata/kapacitor.(*TaskMaster).WritePoints(0xc820064c00, 0xc8205a6b0c, 0x8, 0xc8205a6b25, 0x7, 0x3, 0xc8201ae000, 0x10, 0x10, 0x0, ...)
    /root/go/src/github.com/influxdata/kapacitor/task_master.go:533 +0x294
    github.com/influxdata/kapacitor/services/httpd.(*Handler).serveWriteLine(0xc8201c8230, 0x7fbb72003320, 0xc8251c2180, 0xc8205862a0, 0xc820a84000, 0xeef, 0x1e00, 0x0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:430 +0x862
    github.com/influxdata/kapacitor/services/httpd.(*Handler).serveWrite(0xc8201c8230, 0x7fbb72003320, 0xc8251c2180, 0xc8205862a0, 0x0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:393 +0x680
    github.com/influxdata/kapacitor/services/httpd.(*Handler).(github.com/influxdata/kapacitor/services/httpd.serveWrite)-fm(0x7fbb72003320, 0xc8251c2180, 0xc8205862a0, 0x0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:132 +0x48
    github.com/influxdata/kapacitor/services/httpd.authenticate.func1(0x7fbb72003320, 0xc8251c2180, 0xc8205862a0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:532 +0x52
    net/http.HandlerFunc.ServeHTTP(0xc8201dde60, 0x7fbb72003320, 0xc8251c2180, 0xc8205862a0)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.jsonContent.func1(0x7fbb72003320, 0xc8251c2180, 0xc8205862a0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:570 +0xaa
    net/http.HandlerFunc.ServeHTTP(0xc8201dde80, 0x7fbb72003320, 0xc8251c2180, 0xc8205862a0)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.gzipFilter.func1(0x7fbb72003298, 0xc8251c2120, 0xc8205862a0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:563 +0x2af
    net/http.HandlerFunc.ServeHTTP(0xc8201ddea0, 0x7fbb72003298, 0xc8251c2120, 0xc8205862a0)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.versionHeader.func1(0x7fbb72003298, 0xc8251c2120, 0xc8205862a0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:579 +0xc0
    net/http.HandlerFunc.ServeHTTP(0xc8201ddec0, 0x7fbb72003298, 0xc8251c2120, 0xc8205862a0)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.cors.func1(0x7fbb72003298, 0xc8251c2120, 0xc8205862a0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:612 +0x329
    net/http.HandlerFunc.ServeHTTP(0xc8201ddee0, 0x7fbb72003298, 0xc8251c2120, 0xc8205862a0)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.requestID.func1(0x7fbb72003298, 0xc8251c2120, 0xc8205862a0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:622 +0x15d
    net/http.HandlerFunc.ServeHTTP(0xc8201ddf00, 0x7fbb72003298, 0xc8251c2120, 0xc8205862a0)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.recovery.func1(0x7fbb72003238, 0xc820370410, 0xc8205862a0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:639 +0x114
    net/http.HandlerFunc.ServeHTTP(0xc8201ddf20, 0x7fbb72003238, 0xc820370410, 0xc8205862a0)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.(*ServeMux).ServeHTTP(0xc8201cda10, 0x7fbb72003238, 0xc820370410, 0xc8205862a0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/mux.go:163 +0x17d
    github.com/influxdata/kapacitor/services/httpd.(*Handler).ServeHTTP(0xc8201c8230, 0x7fbb72003238, 0xc820370410, 0xc8205862a0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:306 +0xe5
    net/http.serverHandler.ServeHTTP(0xc8201d6680, 0x7fbb72003238, 0xc820370410, 0xc8205862a0)
    /usr/local/go/src/net/http/server.go:2081 +0x19e
    net/http.(*conn).serve(0xc820670080)
    /usr/local/go/src/net/http/server.go:1472 +0xf2e
    created by net/http.(*Server).Serve
    /usr/local/go/src/net/http/server.go:2137 +0x44e

goroutine 55165 [chan receive, 10 minutes]:
    github.com/influxdata/kapacitor.(*node).Wait(0xc8200ccd00, 0x0, 0x0)
    /root/go/src/github.com/influxdata/kapacitor/node.go:152 +0xc6
    github.com/influxdata/kapacitor.(*ExecutingTask).Wait.func1(0x7fbb71fb3c28, 0xc8200ccd00, 0x0, 0x0)
    /root/go/src/github.com/influxdata/kapacitor/task.go:317 +0x33
    github.com/influxdata/kapacitor.(*ExecutingTask).rwalk(0xc82079d3b0, 0x1024598, 0x0, 0x0)
    /root/go/src/github.com/influxdata/kapacitor/task.go:147 +0x91
    github.com/influxdata/kapacitor.(*ExecutingTask).Wait(0xc82079d3b0, 0x0, 0x0)
    /root/go/src/github.com/influxdata/kapacitor/task.go:318 +0x39
    github.com/influxdata/kapacitor/services/task_store.(*Service).startTask.func1(0xc82079d3b0, 0xc82017f0e0, 0xc820064c00, 0xc8212531c0)
    /root/go/src/github.com/influxdata/kapacitor/services/task_store/service.go:1853 +0x30
    created by github.com/influxdata/kapacitor/services/task_store.(*Service).startTask
    /root/go/src/github.com/influxdata/kapacitor/services/task_store/service.go:1867 +0x324

goroutine 55232 [select, 6 minutes]:
    github.com/influxdata/kapacitor.(*Edge).CollectPoint(0xc8201c81c0, 0xc82077a8c8, 0x8, 0xc82059629c, 0x9, 0xc8205962b6, 0x7, 0xe56498, 0x0, 0x0, ...)
    /root/go/src/github.com/influxdata/kapacitor/edge.go:171 +0x1c6
    github.com/influxdata/kapacitor.(*TaskMaster).WritePoints(0xc820064c00, 0xc82059629c, 0x9, 0xc8205962b6, 0x7, 0x3, 0xc8204da000, 0x69, 0x80, 0x0, ...)
    /root/go/src/github.com/influxdata/kapacitor/task_master.go:533 +0x294
    github.com/influxdata/kapacitor/services/httpd.(*Handler).serveWriteLine(0xc8201c8230, 0x7fbb72003320, 0xc8210c40c0, 0xc822f0e0e0, 0xc820944000, 0x554e, 0x7e00, 0x0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:430 +0x862
    github.com/influxdata/kapacitor/services/httpd.(*Handler).serveWrite(0xc8201c8230, 0x7fbb72003320, 0xc8210c40c0, 0xc822f0e0e0, 0x0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:393 +0x680
    github.com/influxdata/kapacitor/services/httpd.(*Handler).(github.com/influxdata/kapacitor/services/httpd.serveWrite)-fm(0x7fbb72003320, 0xc8210c40c0, 0xc822f0e0e0, 0x0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:132 +0x48
    github.com/influxdata/kapacitor/services/httpd.authenticate.func1(0x7fbb72003320, 0xc8210c40c0, 0xc822f0e0e0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:532 +0x52
    net/http.HandlerFunc.ServeHTTP(0xc8201dde60, 0x7fbb72003320, 0xc8210c40c0, 0xc822f0e0e0)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.jsonContent.func1(0x7fbb72003320, 0xc8210c40c0, 0xc822f0e0e0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:570 +0xaa
    net/http.HandlerFunc.ServeHTTP(0xc8201dde80, 0x7fbb72003320, 0xc8210c40c0, 0xc822f0e0e0)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.gzipFilter.func1(0x7fbb72003298, 0xc8210c4080, 0xc822f0e0e0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:563 +0x2af
    net/http.HandlerFunc.ServeHTTP(0xc8201ddea0, 0x7fbb72003298, 0xc8210c4080, 0xc822f0e0e0)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.versionHeader.func1(0x7fbb72003298, 0xc8210c4080, 0xc822f0e0e0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:579 +0xc0
    net/http.HandlerFunc.ServeHTTP(0xc8201ddec0, 0x7fbb72003298, 0xc8210c4080, 0xc822f0e0e0)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.cors.func1(0x7fbb72003298, 0xc8210c4080, 0xc822f0e0e0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:612 +0x329
    net/http.HandlerFunc.ServeHTTP(0xc8201ddee0, 0x7fbb72003298, 0xc8210c4080, 0xc822f0e0e0)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.requestID.func1(0x7fbb72003298, 0xc8210c4080, 0xc822f0e0e0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:622 +0x15d
    net/http.HandlerFunc.ServeHTTP(0xc8201ddf00, 0x7fbb72003298, 0xc8210c4080, 0xc822f0e0e0)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.recovery.func1(0x7fbb72003238, 0xc8209900d0, 0xc822f0e0e0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:639 +0x114
    net/http.HandlerFunc.ServeHTTP(0xc8201ddf20, 0x7fbb72003238, 0xc8209900d0, 0xc822f0e0e0)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.(*ServeMux).ServeHTTP(0xc8201cda10, 0x7fbb72003238, 0xc8209900d0, 0xc822f0e0e0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/mux.go:163 +0x17d
    github.com/influxdata/kapacitor/services/httpd.(*Handler).ServeHTTP(0xc8201c8230, 0x7fbb72003238, 0xc8209900d0, 0xc822f0e0e0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:306 +0xe5
    net/http.serverHandler.ServeHTTP(0xc8201d6680, 0x7fbb72003238, 0xc8209900d0, 0xc822f0e0e0)
    /usr/local/go/src/net/http/server.go:2081 +0x19e
    net/http.(*conn).serve(0xc8208a2180)
    /usr/local/go/src/net/http/server.go:1472 +0xf2e
    created by net/http.(*Server).Serve
    /usr/local/go/src/net/http/server.go:2137 +0x44e

goroutine 46885 [chan send, 261 minutes]:
    github.com/influxdata/kapacitor/tick/ast.lexToken(0xc820922c60, 0x1024920)
    /root/go/src/github.com/influxdata/kapacitor/tick/ast/lex.go:343 +0x335
    github.com/influxdata/kapacitor/tick/ast.(*lexer).run(0xc820922c60)
    /root/go/src/github.com/influxdata/kapacitor/tick/ast/lex.go:238 +0x32
    created by github.com/influxdata/kapacitor/tick/ast.lex
    /root/go/src/github.com/influxdata/kapacitor/tick/ast/lex.go:230 +0xc1

goroutine 55306 [select, 4 minutes]:
    github.com/influxdata/kapacitor.(*Edge).CollectPoint(0xc8201c81c0, 0xc82059d3e0, 0x4, 0xc82066ed3c, 0x8, 0xc82066ed55, 0x7, 0xe56498, 0x0, 0x0, ...)
    /root/go/src/github.com/influxdata/kapacitor/edge.go:171 +0x1c6
    github.com/influxdata/kapacitor.(*TaskMaster).WritePoints(0xc820064c00, 0xc82066ed3c, 0x8, 0xc82066ed55, 0x7, 0x3, 0xc820911400, 0x14, 0x20, 0x0, ...)
    /root/go/src/github.com/influxdata/kapacitor/task_master.go:533 +0x294
    github.com/influxdata/kapacitor/services/httpd.(*Handler).serveWriteLine(0xc8201c8230, 0x7fbb72003320, 0xc8205a4f80, 0xc822f0f0a0, 0xc820846000, 0x13ca, 0x1e00, 0x0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:430 +0x862
    github.com/influxdata/kapacitor/services/httpd.(*Handler).serveWrite(0xc8201c8230, 0x7fbb72003320, 0xc8205a4f80, 0xc822f0f0a0, 0x0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:393 +0x680
    github.com/influxdata/kapacitor/services/httpd.(*Handler).(github.com/influxdata/kapacitor/services/httpd.serveWrite)-fm(0x7fbb72003320, 0xc8205a4f80, 0xc822f0f0a0, 0x0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:132 +0x48
    github.com/influxdata/kapacitor/services/httpd.authenticate.func1(0x7fbb72003320, 0xc8205a4f80, 0xc822f0f0a0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:532 +0x52
    net/http.HandlerFunc.ServeHTTP(0xc8201dde60, 0x7fbb72003320, 0xc8205a4f80, 0xc822f0f0a0)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.jsonContent.func1(0x7fbb72003320, 0xc8205a4f80, 0xc822f0f0a0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:570 +0xaa
    net/http.HandlerFunc.ServeHTTP(0xc8201dde80, 0x7fbb72003320, 0xc8205a4f80, 0xc822f0f0a0)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.gzipFilter.func1(0x7fbb72003298, 0xc8205a4f20, 0xc822f0f0a0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:563 +0x2af
    net/http.HandlerFunc.ServeHTTP(0xc8201ddea0, 0x7fbb72003298, 0xc8205a4f20, 0xc822f0f0a0)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.versionHeader.func1(0x7fbb72003298, 0xc8205a4f20, 0xc822f0f0a0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:579 +0xc0
    net/http.HandlerFunc.ServeHTTP(0xc8201ddec0, 0x7fbb72003298, 0xc8205a4f20, 0xc822f0f0a0)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.cors.func1(0x7fbb72003298, 0xc8205a4f20, 0xc822f0f0a0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:612 +0x329
    net/http.HandlerFunc.ServeHTTP(0xc8201ddee0, 0x7fbb72003298, 0xc8205a4f20, 0xc822f0f0a0)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.requestID.func1(0x7fbb72003298, 0xc8205a4f20, 0xc822f0f0a0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:622 +0x15d
    net/http.HandlerFunc.ServeHTTP(0xc8201ddf00, 0x7fbb72003298, 0xc8205a4f20, 0xc822f0f0a0)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.recovery.func1(0x7fbb72003238, 0xc820371790, 0xc822f0f0a0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:639 +0x114
    net/http.HandlerFunc.ServeHTTP(0xc8201ddf20, 0x7fbb72003238, 0xc820371790, 0xc822f0f0a0)
    /usr/local/go/src/net/http/server.go:1618 +0x3a
    github.com/influxdata/kapacitor/services/httpd.(*ServeMux).ServeHTTP(0xc8201cda10, 0x7fbb72003238, 0xc820371790, 0xc822f0f0a0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/mux.go:163 +0x17d
    github.com/influxdata/kapacitor/services/httpd.(*Handler).ServeHTTP(0xc8201c8230, 0x7fbb72003238, 0xc820371790, 0xc822f0f0a0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:306 +0xe5
    net/http.serverHandler.ServeHTTP(0xc8201d6680, 0x7fbb72003238, 0xc820371790, 0xc822f0f0a0)
    /usr/local/go/src/net/http/server.go:2081 +0x19e
    net/http.(*conn).serve(0xc8208a3580)
    /usr/local/go/src/net/http/server.go:1472 +0xf2e
    created by net/http.(*Server).Serve
    /usr/local/go/src/net/http/server.go:2137 +0x44e

goroutine 55219 [select, 8 minutes]:
    github.com/influxdata/kapacitor.(*Edge).CollectPoint(0xc8201c81c0, 0xc82024cd40, 0x8, 0xc8205a6a6c, 0x9, 0xc8205a6a86, 0x7, 0xe56498, 0x0, 0x0, ...)
    /root/go/src/github.com/influxdata/kapacitor/edge.go:171 +0x1c6
    github.com/influxdata/kapacitor.(*TaskMaster).WritePoints(0xc820064c00, 0xc8205a6a6c, 0x9, 0xc8205a6a86, 0x7, 0x3, 0xc821251800, 0x69, 0x80, 0x0, ...)
    /root/go/src/github.com/influxdata/kapacitor/task_master.go:533 +0x294
    github.com/influxdata/kapacitor/services/httpd.(*Handler).serveWriteLine(0xc8201c8230, 0x7fbb72003320, 0xc82054c420, 0xc8205869a0, 0xc8202b8000, 0x554e, 0x7e00, 0x0)
    /root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:430 +0x862
    github.com/influxdata/kapacitor/services/httpd.(*Handler).serveWrite(0xc8201c8230, 0x7fbb72003320, 0xc82054c420, 0xc8205869a0, 0x0)
@nathanielc
Copy link
Contributor

@phemmer This only happens if the task already exists and is running.

Specifically the kapacitor disable foo command hangs on a running task.

This should be fixed.

@phemmer
Copy link
Author

phemmer commented Jul 25, 2016

@nathanielc as of when? i just downloaded the nightly from https://influxdata.com/downloads/#kapacitor and can still reproduce it.

@nathanielc
Copy link
Contributor

nathanielc commented Jul 25, 2016

@phemmer I couldn't reproduce the issue until after the task was enabled, using 1.0 beta3 or current master.

kapacitor delete tasks foo #just in case it already exists. If this hangs restart Kapacitor and try again.
cat > /tmp/foo <<'EOF'
var foo = 'bar'
EOF

kapacitor define foo -type stream -tick /tmp/foo -dbrp telegraf.autogen # works fine
kapacitor enable foo
kapacitor define foo -type stream -tick /tmp/foo -dbrp telegraf.autogen #hangs
# restart Kapacitor
kapacitor enable foo
kapacitor disable foo # also hangs

If kapacitor is not in a broken state you can also do

kapacitor define foo -type stream -tick /tmp/foo -dbrp telegraf.autogen -no-reload

and it will successfully update the task.

I replied before I saw your edit, I can confirm that other API requests hang as well. I'll look into it, I have a good idea as to what might be going on.

@nathanielc nathanielc changed the title Kapacitor hangs loading script with no input Kapacitor hangs on most requests after disabling script which consumes no data. Jul 25, 2016
@nathanielc nathanielc mentioned this issue Jul 25, 2016
3 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants