Influxdb 0.11 Slice index out of bounds

[Madhu94]

I'm trying to bulk insert about 1.3m records. I should mention that some of them are duplicates(points with same tagset, same timestamp). Influxdb crashes and its hard to get it running again, unless I delete the .influxdb file.What could cause this problem? This is the error I'm getting.

panic: runtime error: slice bounds out of range

goroutine 405 [running]:
github.com/influxdata/influxdb/tsdb/engine/tsm1.(*indirectIndex).UnmarshalBinary(0xc2bbb21600, 0x7f89ecd8a1c8, 0xb8a68e7, 0xb8a68ef, 0x0, 0x0)
    /tmp/influxdb/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/reader.go:673 +0x466
github.com/influxdata/influxdb/tsdb/engine/tsm1.(*mmapAccessor).init(0xc2bb9a53b0, 0x0, 0x0, 0x0, 0x0)
    /tmp/influxdb/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/reader.go:970 +0x4e8
github.com/influxdata/influxdb/tsdb/engine/tsm1.NewTSMReaderWithOptions(0x0, 0x0, 0xc2bba0afc0, 0x0, 0x0, 0x0)
    /tmp/influxdb/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/reader.go:158 +0x2e5
github.com/influxdata/influxdb/tsdb/engine/tsm1.(*FileStore).Replace(0xc24e73c180, 0x0, 0x0, 0x0, 0xc254807d20, 0x1, 0x1, 0x0, 0x0)
    /tmp/influxdb/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/file_store.go:409 +0x4ff
github.com/influxdata/influxdb/tsdb/engine/tsm1.(*Engine).writeSnapshotAndCommit(0xc25725c000, 0xc2865ba6c0, 0x3, 0x4, 0xc22fe22100, 0xc2865b4330, 0x0, 0x0)
    /tmp/influxdb/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/engine.go:481 +0x2c0
github.com/influxdata/influxdb/tsdb/engine/tsm1.(*Engine).WriteSnapshot(0xc25725c000, 0x0, 0x0)
    /tmp/influxdb/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/engine.go:459 +0x1b5
github.com/influxdata/influxdb/tsdb/engine/tsm1.(*Engine).compactCache(0xc25725c000)
    /tmp/influxdb/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/engine.go:507 +0x124
created by github.com/influxdata/influxdb/tsdb/engine/tsm1.(*Engine).Open
    /tmp/influxdb/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/engine.go:153 +0x251

goroutine 1 [chan receive, 16 minutes]:
main.(*Main).Run(0xc208039700, 0xc20800a000, 0x0, 0x0, 0x0, 0x0)
    /tmp/influxdb/src/github.com/influxdata/influxdb/cmd/influxd/main.go:94 +0x765
main.main()
    /tmp/influxdb/src/github.com/influxdata/influxdb/cmd/influxd/main.go:45 +0xdc

goroutine 6 [syscall, 16 minutes]:
os/signal.loop()
    /root/.gvm/gos/go1.4.3/src/os/signal/signal_unix.go:21 +0x1f
created by os/signal.init·1
    /root/.gvm/gos/go1.4.3/src/os/signal/signal_unix.go:27 +0x35

goroutine 18 [IO wait, 16 minutes]:
net.(*pollDesc).Wait(0xc2080113a0, 0x72, 0x0, 0x0)
    /root/.gvm/gos/go1.4.3/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc2080113a0, 0x0, 0x0)
    /root/.gvm/gos/go1.4.3/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).accept(0xc208011340, 0x0, 0x7f8a4f8af0d0, 0xc2080ec168)
    /root/.gvm/gos/go1.4.3/src/net/fd_unix.go:419 +0x40b
net.(*TCPListener).AcceptTCP(0xc20802a0e8, 0xc208053db8, 0x0, 0x0)
    /root/.gvm/gos/go1.4.3/src/net/tcpsock_posix.go:234 +0x4e
net.(*TCPListener).Accept(0xc20802a0e8, 0x0, 0x0, 0x0, 0x0)
    /root/.gvm/gos/go1.4.3/src/net/tcpsock_posix.go:244 +0x4c
net/http.(*Server).Serve(0xc208124000, 0x7f8a4f8b46e8, 0xc20802a0e8, 0x0, 0x0)
    /root/.gvm/gos/go1.4.3/src/net/http/server.go:1728 +0x92
net/http.Serve(0x7f8a4f8b46e8, 0xc20802a0e8, 0x7f8a4f8b57d0, 0xc208108120, 0x0, 0x0)
    /root/.gvm/gos/go1.4.3/src/net/http/server.go:1606 +0xa1
github.com/influxdata/influxdb/services/meta.(*Service).serve(0xc208030d20)
    /tmp/influxdb/src/github.com/influxdata/influxdb/services/meta/service.go:152 +0x7a
created by github.com/influxdata/influxdb/services/meta.(*Service).Open
    /tmp/influxdb/src/github.com/influxdata/influxdb/services/meta/service.go:127 +0xa82

goroutine 17 [syscall, 16 minutes, locked to thread]:
runtime.goexit()
    /root/.gvm/gos/go1.4.3/src/runtime/asm_amd64.s:2232 +0x1

goroutine 8 [IO wait, 16 minutes]:
net.(*pollDesc).Wait(0xc208011330, 0x72, 0x0, 0x0)
    /root/.gvm/gos/go1.4.3/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc208011330, 0x0, 0x0)
    /root/.gvm/gos/go1.4.3/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).accept(0xc2080112d0, 0x0, 0x7f8a4f8af0d0, 0xc208104000)
    /root/.gvm/gos/go1.4.3/src/net/fd_unix.go:419 +0x40b
net.(*TCPListener).AcceptTCP(0xc20802a0e0, 0x0, 0x0, 0x0)
    /root/.gvm/gos/go1.4.3/src/net/tcpsock_posix.go:234 +0x4e
net.(*TCPListener).Accept(0xc20802a0e0, 0x0, 0x0, 0x0, 0x0)
    /root/.gvm/gos/go1.4.3/src/net/tcpsock_posix.go:244 +0x4c
github.com/influxdata/influxdb/tcp.(*Mux).Serve(0xc20805c840, 0x7f8a4f8b46e8, 0xc20802a0e0, 0x0, 0x0)
    /tmp/influxdb/src/github.com/influxdata/influxdb/tcp/mux.go:52 +0xc7
created by github.com/influxdata/influxdb/cmd/influxd/run.(*Server).Open
    /tmp/influxdb/src/github.com/influxdata/influxdb/cmd/influxd/run/server.go:392 +0x25e

goroutine 19 [chan receive, 16 minutes]:
github.com/influxdata/influxdb/tcp.(*listener).Accept(0xc2080ecc30, 0x0, 0x0, 0x0, 0x0)
    /tmp/influxdb/src/github.com/influxdata/influxdb/tcp/mux.go:135 +0x80
github.com/influxdata/influxdb/services/meta.(*raftLayer).Accept(0xc20810c150, 0x0, 0x0, 0x0, 0x0)
    /tmp/influxdb/src/github.com/influxdata/influxdb/services/meta/raft_state.go:330 +0x65
github.com/hashicorp/raft.(*NetworkTransport).listen(0xc20812c070)
    /tmp/influxdb/src/github.com/hashicorp/raft/net_transport.go:346 +0x50
created by github.com/hashicorp/raft.NewNetworkTransport
    /tmp/influxdb/src/github.com/hashicorp/raft/net_transport.go:138 +0x2c4

goroutine 10 [select]:
github.com/hashicorp/raft.(*Raft).leaderLoop(0xc2081361c0)
    /tmp/influxdb/src/github.com/hashicorp/raft/raft.go:807 +0xc7c
github.com/hashicorp/raft.(*Raft).runLeader(0xc2081361c0)
    /tmp/influxdb/src/github.com/hashicorp/raft/raft.go:779 +0x74e
github.com/hashicorp/raft.(*Raft).run(0xc2081361c0)
    /tmp/influxdb/src/github.com/hashicorp/raft/raft.go:576 +0xc9
github.com/hashicorp/raft.*Raft.(github.com/hashicorp/raft.run)·fm()
    /tmp/influxdb/src/github.com/hashicorp/raft/raft.go:241 +0x27
github.com/hashicorp/raft.func·011()
    /tmp/influxdb/src/github.com/hashicorp/raft/state.go:152 +0x51
created by github.com/hashicorp/raft.(*raftState).goFunc
    /tmp/influxdb/src/github.com/hashicorp/raft/state.go:153 +0xe3

goroutine 11 [select, 1 minutes]:
github.com/hashicorp/raft.(*Raft).runFSM(0xc2081361c0)
    /tmp/influxdb/src/github.com/hashicorp/raft/raft.go:490 +0xf74
github.com/hashicorp/raft.*Raft.(github.com/hashicorp/raft.runFSM)·fm()
    /tmp/influxdb/src/github.com/hashicorp/raft/raft.go:242 +0x27
github.com/hashicorp/raft.func·011()
    /tmp/influxdb/src/github.com/hashicorp/raft/state.go:152 +0x51
created by github.com/hashicorp/raft.(*raftState).goFunc
    /tmp/influxdb/src/github.com/hashicorp/raft/state.go:153 +0xe3

goroutine 12 [select, 1 minutes]:
github.com/hashicorp/raft.(*Raft).runSnapshots(0xc2081361c0)
    /tmp/influxdb/src/github.com/hashicorp/raft/raft.go:1594 +0x466
github.com/hashicorp/raft.*Raft.(github.com/hashicorp/raft.runSnapshots)·fm()
    /tmp/influxdb/src/github.com/hashicorp/raft/raft.go:243 +0x27
github.com/hashicorp/raft.func·011()
    /tmp/influxdb/src/github.com/hashicorp/raft/state.go:152 +0x51
created by github.com/hashicorp/raft.(*raftState).goFunc
    /tmp/influxdb/src/github.com/hashicorp/raft/state.go:153 +0xe3

goroutine 13 [select, 16 minutes]:
github.com/influxdata/influxdb/services/meta.(*raftState).logLeaderChanges(0xc2081061b0)
    /tmp/influxdb/src/github.com/influxdata/influxdb/services/meta/raft_state.go:138 +0x459
created by github.com/influxdata/influxdb/services/meta.(*raftState).open
    /tmp/influxdb/src/github.com/influxdata/influxdb/services/meta/raft_state.go:128 +0xf35

goroutine 34 [select, 16 minutes]:
github.com/influxdata/influxdb/cmd/influxd/run.(*Server).monitorErrorChan(0xc2080f4000, 0xc20805c5a0)
    /tmp/influxdb/src/github.com/influxdata/influxdb/cmd/influxd/run/server.go:611 +0x14a
created by github.com/influxdata/influxdb/cmd/influxd/run.(*Server).Open
    /tmp/influxdb/src/github.com/influxdata/influxdb/cmd/influxd/run/server.go:400 +0x44c

goroutine 16 [IO wait, 1 minutes]:
net.(*pollDesc).Wait(0xc20816e060, 0x72, 0x0, 0x0)
    /root/.gvm/gos/go1.4.3/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc20816e060, 0x0, 0x0)
    /root/.gvm/gos/go1.4.3/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).Read(0xc20816e000, 0xc20818e000, 0x8000, 0x8000, 0x0, 0x7f8a4f8af0d0, 0xc25c2b42e8)
    /root/.gvm/gos/go1.4.3/src/net/fd_unix.go:242 +0x40f
net.(*conn).Read(0xc208170000, 0xc20818e000, 0x8000, 0x8000, 0x0, 0x0, 0x0)
    /root/.gvm/gos/go1.4.3/src/net/net.go:121 +0xdc
io.Copy(0x7f8a4f8b9a70, 0xc20802a180, 0x7f8a4f8b9610, 0xc208170000, 0x7d8, 0x0, 0x0)
    /root/.gvm/gos/go1.4.3/src/io/io.go:362 +0x1f6
net/http.func·008()
    /root/.gvm/gos/go1.4.3/src/net/http/server.go:171 +0x7d
created by net/http.(*conn).closeNotify
    /root/.gvm/gos/go1.4.3/src/net/http/server.go:177 +0x2e2

goroutine 14 [IO wait, 1 minutes]:
net.(*pollDesc).Wait(0xc20812cb50, 0x72, 0x0, 0x0)
    /root/.gvm/gos/go1.4.3/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc20812cb50, 0x0, 0x0)
    /root/.gvm/gos/go1.4.3/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).Read(0xc20812caf0, 0xc208174000, 0x1000, 0x1000, 0x0, 0x7f8a4f8af0d0, 0xc24e9aa000)
    /root/.gvm/gos/go1.4.3/src/net/fd_unix.go:242 +0x40f
net.(*conn).Read(0xc20802a148, 0xc208174000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
    /root/.gvm/gos/go1.4.3/src/net/net.go:121 +0xdc
net/http.noteEOFReader.Read(0x7f8a4f8b9610, 0xc20802a148, 0xc208060058, 0xc208174000, 0x1000, 0x1000, 0xaa9760, 0x0, 0x0)
    /root/.gvm/gos/go1.4.3/src/net/http/transport.go:1270 +0x6e
net/http.(*noteEOFReader).Read(0xc20801fa40, 0xc208174000, 0x1000, 0x1000, 0xc2080d2000, 0x0, 0x0)
    <autogenerated>:125 +0xd4
bufio.(*Reader).fill(0xc208124b40)
    /root/.gvm/gos/go1.4.3/src/bufio/bufio.go:97 +0x1ce
bufio.(*Reader).Peek(0xc208124b40, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0)
    /root/.gvm/gos/go1.4.3/src/bufio/bufio.go:132 +0xf0
net/http.(*persistConn).readLoop(0xc208060000)
    /root/.gvm/gos/go1.4.3/src/net/http/transport.go:842 +0xa4
created by net/http.(*Transport).dialConn
    /root/.gvm/gos/go1.4.3/src/net/http/transport.go:660 +0xc9f

goroutine 15 [select, 1 minutes]:
net/http.(*persistConn).writeLoop(0xc208060000)
    /root/.gvm/gos/go1.4.3/src/net/http/transport.go:945 +0x41d
created by net/http.(*Transport).dialConn
    /root/.gvm/gos/go1.4.3/src/net/http/transport.go:661 +0xcbc

goroutine 50 [select, 1 minutes]:
github.com/influxdata/influxdb/services/meta.(*handler).serveSnapshot(0xc208108120, 0x7f8a4f8b9940, 0xc2473d5e00, 0xc250ac60d0)
    /tmp/influxdb/src/github.com/influxdata/influxdb/services/meta/handler.go:250 +0x5a1
github.com/influxdata/influxdb/services/meta.*handler.(github.com/influxdata/influxdb/services/meta.serveSnapshot)·fm(0x7f8a4f8b9940, 0xc2473d5e00, 0xc250ac60d0)
    /tmp/influxdb/src/github.com/influxdata/influxdb/services/meta/handler.go:91 +0x45
net/http.HandlerFunc.ServeHTTP(0xc24e03aaa0, 0x7f8a4f8b9940, 0xc2473d5e00, 0xc250ac60d0)
    /root/.gvm/gos/go1.4.3/src/net/http/server.go:1265 +0x41
github.com/influxdata/influxdb/services/meta.func·001(0x7f8a4f8b9868, 0xc2473d5d60, 0xc250ac60d0)
    /tmp/influxdb/src/github.com/influxdata/influxdb/services/meta/handler.go:424 +0x288
net/http.HandlerFunc.ServeHTTP(0xc24e03aac0, 0x7f8a4f8b9868, 0xc2473d5d60, 0xc250ac60d0)
    /root/.gvm/gos/go1.4.3/src/net/http/server.go:1265 +0x41
github.com/influxdata/influxdb/services/meta.func·002(0x7f8a4f8b9868, 0xc2473d5d60, 0xc250ac60d0)
    /tmp/influxdb/src/github.com/influxdata/influxdb/services/meta/handler.go:433 +0xbe
net/http.HandlerFunc.ServeHTTP(0xc2473d5cc0, 0x7f8a4f8b9868, 0xc2473d5d60, 0xc250ac60d0)
    /root/.gvm/gos/go1.4.3/src/net/http/server.go:1265 +0x41
github.com/influxdata/influxdb/services/meta.func·003(0x7f8a4f8b9868, 0xc2473d5d60, 0xc250ac60d0)
    /tmp/influxdb/src/github.com/influxdata/influxdb/services/meta/handler.go:443 +0x179
net/http.HandlerFunc.ServeHTTP(0xc24e03abd0, 0x7f8a4f8b9868, 0xc2473d5d60, 0xc250ac60d0)
    /root/.gvm/gos/go1.4.3/src/net/http/server.go:1265 +0x41
github.com/influxdata/influxdb/services/meta.func·006(0x7f8a4f8b9830, 0xc24b882320, 0xc250ac60d0)
    /tmp/influxdb/src/github.com/influxdata/influxdb/services/meta/handler.go:472 +0x197
net/http.HandlerFunc.ServeHTTP(0xc2473d5d40, 0x7f8a4f8b9830, 0xc24b882320, 0xc250ac60d0)
    /root/.gvm/gos/go1.4.3/src/net/http/server.go:1265 +0x41
github.com/influxdata/influxdb/services/meta.(*handler).ServeHTTP(0xc208108120, 0x7f8a4f8b9830, 0xc24b882320, 0xc250ac60d0)
    /tmp/influxdb/src/github.com/influxdata/influxdb/services/meta/handler.go:91 +0x462
net/http.serverHandler.ServeHTTP(0xc208124000, 0x7f8a4f8b9830, 0xc24b882320, 0xc250ac60d0)
    /root/.gvm/gos/go1.4.3/src/net/http/server.go:1703 +0x19a
net/http.(*conn).serve(0xc208172000)
    /root/.gvm/gos/go1.4.3/src/net/http/server.go:1204 +0xb57
created by net/http.(*Server).Serve
    /root/.gvm/gos/go1.4.3/src/net/http/server.go:1751 +0x35e

goroutine 20 [select, 1 minutes]:
net/http.(*persistConn).roundTrip(0xc208060000, 0xc24e70a410, 0x0, 0x0, 0x0)
    /root/.gvm/gos/go1.4.3/src/net/http/transport.go:1082 +0x7ad
net/http.(*Transport).RoundTrip(0xc208074000, 0xc250910270, 0xc242954a20, 0x0, 0x0)
    /root/.gvm/gos/go1.4.3/src/net/http/transport.go:235 +0x558
net/http.send(0xc250910270, 0x7f8a4f8af278, 0xc208074000, 0x15, 0x0, 0x0)
    /root/.gvm/gos/go1.4.3/src/net/http/client.go:219 +0x4fc
net/http.(*Client).send(0x10acae0, 0xc250910270, 0x15, 0x0, 0x0)
    /root/.gvm/gos/go1.4.3/src/net/http/client.go:142 +0x15b
net/http.(*Client).doFollowingRedirects(0x10acae0, 0xc250910270, 0xdbef90, 0x0, 0x0, 0x0)
    /root/.gvm/gos/go1.4.3/src/net/http/client.go:367 +0xb25
net/http.(*Client).Get(0x10acae0, 0xc242954a00, 0x15, 0xc24e9b7db8, 0x0, 0x0)
    /root/.gvm/gos/go1.4.3/src/net/http/client.go:299 +0xba
net/http.Get(0xc242954a00, 0x15, 0xc24e70a400, 0x0, 0x0)
    /root/.gvm/gos/go1.4.3/src/net/http/client.go:276 +0x50
github.com/influxdata/influxdb/services/meta.(*Client).getSnapshot(0xc2080111f0, 0xc19140, 0x5, 0x16, 0x0, 0x0, 0x0)
    /tmp/influxdb/src/github.com/influxdata/influxdb/services/meta/client.go:1135 +0x1d1
github.com/influxdata/influxdb/services/meta.(*Client).retryUntilSnapshot(0xc2080111f0, 0x16, 0xc24e73e540)
    /tmp/influxdb/src/github.com/influxdata/influxdb/services/meta/client.go:1220 +0x170
github.com/influxdata/influxdb/services/meta.(*Client).pollForUpdates(0xc2080111f0)
    /tmp/influxdb/src/github.com/influxdata/influxdb/services/meta/client.go:1114 +0x40
created by github.com/influxdata/influxdb/services/meta.(*Client).Open
    /tmp/influxdb/src/github.com/influxdata/influxdb/services/meta/client.go:88 +0x101

goroutine 66 [IO wait, 1 minutes]:
net.(*pollDesc).Wait(0xc208010370, 0x72, 0x0, 0x0)
    /root/.gvm/gos/go1.4.3/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc208010370, 0x0, 0x0)
    /root/.gvm/gos/go1.4.3/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).Read(0xc208010310, 0xc208086000, 0x1000, 0x1000, 0x0, 0x7f8a4f8af0d0, 0xc25c2b4148)
    /root/.gvm/gos/go1.4.3/src/net/fd_unix.go:242 +0x40f
net.(*conn).Read(0xc20802a040, 0xc208086000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
    /root/.gvm/gos/go1.4.3/src/net/net.go:121 +0xdc
net/http.(*liveSwitchReader).Read(0xc208030408, 0xc208086000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
    /root/.gvm/gos/go1.4.3/src/net/http/server.go:214 +0xab
io.(*LimitedReader).Read(0xc20801eae0, 0xc208086000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
    /root/.gvm/gos/go1.4.3/src/io/io.go:408 +0xce
bufio.(*Reader).fill(0xc208124420)
    /root/.gvm/gos/go1.4.3/src/bufio/bufio.go:97 +0x1ce
bufio.(*Reader).ReadSlice(0xc208124420, 0xa, 0x0, 0x0, 0x0, 0x0, 0x0)
    /root/.gvm/gos/go1.4.3/src/bufio/bufio.go:295 +0x257
bufio.(*Reader).ReadLine(0xc208124420, 0x0, 0x0, 0x0, 0xc207658e00, 0x0, 0x0)
    /root/.gvm/gos/go1.4.3/src/bufio/bufio.go:324 +0x62
net/textproto.(*Reader).readLineSlice(0xc211a71800, 0x0, 0x0, 0x0, 0x0, 0x0)
    /root/.gvm/gos/go1.4.3/src/net/textproto/reader.go:55 +0x9e
net/textproto.(*Reader).ReadLine(0xc211a71800, 0x0, 0x0, 0x0, 0x0)
    /root/.gvm/gos/go1.4.3/src/net/textproto/reader.go:36 +0x4f
net/http.ReadRequest(0xc208124420, 0xc250b5e680, 0x0, 0x0)
    /root/.gvm/gos/go1.4.3/src/net/http/request.go:598 +0xcb
net/http.(*conn).readRequest(0xc2080303c0, 0x0, 0x0, 0x0)
    /root/.gvm/gos/go1.4.3/src/net/http/server.go:586 +0x26f
net/http.(*conn).serve(0xc2080303c0)
    /root/.gvm/gos/go1.4.3/src/net/http/server.go:1162 +0x69e
created by net/http.(*Server).Serve
    /root/.gvm/gos/go1.4.3/src/net/http/server.go:1751 +0x35e

goroutine 36 [IO wait, 1 minutes]:
net.(*pollDesc).Wait(0xc2080c2060, 0x72, 0x0, 0x0)
    /root/.gvm/gos/go1.4.3/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc2080c2060, 0x0, 0x0)
    /root/.gvm/gos/go1.4.3/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).Read(0xc2080c2000, 0xc2080ea000, 0x1000, 0x1000, 0x0, 0x7f8a4f8af0d0, 0xc21122e6a0)
    /root/.gvm/gos/go1.4.3/src/net/fd_unix.go:242 +0x40f
net.(*conn).Read(0xc2080e8000, 0xc2080ea000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
    /root/.gvm/gos/go1.4.3/src/net/net.go:121 +0xdc
net/http.noteEOFReader.Read(0x7f8a4f8b9610, 0xc2080e8000, 0xc2080601b8, 0xc2080ea000, 0x1000, 0x1000, 0xaa9760, 0x0, 0x0)
    /root/.gvm/gos/go1.4.3/src/net/http/transport.go:1270 +0x6e
net/http.(*noteEOFReader).Read(0xc20810e0c0, 0xc2080ea000, 0x1000, 0x1000, 0xc2080cfb00, 0x0, 0x0)
    <autogenerated>:125 +0xd4
bufio.(*Reader).fill(0xc208108540)
    /root/.gvm/gos/go1.4.3/src/bufio/bufio.go:97 +0x1ce
bufio.(*Reader).Peek(0xc208108540, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0)
    /root/.gvm/gos/go1.4.3/src/bufio/bufio.go:132 +0xf0
net/http.(*persistConn).readLoop(0xc208060160)
    /root/.gvm/gos/go1.4.3/src/net/http/transport.go:842 +0xa4
created by net/http.(*Transport).dialConn
    /root/.gvm/gos/go1.4.3/src/net/http/transport.go:660 +0xc9f

goroutine 37 [select, 1 minutes]:
net/http.(*persistConn).writeLoop(0xc208060160)
    /root/.gvm/gos/go1.4.3/src/net/http/transport.go:945 +0x41d
created by net/http.(*Transport).dialConn
    /root/.gvm/gos/go1.4.3/src/net/http/transport.go:661 +0xcbc

goroutine 67 [select, 16 minutes]:
github.com/influxdata/influxdb/services/hh.(*Service).purgeInactiveProcessors(0xc208035ba0)
    /tmp/influxdb/src/github.com/influxdata/influxdb/services/hh/service.go:227 +0x199
created by github.com/influxdata/influxdb/services/hh.(*Service).Open
    /tmp/influxdb/src/github.com/influxdata/influxdb/services/hh/service.go:121 +0x601

goroutine 68 [chan receive]:
github.com/influxdata/influxdb/services/subscriber.(*Service).writePoints(0xc208011260)
    /tmp/influxdb/src/github.com/influxdata/influxdb/services/subscriber/service.go:225 +0x99
created by github.com/influxdata/influxdb/services/subscriber.(*Service).Open
    /tmp/influxdb/src/github.com/influxdata/influxdb/services/subscriber/service.go:85 +0x167

goroutine 69 [select, 1 minutes]:
github.com/influxdata/influxdb/services/subscriber.(*Service).waitForMetaUpdates(0xc208011260)
    /tmp/influxdb/src/github.com/influxdata/influxdb/services/subscriber/service.go:119 +0x2ad
created by github.com/influxdata/influxdb/services/subscriber.(*Service).Open
    /tmp/influxdb/src/github.com/influxdata/influxdb/services/subscriber/service.go:87 +0x181

goroutine 70 [select]:
github.com/influxdata/influxdb/monitor.(*Monitor).storeStatistics(0xc2080f2000)
    /tmp/influxdb/src/github.com/influxdata/influxdb/monitor/service.go:324 +0xec2
created by github.com/influxdata/influxdb/monitor.(*Monitor).Open
    /tmp/influxdb/src/github.com/influxdata/influxdb/monitor/service.go:105 +0x33c

goroutine 71 [chan receive, 16 minutes]:
github.com/influxdata/influxdb/tcp.(*listener).Accept(0xc2081048d0, 0x0, 0x0, 0x0, 0x0)
    /tmp/influxdb/src/github.com/influxdata/influxdb/tcp/mux.go:135 +0x80
github.com/influxdata/influxdb/cluster.(*Service).serve(0xc2080ae080)
    /tmp/influxdb/src/github.com/influxdata/influxdb/cluster/service.go:100 +0x102
created by github.com/influxdata/influxdb/cluster.(*Service).Open
    /tmp/influxdb/src/github.com/influxdata/influxdb/cluster/service.go:77 +0x138

goroutine 72 [select, 6 minutes]:
github.com/influxdata/influxdb/services/precreator.(*Service).runPrecreation(0xc20802caa0)
    /tmp/influxdb/src/github.com/influxdata/influxdb/services/precreator/service.go:75 +0x3b7
created by github.com/influxdata/influxdb/services/precreator.(*Service).Open
    /tmp/influxdb/src/github.com/influxdata/influxdb/services/precreator/service.go:53 +0x20a

goroutine 73 [chan receive, 16 minutes]:
github.com/influxdata/influxdb/tcp.(*listener).Accept(0xc2081048e0, 0x0, 0x0, 0x0, 0x0)
    /tmp/influxdb/src/github.com/influxdata/influxdb/tcp/mux.go:135 +0x80
github.com/influxdata/influxdb/services/snapshotter.(*Service).serve(0xc208124d20)
    /tmp/influxdb/src/github.com/influxdata/influxdb/services/snapshotter/service.go:88 +0xad
created by github.com/influxdata/influxdb/services/snapshotter.(*Service).Open
    /tmp/influxdb/src/github.com/influxdata/influxdb/services/snapshotter/service.go:61 +0x133

goroutine 74 [chan receive, 16 minutes]:
github.com/influxdata/influxdb/tcp.(*listener).Accept(0xc2081048f0, 0x0, 0x0, 0x0, 0x0)
    /tmp/influxdb/src/github.com/influxdata/influxdb/tcp/mux.go:135 +0x80
github.com/influxdata/influxdb/services/copier.(*Service).serve(0xc20802ceb0)
    /tmp/influxdb/src/github.com/influxdata/influxdb/services/copier/service.go:78 +0xad
created by github.com/influxdata/influxdb/services/copier.(*Service).Open
    /tmp/influxdb/src/github.com/influxdata/influxdb/services/copier/service.go:51 +0x133

goroutine 75 [IO wait, 16 minutes]:
net.(*pollDesc).Wait(0xc2080106f0, 0x72, 0x0, 0x0)
    /root/.gvm/gos/go1.4.3/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc2080106f0, 0x0, 0x0)
    /root/.gvm/gos/go1.4.3/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).accept(0xc208010690, 0x0, 0x7f8a4f8af0d0, 0xc2084e2320)
    /root/.gvm/gos/go1.4.3/src/net/fd_unix.go:419 +0x40b
net.(*TCPListener).AcceptTCP(0xc2080e81a8, 0xc208056d60, 0x0, 0x0)
    /root/.gvm/gos/go1.4.3/src/net/tcpsock_posix.go:234 +0x4e
net.(*TCPListener).Accept(0xc2080e81a8, 0x0, 0x0, 0x0, 0x0)
    /root/.gvm/gos/go1.4.3/src/net/tcpsock_posix.go:244 +0x4c
net/http.(*Server).Serve(0xc20805cae0, 0x7f8a4f8b46e8, 0xc2080e81a8, 0x0, 0x0)
    /root/.gvm/gos/go1.4.3/src/net/http/server.go:1728 +0x92
net/http.Serve(0x7f8a4f8b46e8, 0xc2080e81a8, 0x7f8a4f8af2c8, 0xc20801f020, 0x0, 0x0)
    /root/.gvm/gos/go1.4.3/src/net/http/server.go:1606 +0xa1
github.com/influxdata/influxdb/services/admin.(*Service).serve(0xc208124e40)
    /tmp/influxdb/src/github.com/influxdata/influxdb/services/admin/service.go:116 +0x216
created by github.com/influxdata/influxdb/services/admin.(*Service).Open
    /tmp/influxdb/src/github.com/influxdata/influxdb/services/admin/service.go:72 +0x472

goroutine 76 [select]:
github.com/influxdata/influxdb/services/continuous_querier.(*Service).backgroundLoop(0xc2080ae400)
    /tmp/influxdb/src/github.com/influxdata/influxdb/services/continuous_querier/service.go:185 +0x5a9
created by github.com/influxdata/influxdb/services/continuous_querier.(*Service).Open
    /tmp/influxdb/src/github.com/influxdata/influxdb/services/continuous_querier/service.go:115 +0x291

goroutine 77 [IO wait]:
net.(*pollDesc).Wait(0xc208010a00, 0x72, 0x0, 0x0)
    /root/.gvm/gos/go1.4.3/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc208010a00, 0x0, 0x0)
    /root/.gvm/gos/go1.4.3/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).accept(0xc2080109a0, 0x0, 0x7f8a4f8af0d0, 0xc25dfa60b0)
    /root/.gvm/gos/go1.4.3/src/net/fd_unix.go:419 +0x40b
net.(*TCPListener).AcceptTCP(0xc2080e81b0, 0xc2082a7dc8, 0x0, 0x0)
    /root/.gvm/gos/go1.4.3/src/net/tcpsock_posix.go:234 +0x4e
net.(*TCPListener).Accept(0xc2080e81b0, 0x0, 0x0, 0x0, 0x0)
    /root/.gvm/gos/go1.4.3/src/net/tcpsock_posix.go:244 +0x4c
net/http.(*Server).Serve(0xc208186060, 0x7f8a4f8b46e8, 0xc2080e81b0, 0x0, 0x0)
    /root/.gvm/gos/go1.4.3/src/net/http/server.go:1728 +0x92
net/http.Serve(0x7f8a4f8b46e8, 0xc2080e81b0, 0x7f8a4f8bebc8, 0xc2080ae480, 0x0, 0x0)
    /root/.gvm/gos/go1.4.3/src/net/http/server.go:1606 +0xa1
github.com/influxdata/influxdb/services/httpd.(*Service).serve(0xc208124f60)
    /tmp/influxdb/src/github.com/influxdata/influxdb/services/httpd/service.go:153 +0x79
created by github.com/influxdata/influxdb/services/httpd.(*Service).Open
    /tmp/influxdb/src/github.com/influxdata/influxdb/services/httpd/service.go:121 +0x73b

goroutine 78 [select, 16 minutes]:
github.com/influxdata/influxdb/services/retention.(*Service).deleteShardGroups(0xc208125020)
    /tmp/influxdb/src/github.com/influxdata/influxdb/services/retention/service.go:68 +0xa08
created by github.com/influxdata/influxdb/services/retention.(*Service).Open
    /tmp/influxdb/src/github.com/influxdata/influxdb/services/retention/service.go:44 +0x19b

goroutine 79 [select, 16 minutes]:
github.com/influxdata/influxdb/services/retention.(*Service).deleteShards(0xc208125020)
    /tmp/influxdb/src/github.com/influxdata/influxdb/services/retention/service.go:102 +0xd03
created by github.com/influxdata/influxdb/services/retention.(*Service).Open
    /tmp/influxdb/src/github.com/influxdata/influxdb/services/retention/service.go:45 +0x1b8

goroutine 80 [chan receive, 16 minutes]:
github.com/influxdata/influxdb/cmd/influxd/run.(*Server).startServerReporting(0xc2080f4000)
    /tmp/influxdb/src/github.com/influxdata/influxdb/cmd/influxd/run/server.go:547 +0x94
created by github.com/influxdata/influxdb/cmd/influxd/run.(*Server).Open
    /tmp/influxdb/src/github.com/influxdata/influxdb/cmd/influxd/run/server.go:483 +0x15a1

goroutine 81 [select, 16 minutes]:
github.com/influxdata/influxdb/cmd/influxd/run.(*Command).monitorServerErrors(0xc208074480)
    /tmp/influxdb/src/github.com/influxdata/influxdb/cmd/influxd/run/command.go:157 +0x2dd
created by github.com/influxdata/influxdb/cmd/influxd/run.(*Command).Run
    /tmp/influxdb/src/github.com/influxdata/influxdb/cmd/influxd/run/command.go:139 +0x1003

goroutine 170 [sleep]:
github.com/influxdata/influxdb/tsdb/engine/tsm1.(*Engine).compactTSMLevel(0xc233167d90, 0x1, 0x2)
    /tmp/influxdb/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/engine.go:542 +0x181
created by github.com/influxdata/influxdb/tsdb/engine/tsm1.(*Engine).Open
    /tmp/influxdb/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/engine.go:156 +0x2c4

goroutine 114 [sleep]:
github.com/influxdata/influxdb/tsdb/engine/tsm1.(*Engine).compactCache(0xc20c54e0b0)
    /tmp/influxdb/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/engine.go:513 +0x20c
created by github.com/influxdata/influxdb/tsdb/engine/tsm1.(*Engine).Open
    /tmp/influxdb/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/engine.go:153 +0x251

goroutine 281 [sleep]:
github.com/influxdata/influxdb/tsdb/engine/tsm1.(*Engine).compactTSMFull(0xc23dab6b00)
    /tmp/influxdb/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/engine.go:606 +0x16d
created by github.com/influxdata/influxdb/tsdb/engine/tsm1.(*Engine).Open
    /tmp/influxdb/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/engine.go:154 +0x26e

goroutine 171 [sleep]:
github.com/influxdata/influxdb/tsdb/engine/tsm1.(*Engine).compactTSMLevel(0xc233167d90, 0x0, 0x3)
    /tmp/influxdb/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/engine.go:542 +0x181
created by github.com/influxdata/influxdb/tsdb/engine/tsm1.(*Engine).Open
    /tmp/influxdb/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/engine.go:157 +0x2ef

goroutine 168 [sleep]:
github.com/influxdata/influxdb/tsdb/engine/tsm1.(*Engine).compactTSMFull(0xc233167d90)
    /tmp/influxdb/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/engine.go:606 +0x16d
created by github.com/influxdata/influxdb/tsdb/engine/tsm1.(*Engine).Open
    /tmp/influxdb/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/engine.go:154 +0x26e

goroutine 43 [sleep]:
github.com/influxdata/influxdb/tsdb/engine/tsm1.(*Engine).compactCache(0xc2082ae000)
    /tmp/influxdb/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/engine.go:513 +0x20c
created by github.com/influxdata/influxdb/tsdb/engine/tsm1.(*Engine).Open
    /tmp/influxdb/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/engine.go:153 +0x251

goroutine 102 [sleep]:
github.com/influxdata/influxdb/tsdb/engine/tsm1.(*Engine).compactCache(0xc2080600b0)
    /tmp/influxdb/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/engine.go:513 +0x20c
created by github.com/influxdata/influxdb/tsdb/engine/tsm1.(*Engine).Open
    /tmp/influxdb/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/engine.go:153 +0x251

goroutine 103 [sleep]:
github.com/influxdata/influxdb/tsdb/engine/tsm1.(*Engine).compactTSMFull(0xc2080600b0)
    /tmp/influxdb/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/engine.go:606 +0x16d
created by github.com/influxdata/influxdb/tsdb/engine/tsm1.(*Engine).Open
    /tmp/influxdb/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/engine.go:154 +0x26e

goroutine 104 [sleep]:
github.com/influxdata/influxdb/tsdb/engine/tsm1.(*Engine).compactTSMLevel(0xc2080600b0, 0x1, 0x1)
    /tmp/influxdb/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/engine.go:542 +0x181
created by github.com/influxdata/influxdb/tsdb/engine/tsm1.(*Engine).Open
    /tmp/influxdb/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/engine.go:155 +0x299

goroutine 105 [sleep]:
github.com/influxdata/influxdb/tsdb/engine/tsm1.(*Engine).compactTSMLevel(0xc2080600b0, 0x1, 0x2)
    /tmp/influxdb/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/engine.go:542 +0x181
created by github.com/influxdata/influxdb/tsdb/engine/tsm1.(*Engine).Open
    /tmp/influxdb/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/engine.go:156 +0x2c4

goroutine 106 [sleep]:
github.com/influxdata/influxdb/tsdb/engine/tsm1.(*Engine).compactTSMLevel(0xc2080600b0, 0x0, 0x3)
    /tmp/influxdb/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/engine.go:542 +0x181
created by github.com/influxdata/influxdb/tsdb/engine/tsm1.(*Engine).Open
    /tmp/influxdb/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/engine.go:157 +0x2ef

goroutine 116 [sleep]:
github.com/influxdata/influxdb/tsdb/engine/tsm1.(*Engine).compactTSMLevel(0xc20c54e0b0, 0x1, 0x1)
    /tmp/influxdb/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/engine.go:542 +0x181
created by github.com/influxdata/influxdb/tsdb/engine/tsm1.(*Engine).Open
    /tmp/influxdb/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/engine.go:155 +0x299

goroutine 339 [sleep]:
github.com/influxdata/influxdb/tsdb/engine/tsm1.(*Engine).compactCache(0xc24aac84d0)
    /tmp/influxdb/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/engine.go:513 +0x20c
created by github.com/influxdata/influxdb/tsdb/engine/tsm1.(*Engine).Open
    /tmp/influxdb/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/engine.go:153 +0x251

goroutine 115 [sleep]:
github.com/influxdata/influxdb/tsdb/engine/tsm1.(*Engine).compactTSMFull(0xc20c54e0b0)
    /tmp/influxdb/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/engine.go:606 +0x16d
created by github.com/influxdata/influxdb/tsdb/engine/tsm1.(*Engine).Open
    /tmp/influxdb/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/engine.go:154 +0x26e

goroutine 117 [sleep]:
github.com/influxdata/influxdb/tsdb/engine/tsm1.(*Engine).compactTSMLevel(0xc20c54e0b0, 0x1, 0x2)
    /tmp/influxdb/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/engine.go:542 +0x181
created by github.com/influxdata/influxdb/tsdb/engine/tsm1.(*Engine).Open
    /tmp/influxdb/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/engine.go:156 +0x2c4

goroutine 196 [sleep]:
github.com/influxdata/influxdb/tsdb/engine/tsm1.(*Engine).compactTSMLevel(0xc21a6876b0, 0x0, 0x3)
    /tmp/influxdb/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/engine.go:542 +0x181
created by github.com/influxdata/influxdb/tsdb/engine/tsm1.(*Engine).Open
    /tmp/influxdb/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/engine.go:157 +0x2ef

goroutine 126 [sleep]:
github.com/influxdata/influxdb/tsdb/engine/tsm1.(*Engine).compactTSMFull(0xc21a686630)
    /tmp/influxdb/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/engine.go:606 +0x16d
created by github.com/influxdata/influxdb/tsdb/engine/tsm1.(*Engine).Open
    /tmp/influxdb/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/engine.go:154 +0x26e

goroutine 44 [sleep]:
github.com/influxdata/influxdb/tsdb/engine/tsm1.(*Engine).compactTSMFull(0xc2082ae000)
    /tmp/influxdb/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/engine.go:606 +0x16d
created by github.com/influxdata/influxdb/tsdb/engine/tsm1.(*Engine).Open
    /tmp/influxdb/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/engine.go:154 +0x26e

goroutine 45 [sleep]:
github.com/influxdata/influxdb/tsdb/engine/tsm1.(*Engine).compactTSMLevel(0xc2082ae000, 0x1, 0x1)
    /tmp/influxdb/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/engine.go:542 +0x181
created by github.com/influxdata/influxdb/tsdb/engine/tsm1.(*Engine).Open
    /tmp/influxdb/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/engine.go:155 +0x299

goroutine 46 [sleep]:
github.com/influxdata/influxdb/tsdb/engine/tsm1.(*Engine).compactTSMLevel(0xc2082ae000, 0x1, 0x2)
    /tmp/influxdb/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/engine.go:542 +0x181
created by github.com/influxdata/influxdb/tsdb/engine/tsm1.(*Engine).Open
    /tmp/influxdb/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/engine.go:156 +0x2c4

goroutine 47 [sleep]:
github.com/influxdata/influxdb/tsdb/engine/tsm1.(*Engine).compactTSMLevel(0xc2082ae000, 0x0, 0x3)
    /tmp/influxdb/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/engine.go:542 +0x181
created by github.com/influxdata/influxdb/tsdb/engine/tsm1.(*Engine).Open
    /tmp/influxdb/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/engine.go:157 +0x2ef

goroutine 118 [sleep]:
github.com/influxdata/influxdb/tsdb/engine/tsm1.(*Engine).compactTSMLevel(0xc20c54e0b0, 0x0, 0x3)
    /tmp/influxdb/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/engine.go:542 +0x181
created by github.com/influxdata/influxdb/tsdb/engine/tsm1.(*Engine).Open
    /tmp/influxdb/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/engine.go:157 +0x2ef

goroutine 125 [sleep]:
github.com/influxdata/influxdb/tsdb/engine/tsm1.(*Engine).compactCache(0xc21a686630)
    /tmp/influxdb/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/engine.go:513 +0x20c
created by github.com/influxdata/influxdb/tsdb/engine/tsm1.(*Engine).Open
    /tmp/influxdb/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/engine.go:153 +0x251

goroutine 127 [sleep]:
github.com/influxdata/influxdb/tsdb/engine/tsm1.(*Engine).compactTSMLevel(0xc21a686630, 0x1, 0x1)
    /tmp/influxdb/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/engine.go:542 +0x181
created by github.com/influxdata/influxdb/tsdb/engine/tsm1.(*Engine).Open
    /tmp/influxdb/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/engine.go:155 +0x299

goroutine 128 [sleep]:
github.com/influxdata/influxdb/tsdb/engine/tsm1.(*Engine).compactTSMLevel(0xc21a686630, 0x1, 0x2)
    /tmp/influxdb/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/engine.go:542 +0x181
created by github.com/influxdata/influxdb/tsdb/engine/tsm1.(*Engine).Open
    /tmp/influxdb/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/engine.go:156 +0x2c4

goroutine 129 [sleep]:
github.com/influxdata/influxdb/tsdb/engine/tsm1.(*Engine).compactTSMLevel(0xc21a686630, 0x0, 0x3)
    /tmp/influxdb/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/engine.go:542 +0x181
created by github.com/influxdata/influxdb/tsdb/engine/tsm1.(*Engine).Open
    /tmp/influxdb/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/engine.go:157 +0x2ef

goroutine 161 [sleep]:
github.com/influxdata/influxdb/tsdb/engine/tsm1.(*Engine).compactTSMFull(0xc21a6876b0)
    /tmp/influxdb/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/engine.go:606 +0x16d
created by github.com/influxdata/influxdb/tsdb/engine/tsm1.(*Engine).Open
    /tmp/influxdb/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/engine.go:154 +0x26e

goroutine 33 [sleep]:
github.com/influxdata/influxdb/tsdb/engine/tsm1.(*Engine).compactCache(0xc22e605600)
    /tmp/influxdb/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/engine.go:513 +0x20c
created by github.com/influxdata/influxdb/tsdb/engine/tsm1.(*Engine).Open
    /tmp/influxdb/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/engine.go:153 +0x251

goroutine 178 [sleep]:
github.com/influxdata/influxdb/tsdb/engine/tsm1.(*Engine).compactTSMFull(0xc22e605600)
    /tmp/influxdb/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/engine.go:606 +0x16d
created by github.com/influxdata/influxdb/tsdb/engine/tsm1.(*Engine).Open
    /tmp/influxdb/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/engine.go:154 +0x26e

goroutine 179 [sleep]:
github.com/influxdata/influxdb/tsdb/engine/tsm1.(*Engine).compactTSMLevel(0xc22e605600, 0x1, 0x1)
    /tmp/influxdb/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/engine.go:542 +0x181
created by github.com/influxdata/influxdb/tsdb/engine/tsm1.(*Engine).Open
    /tmp/influxdb/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/engine.go:155 +0x299

goroutine 180 [sleep]:
github.com/influxdata/influxdb/tsdb/engine/tsm1.(*Engine).compactTSMLevel(0xc22e605600, 0x1, 0x2)
    /tmp/influxdb/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/engine.go:542 +0x181
created by github.com/influxdata/influxdb/tsdb/engine/tsm1.(*Engine).Open
    /tmp/influxdb/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/engine.go:156 +0x2c4

goroutine 181 [sleep]:
github.com/influxdata/influxdb/tsdb/engine/tsm1.(*Engine).compactTSMLevel(0xc22e605600, 0x0, 0x3)
    /tmp/influxdb/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/engine.go:542 +0x181
created by github.com/influxdata/influxdb/tsdb/engine/tsm1.(*Engine).Open
    /tmp/influxdb/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/engine.go:157 +0x2ef

goroutine 195 [sleep]:
github.com/influxdata/influxdb/tsdb/engine/tsm1.(*Engine).compactTSMLevel(0xc21a6876b0, 0x1, 0x2)
    /tmp/influxdb/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/engine.go:542 +0x181
created by github.com/influxdata/influxdb/tsdb/engine/tsm1.(*Engine).Open
    /tmp/influxdb/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/engine.go:156 +0x2c4

goroutine 194 [sleep]:
github.com/influxdata/influxdb/tsdb/engine/tsm1.(*Engine).compactTSMLevel(0xc21a6876b0, 0x1, 0x1)
    /tmp/influxdb/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/engine.go:542 +0x181
created by github.com/influxdata/influxdb/tsdb/engine/tsm1.(*Engine).Open
    /tmp/influxdb/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/engine.go:155 +0x299

goroutine 160 [sleep]:
github.com/influxdata/influxdb/tsdb/engine/tsm1.(*Engine).compactCache(0xc21a6876b0)
    /tmp/influxdb/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/engine.go:513 +0x20c
created by github.com/influxdata/influxdb/tsdb/engine/tsm1.(*Engine).Open
    /tmp/influxdb/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/engine.go:153 +0x251

goroutine 284 [sleep]:
github.com/influxdata/influxdb/tsdb/engine/tsm1.(*Engine).compactTSMLevel(0xc23dab6b00, 0x0, 0x3)
    /tmp/influxdb/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/engine.go:542 +0x181
created by github.com/influxdata/influxdb/tsdb/engine/tsm1.(*Engine).Open
    /tmp/influxdb/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/engine.go:157 +0x2ef

goroutine 283 [sleep]:
github.com/influxdata/influxdb/tsdb/engine/tsm1.(*Engine).compactTSMLevel(0xc23dab6b00, 0x1, 0x2)
    /tmp/influxdb/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/engine.go:542 +0x181
created by github.com/influxdata/influxdb/tsdb/engine/tsm1.(*Engine).Open
    /tmp/influxdb/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/engine.go:156 +0x2c4

goroutine 169 [sleep]:
github.com/influxdata/influxdb/tsdb/engine/tsm1.(*Engine).compactTSMLevel(0xc233167d90, 0x1, 0x1)
    /tmp/influxdb/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/engine.go:542 +0x181
created by github.com/influxdata/influxdb/tsdb/engine/tsm1.(*Engine).Open
    /tmp/influxdb/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/engine.go:155 +0x299

goroutine 57 [sleep]:
github.com/influxdata/influxdb/tsdb/engine/tsm1.(*Engine).compactCache(0xc22e6040b0)
    /tmp/influxdb/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/engine.go:513 +0x20c
created by github.com/influxdata/influxdb/tsdb/engine/tsm1.(*Engine).Open
    /tmp/influxdb/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/engine.go:153 +0x251

goroutine 58 [sleep]:
github.com/influxdata/influxdb/tsdb/engine/tsm1.(*Engine).compactTSMFull(0xc22e6040b0)
    /tmp/influxdb/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/engine.go:606 +0x16d
created by github.com/influxdata/influxdb/tsdb/engine/tsm1.(*Engine).Open
    /tmp/influxdb/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/engine.go:154 +0x26e

goroutine 59 [sleep]:
github.com/influxdata/influxdb/tsdb/engine/tsm1.(*Engine).compactTSMLevel(0xc22e6040b0, 0x1, 0x1)
    /tmp/influxdb/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/engine.go:542 +0x181
created by github.com/influxdata/influxdb/tsdb/engine/tsm1.(*Engine).Open
    /tmp/influxdb/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/engine.go:155 +0x299

goroutine 60 [sleep]:
github.com/influxdata/influxdb/tsdb/engine/tsm1.(*Engine).compactTSMLevel(0xc22e6040b0, 0x1, 0x2)
    /tmp/influxdb/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/engine.go:542 +0x181
created by github.com/influxdata/influxdb/tsdb/engine/tsm1.(*Engine).Open
    /tmp/influxdb/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/engine.go:156 +0x2c4

goroutine 61 [sleep]:
github.com/influxdata/influxdb/tsdb/engine/tsm1.(*Engine).compactTSMLevel(0xc22e6040b0, 0x0, 0x3)
    /tmp/influxdb/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/engine.go:542 +0x181
created by github.com/influxdata/influxdb/tsdb/engine/tsm1.(*Engine).Open
    /tmp/influxdb/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/engine.go:157 +0x2ef

goroutine 282 [sleep]:
github.com/influxdata/influxdb/tsdb/engine/tsm1.(*Engine).compactTSMLevel(0xc23dab6b00, 0x1, 0x1)
    /tmp/influxdb/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/engine.go:542 +0x181
created by github.com/influxdata/influxdb/tsdb/engine/tsm1.(*Engine).Open
    /tmp/influxdb/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/engine.go:155 +0x299

goroutine 167 [sleep]:
github.com/influxdata/influxdb/tsdb/engine/tsm1.(*Engine).compactCache(0xc233167d90)
    /tmp/influxdb/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/engine.go:513 +0x20c
created by github.com/influxdata/influxdb/tsdb/engine/tsm1.(*Engine).Open
    /tmp/influxdb/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/engine.go:153 +0x251

goroutine 280 [sleep]:
github.com/influxdata/influxdb/tsdb/engine/tsm1.(*Engine).compactCache(0xc23dab6b00)
    /tmp/influxdb/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/engine.go:513 +0x20c
created by github.com/influxdata/influxdb/tsdb/engine/tsm1.(*Engine).Open
    /tmp/influxdb/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/engine.go:153 +0x251

goroutine 325 [sleep]:
github.com/influxdata/influxdb/tsdb/engine/tsm1.(*Engine).compactCache(0xc21ff9a0b0)
    /tmp/influxdb/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/engine.go:513 +0x20c
created by github.com/influxdata/influxdb/tsdb/engine/tsm1.(*Engine).Open
    /tmp/influxdb/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/engine.go:153 +0x251

goroutine 326 [sleep]:
github.com/influxdata/influxdb/tsdb/engine/tsm1.(*Engine).compactTSMFull(0xc21ff9a0b0)
    /tmp/influxdb/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/engine.go:606 +0x16d
created by github.com/influxdata/influxdb/tsdb/engine/tsm1.(*Engine).Open
    /tmp/influxdb/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/engine.go:154 +0x26e

goroutine 327 [sleep]:
github.com/influxdata/influxdb/tsdb/engine/tsm1.(*Engine).compactTSMLevel(0xc21ff9a0b0, 0x1, 0x1)
    /tmp/influxdb/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/engine.go:542 +0x181
created by github.com/influxdata/influxdb/tsdb/engine/tsm1.(*Engine).Open
    /tmp/influxdb/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/engine.go:155 +0x299

goroutine 328 [sleep]:
github.com/influxdata/influxdb/tsdb/engine/tsm1.(*Engine).compactTSMLevel(0xc21ff9a0b0, 0x1, 0x2)
    /tmp/influxdb/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/engine.go:542 +0x181
created by github.com/influxdata/influxdb/tsdb/engine/tsm1.(*Engine).Open
    /tmp/influxdb/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/engine.go:156 +0x2c4

goroutine 329 [sleep]:
github.com/influxdata/influxdb/tsdb/engine/tsm1.(*Engine).compactTSMLevel(0xc21ff9a0b0, 0x0, 0x3)
    /tmp/influxdb/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/engine.go:542 +0x181
created by github.com/influxdata/influxdb/tsdb/engine/tsm1.(*Engine).Open
    /tmp/influxdb/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/engine.go:157 +0x2ef

goroutine 340 [sleep]:
github.com/influxdata/influxdb/tsdb/engine/tsm1.(*Engine).compactTSMFull(0xc24aac84d0)
    /tmp/influxdb/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/engine.go:606 +0x16d
created by github.com/influxdata/influxdb/tsdb/engine/tsm1.(*Engine).Open
    /tmp/influxdb/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/engine.go:154 +0x26e

goroutine 341 [sleep]:
github.com/influxdata/influxdb/tsdb/engine/tsm1.(*Engine).compactTSMLevel(0xc24aac84d0, 0x1, 0x1)
    /tmp/influxdb/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/engine.go:542 +0x181
created by github.com/influxdata/influxdb/tsdb/engine/tsm1.(*Engine).Open
    /tmp/influxdb/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/engine.go:155 +0x299

goroutine 342 [sleep]:
github.com/influxdata/influxdb/tsdb/engine/tsm1.(*Engine).compactTSMLevel(0xc24aac84d0, 0x1, 0x2)
    /tmp/influxdb/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/engine.go:542 +0x181
created by github.com/influxdata/influxdb/tsdb/engine/tsm1.(*Engine).Open
    /tmp/influxdb/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/engine.go:156 +0x2c4

goroutine 343 [sleep]:
github.com/influxdata/influxdb/tsdb/engine/tsm1.(*Engine).compactTSMLevel(0xc24aac84d0, 0x0, 0x3)
    /tmp/influxdb/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/engine.go:542 +0x181
created by github.com/influxdata/influxdb/tsdb/engine/tsm1.(*Engine).Open
    /tmp/influxdb/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/engine.go:157 +0x2ef

goroutine 333 [sleep]:
github.com/influxdata/influxdb/tsdb/engine/tsm1.(*Engine).compactCache(0xc246e1a0b0)
    /tmp/influxdb/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/engine.go:513 +0x20c
created by github.com/influxdata/influxdb/tsdb/engine/tsm1.(*Engine).Open
    /tmp/influxdb/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/engine.go:153 +0x251

goroutine 334 [sleep]:
github.com/influxdata/influxdb/tsdb/engine/tsm1.(*Engine).compactTSMFull(0xc246e1a0b0)
    /tmp/influxdb/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/engine.go:606 +0x16d
created by github.com/influxdata/influxdb/tsdb/engine/tsm1.(*Engine).Open
    /tmp/influxdb/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/engine.go:154 +0x26e

goroutine 335 [sleep]:
github.com/influxdata/influxdb/tsdb/engine/tsm1.(*Engine).compactTSMLevel(0xc246e1a0b0, 0x1, 0x1)
    /tmp/influxdb/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/engine.go:542 +0x181
created by github.com/influxdata/influxdb/tsdb/engine/tsm1.(*Engine).Open
    /tmp/influxdb/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/engine.go:155 +0x299

goroutine 336 [sleep]:
github.com/influxdata/influxdb/tsdb/engine/tsm1.(*Engine).compactTSMLevel(0xc246e1a0b0, 0x1, 0x2)
    /tmp/influxdb/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/engine.go:542 +0x181
created by github.com/influxdata/influxdb/tsdb/engine/tsm1.(*Engine).Open
    /tmp/influxdb/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/engine.go:156 +0x2c4

goroutine 337 [sleep]:
github.com/influxdata/influxdb/tsdb/engine/tsm1.(*Engine).compactTSMLevel(0xc246e1a0b0, 0x0, 0x3)
    /tmp/influxdb/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/engine.go:542 +0x181
created by github.com/influxdata/influxdb/tsdb/engine/tsm1.(*Engine).Open
    /tmp/influxdb/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/engine.go:157 +0x2ef

goroutine 406 [sleep]:
github.com/influxdata/influxdb/tsdb/engine/tsm1.(*Engine).compactTSMFull(0xc25725c000)
    /tmp/influxdb/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/engine.go:606 +0x16d
created by github.com/influxdata/influxdb/tsdb/engine/tsm1.(*Engine).Open
    /tmp/influxdb/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/engine.go:154 +0x26e

goroutine 407 [sleep]:
github.com/influxdata/influxdb/tsdb/engine/tsm1.(*Engine).compactTSMLevel(0xc25725c000, 0x1, 0x1)
    /tmp/influxdb/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/engine.go:542 +0x181
created by github.com/influxdata/influxdb/tsdb/engine/tsm1.(*Engine).Open
    /tmp/influxdb/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/engine.go:155 +0x299

goroutine 408 [sleep]:
github.com/influxdata/influxdb/tsdb/engine/tsm1.(*Engine).compactTSMLevel(0xc25725c000, 0x1, 0x2)
    /tmp/influxdb/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/engine.go:542 +0x181
created by github.com/influxdata/influxdb/tsdb/engine/tsm1.(*Engine).Open
    /tmp/influxdb/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/engine.go:156 +0x2c4

goroutine 409 [sleep]:
github.com/influxdata/influxdb/tsdb/engine/tsm1.(*Engine).compactTSMLevel(0xc25725c000, 0x0, 0x3)
    /tmp/influxdb/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/engine.go:542 +0x181
created by github.com/influxdata/influxdb/tsdb/engine/tsm1.(*Engine).Open
    /tmp/influxdb/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/engine.go:157 +0x2ef

[max3163]

Hi,

I think it's related to this report too : 6022

[Madhu94]

Update : Removing a field which held strings of varying length, stopped this behaviour. Any reason why that seemed to solve the problem? I would like to know, to avoid this error in the future.
Thanks.

[beckettsean]

@Madhu94

What was the name of the field?
How long were the strings?
How did you remove the field?

I think the answers to those questions will help us understand what happened.

[Madhu94]

I have a script to populate the influxdb from an ms-sql database.I deleted my original influxdb database, and modified the script so that the string field(let's call it "miscellaneous") was NOT inserted this time. Some of the strings were really long-the longest of them was 800k characters long.