influxdb 0.9.4.2 constantly crashes

[azrdev]

Fedora 22 Server, installed the official influxdb-0.9.4.2-1.x86_64.rpm, (re)moved /var/opt/influxdb/ and set engine = "tsm1" in the config file. Starting influxdb via systemd, with the service file from the rpm.
Today I noticed every few seconds the database crashes. journalctl -e -u influxdb says:

Okt 27 11:23:15 host systemd[1]: Started InfluxDB is an open-source, distributed, time series database.
Okt 27 11:23:15 host systemd[1]: Starting InfluxDB is an open-source, distributed, time series database...
Okt 27 11:23:15 host influxd[6779]: 8888888           .d888 888                   8888888b.  888888b.
Okt 27 11:23:15 host influxd[6779]: 888            d88P"  888                   888  "Y88b 888  "88b
Okt 27 11:23:15 host influxd[6779]: 888            888    888                   888    888 888  .88P
Okt 27 11:23:15 host influxd[6779]: 888   88888b.  888888 888 888  888 888  888 888    888 8888888K.
Okt 27 11:23:15 host influxd[6779]: 888   888 "88b 888    888 888  888  Y8bd8P' 888    888 888  "Y88b
Okt 27 11:23:15 host influxd[6779]: 888   888  888 888    888 888  888   X88K   888    888 888    888
Okt 27 11:23:15 host influxd[6779]: 888   888  888 888    888 Y88b 888 .d8""8b. 888  .d88P 888   d88P
Okt 27 11:23:15 host influxd[6779]: 8888888 888  888 888    888  "Y88888 888  888 8888888P"  8888888P"
Okt 27 11:23:15 host influxd[6779]: 2015/10/27 11:23:15 InfluxDB starting, version 0.9.4.2, branch 0.9.4, commit da1b59e7d7764d36786253b2db13b97f42ed4e1d
Okt 27 11:23:15 host influxd[6779]: 2015/10/27 11:23:15 Go version go1.5, GOMAXPROCS set to 2
Okt 27 11:23:15 host influxd[6779]: 2015/10/27 11:23:15 Using configuration at: /etc/opt/influxdb/influxdb.conf
Okt 27 11:23:15 host influxd[6779]: [metastore] 2015/10/27 11:23:15 Using data dir: /var/opt/influxdb/meta
Okt 27 11:23:15 host influxd[6779]: [metastore] 2015/10/27 11:23:15 Skipping cluster join: already member of cluster: nodeId=1 raftEnabled=true peers=[localh
Okt 27 11:23:15 host influxd[6779]: [metastore] 2015/10/27 11:23:15 Node at localhost:8088 [Follower]
Okt 27 11:23:17 host influxd[6779]: [metastore] 2015/10/27 11:23:17 Node at localhost:8088 [Leader]. peers=[localhost:8088]
Okt 27 11:23:17 host influxd[6779]: [metastore] 2015/10/27 11:23:17 Updated node id=1 hostname=localhost:8088
Okt 27 11:23:17 host influxd[6779]: [monitor] 2015/10/27 11:23:17 Starting monitor system
Okt 27 11:23:17 host influxd[6779]: [monitor] 2015/10/27 11:23:17 'build' registered for diagnostics monitoring
Okt 27 11:23:17 host influxd[6779]: [monitor] 2015/10/27 11:23:17 'runtime' registered for diagnostics monitoring
Okt 27 11:23:17 host influxd[6779]: [monitor] 2015/10/27 11:23:17 'network' registered for diagnostics monitoring
Okt 27 11:23:17 host influxd[6779]: [monitor] 2015/10/27 11:23:17 'system' registered for diagnostics monitoring
Okt 27 11:23:17 host influxd[6779]: [store] 2015/10/27 11:23:17 Using data dir: /var/opt/influxdb/data
Okt 27 11:23:17 host influxd[6779]: [handoff] 2015/10/27 11:23:17 Starting hinted handoff service
Okt 27 11:23:17 host influxd[6779]: [handoff] 2015/10/27 11:23:17 Using data dir: /var/opt/influxdb/hh
Okt 27 11:23:17 host influxd[6779]: [tcp] 2015/10/27 11:23:17 Starting cluster service
Okt 27 11:23:17 host influxd[6779]: [shard-precreation] 2015/10/27 11:23:17 Starting precreation service with check interval of 10m0s, advance period of 30m0
Okt 27 11:23:17 host influxd[6779]: [snapshot] 2015/10/27 11:23:17 Starting snapshot service
Okt 27 11:23:17 host influxd[6779]: [copier] 2015/10/27 11:23:17 Starting copier service
Okt 27 11:23:17 host influxd[6779]: [admin] 2015/10/27 11:23:17 Starting admin service
Okt 27 11:23:17 host influxd[6779]: [admin] 2015/10/27 11:23:17 Listening on HTTP: [::]:8083
Okt 27 11:23:17 host influxd[6779]: [continuous_querier] 2015/10/27 11:23:17 Starting continuous query service
Okt 27 11:23:17 host influxd[6779]: [httpd] 2015/10/27 11:23:17 Starting HTTP service
Okt 27 11:23:17 host influxd[6779]: [httpd] 2015/10/27 11:23:17 Authentication enabled: false
Okt 27 11:23:17 host influxd[6779]: [httpd] 2015/10/27 11:23:17 Listening on HTTP: [::]:8086
Okt 27 11:23:17 host influxd[6779]: [retention] 2015/10/27 11:23:17 Starting retention policy enforcement service with check interval of 30m0s
Okt 27 11:23:17 host influxd[6779]: [run] 2015/10/27 11:23:17 Listening for signals
Okt 27 11:23:17 host influxd[6779]: [monitor] 2015/10/27 11:23:17 Storing statistics in database '_internal' retention policy '', at interval 10s
Okt 27 11:23:30 host influxd[6779]: panic: runtime error: invalid memory address or nil pointer dereference
Okt 27 11:23:30 host influxd[6779]: [signal 0xb code=0x1 addr=0x0 pc=0x5d1a5e]
Okt 27 11:23:30 host influxd[6779]: goroutine 50 [running]:
Okt 27 11:23:30 host influxd[6779]: github.com/influxdb/influxdb/tsdb.NewEngine(0xc8203ff0b0, 0x2a, 0xc8203ff020, 0x29, 0xc8201093a0, 0x4, 0x6400000, 0x8bb2c
Okt 27 11:23:30 host influxd[6779]: /tmp/build/src/github.com/influxdb/influxdb/tsdb/engine.go:56 +0x11e
Okt 27 11:23:30 host influxd[6779]: github.com/influxdb/influxdb/tsdb.(*Shard).Open.func1(0xc82008ac60, 0x0, 0x0)
Okt 27 11:23:30 host influxd[6779]: /tmp/build/src/github.com/influxdb/influxdb/tsdb/shard.go:108 +0x16c
Okt 27 11:23:30 host influxd[6779]: github.com/influxdb/influxdb/tsdb.(*Shard).Open(0xc82008ac60, 0x0, 0x0)
Okt 27 11:23:30 host influxd[6779]: /tmp/build/src/github.com/influxdb/influxdb/tsdb/shard.go:95 +0x2d
Okt 27 11:23:30 host influxd[6779]: github.com/influxdb/influxdb/tsdb.(*Store).CreateShard(0xc82014e000, 0xc820136030, 0x9, 0xc82013aec0, 0x7, 0x3, 0x0, 0x0)
Okt 27 11:23:30 host influxd[6779]: /tmp/build/src/github.com/influxdb/influxdb/tsdb/store.go:102 +0x9cc
Okt 27 11:23:30 host influxd[6779]: github.com/influxdb/influxdb/cluster.(*PointsWriter).writeToShard.func1(0xc82009b100, 0xc820136030, 0x9, 0xc82013aec0, 0x
Okt 27 11:23:30 host influxd[6779]: /tmp/build/src/github.com/influxdb/influxdb/cluster/points_writer.go:271 +0x1d5
Okt 27 11:23:30 host influxd[6779]: created by github.com/influxdb/influxdb/cluster.(*PointsWriter).writeToShard
Okt 27 11:23:30 host influxd[6779]: /tmp/build/src/github.com/influxdb/influxdb/cluster/points_writer.go:299 +0x1c5
Okt 27 11:23:30 host influxd[6779]: goroutine 1 [chan receive]:
Okt 27 11:23:30 host influxd[6779]: main.(*Main).Run(0xc820043f10, 0xc820060160, 0x2, 0x2, 0x0, 0x0)
Okt 27 11:23:30 host influxd[6779]: /tmp/build/src/github.com/influxdb/influxdb/cmd/influxd/main.go:91 +0xa16
Okt 27 11:23:30 host systemd[1]: influxdb.service: main process exited, code=exited, status=2/INVALIDARGUMENT
Okt 27 11:23:30 host systemd[1]: Unit influxdb.service entered failed state.
Okt 27 11:23:30 host systemd[1]: influxdb.service failed.
Okt 27 11:23:31 host systemd[1]: influxdb.service holdoff time over, scheduling restart.
Okt 27 11:23:31 host systemd[1]: Started InfluxDB is an open-source, distributed, time series database.
Okt 27 11:23:31 host systemd[1]: Starting InfluxDB is an open-source, distributed, time series database...

(after that, it repeats)

The only clients to the database are telegraf 0.1.9 and grafana 2.1.3-1, however, influxdb's behaviour is the same when those are not running.

[otoolep]

Can we see your config file? It looks like your engine can't initialize.

[otoolep]

This appears to be the problematic line:

https://github.com/influxdb/influxdb/blob/da1b59e7d7764d36786253b2db13b97f42ed4e1d/tsdb/engine.go#L56

[beckettsean]

@azrdev the tsm1 engine was introduced with the 0.9.5 branch. With the 0.9.4.2 release only the b1 or bz1 engines can be used.

If you want to run the tsm1 engine, please do a fresh install of an 0.9.5 nightly build.