InfluxDB high memory usage and very slow to start

[MatMeredith]

I've been running a POC InfluxDB system (InfluxDB v0.12.1 git: 0.12 e094138) for ~a month. Over that time the DB has grown to ~15GB.
I'm logging 1000 measurements with just one tag (the name of the server that the measurement was collected from) and using Grafana to generate dashboards.

Looking at memory usage I noticed that InfluxDB was using ~5.5 GB so tried restarting the process to see if memory usage would be reduced.

"service influxdb restart" completed OK, but InfluxDB never started listening on port 8083 or responding to requests. I left it for ~30 minutes before giving up and deleting the DB. InfluxDB then started instantly.

Before deleting the DB I ran influxdb with the cpuprofile and memprofile options and left it running for a minute or so. Outupt attached if this is helpful.

This is obviously a critical issue that we need to resolve before we can take InfluxDB into production. Any ideas?

profile.zip

[jwilder]

@MatMeredith There are number of startup improvements in the currently nightlies. Can you try running a nightly?

[MatMeredith]

@jwilder Thank you -- I'll try this. Does the memory usage seem right though? When I restarted (with no database) memory use was negligible. Memory use has since been steadily growing. The database size has reached 4GB so far, and memory usage has increased to ~2GB. I assume it will continue increasing until it reaches the 5.5GB it was at previously. Is this expected? Is RAM usage expected to increase with total DB size?

I do also have a couple of continuous queries running, aggregating / downsampling all of the data received. I don't know if this is relevant to any of the issues I'm seeing.

[jwilder]

It depends on the shape of your data and what queries are running. The more series and kinds of queries running will incur more RAM usage.

[MatMeredith]

Seems reasonable. But my number of series and queries haven’t changed since I restarted InfluxDB and yet RAM usage has increased steadily from ~500MB to 2GB and (based on my previous run) will continue to increase to 5.5GB. Would you expect this increase over time?

From: Jason Wilder [mailto:notifications@github.com]
Sent: 02 June 2016 15:07
To: influxdata/influxdb
Cc: Matthew Meredith; Mention
Subject: Re: [influxdata/influxdb] InfluxDB high memory usage and very slow to start (#6759)

It depends on the shape of your data and what queries are running. The more series and kinds of queries running will incur more RAM usage.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHubhttps://github.com//issues/6759#issuecomment-223302124, or mute the threadhttps://github.com/notifications/unsubscribe/ADSd0pGrn4n-cD64bGdLMvKv5K2Ak3S6ks5qHuN-gaJpZM4IrTpt.

[jwilder]

Slow startup should be fixed by #6743