Track stats for number of series, measurements

[mark-rushakoff]

Track stats for number of series, measurements

Per database: track number of series and measurements
Per measurement: track number of series

Now you can run queries like:

> select sum(numSeries) from _internal.."measurement" where time > now() - 30s group by time(10s), "database"
name: measurement
tags: database=_internal
time            sum
----            ---
2016-02-24T00:35:30Z
2016-02-24T00:35:40Z    50
2016-02-24T00:35:50Z    50
2016-02-24T00:36:00Z    50


name: measurement
tags: database=telegraf
time            sum
----            ---
2016-02-24T00:35:30Z
2016-02-24T00:35:40Z    61
2016-02-24T00:35:50Z    61
2016-02-24T00:36:00Z    61

> select * from _internal.."database" where time > now() - 30s group by "database"
name: database
tags: database=_internal
time            clusterID       database    hostname        nodeID  numMeasurements numSeries
----            ---------       --------    --------        ------  --------------- ---------
2016-02-24T00:36:00Z    3699771861689734216 _internal   Marks-MacBook-Pro.local 0   11      50
2016-02-24T00:36:10Z    3699771861689734216 _internal   Marks-MacBook-Pro.local 0   11      50
2016-02-24T00:36:20Z    3699771861689734216 _internal   Marks-MacBook-Pro.local 0   11      50


name: database
tags: database=telegraf
time            clusterID       database    hostname        nodeID  numMeasurements numSeries
----            ---------       --------    --------        ------  --------------- ---------
2016-02-24T00:36:00Z    3699771861689734216 telegraf    Marks-MacBook-Pro.local 0   14      61
2016-02-24T00:36:10Z    3699771861689734216 telegraf    Marks-MacBook-Pro.local 0   14      61
2016-02-24T00:36:20Z    3699771861689734216 telegraf    Marks-MacBook-Pro.local 0   14      61

> select "database", "measurement", top(numSeries, 10) from _internal.."measurement"  where time > now() - 10s
name: measurement
-----------------
time            database    measurement     top
2016-02-24T04:07:32Z    telegraf    influxdb_measurement    25
2016-02-24T04:07:32Z    _internal   measurement     25
2016-02-24T04:07:32Z    _internal   shard           7
2016-02-24T04:07:32Z    telegraf    influxdb_shard      7
2016-02-24T04:07:32Z    telegraf    net         6
2016-02-24T04:07:32Z    telegraf    cpu         4
2016-02-24T04:07:32Z    _internal   tsm1_wal        4
2016-02-24T04:07:32Z    telegraf    influxdb_tsm1_cache 4
2016-02-24T04:07:32Z    telegraf    influxdb_tsm1_wal   4
2016-02-24T04:07:32Z    _internal   tsm1_cache      4

[e-dard]

Since this is NewDatabaseIndex, name will suffice for the argument.

[e-dard]

just a couple of small questions.

[mark-rushakoff]

@e-dard variable renamed as requested; other question is a non-issue, I believe. Good to merge on green?

[e-dard]

@mark-rushakoff needs another review from Core.

[panda87]

@mark-rushakoff can you add these stats (number of series, measurements) to the http://127.0.0.1:8086/debug/vars? Im using telegraf input for influxdb and see these stats as metrics will be great to monitor my no. of series - for performance purposes.

[benbjohnson]

👍

[mark-rushakoff]

@panda87 any stats (i.e. anything tracked via influxdb.NewStatistics automatically show up in the /debug/vars endpoint. The next nightly build will contain that functionality.

[panda87]

This is awesome, thanks

[adrianlzt]

@mark-rushakoff, I see that you use the trick where time > now() - 10s to only get the last values. But it will fail if it is executed just in the moment it is getting updated.

Sorry to ask this here.

[e-dard]

@adrianlzt a good place to ask questions like that is https://community.influxdata.com

[Codelica]

Sorry to revive this merge, but was it fully committed? I'm interested in getting numSeries totals for each measurement in each database, which from the last example in the original posting looks like it should be possible?

select "database", "measurement", top(numSeries, 10) from _internal.."measurement" where time > now() - 10s

But _internal.."measurement" doesn't seem to exist on my system (Influx 1.2.2) and I don't see any other measurement in _internal that would have series counts at a db/measurement level.

Any info would be greatly appreciated. Thanks!

[e-dard]

@Codelica the measurement name that holds the numSeries field is called "database".

select "database", "measurement", top("numSeries", 10) from _internal.."database" where time > now() - 10s

[Codelica]

@e-dard thanks. I did find that, but doesn't include a measurement tag, so it's only possible to get the total series count per db. I looked to see if it might be some config option, but didn't see anything. I'm trying to get down to the per measurement level to monitor serveral measurements within our db's that have the potential to grow quickly. It just seems like the per measurement capabilities described by @mark-rushakoff in the original post here got trimmed somewhere along the way? Thanks again.

[jwilder]

The per measurement stats had to be removed in #6168 because it caused #6131. There are more details discussed in those links.

[Codelica]

@jwilder Thanks.. that clears up my mystery. 👍 Wish it could have been left as an option for those of us without a large number of measurements, but I understand.

One related question and I'm out of everyone's hair. :) The only way I've been able to track this is with an external script executing "SHOW SERIES FROM XXX" and then processing the output to a count. (which actually seems to randomly kill the Influx daemon on large counts) Am I overlooking any other way to do this ? (where a count is returned rather than 150k+ series to count)

Thanks again.

[jwilder]

@Codelica show series is pretty expensive unfortunately. There isn't a way to get cardinality per measurement through the query language right now. There is a proposal in #7195, but that is not implemented yet.

While not ideal, you can run influx_inspect report -detailed /path/to/shard/data/dir and it will output some series cardinality estimates. The output format is not stable and very likely to change in the future, but you might be able to script something to get your counts.

[Codelica]

Thanks for all the info... 👍