Running metrics on buckets by doc_count

[jmacmahon]

It would be really useful to run metrics on the doc_count field of buckets returned by aggregations, rather than just on fields of the documents in the buckets themselves.

My particular use-case is that I have an index of timestamped log entries, and I want to find the distrubution of log rate per unit time. In particular I might want to find the 95th-percentile logs per second. It's easy enough to partition the data into second-size buckets using date_histogram, and then get a doc_count for each second in the month, but the only way to split these into percentiles is to sort by doc_count and then download the data to the client and do the percentile calculations there.

Maybe I'm missing some functionality and there is a way to do this, but this forum post in May 2014 says that it's impossible: http://elasticsearch-users.115913.n3.nabble.com/stats-extended-stats-percentiles-for-doc-count-in-aggregations-td4055201.html

Sorry if this has been discussed and ruled out.

[clintongormley]

Hi @jmacmahon

Agreed this would be nice to have. I'm going to close this issue in favour of #4404, which is similar.

thanks