Paginate results from /metrics/search

[ksheedlo]

I develop Rackspace Intelligence, which uses Blueflood extensively via Rackspace Monitoring and Cloud Metrics. I've been debugging a performance problem where we need to find a device that generates metrics so we can display it on a particular graph page, not knowing which device it will be ahead of time.

Currently we call an API in Rackspace Monitoring that delegates to Metrics (which is Blueflood). The Blueflood API does not support any limits on the size of the response beyond the hard coded limit of 100,000 in the code. I believe this is causing the time to respond to go up without a reasonable bound on large accounts. On the customer's account that alerted us to the issue, I saw response times from Blueflood up to 20 seconds.

We would like to speed up this operation by calling Rackspace Cloud Metrics directly, but we need an upper bound on the latency of this operation for it to make sense. It would be nice to expose the from and size parameters to Elasticsearch as queryable parameters to the /metrics/search API.