Skip to content
This repository has been archived by the owner on Mar 27, 2021. It is now read-only.

Dynamically determine metadata read indices based upon query range #745

Closed
adsail opened this issue Jan 21, 2021 · 0 comments · Fixed by #746
Closed

Dynamically determine metadata read indices based upon query range #745

adsail opened this issue Jan 21, 2021 · 0 comments · Fixed by #746
Assignees

Comments

@adsail
Copy link
Contributor

adsail commented Jan 21, 2021

While the rotation interval of the ES write index—in other words, how frequently Heroic consumers stop writing to an elasticsearch index, create a new one and start writing to it instead—is configurable, the maximum number of read indices is not configurable and not dynamic.

maxReadIndices should be dynamically determined based upon the date range of a query and the rotation interval of Elasticsearch indices. We can then narrow the metadata response to only return documents relevant to the query as opposed to all documents. This will narrow the query to BigTable and lay the groundwork for future guardrails to enable malleable series limits that take query range into consideration.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant