Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Split Discover query into three #69134

Closed
timroes opened this issue Jun 15, 2020 · 4 comments · Fixed by #104818
Closed

Split Discover query into three #69134

timroes opened this issue Jun 15, 2020 · 4 comments · Fixed by #104818
Assignees
Labels
blocked enhancement New value added to drive a business result Feature:Discover Discover Application performance Team:Visualizations Visualization editors, elastic-charts and infrastructure

Comments

@timroes
Copy link
Contributor

timroes commented Jun 15, 2020

It would be better for performance reasons, if we would split the Discover query which currently loads the data for the chart on top and the document list into two queries (one for the documents, one for the chart on top). That aggregation query can then run potentially slower and doesn't block us from seeing the documents already.

This will also be a requirement for #55975, because if we're doing the aggregation in the same request, we will not have much performance win, from not requesting the total hits.

@timroes timroes added Feature:Discover Discover Application performance enhancement New value added to drive a business result Team:Visualizations Visualization editors, elastic-charts and infrastructure labels Jun 15, 2020
@elasticmachine
Copy link
Contributor

Pinging @elastic/kibana-app (Team:KibanaApp)

@timroes
Copy link
Contributor Author

timroes commented Nov 2, 2020

This comment (#76307 (comment)) has a couple of more up-to-date details on the splitting of queries.

@timroes timroes changed the title Split Discover query into two Split Discover query into three Nov 2, 2020
@timroes
Copy link
Contributor Author

timroes commented Nov 5, 2020

Technical notes

After splitting queries (and #55975 which I will close to sum them both up in this issue) the logic of Discover should be as follows:

  1. Request the documents with a size: x (x is at the moment 500) and a track_total_hits: false query. This query should be super fast.
  2. When this query returns we're showing the documents retrieved via it and we show the hit count in Discover. There are two potential cases:
    1. The amount of documents found are all documents, thus we get an exact hit count (eq) returned in the request. We simply show that as the hit count.
    2. The amount of documents found are not all documents, thus we only get an estimated hit count (like > 500). We show that hit count in the UI as > 500 and send out a separate request with size: 0 and track_total_hits: true to get the exact hit count. We show a loading spinner in the UI beside the hit count. Once this request is done we replace the estimated hit count by the exact hit count and remove the loading spinner.
  3. If the index pattern is time based and we want to show a chart above the documents, start in parallel to (1) send out a request with just the date_histogram and size: 0 and show a loading spinner in place of the chart until that request returned and we can draw the chart.

@timroes
Copy link
Contributor Author

timroes commented Dec 17, 2020

Blocked on #55974 currently

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
blocked enhancement New value added to drive a business result Feature:Discover Discover Application performance Team:Visualizations Visualization editors, elastic-charts and infrastructure
Projects
None yet
3 participants