Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support multi ranges traversal when doing date histogram rewrite optimization #13317

Merged
merged 18 commits into from
May 3, 2024

Conversation

bowenlan-amzn
Copy link
Member

@bowenlan-amzn bowenlan-amzn commented Apr 21, 2024

Description

When intersect and visit a BKD tree, the value of the point or the min/max of the inner node are ever-increasing.
The ranges rewritten from date histogram are also every-increasing, and connected with each other.
So, we can do a single traversal to populate the results for multiple ranges which makes sure that any points or inner node are only visited once.
And we can do it in a clever way that only when we meet a new value that not inside current range, we iterate to the next range, and we only need to collect into one range at any point of time.
This new value is possibly met when visiting a new node or visiting inside a leaf node.

Note it's hard to cover the traversal logic, it will need to ingest thousands or more documents (every leaf node contains 512 docs).

Benchmark Results

The threshold of doing filter rewrite is set to 70k, so every operation is using the optimization.
The JVM memory is 1g
r6g.xlarge instance (4vCPU, 32G memory), one node cluster

For each workload, old optimized results firt, then the new opimitzed results with multi range traversal, last one if exists would be the optimization disabled results which falls back to default aggregation workflow.

pmc

|                                                 Min Throughput | articles_monthly_agg_uncached |       19.95 |  ops/s |
|                                                Mean Throughput | articles_monthly_agg_uncached |       19.97 |  ops/s |
|                                              Median Throughput | articles_monthly_agg_uncached |       19.98 |  ops/s |
|                                                 Max Throughput | articles_monthly_agg_uncached |       19.98 |  ops/s |
|                                        50th percentile latency | articles_monthly_agg_uncached |     53.4486 |     ms |
|                                        90th percentile latency | articles_monthly_agg_uncached |     75.4133 |     ms |
|                                        99th percentile latency | articles_monthly_agg_uncached |     90.6223 |     ms |
|                                       100th percentile latency | articles_monthly_agg_uncached |     101.146 |     ms |
|                                   50th percentile service time | articles_monthly_agg_uncached |     46.0403 |     ms |
|                                   90th percentile service time | articles_monthly_agg_uncached |     56.9184 |     ms |
|                                   99th percentile service time | articles_monthly_agg_uncached |     65.0497 |     ms |
|                                  100th percentile service time | articles_monthly_agg_uncached |     65.5977 |     ms |
|                                                     error rate | articles_monthly_agg_uncached |           0 |      % |
|                                                 Min Throughput |   articles_monthly_agg_cached |       19.97 |  ops/s |
|                                                Mean Throughput |   articles_monthly_agg_cached |       19.97 |  ops/s |
|                                              Median Throughput |   articles_monthly_agg_cached |       19.97 |  ops/s |
|                                                 Max Throughput |   articles_monthly_agg_cached |       19.98 |  ops/s |
|                                        50th percentile latency |   articles_monthly_agg_cached |     5.03077 |     ms |
|                                        90th percentile latency |   articles_monthly_agg_cached |     5.44521 |     ms |
|                                        99th percentile latency |   articles_monthly_agg_cached |     5.62429 |     ms |
|                                       100th percentile latency |   articles_monthly_agg_cached |     5.69051 |     ms |
|                                   50th percentile service time |   articles_monthly_agg_cached |     4.36067 |     ms |
|                                   90th percentile service time |   articles_monthly_agg_cached |     4.52949 |     ms |
|                                   99th percentile service time |   articles_monthly_agg_cached |     4.73908 |     ms |
|                                  100th percentile service time |   articles_monthly_agg_cached |     4.81191 |     ms |
|                                                     error rate |   articles_monthly_agg_cached |           0 |      % |
|                                        50th percentile latency | articles_monthly_agg_uncached |     11.1038 |     ms |
|                                        90th percentile latency | articles_monthly_agg_uncached |     12.4704 |     ms |
|                                        99th percentile latency | articles_monthly_agg_uncached |     12.9994 |     ms |
|                                       100th percentile latency | articles_monthly_agg_uncached |     15.2319 |     ms |
|                                   50th percentile service time | articles_monthly_agg_uncached |     10.3322 |     ms |
|                                   90th percentile service time | articles_monthly_agg_uncached |     11.7222 |     ms |
|                                   99th percentile service time | articles_monthly_agg_uncached |     12.2619 |     ms |
|                                  100th percentile service time | articles_monthly_agg_uncached |     14.4408 |     ms |
|                                                     error rate | articles_monthly_agg_uncached |           0 |      % |
|                                                 Min Throughput |   articles_monthly_agg_cached |       20.02 |  ops/s |
|                                                Mean Throughput |   articles_monthly_agg_cached |       20.02 |  ops/s |
|                                              Median Throughput |   articles_monthly_agg_cached |       20.02 |  ops/s |
|                                                 Max Throughput |   articles_monthly_agg_cached |       20.02 |  ops/s |
|                                        50th percentile latency |   articles_monthly_agg_cached |     3.87232 |     ms |
|                                        90th percentile latency |   articles_monthly_agg_cached |     4.33174 |     ms |
|                                        99th percentile latency |   articles_monthly_agg_cached |     4.64391 |     ms |
|                                       100th percentile latency |   articles_monthly_agg_cached |      4.8993 |     ms |
|                                   50th percentile service time |   articles_monthly_agg_cached |     3.15404 |     ms |
|                                   90th percentile service time |   articles_monthly_agg_cached |     3.30391 |     ms |
|                                   99th percentile service time |   articles_monthly_agg_cached |     3.73018 |     ms |
|                                  100th percentile service time |   articles_monthly_agg_cached |     3.98568 |     ms |
|                                                     error rate |   articles_monthly_agg_cached |           0 |      % |
|                                                 Min Throughput | articles_monthly_agg_uncached |       19.99 |  ops/s |
|                                                Mean Throughput | articles_monthly_agg_uncached |       19.99 |  ops/s |
|                                              Median Throughput | articles_monthly_agg_uncached |       19.99 |  ops/s |
|                                                 Max Throughput | articles_monthly_agg_uncached |       19.99 |  ops/s |
|                                        50th percentile latency | articles_monthly_agg_uncached |     19.5404 |     ms |
|                                        90th percentile latency | articles_monthly_agg_uncached |      20.037 |     ms |
|                                        99th percentile latency | articles_monthly_agg_uncached |      27.151 |     ms |
|                                       100th percentile latency | articles_monthly_agg_uncached |     28.0157 |     ms |
|                                   50th percentile service time | articles_monthly_agg_uncached |     18.8363 |     ms |
|                                   90th percentile service time | articles_monthly_agg_uncached |     19.0544 |     ms |
|                                   99th percentile service time | articles_monthly_agg_uncached |     26.6789 |     ms |
|                                  100th percentile service time | articles_monthly_agg_uncached |     26.8235 |     ms |
|                                                     error rate | articles_monthly_agg_uncached |           0 |      % |
|                                                 Min Throughput |   articles_monthly_agg_cached |       20.01 |  ops/s |
|                                                Mean Throughput |   articles_monthly_agg_cached |       20.02 |  ops/s |
|                                              Median Throughput |   articles_monthly_agg_cached |       20.02 |  ops/s |
|                                                 Max Throughput |   articles_monthly_agg_cached |       20.02 |  ops/s |
|                                        50th percentile latency |   articles_monthly_agg_cached |     4.13489 |     ms |
|                                        90th percentile latency |   articles_monthly_agg_cached |     4.53141 |     ms |
|                                        99th percentile latency |   articles_monthly_agg_cached |     4.81172 |     ms |
|                                       100th percentile latency |   articles_monthly_agg_cached |     5.01492 |     ms |
|                                   50th percentile service time |   articles_monthly_agg_cached |     3.43127 |     ms |
|                                   90th percentile service time |   articles_monthly_agg_cached |      3.5448 |     ms |
|                                   99th percentile service time |   articles_monthly_agg_cached |     3.65492 |     ms |
|                                  100th percentile service time |   articles_monthly_agg_cached |     4.09986 |     ms |
|                                                     error rate |   articles_monthly_agg_cached |           0 |      % |

big5

|                                                 Min Throughput | composite-date_histogram-daily |        2.01 |  ops/s |
|                                                Mean Throughput | composite-date_histogram-daily |        2.01 |  ops/s |
|                                              Median Throughput | composite-date_histogram-daily |        2.01 |  ops/s |
|                                                 Max Throughput | composite-date_histogram-daily |        2.01 |  ops/s |
|                                        50th percentile latency | composite-date_histogram-daily |     11.7901 |     ms |
|                                        90th percentile latency | composite-date_histogram-daily |     12.2404 |     ms |
|                                        99th percentile latency | composite-date_histogram-daily |     12.9297 |     ms |
|                                       100th percentile latency | composite-date_histogram-daily |     13.0813 |     ms |
|                                   50th percentile service time | composite-date_histogram-daily |     10.5722 |     ms |
|                                   90th percentile service time | composite-date_histogram-daily |     10.7189 |     ms |
|                                   99th percentile service time | composite-date_histogram-daily |     11.1253 |     ms |
|                                  100th percentile service time | composite-date_histogram-daily |     11.6602 |     ms |
|                                                     error rate | composite-date_histogram-daily |           0 |      % |
|                                                 Min Throughput |      date_histogram_hourly_agg |           2 |  ops/s |
|                                                Mean Throughput |      date_histogram_hourly_agg |        2.01 |  ops/s |
|                                              Median Throughput |      date_histogram_hourly_agg |        2.01 |  ops/s |
|                                                 Max Throughput |      date_histogram_hourly_agg |        2.01 |  ops/s |
|                                        50th percentile latency |      date_histogram_hourly_agg |     157.577 |     ms |
|                                        90th percentile latency |      date_histogram_hourly_agg |     159.375 |     ms |
|                                        99th percentile latency |      date_histogram_hourly_agg |     163.655 |     ms |
|                                       100th percentile latency |      date_histogram_hourly_agg |     178.386 |     ms |
|                                   50th percentile service time |      date_histogram_hourly_agg |     156.441 |     ms |
|                                   90th percentile service time |      date_histogram_hourly_agg |     158.257 |     ms |
|                                   99th percentile service time |      date_histogram_hourly_agg |     162.534 |     ms |
|                                  100th percentile service time |      date_histogram_hourly_agg |      177.27 |     ms |
|                                                     error rate |      date_histogram_hourly_agg |           0 |      % |
|                                                 Min Throughput |      date_histogram_minute_agg |        1.27 |  ops/s |
|                                                Mean Throughput |      date_histogram_minute_agg |        1.27 |  ops/s |
|                                              Median Throughput |      date_histogram_minute_agg |        1.27 |  ops/s |
|                                                 Max Throughput |      date_histogram_minute_agg |        1.27 |  ops/s |
|                                        50th percentile latency |      date_histogram_minute_agg |       72008 |     ms |
|                                        90th percentile latency |      date_histogram_minute_agg |     83222.9 |     ms |
|                                        99th percentile latency |      date_histogram_minute_agg |     85749.4 |     ms |
|                                       100th percentile latency |      date_histogram_minute_agg |     86022.3 |     ms |
|                                   50th percentile service time |      date_histogram_minute_agg |     783.015 |     ms |
|                                   90th percentile service time |      date_histogram_minute_agg |     786.134 |     ms |
|                                   99th percentile service time |      date_histogram_minute_agg |     790.022 |     ms |
|                                  100th percentile service time |      date_histogram_minute_agg |     790.282 |     ms |
|                                                     error rate |      date_histogram_minute_agg |           0 |      % |
|                                                 Min Throughput | composite-date_histogram-daily |           2 |  ops/s |
|                                                Mean Throughput | composite-date_histogram-daily |        2.01 |  ops/s |
|                                              Median Throughput | composite-date_histogram-daily |        2.01 |  ops/s |
|                                                 Max Throughput | composite-date_histogram-daily |        2.01 |  ops/s |
|                                        50th percentile latency | composite-date_histogram-daily |     10.2503 |     ms |
|                                        90th percentile latency | composite-date_histogram-daily |     10.7774 |     ms |
|                                        99th percentile latency | composite-date_histogram-daily |     11.0552 |     ms |
|                                       100th percentile latency | composite-date_histogram-daily |     11.1781 |     ms |
|                                   50th percentile service time | composite-date_histogram-daily |      8.9753 |     ms |
|                                   90th percentile service time | composite-date_histogram-daily |     9.26641 |     ms |
|                                   99th percentile service time | composite-date_histogram-daily |      10.086 |     ms |
|                                  100th percentile service time | composite-date_histogram-daily |     10.3218 |     ms |
|                                                     error rate | composite-date_histogram-daily |           0 |      % |
|                                                 Min Throughput |      date_histogram_hourly_agg |        2.01 |  ops/s |
|                                                Mean Throughput |      date_histogram_hourly_agg |        2.01 |  ops/s |
|                                              Median Throughput |      date_histogram_hourly_agg |        2.01 |  ops/s |
|                                                 Max Throughput |      date_histogram_hourly_agg |        2.01 |  ops/s |
|                                        50th percentile latency |      date_histogram_hourly_agg |      19.289 |     ms |
|                                        90th percentile latency |      date_histogram_hourly_agg |     19.9434 |     ms |
|                                        99th percentile latency |      date_histogram_hourly_agg |     21.8219 |     ms |
|                                       100th percentile latency |      date_histogram_hourly_agg |     29.2806 |     ms |
|                                   50th percentile service time |      date_histogram_hourly_agg |     18.2352 |     ms |
|                                   90th percentile service time |      date_histogram_hourly_agg |     18.5031 |     ms |
|                                   99th percentile service time |      date_histogram_hourly_agg |     20.5678 |     ms |
|                                  100th percentile service time |      date_histogram_hourly_agg |     28.1168 |     ms |
|                                                     error rate |      date_histogram_hourly_agg |           0 |      % |
|                                                 Min Throughput |      date_histogram_minute_agg |        2.01 |  ops/s |
|                                                Mean Throughput |      date_histogram_minute_agg |        2.01 |  ops/s |
|                                              Median Throughput |      date_histogram_minute_agg |        2.01 |  ops/s |
|                                                 Max Throughput |      date_histogram_minute_agg |        2.01 |  ops/s |
|                                        50th percentile latency |      date_histogram_minute_agg |     71.7014 |     ms |
|                                        90th percentile latency |      date_histogram_minute_agg |     72.2578 |     ms |
|                                        99th percentile latency |      date_histogram_minute_agg |     72.9293 |     ms |
|                                       100th percentile latency |      date_histogram_minute_agg |     94.9371 |     ms |
|                                   50th percentile service time |      date_histogram_minute_agg |     70.4787 |     ms |
|                                   90th percentile service time |      date_histogram_minute_agg |      70.908 |     ms |
|                                   99th percentile service time |      date_histogram_minute_agg |     71.8165 |     ms |
|                                  100th percentile service time |      date_histogram_minute_agg |     94.2959 |     ms |
|                                                     error rate |      date_histogram_minute_agg |           0 |      % |

nyc_taxis

|                                                 Min Throughput |      autohisto_agg |        1.51 |  ops/s |
|                                                Mean Throughput |      autohisto_agg |        1.51 |  ops/s |
|                                              Median Throughput |      autohisto_agg |        1.51 |  ops/s |
|                                                 Max Throughput |      autohisto_agg |        1.52 |  ops/s |
|                                        50th percentile latency |      autohisto_agg |     25.4275 |     ms |
|                                        90th percentile latency |      autohisto_agg |     26.1543 |     ms |
|                                        99th percentile latency |      autohisto_agg |     26.9703 |     ms |
|                                       100th percentile latency |      autohisto_agg |     27.9551 |     ms |
|                                   50th percentile service time |      autohisto_agg |     24.0695 |     ms |
|                                   90th percentile service time |      autohisto_agg |     24.6367 |     ms |
|                                   99th percentile service time |      autohisto_agg |     25.8091 |     ms |
|                                  100th percentile service time |      autohisto_agg |     26.3474 |     ms |
|                                                     error rate |      autohisto_agg |           0 |      % |
|                                                 Min Throughput | date_histogram_agg |        1.51 |  ops/s |
|                                                Mean Throughput | date_histogram_agg |        1.52 |  ops/s |
|                                              Median Throughput | date_histogram_agg |        1.51 |  ops/s |
|                                                 Max Throughput | date_histogram_agg |        1.53 |  ops/s |
|                                        50th percentile latency | date_histogram_agg |     24.7382 |     ms |
|                                        90th percentile latency | date_histogram_agg |     25.1863 |     ms |
|                                        99th percentile latency | date_histogram_agg |     26.5017 |     ms |
|                                       100th percentile latency | date_histogram_agg |     28.9566 |     ms |
|                                   50th percentile service time | date_histogram_agg |      23.325 |     ms |
|                                   90th percentile service time | date_histogram_agg |      23.514 |     ms |
|                                   99th percentile service time | date_histogram_agg |     25.2249 |     ms |
|                                  100th percentile service time | date_histogram_agg |     27.3667 |     ms |
|                                                     error rate | date_histogram_agg |           0 |      % |
|                                                 Min Throughput |      autohisto_agg |        1.51 |  ops/s |
|                                                Mean Throughput |      autohisto_agg |        1.52 |  ops/s |
|                                              Median Throughput |      autohisto_agg |        1.51 |  ops/s |
|                                                 Max Throughput |      autohisto_agg |        1.53 |  ops/s |
|                                        50th percentile latency |      autohisto_agg |     13.9074 |     ms |
|                                        90th percentile latency |      autohisto_agg |     14.2888 |     ms |
|                                        99th percentile latency |      autohisto_agg |     15.2199 |     ms |
|                                       100th percentile latency |      autohisto_agg |     15.9146 |     ms |
|                                   50th percentile service time |      autohisto_agg |     12.4878 |     ms |
|                                   90th percentile service time |      autohisto_agg |     12.7325 |     ms |
|                                   99th percentile service time |      autohisto_agg |     13.6324 |     ms |
|                                  100th percentile service time |      autohisto_agg |     14.7879 |     ms |
|                                                     error rate |      autohisto_agg |           0 |      % |
|                                                 Min Throughput | date_histogram_agg |        1.51 |  ops/s |
|                                                Mean Throughput | date_histogram_agg |        1.52 |  ops/s |
|                                              Median Throughput | date_histogram_agg |        1.51 |  ops/s |
|                                                 Max Throughput | date_histogram_agg |        1.53 |  ops/s |
|                                        50th percentile latency | date_histogram_agg |     14.2494 |     ms |
|                                        90th percentile latency | date_histogram_agg |     14.6327 |     ms |
|                                        99th percentile latency | date_histogram_agg |     14.8823 |     ms |
|                                       100th percentile latency | date_histogram_agg |     16.1273 |     ms |
|                                   50th percentile service time | date_histogram_agg |     12.9227 |     ms |
|                                   90th percentile service time | date_histogram_agg |     13.1052 |     ms |
|                                   99th percentile service time | date_histogram_agg |     13.4257 |     ms |
|                                  100th percentile service time | date_histogram_agg |     14.2286 |     ms |
|                                                     error rate | date_histogram_agg |           0 |      % |

http_logs

|                                                 Min Throughput | hourly_agg |         0.2 |  ops/s |
|                                                Mean Throughput | hourly_agg |         0.2 |  ops/s |
|                                              Median Throughput | hourly_agg |         0.2 |  ops/s |
|                                                 Max Throughput | hourly_agg |         0.2 |  ops/s |
|                                        50th percentile latency | hourly_agg |     283.503 |     ms |
|                                        90th percentile latency | hourly_agg |     322.241 |     ms |
|                                        99th percentile latency | hourly_agg |     342.808 |     ms |
|                                       100th percentile latency | hourly_agg |     352.563 |     ms |
|                                   50th percentile service time | hourly_agg |     280.674 |     ms |
|                                   90th percentile service time | hourly_agg |     318.865 |     ms |
|                                   99th percentile service time | hourly_agg |     336.963 |     ms |
|                                  100th percentile service time | hourly_agg |      346.66 |     ms |
|                                                     error rate | hourly_agg |           0 |      % |
|                                                 Min Throughput | hourly_agg |         0.2 |  ops/s |
|                                                Mean Throughput | hourly_agg |         0.2 |  ops/s |
|                                              Median Throughput | hourly_agg |         0.2 |  ops/s |
|                                                 Max Throughput | hourly_agg |         0.2 |  ops/s |
|                                        50th percentile latency | hourly_agg |     41.3073 |     ms |
|                                        90th percentile latency | hourly_agg |     44.4373 |     ms |
|                                        99th percentile latency | hourly_agg |     48.0578 |     ms |
|                                       100th percentile latency | hourly_agg |     53.0044 |     ms |
|                                   50th percentile service time | hourly_agg |     37.6982 |     ms |
|                                   90th percentile service time | hourly_agg |     41.1426 |     ms |
|                                   99th percentile service time | hourly_agg |     44.6609 |     ms |
|                                  100th percentile service time | hourly_agg |     46.2757 |     ms |
|                                                     error rate | hourly_agg |           0 |      % |

Related Issues

Resolves #13171 #13345

Check List

  • New functionality includes testing.
    • All tests pass
  • New functionality has been documented.
    • New functionality has javadoc added
  • Failing checks are inspected and point to the corresponding known issue(s) (See: Troubleshooting Failing Builds)
  • Commits are signed per the DCO using --signoff
  • Commit changes are listed out in CHANGELOG.md file (See: Changelog)
  • [ ] Public documentation issue/PR created

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com>
@github-actions github-actions bot added bug Something isn't working Performance This is for any performance related enhancements or bugs Search:Aggregations labels Apr 21, 2024
@bowenlan-amzn bowenlan-amzn changed the title multi range traversal in BKD Support multi ranges traversal when doing date histogram rewrite optimization Apr 21, 2024
Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com>
@bowenlan-amzn bowenlan-amzn added v2.14.0 backport 2.x Backport to 2.x branch labels Apr 21, 2024
Copy link
Contributor

❌ Gradle check result for c4d9055: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Copy link
Contributor

❌ Gradle check result for 02f5c14: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

@bowenlan-amzn bowenlan-amzn force-pushed the issue13171-main branch 2 times, most recently from 76cc193 to aaa7414 Compare April 21, 2024 18:07
Copy link
Contributor

❌ Gradle check result for 76dcb65: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Copy link
Contributor

❌ Gradle check result for 76cc193: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Copy link
Contributor

❌ Gradle check result for aaa7414: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Copy link
Contributor

❌ Gradle check result for aaa7414:

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Copy link
Contributor

❌ Gradle check result for aaa7414: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com>
Copy link
Contributor

✅ Gradle check result for 1ee1070: SUCCESS

Copy link

codecov bot commented Apr 22, 2024

Codecov Report

Attention: Patch coverage is 81.12245% with 37 lines in your changes are missing coverage. Please review.

Project coverage is 71.55%. Comparing base (b15cb0c) to head (d739534).
Report is 258 commits behind head on main.

Files Patch % Lines
...h/aggregations/bucket/FastFilterRewriteHelper.java 88.82% 8 Missing and 11 partials ⚠️
.../bucket/histogram/AutoDateHistogramAggregator.java 22.22% 7 Missing ⚠️
...egations/bucket/composite/CompositeAggregator.java 40.00% 6 Missing ⚠️
...ions/bucket/histogram/DateHistogramAggregator.java 28.57% 5 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff              @@
##               main   #13317      +/-   ##
============================================
+ Coverage     71.42%   71.55%   +0.13%     
- Complexity    59978    60991    +1013     
============================================
  Files          4985     5050      +65     
  Lines        282275   286987    +4712     
  Branches      40946    41591     +645     
============================================
+ Hits         201603   205347    +3744     
- Misses        63999    64618     +619     
- Partials      16673    17022     +349     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com>
Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com>
Copy link
Contributor

❌ Gradle check result for ba7c549: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Copy link
Contributor

❌ Gradle check result for 50126be: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Copy link
Contributor

github-actions bot commented May 3, 2024

❌ Gradle check result for d739534: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Copy link
Contributor

github-actions bot commented May 3, 2024

✅ Gradle check result for d739534: SUCCESS

@msfroh msfroh merged commit ed33488 into opensearch-project:main May 3, 2024
28 checks passed
@opensearch-trigger-bot
Copy link
Contributor

The backport to 2.x failed:

The process '/usr/bin/git' failed with exit code 128

To backport manually, run these commands in your terminal:

# Navigate to the root of your repository
cd $(git rev-parse --show-toplevel)
# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add ../.worktrees/OpenSearch/backport-2.x 2.x
# Navigate to the new working tree
pushd ../.worktrees/OpenSearch/backport-2.x
# Create a new branch
git switch --create backport/backport-13317-to-2.x
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 ed33488aa426bd618685729fc638adad763f6ff7
# Push it to GitHub
git push --set-upstream origin backport/backport-13317-to-2.x
# Go back to the original working tree
popd
# Delete the working tree
git worktree remove ../.worktrees/OpenSearch/backport-2.x

Then, create a pull request where the base branch is 2.x and the compare/head branch is backport/backport-13317-to-2.x.

bowenlan-amzn added a commit to bowenlan-amzn/OpenSearch that referenced this pull request May 3, 2024
bowenlan-amzn added a commit to bowenlan-amzn/OpenSearch that referenced this pull request May 3, 2024
…mization (opensearch-project#13317)

Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com>
bowenlan-amzn added a commit to bowenlan-amzn/OpenSearch that referenced this pull request May 3, 2024
…mization (opensearch-project#13317)

Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com>

remove unnecessary change log

Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com>
@bowenlan-amzn bowenlan-amzn deleted the issue13171-main branch May 3, 2024 05:24
bowenlan-amzn added a commit to bowenlan-amzn/OpenSearch that referenced this pull request May 3, 2024
…mization (opensearch-project#13317)

Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com>

remove unnecessary change log

Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com>
dbwiddis pushed a commit that referenced this pull request May 3, 2024
…ram rewrite optimization (#13317) (#13533)

* Support multi ranges traversal when doing date histogram rewrite optimization (#13317)

Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com>

remove unnecessary change log

Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com>

* Update rest tests skip version to include 2.15

Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com>

---------

Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com>
msfroh pushed a commit that referenced this pull request May 3, 2024
…am rewrite optimization (#13317) (#13522)

* Support multi ranges traversal when doing date histogram rewrite optimization (#13317)

Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com>

remove unnecessary change log

Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com>

* Update rest tests skip version to include 2.15

Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com>

---------

Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com>
@bowenlan-amzn
Copy link
Member Author

Confirmed the performance improvements from all 4 workloads

finnegancarroll pushed a commit to finnegancarroll/OpenSearch that referenced this pull request May 10, 2024
deshsidd pushed a commit to deshsidd/OpenSearch that referenced this pull request May 17, 2024
parv0201 pushed a commit to parv0201/OpenSearch that referenced this pull request Jun 10, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport 2.x Backport to 2.x branch backport-failed bug Something isn't working Performance This is for any performance related enhancements or bugs Search:Aggregations v2.14.0
Projects
Status: 2.14.0 (Launched)
Development

Successfully merging this pull request may close these issues.

Possible regression of date histogram starting from 2.12
5 participants