Skip to content

Conversation

@LantaoJin
Copy link
Member

@LantaoJin LantaoJin commented Jun 26, 2025

Description

Add big5 (https://github.com/opensearch-project/opensearch-benchmark-workloads/tree/main/big5/queries) to PPL IT suite.
Only one documentation in big5 index. The target of this IT suite is displaying the end-to-end execution times (analyzing + optimizing + codegen + compiling).
It prints the end2end execution times, on my laptop is:

Summary of PPLBig5IT:
range_field_conjunction_big_range_big_term_query: 94 ms
range: 13 ms
query_string_on_message: 38 ms
composite_date_histogram_daily: 58 ms
desc_sort_timestamp: 14 ms
desc_sort_with_after_timestamp: 5 ms
terms_significant_1: 11 ms
range_numeric: 6 ms
default: 6 ms
asc_sort_with_after_timestamp: 5 ms
sort_numeric_asc_with_match: 6 ms
sort_numeric_desc_with_match: 5 ms
composite_terms_keyword: 9 ms
sort_numeric_asc: 5 ms
keyword_terms_low_cardinality: 11 ms
range_field_conjunction_small_range_big_term_query: 5 ms
keyword_in_range: 6 ms
range_with_desc_sort: 5 ms
range_with_asc_sort: 6 ms
multi_terms_keyword: 7 ms
range_field_disjunction_big_range_small_term_query: 11 ms
query_string_on_message_filtered: 5 ms
term: 4 ms
sort_keyword_can_match_shortcut: 4 ms
range_field_conjunction_small_range_small_term_query: 6 ms
scroll: 5 ms
terms_significant_2: 6 ms
keyword_terms: 4 ms
date_histogram_minute_agg: 7 ms
asc_sort_timestamp_can_match_shortcut: 4 ms
sort_numeric_desc: 4 ms
desc_sort_timestamp_no_can_match_shortcut: 5 ms
composite_terms: 6 ms
sort_keyword_no_can_match_shortcut: 4 ms
asc_sort_timestamp: 4 ms
asc_sort_timestamp_no_can_match_shortcut: 4 ms
query_string_on_message_filtered_sorted_num: 6 ms
date_histogram_hourly_agg: 5 ms
desc_sort_timestamp_can_match_shortcut: 4 ms
Total 39 queries succeed. Average duration: 10 ms
Summary of CalcitePPLBig5IT:
asc_sort_timestamp_can_match_shortcut: 84 ms
composite_date_histogram_daily: 150 ms
date_histogram_hourly_agg: 50 ms
terms_significant_1: 37 ms
range_numeric: 25 ms
asc_sort_timestamp: 17 ms
multi_terms_keyword: 38 ms
desc_sort_with_after_timestamp: 12 ms
range_field_conjunction_small_range_small_term_query: 40 ms
sort_numeric_desc_with_match: 17 ms
desc_sort_timestamp: 11 ms
composite_terms_keyword: 30 ms
query_string_on_message: 34 ms
range_with_asc_sort: 21 ms
scroll: 11 ms
query_string_on_message_filtered_sorted_num: 36 ms
range_field_disjunction_big_range_small_term_query: 29 ms
keyword_in_range: 38 ms
default: 9 ms
asc_sort_with_after_timestamp: 16 ms
asc_sort_timestamp_no_can_match_shortcut: 21 ms
range_field_conjunction_big_range_big_term_query: 22 ms
sort_numeric_asc: 12 ms
sort_numeric_asc_with_match: 11 ms
keyword_terms_low_cardinality: 21 ms
date_histogram_minute_agg: 25 ms
query_string_on_message_filtered: 23 ms
terms_significant_2: 34 ms
keyword_terms: 14 ms
term: 8 ms
range_field_conjunction_small_range_big_term_query: 17 ms
composite_terms: 25 ms
range_with_desc_sort: 19 ms
sort_numeric_desc: 10 ms
desc_sort_timestamp_can_match_shortcut: 11 ms
sort_keyword_can_match_shortcut: 12 ms
range: 17 ms
sort_keyword_no_can_match_shortcut: 11 ms
desc_sort_timestamp_no_can_match_shortcut: 10 ms
Total 39 queries succeed. Average duration: 26 ms

Related Issues

Resolves #[Issue number to be closed when this PR is merged]

Check List

  • New functionality includes testing.
  • New functionality has been documented.
  • New functionality has javadoc added.
  • New functionality has a user manual doc added.
  • API changes companion pull request created.
  • Commits are signed per the DCO using --signoff.
  • Public documentation issue/PR created.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Signed-off-by: Lantao Jin <ltjin@amazon.com>
@LantaoJin LantaoJin added the testing Related to improving software testing label Jun 26, 2025
Signed-off-by: Lantao Jin <ltjin@amazon.com>
Signed-off-by: Lantao Jin <ltjin@amazon.com>
Signed-off-by: Lantao Jin <ltjin@amazon.com>
@LantaoJin LantaoJin added the enhancement New feature or request label Jul 2, 2025
Copy link
Collaborator

@dai-chen dai-chen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The target of this IT suite is displaying the end-to-end execution times

I think this is benchmark instead of IT right? If so, we may put all these into benchmarks module?

@LantaoJin
Copy link
Member Author

The target of this IT suite is displaying the end-to-end execution times

I think this is benchmark instead of IT right? If so, we may put all these into benchmarks module?

Unlike jmh micro-benchmark, Big5 Suite requires a remote cluster to be launched, so it located in integ-test module, or we have to move the integ-test configs to benchmarks module.

@penghuo penghuo merged commit d94e4de into opensearch-project:main Jul 8, 2025
27 checks passed
opensearch-trigger-bot bot pushed a commit that referenced this pull request Jul 8, 2025
* Add big5 to IT Suite

Signed-off-by: Lantao Jin <ltjin@amazon.com>

* ignore client error

Signed-off-by: Lantao Jin <ltjin@amazon.com>

* add tq1

Signed-off-by: Lantao Jin <ltjin@amazon.com>

* refactor and ppl rewriting

Signed-off-by: Lantao Jin <ltjin@amazon.com>

* update the relevance queries

Signed-off-by: Lantao Jin <ltjin@amazon.com>

---------

Signed-off-by: Lantao Jin <ltjin@amazon.com>
(cherry picked from commit d94e4de)
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
qianheng-aws pushed a commit that referenced this pull request Jul 9, 2025
* Add big5 to IT Suite



* ignore client error



* add tq1



* refactor and ppl rewriting



* update the relevance queries



---------


(cherry picked from commit d94e4de)

Signed-off-by: Lantao Jin <ltjin@amazon.com>
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
@LantaoJin LantaoJin mentioned this pull request Jul 9, 2025
7 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backport 2.19-dev enhancement New feature or request testing Related to improving software testing

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants