[Backport 2.19-dev] Support Bin command with Calcite
#4169
Closed
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Signed-off-by: opensearch-ci-bot <opensearch-infra@amazon.com> Co-authored-by: opensearch-ci-bot <opensearch-infra@amazon.com>
Signed-off-by: opensearch-ci-bot <opensearch-infra@amazon.com> Signed-off-by: Zelin Hao <zelinhao@amazon.com> Co-authored-by: opensearch-ci-bot <opensearch-infra@amazon.com>
…oject#3160) (opensearch-project#3597) * Test cases * Minimise code changes * Format * Update integration test * Update unit test --------- (cherry picked from commit eb88e3c) Signed-off-by: Andy Kwok <andy.kwok@improving.com> Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Signed-off-by: opensearch-ci-bot <opensearch-infra@amazon.com> Co-authored-by: opensearch-ci-bot <opensearch-infra@amazon.com>
…t#3110) Signed-off-by: currantw <taylor.curran@improving.com> Signed-off-by: xinyual <xinyual@amazon.com>
…ject#3075) Signed-off-by: Norman Jordan <norman.jordan@improving.com> Signed-off-by: normanj-bitquill <78755797+normanj-bitquill@users.noreply.github.com> Co-authored-by: Andrew Carbonetto <andrew.carbonetto@improving.com> Signed-off-by: xinyual <xinyual@amazon.com>
* Add trendline (With SWA) PPL command --------- Signed-off-by: James Duong <james.duong@improving.com> Signed-off-by: Andrew Carbonetto <andrew.carbonetto@improving.com> Co-authored-by: Andrew Carbonetto <andrew.carbonetto@improving.com> Signed-off-by: xinyual <xinyual@amazon.com>
Signed-off-by: currantw <taylor.curran@improving.com> * Add support for casting (`cast(field_name to ip)`) and remove existing unused sorting syntax. Signed-off-by: currantw <taylor.curran@improving.com> * Update comparison logic to compare in IPv6 Signed-off-by: currantw <taylor.curran@improving.com> * Fix bug casting to IP Signed-off-by: currantw <taylor.curran@improving.com> * Fix failing tests Signed-off-by: currantw <taylor.curran@improving.com> * Assert that comparison only valid if same type, update tests accordingly Signed-off-by: currantw <taylor.curran@improving.com> * Add additional tests to increase code coverage Signed-off-by: currantw <taylor.curran@improving.com> * Integrate `cidrmatch` changes Signed-off-by: currantw <taylor.curran@improving.com> * Remove `OpenSearchIPType` data type Signed-off-by: currantw <taylor.curran@improving.com> * Fix more failing tests Signed-off-by: currantw <taylor.curran@improving.com> * Minor cleanup Signed-off-by: currantw <taylor.curran@improving.com> * Add new tests for IP data type to `SortCommandIT`, and update `weblogs` test data. Signed-off-by: currantw <taylor.curran@improving.com> * Fixing IT test failure. Signed-off-by: currantw <taylor.curran@improving.com> * Spotless and update test to sort in SQL Signed-off-by: currantw <taylor.curran@improving.com> * Fix broken link Signed-off-by: currantw <taylor.curran@improving.com> * Fix failing code coverage Signed-off-by: currantw <taylor.curran@improving.com> * Fix failing doctest Signed-off-by: currantw <taylor.curran@improving.com> * Fix failing `ip.rst` doctest Signed-off-by: currantw <taylor.curran@improving.com> * Fix test failure due to merge. Signed-off-by: currantw <taylor.curran@improving.com> * Fix spotless Signed-off-by: currantw <taylor.curran@improving.com> * Add missing `url` field Signed-off-by: currantw <taylor.curran@improving.com> * Address minor review comments. Signed-off-by: currantw <taylor.curran@improving.com> * Revert sort syntax changes Signed-off-by: currantw <taylor.curran@improving.com> * Minor doc update Signed-off-by: currantw <taylor.curran@improving.com> * FIx failing `ip.rst` doctest Signed-off-by: currantw <taylor.curran@improving.com> * Add `IPComparisonIT` tests for comparison operators, rename modules and weblogs test index to make plural for consistency. Signed-off-by: currantw <taylor.curran@improving.com> --------- Signed-off-by: currantw <taylor.curran@improving.com> Signed-off-by: xinyual <xinyual@amazon.com>
…er (opensearch-project#3249) * First commit for Calcite integration Signed-off-by: Lantao Jin <ltjin@amazon.com> * disable java security manager in IT Signed-off-by: Lantao Jin <ltjin@amazon.com> --------- Signed-off-by: Lantao Jin <ltjin@amazon.com> Signed-off-by: xinyual <xinyual@amazon.com>
…ject#3258) * [POC] Make Calcite execute successfully Signed-off-by: Heng Qian <qianheng@amazon.com> * [POC] Change caching schema to simple schema and avoid registering table when visitRelation. Signed-off-by: Heng Qian <qianheng@amazon.com> * spotlessApply Signed-off-by: Heng Qian <qianheng@amazon.com> * address comments Signed-off-by: Heng Qian <qianheng@amazon.com> --------- Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: xinyual <xinyual@amazon.com>
* Make basic aggregation working (partial) Signed-off-by: Lantao Jin <ltjin@amazon.com> * add a settings to enable calcite Signed-off-by: Lantao Jin <ltjin@amazon.com> * add more UTs Signed-off-by: Lantao Jin <ltjin@amazon.com> --------- Signed-off-by: Lantao Jin <ltjin@amazon.com> Signed-off-by: xinyual <xinyual@amazon.com>
opensearch-project#3327) * Support Filter and Project pushdown Signed-off-by: Heng Qian <qianheng@amazon.com> * Support Filter and Project pushdown v2 Signed-off-by: Heng Qian <qianheng@amazon.com> * Address comments Signed-off-by: Heng Qian <qianheng@amazon.com> * Add original license for PredicateAnalyzer Signed-off-by: Heng Qian <qianheng@amazon.com> --------- Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: xinyual <xinyual@amazon.com>
* Build integration test framework Signed-off-by: Lantao Jin <ltjin@amazon.com> * make local work Signed-off-by: Lantao Jin <ltjin@amazon.com> * Fix the timestamp issue Signed-off-by: Lantao Jin <ltjin@amazon.com> * address comments Signed-off-by: Lantao Jin <ltjin@amazon.com> * fix java style and rename CalcitePPLTestCase back to CalcitePPLIntegTestCase Signed-off-by: Lantao Jin <ltjin@amazon.com> --------- Signed-off-by: Lantao Jin <ltjin@amazon.com> Signed-off-by: xinyual <xinyual@amazon.com>
…nsearch-project#3349) Signed-off-by: Lantao Jin <ltjin@amazon.com> Signed-off-by: xinyual <xinyual@amazon.com>
…oject#3355) * Add more aggregation tests Signed-off-by: Lantao Jin <ltjin@amazon.com> * delete unrelavant code Signed-off-by: Lantao Jin <ltjin@amazon.com> --------- Signed-off-by: Lantao Jin <ltjin@amazon.com> Signed-off-by: xinyual <xinyual@amazon.com>
* Transform to calcite plan before executing Signed-off-by: Heng Qian <qianheng@amazon.com> * Fix bug for single column row Signed-off-by: Heng Qian <qianheng@amazon.com> * Add settings for calcite pushdown Signed-off-by: Heng Qian <qianheng@amazon.com> * Lazily construct OpenSearchRequestBuilder and do push down Signed-off-by: Heng Qian <qianheng@amazon.com> * Address comments and disable push down Signed-off-by: Heng Qian <qianheng@amazon.com> --------- Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: xinyual <xinyual@amazon.com>
Signed-off-by: Lantao Jin <ltjin@amazon.com> Signed-off-by: xinyual <xinyual@amazon.com>
Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: xinyual <xinyual@amazon.com>
* Fix PredicateAnalyzer for in and notIn Signed-off-by: Heng Qian <qianheng@amazon.com> * Change text field to keyword since we don't support push down for that type Signed-off-by: Heng Qian <qianheng@amazon.com> --------- Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: xinyual <xinyual@amazon.com>
…3371) Signed-off-by: xinyual <xinyual@amazon.com>
Signed-off-by: xinyual <xinyual@amazon.com>
Signed-off-by: xinyual <xinyual@amazon.com>
…3376) * [BugFix] Fix text field push down Signed-off-by: Heng Qian <qianheng@amazon.com> * Ignore CalciteSortCommandIT.testSortWithNullValue Signed-off-by: Heng Qian <qianheng@amazon.com> * Refine code: only get keyword subfield for termQuery builder Signed-off-by: Heng Qian <qianheng@amazon.com> * Refine code Signed-off-by: Heng Qian <qianheng@amazon.com> * remove ignore tests in CalcitePPLInSubqueryIT Signed-off-by: Heng Qian <qianheng@amazon.com> --------- Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: xinyual <xinyual@amazon.com>
* add udf/udaf interface and take/sqrt function Signed-off-by: xinyual <xinyual@amazon.com> * add UT Signed-off-by: xinyual <xinyual@amazon.com> * add POW, Atan, Atan2 and corresponding UT Signed-off-by: xinyual <xinyual@amazon.com> * apply spotless Signed-off-by: xinyual <xinyual@amazon.com> * fix table for join it Signed-off-by: xinyual <xinyual@amazon.com> * add java doc Signed-off-by: xinyual <xinyual@amazon.com> * apply spotless Signed-off-by: xinyual <xinyual@amazon.com> --------- Signed-off-by: xinyual <xinyual@amazon.com>
…ry impl (opensearch-project#3377) Signed-off-by: xinyual <xinyual@amazon.com>
…ect#3378) Signed-off-by: Lantao Jin <ltjin@amazon.com> Signed-off-by: xinyual <xinyual@amazon.com>
…t#3388) Signed-off-by: xinyual <xinyual@amazon.com>
…t#3392) * Implement ppl scalar subquery command with Calcite Signed-off-by: Lantao Jin <ltjin@amazon.com> * more general subquery checker Signed-off-by: Lantao Jin <ltjin@amazon.com> * support correlated IN subquery Signed-off-by: Lantao Jin <ltjin@amazon.com> --------- Signed-off-by: Lantao Jin <ltjin@amazon.com> Signed-off-by: xinyual <xinyual@amazon.com>
* Change push down to logical index scan Signed-off-by: Heng Qian <qianheng@amazon.com> * Support Aggregate Push Down Signed-off-by: Heng Qian <qianheng@amazon.com> * Rebase and resolve conflict Signed-off-by: Heng Qian <qianheng@amazon.com> * Add TODO Signed-off-by: Heng Qian <qianheng@amazon.com> * Address comments Signed-off-by: Heng Qian <qianheng@amazon.com> --------- Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: xinyual <xinyual@amazon.com>
* add string udfs Signed-off-by: xinyual <xinyual@amazon.com> * add it to string Signed-off-by: xinyual <xinyual@amazon.com> * add IT for string function Signed-off-by: xinyual <xinyual@amazon.com> * remove change for local test Signed-off-by: xinyual <xinyual@amazon.com> * revert change Signed-off-by: xinyual <xinyual@amazon.com> --------- Signed-off-by: xinyual <xinyual@amazon.com>
…opensearch-project#3966) * Append limit operator for QUEERY_SIZE_LIMIT * Add LogicalSystemLimit * Revert part of opensearch-project#3880 * Fix IT after merging main --------- (cherry picked from commit 5316c0a) Signed-off-by: Heng Qian <qianheng@amazon.com>
…h-project#3960) (cherry picked from commit 0da93a3) Signed-off-by: Peng Huo <penghuo@gmail.com>
…ect#3948) * Fix create PIT permissions issue (opensearch-project#3921) Signed-off-by: Vamsi Manohar <reddyvam@amazon.com> (cherry picked from commit 739c4df) Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> * Fix json string formatting with earlier java versions Signed-off-by: Vamsi Manohar <reddyvam@amazon.com> --------- Signed-off-by: Vamsi Manohar <reddyvam@amazon.com> Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: Vamsi Manohar <reddyvam@amazon.com>
…pensearch-project#3976) (cherry picked from commit db2a8bf) Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
…ect#3952) (opensearch-project#3977) * Disable a failed PPL query fallback to v2 by default * Workaround the permissionIT --------- (cherry picked from commit 7892b90) Signed-off-by: Lantao Jin <ltjin@amazon.com>
) (opensearch-project#3968) * Change the use of SqlTypeFamily.STRING to SqlTypeFamily.CHARACTER as the string family contains binary, which is not expected for most functions * Implement basic argument type coercion at RelNode level * Conform type checkers with their definition in documentation - string as an input is removed if it is not in the document - string as an input is kept if it is in the document, even if it can be implicitly cast - use PPLOperandTypes as much as possible * Implement type widening for comparator functions - Add COMPARATORS set to BuiltinFunctionName for identifying comparison operators - Implement widenArguments method in CoercionUtils to find widest compatible type - Apply type widening to comparator functions before applying type casting - Add detailed JavaDoc to explain coercion methods * Update error messages of datetime functions with invalid args * Simplify datetime-string compare logic with implict coercion * Refactor resolve with coercion * Move down argument cast for reduce function * Merge comparators and their IP variants so that coercion works for IP comparison - when not merging, ip comparing will also pass the type checker of Calcite's comparators * Refactor ip comparator to comparator * Revert "Refactor ip comparator to comparator" This reverts commit c539056. * Revert "Merge comparators and their IP variants so that coercion works for IP comparison" This reverts commit bd9f3bb. * Rule out ip from built-in comparator via its type checker * Restrict CompareIP's parameter type * Revert to previous implementation of CompareIpFunction to temporarily fix ip comparison pushdown problems (udt not correctly serialized; ip comparison is not converted to range query) * Test argument coercion explain * Fix error msg in CalcitePPLFunctionTypeTest --------- (cherry picked from commit 484f49e) Signed-off-by: Yuanchun Shen <yuanchu@amazon.com>
…-project#3806) (opensearch-project#3979) (cherry picked from commit 7ccdcd1) Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: Sayali Gaikawad <gaiksaya@amazon.com>
…nsearch-project#3991) * Implement reverse * Add reverse integ tests and unit tests * Add reverse documentation * Modify reverse test and documentation * Fix limit pushdown bug when reverse comes before head * Revert "Fix limit pushdown bug when reverse comes before head" This reverts commit 087c936. * Fix grammar, naming, and test cases. Pushdown reverted will be in 2nd PR. * Fix reverse tests: update logical plans, format with Spotless - Updated expected logical plans and Spark SQL in reverse tests - Applied Spotless to fix formatting * Fix OS version in build 3.1.0 * Add note on limitation to rst * Move explain IT to correct file, add Anonymizer test * Add reverse to index.rst --------- (cherry picked from commit c05a58c) Signed-off-by: Selina Song <selsong@amazon.com> Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: Selina Song <selsong@amazon.com>
…4011) * add math udfs * fix decimal bug * make general udf adapter * add math IT * fix * add rst * fix error * change signum IT * add javadoc --------- (cherry picked from commit d6aac3c) Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
…ilter (opensearch-project#4022) * Prevent aggregation push down when it has inner filter (opensearch-project#4002) * Prevent aggregation push down when it has inner filter Signed-off-by: Heng Qian <qianheng@amazon.com> * Fix IT & Remove log Signed-off-by: Heng Qian <qianheng@amazon.com> * Fix 4009 Signed-off-by: Heng Qian <qianheng@amazon.com> * Fix IT Signed-off-by: Heng Qian <qianheng@amazon.com> --------- Signed-off-by: Heng Qian <qianheng@amazon.com> (cherry picked from commit 6e3329f) Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> * Fix IT Signed-off-by: Heng Qian <qianheng@amazon.com> --------- Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: Heng Qian <qianheng@amazon.com>
…earch-project#4024) (opensearch-project#4054) * Remove getTypeChecker from FunctionImp interface * Refactor registerExternalFunction to registerExternalOperator * Do not register GEOIP function if got incompatible client * Create scaffold for type checking of aggregation functions * Add type checkers for aggregation functions * Test type checking for aggregation functions --------- (cherry picked from commit d758163) Signed-off-by: Yuanchun Shen <yuanchu@amazon.com>
…project#4028) * Fix span on negative timestamp * Fix span on negative timestamp * typo * Refine code --------- (cherry picked from commit ad3fc1f) Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
…oject#4059) (cherry picked from commit 8368e60) Signed-off-by: Vamsi Manohar <reddyvam@amazon.com> Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
opensearch-project#3982) (opensearch-project#4062) * Fix casting double 0.0 to string * Fix float to string casting precision lost with custom FormatNumberFunction This commit fixes float to string casting by replacing the use of SqlLibraryOperators.FORMAT_NUMBER with a custom FormatNumberFunction implementation. The new implementation converts the number to a BigDecimal before formatting to preserve precision and avoid issues like 6.2 becoming 6.199999809265137. * Simplify the implementation of fp number to string cast * Update implementation of NumberToStringFunction * Cast decimal with NUMBER_TO_STRING function * Test cast decimal --------- (cherry picked from commit 1977083) Signed-off-by: Yuanchun Shen <yuanchu@amazon.com>
…-project#4020) (opensearch-project#4068) * eval command support * improvment * Refactor * fix CI * fix CI * fix CI * fixes * fix * Add IT * remove redundant tests --------- (cherry picked from commit f40d870) Signed-off-by: Kai Huang <ahkcs@amazon.com> Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
…ject#4074) * enhance sort command * update failing tests * fix formatting * add integ tests * update documentation * fix failing test * update default and tests * fix tests * update analyzer test * update reverse sort direction * update formatting * update docs * add javadoc * add tests * fix failing tests * fix failing tests * update integ tests for query size limit change * add explainit for desc and type cast * add tests for desc * fix formatting * make count optional * add cross cluster tests * fix tests * normalize count in AST node * default null count to 0 * update logicalsort default constructor --------- (cherry picked from commit c8d2694) Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
… (opensearch-project#4064) * Remove named function arg from functions other than table functions * Test eval if function with equal as condition --------- (cherry picked from commit 9f2b19a) Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
…ct#3959) (opensearch-project#4080) * Add reverse op for compare ip to support pushdown * Pushdown ip comparison * Refactor CompareIpFunction to use SqlKind directly * Simplify the overriding of reverse() for IP comparators --------- (cherry picked from commit e2375fe) Signed-off-by: Yuanchun Shen <yuanchu@amazon.com>
Signed-off-by: Lantao Jin <ltjin@amazon.com>
…h-project#4072) * Support script push down on text field (opensearch-project#4010) * Support script push down on text field Signed-off-by: Heng Qian <qianheng@amazon.com> * Fix IT Signed-off-by: Heng Qian <qianheng@amazon.com> * Add UT for struct type push down Signed-off-by: Heng Qian <qianheng@amazon.com> --------- Signed-off-by: Heng Qian <qianheng@amazon.com> (cherry picked from commit bb1a644) Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> * Fix CI failure because of plan having changed (opensearch-project#4077) Signed-off-by: Heng Qian <qianheng@amazon.com> (cherry picked from commit acc055e) * set segment and document as missing PR: opensearch-project/OpenSearch#17927 in core 2.19 Signed-off-by: Heng Qian <qianheng@amazon.com> * Fix IT Signed-off-by: Heng Qian <qianheng@amazon.com> * fix compiling Signed-off-by: Heng Qian <qianheng@amazon.com> --------- Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: qianheng <qianheng@amazon.com>
…roject#3972) (opensearch-project#4081) * Support pushdown dedup with Calcite (opensearch-project#3972) * Support pushdown dedup with Calcite Signed-off-by: Lantao Jin <ltjin@amazon.com> * Fix IT Signed-off-by: Lantao Jin <ltjin@amazon.com> * Address comments Signed-off-by: Lantao Jin <ltjin@amazon.com> * Fix flaky test Signed-off-by: Lantao Jin <ltjin@amazon.com> * Address comment Signed-off-by: Lantao Jin <ltjin@amazon.com> * delete useless codes Signed-off-by: Lantao Jin <ltjin@amazon.com> * Add more ITs Signed-off-by: Lantao Jin <ltjin@amazon.com> --------- Signed-off-by: Lantao Jin <ltjin@amazon.com> (cherry picked from commit f358e5d) * Fix conflicts Signed-off-by: Lantao Jin <ltjin@amazon.com> --------- Signed-off-by: Lantao Jin <ltjin@amazon.com>
…evance function. (opensearch-project#4093) * Make fields parameter optional in multi field relevance queries (opensearch-project#4018) Signed-off-by: Vamsi Manohar <reddyvam@amazon.com> (cherry picked from commit 5c0ed0d) Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> * Fixed text block syntax issue for Java 11 Signed-off-by: Vamsi Manohar <reddyvam@amazon.com> --------- Signed-off-by: Vamsi Manohar <reddyvam@amazon.com> Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: Vamsi Manohar <reddyvam@amazon.com>
…search-project#3989) (opensearch-project#4096) * add UUID regex into template * fix spotless * add integration test --------- (cherry picked from commit 4a3b03a) Signed-off-by: Hailong Cui <ihailong@amazon.com> Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
…roject#4089) * Add flaky retry on CalcitePPLTpchIT (opensearch-project#4060) * Add flaky retry on CalcitePPLTpchIT.testQ7 Signed-off-by: Lantao Jin <ltjin@amazon.com> * Add retry to all tpch queries Signed-off-by: Lantao Jin <ltjin@amazon.com> * address comments Signed-off-by: Lantao Jin <ltjin@amazon.com> --------- Signed-off-by: Lantao Jin <ltjin@amazon.com> (cherry picked from commit be38740) Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> * Fix retry annotation for class Signed-off-by: Lantao Jin <ltjin@amazon.com> * Ignore testQ7 in macOS Signed-off-by: Lantao Jin <ltjin@amazon.com> --------- Signed-off-by: Lantao Jin <ltjin@amazon.com> Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: Lantao Jin <ltjin@amazon.com>
…ection Features (Cal… (opensearch-project#4102) * `fields` Command Enhancement - Advanced Field Selection Features (Calcite) (opensearch-project#3970) * Add support for space-separated fields in addition to comma-separated Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Byte number should treated as Long in doc values (opensearch-project#3928) Signed-off-by: Lantao Jin <ltjin@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Adding table as alias of the fields command Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fix create PIT permissions issue (opensearch-project#3921) Signed-off-by: Vamsi Manohar <reddyvam@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Convert like function call to wildcard query for Calcite filter pushdown (opensearch-project#3915) * Convert like function call to wildcard query for Calcite filter pushdown Signed-off-by: Songkan Tang <songkant@amazon.com> * Fix V2 expression like function bug and match its behavior in Calcite Signed-off-by: Songkan Tang <songkant@amazon.com> * Fix like default escape in Calcite Signed-off-by: Songkan Tang <songkant@amazon.com> * Fix tests Signed-off-by: Songkan Tang <songkant@amazon.com> * Fix spotless check Signed-off-by: Songkan Tang <songkant@amazon.com> * Address comments Signed-off-by: Songkan Tang <songkant@amazon.com> * Fix SQL IT correctness Signed-off-by: Songkan Tang <songkant@amazon.com> * Remove test log Signed-off-by: Songkan Tang <songkant@amazon.com> * Minor improve one CalciteLikeQueryIT Signed-off-by: Songkan Tang <songkant@amazon.com> --------- Signed-off-by: Songkan Tang <songkant@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Update commons-lang exclude rule to exclude it everywhere (opensearch-project#3932) * Update commons-lang exclude rule to exclude it everywhere Signed-off-by: Simeon Widdis <sawiddis@amazon.com> * Undo removal in core Signed-off-by: Simeon Widdis <sawiddis@amazon.com> --------- Signed-off-by: Simeon Widdis <sawiddis@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Adding wildcard support to fields command in Calcite Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Mixed delimiter support - Support both space and comma delimiters in the same command for table and fields Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Adding widlcard support to non-Calcite engine and updating documentation with new features Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fixing formatting issues Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Support function argument coercion with Calcite (opensearch-project#3914) * Change the use of SqlTypeFamily.STRING to SqlTypeFamily.CHARACTER as the string family contains binary, which is not expected for most functions Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Implement basic argument type coercion at RelNode level Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Conform type checkers with their definition in documentation - string as an input is removed if it is not in the document - string as an input is kept if it is in the document, even if it can be implicitly cast - use PPLOperandTypes as much as possible Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Implement type widening for comparator functions - Add COMPARATORS set to BuiltinFunctionName for identifying comparison operators - Implement widenArguments method in CoercionUtils to find widest compatible type - Apply type widening to comparator functions before applying type casting - Add detailed JavaDoc to explain coercion methods Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Update error messages of datetime functions with invalid args Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Simplify datetime-string compare logic with implict coercion Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Refactor resolve with coercion Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Move down argument cast for reduce function Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Merge comparators and their IP variants so that coercion works for IP comparison - when not merging, ip comparing will also pass the type checker of Calcite's comparators Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Refactor ip comparator to comparator Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Revert "Refactor ip comparator to comparator" This reverts commit c539056. Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Revert "Merge comparators and their IP variants so that coercion works for IP comparison" This reverts commit bd9f3bb. Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Rule out ip from built-in comparator via its type checker Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Restrict CompareIP's parameter type Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Revert to previous implementation of CompareIpFunction to temporarily fix ip comparison pushdown problems (udt not correctly serialized; ip comparison is not converted to range query) Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Test argument coercion explain Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Fix error msg in CalcitePPLFunctionTypeTest Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> --------- Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Add missing command in index.rst (opensearch-project#3943) Signed-off-by: Peng Huo <penghuo@gmail.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Append limit operator for QUEERY_SIZE_LIMIT (opensearch-project#3940) * Append limit operator for QUEERY_SIZE_LIMIT Signed-off-by: Heng Qian <qianheng@amazon.com> * Add LogicalSystemLimit Signed-off-by: Heng Qian <qianheng@amazon.com> * Revert part of opensearch-project#3880 Signed-off-by: Heng Qian <qianheng@amazon.com> * Fix IT after merging main Signed-off-by: Heng Qian <qianheng@amazon.com> --------- Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Performing code cleaning and fixing tests Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Performing code maintenance and adding more test cases Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Doing some code cleaning and maintenance Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fixing code and implementation logic Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Add issue template specific for PPL commands and queries (opensearch-project#3962) * Add issue template specific for PPL commands and queries Signed-off-by: Anas Alkouz <aalkouz@amazon.com> * Add section for Dataset/schema information, add reminders for the customers to remove any sensitive datas Signed-off-by: Anas Alkouz <aalkouz@amazon.com> --------- Signed-off-by: Anas Alkouz <aalkouz@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Increase the precision of sum return type (opensearch-project#3974) Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Disable a failed PPL query fallback to v2 by default (opensearch-project#3952) * Disable a failed PPL query fallback to v2 by default Signed-off-by: Lantao Jin <ltjin@amazon.com> * Workaround the permissionIT Signed-off-by: Lantao Jin <ltjin@amazon.com> --------- Signed-off-by: Lantao Jin <ltjin@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Update the maven snapshot publish endpoint and credential (opensearch-project#3806) Co-authored-by: Sayali Gaikawad <gaiksaya@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Add release notes for 3.2.0 (opensearch-project#3985) Signed-off-by: opensearch-ci <opensearch-infra@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fixing documentation Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Taking care of comments left by Tomo Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Adding full wildcard support functionality Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Increment version to 3.2.0-SNAPSHOT (opensearch-project#3819) Signed-off-by: opensearch-ci-bot <opensearch-infra@amazon.com> Co-authored-by: opensearch-ci-bot <opensearch-infra@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Support `reverse` command with Calcite (opensearch-project#3867) * Implement reverse Signed-off-by: Selina Song <selsong@amazon.com> * Add reverse integ tests and unit tests Signed-off-by: Selina Song <selsong@amazon.com> * Add reverse documentation Signed-off-by: Selina Song <selsong@amazon.com> * Modify reverse test and documentation Signed-off-by: Selina Song <selsong@amazon.com> * Fix limit pushdown bug when reverse comes before head * Revert "Fix limit pushdown bug when reverse comes before head" This reverts commit 087c936. Signed-off-by: Selina Song <selsong@amazon.com> * Fix grammar, naming, and test cases. Pushdown reverted will be in 2nd PR. Signed-off-by: Selina Song <selsong@amazon.com> * Fix reverse tests: update logical plans, format with Spotless - Updated expected logical plans and Spark SQL in reverse tests - Applied Spotless to fix formatting Signed-off-by: Selina Song <selsong@amazon.com> * Fix OS version in build 3.1.0 Signed-off-by: Selina Song <selsong@amazon.com> * Add note on limitation to rst Signed-off-by: Selina Song <selsong@amazon.com> * Move explain IT to correct file, add Anonymizer test Signed-off-by: Selina Song <selsong@amazon.com> * Add reverse to index.rst Signed-off-by: Selina Song <selsong@amazon.com> --------- Signed-off-by: Selina Song <selsong@amazon.com> Co-authored-by: Selina Song <selsong@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Pass JOIN_TIME_OUT value to keepalive (opensearch-project#3826) * Fix JOIN_TIME_OUT Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix style Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix CI Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix Signed-off-by: Kai Huang <ahkcs@amazon.com> * fixes Signed-off-by: Kai Huang <ahkcs@amazon.com> * fixes Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix log-rethrow Signed-off-by: Kai Huang <ahkcs@amazon.com> * Restructure Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix CI Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix format Signed-off-by: Kai Huang <ahkcs@amazon.com> * remove unused PIT Signed-off-by: Kai Huang <ahkcs@amazon.com> * remove reflection, add hintConfig Signed-off-by: Kai Huang <ahkcs@amazon.com> * Added Unit test, restructured to use existing methods Signed-off-by: Kai Huang <ahkcs@amazon.com> * fixes Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix Signed-off-by: Kai Huang <ahkcs@amazon.com> * fixes Signed-off-by: Kai Huang <ahkcs@amazon.com> * add IT tests Signed-off-by: Kai Huang <ahkcs@amazon.com> * formatting Signed-off-by: Kai Huang <ahkcs@amazon.com> * Trim IT Signed-off-by: Kai Huang <ahkcs@amazon.com> * format fix Signed-off-by: Kai Huang <ahkcs@amazon.com> * trim IT Signed-off-by: Kai Huang <ahkcs@amazon.com> * formatting Signed-off-by: Kai Huang <ahkcs@amazon.com> * Update legacy/src/main/java/org/opensearch/sql/legacy/query/planner/physical/node/pointInTime/PointInTime.java Co-authored-by: Louis Chu <lingzhichu.clz@gmail.com> Signed-off-by: Kai Huang <105710027+ahkcs@users.noreply.github.com> * fix Signed-off-by: Kai Huang <ahkcs@amazon.com> * update .gitignore Signed-off-by: Kai Huang <ahkcs@amazon.com> * deletion Signed-off-by: Kai Huang <ahkcs@amazon.com> * restore main PIT Signed-off-by: Kai Huang <ahkcs@amazon.com> --------- Signed-off-by: Kai Huang <ahkcs@amazon.com> Signed-off-by: Kai Huang <105710027+ahkcs@users.noreply.github.com> Co-authored-by: Louis Chu <lingzhichu.clz@gmail.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Changed seenFields to Hashset instead of LinkedHashSet Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Creating a rule only for the fields/table commands to avoid any interference Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * This commit is in response of PR comments left by Tomo and Chen Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fixing Integration test failure Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Adding anonymizer tests, wildcard unit tests, etc Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Disabling Calcite for enhance fields features Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Disabling automatic de-deduplication when Calcite is disabled Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Adding cross-cluster IT test Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Adding a dedicated Cross-cluster IT test file for Calcite Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fixing formatting issues Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Improving widlcard logic and exception message Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Addressing comments left by Tomo regarding wildcard logic implementation Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Empty commit Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Add missing udfs in v3 (opensearch-project#3957) * add math udfs Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * fix decimal bug Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * make general udf adapter Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * add math IT Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * fix Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * add rst Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * fix error Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * change signum IT Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * add javadoc Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> --------- Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * fix snapshot uploading (opensearch-project#4006) * fix snapshot uploading Signed-off-by: Kai Huang <ahkcs@amazon.com> * Add comment Signed-off-by: Kai Huang <ahkcs@amazon.com> --------- Signed-off-by: Kai Huang <ahkcs@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fix DOUBLE to STRING cast rendering zero values in scientific notation (opensearch-project#3982) * Fix casting double 0.0 to string Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Fix float to string casting precision lost with custom FormatNumberFunction This commit fixes float to string casting by replacing the use of SqlLibraryOperators.FORMAT_NUMBER with a custom FormatNumberFunction implementation. The new implementation converts the number to a BigDecimal before formatting to preserve precision and avoid issues like 6.2 becoming 6.199999809265137. Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Simplify the implementation of fp number to string cast Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Update implementation of NumberToStringFunction Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Cast decimal with NUMBER_TO_STRING function Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Test cast decimal Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> --------- Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Eliminate reliance on assert in Calcite for integration test (opensearch-project#4016) * Move num-of-column check of in subquery ahead from RexSubQuery.java#L78 because assert is disabled in production Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Increase script.context.filter.max_compilations_rate for SQLCorrectnessIT Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Check script.disable_max_compilations_rate before setting context-specific compilations rate Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Refactor: remove some methods in tests to upper level to reduce duplication Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Replace plugin-level setting strings with private test-specific ones Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> --------- Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Prevent aggregation push down when it has inner filter (opensearch-project#4002) * Prevent aggregation push down when it has inner filter Signed-off-by: Heng Qian <qianheng@amazon.com> * Fix IT & Remove log Signed-off-by: Heng Qian <qianheng@amazon.com> * Fix 4009 Signed-off-by: Heng Qian <qianheng@amazon.com> * Fix IT Signed-off-by: Heng Qian <qianheng@amazon.com> --------- Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fix span on negative timestamp (opensearch-project#4017) * Fix span on negative timestamp Signed-off-by: Heng Qian <qianheng@amazon.com> * Fix span on negative timestamp Signed-off-by: Heng Qian <qianheng@amazon.com> * typo Signed-off-by: Heng Qian <qianheng@amazon.com> * Refine code Signed-off-by: Heng Qian <qianheng@amazon.com> --------- Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Skip script encoding when run explain with 'extended' (opensearch-project#3930) * No need to decode script when run explain command Signed-off-by: Lantao Jin <ltjin@amazon.com> * address comment Signed-off-by: Lantao Jin <ltjin@amazon.com> * Do not encoding when explain format is 'extended' Signed-off-by: Lantao Jin <ltjin@amazon.com> * Rename the thread local var Signed-off-by: Lantao Jin <ltjin@amazon.com> * Fix IT after merge main Signed-off-by: Lantao Jin <ltjin@amazon.com> --------- Signed-off-by: Lantao Jin <ltjin@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Implement type checking for aggregation functions with Calcite (opensearch-project#4024) * Remove getTypeChecker from FunctionImp interface Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Refactor registerExternalFunction to registerExternalOperator Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Do not register GEOIP function if got incompatible client Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Create scaffold for type checking of aggregation functions Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Add type checkers for aggregation functions Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Test type checking for aggregation functions Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> --------- Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Allow equal expression as a function argument (opensearch-project#4001) * Remove named function arg from functions other than table functions Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Test eval if function with equal as condition Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> --------- Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Push down IP comparison as range query with Calcite (opensearch-project#3959) * Add reverse op for compare ip to support pushdown Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Pushdown ip comparison Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Refactor CompareIpFunction to use SqlKind directly Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Simplify the overriding of reverse() for IP comparators Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> --------- Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * eval sum, avg implementation (opensearch-project#3986) Signed-off-by: Vamsi Manohar <reddyvam@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fix PPL eval command string concatenation with + operator (opensearch-project#4020) * eval command support Signed-off-by: Kai Huang <ahkcs@amazon.com> * improvment Signed-off-by: Kai Huang <ahkcs@amazon.com> * Refactor Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix CI Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix CI Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix CI Signed-off-by: Kai Huang <ahkcs@amazon.com> * fixes Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix Signed-off-by: Kai Huang <ahkcs@amazon.com> * Add IT Signed-off-by: Kai Huang <ahkcs@amazon.com> * remove redundant tests Signed-off-by: Kai Huang <ahkcs@amazon.com> --------- Signed-off-by: Kai Huang <ahkcs@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Support script push down on text field (opensearch-project#4010) * Support script push down on text field Signed-off-by: Heng Qian <qianheng@amazon.com> * Fix IT Signed-off-by: Heng Qian <qianheng@amazon.com> * Add UT for struct type push down Signed-off-by: Heng Qian <qianheng@amazon.com> --------- Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Enhance sort command in PPL (opensearch-project#3934) * enhance sort command Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update failing tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * fix formatting Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * add integ tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update documentation Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * fix failing test Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update default and tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * fix tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update analyzer test Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update reverse sort direction Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update formatting Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update docs Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * add javadoc Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * add tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * fix failing tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * fix failing tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update integ tests for query size limit change Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * add explainit for desc and type cast Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * add tests for desc Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * fix formatting Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * make count optional Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * add cross cluster tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * fix tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * normalize count in AST node Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * default null count to 0 Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update logicalsort default constructor Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> --------- Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Add example for String concat in eval.rst (opensearch-project#4075) * Add example for String concat in eval.rst Signed-off-by: Kai Huang <ahkcs@amazon.com> * mention calcite enabling Signed-off-by: Kai Huang <ahkcs@amazon.com> --------- Signed-off-by: Kai Huang <ahkcs@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Support pushdown dedup with Calcite (opensearch-project#3972) * Support pushdown dedup with Calcite Signed-off-by: Lantao Jin <ltjin@amazon.com> * Fix IT Signed-off-by: Lantao Jin <ltjin@amazon.com> * Address comments Signed-off-by: Lantao Jin <ltjin@amazon.com> * Fix flaky test Signed-off-by: Lantao Jin <ltjin@amazon.com> * Address comment Signed-off-by: Lantao Jin <ltjin@amazon.com> * delete useless codes Signed-off-by: Lantao Jin <ltjin@amazon.com> * Add more ITs Signed-off-by: Lantao Jin <ltjin@amazon.com> --------- Signed-off-by: Lantao Jin <ltjin@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fix CI failure because of plan having changed (opensearch-project#4077) Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Empty commit Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Empty commit Signed-off-by: Aaron Alvarez <aaarone@amazon.com> --------- Signed-off-by: Aaron Alvarez <aaarone@amazon.com> Signed-off-by: Lantao Jin <ltjin@amazon.com> Signed-off-by: Vamsi Manohar <reddyvam@amazon.com> Signed-off-by: Songkan Tang <songkant@amazon.com> Signed-off-by: Simeon Widdis <sawiddis@amazon.com> Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Peng Huo <penghuo@gmail.com> Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Anas Alkouz <aalkouz@amazon.com> Signed-off-by: opensearch-ci <opensearch-infra@amazon.com> Signed-off-by: opensearch-ci-bot <opensearch-infra@amazon.com> Signed-off-by: Selina Song <selsong@amazon.com> Signed-off-by: Kai Huang <ahkcs@amazon.com> Signed-off-by: Kai Huang <105710027+ahkcs@users.noreply.github.com> Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> Signed-off-by: Aaron Alvarez <900908alvarezaaron@gmail.com> Co-authored-by: Aaron Alvarez <aaarone@amazon.com> Co-authored-by: Lantao Jin <ltjin@amazon.com> Co-authored-by: Vamsi Manohar <reddyvam@amazon.com> Co-authored-by: Songkan Tang <songkant@amazon.com> Co-authored-by: Simeon Widdis <sawiddis@gmail.com> Co-authored-by: Yuanchun Shen <yuanchu@amazon.com> Co-authored-by: Peng Huo <penghuo@gmail.com> Co-authored-by: qianheng <qianheng@amazon.com> Co-authored-by: Anas Alkouz <aalkouz@amazon.com> Co-authored-by: Zelin Hao <zelinhao@amazon.com> Co-authored-by: Sayali Gaikawad <gaiksaya@amazon.com> Co-authored-by: opensearch-ci <83309141+opensearch-ci-bot@users.noreply.github.com> Co-authored-by: opensearch-trigger-bot[bot] <98922864+opensearch-trigger-bot[bot]@users.noreply.github.com> Co-authored-by: opensearch-ci-bot <opensearch-infra@amazon.com> Co-authored-by: Selina Song <selinasong6@gmail.com> Co-authored-by: Selina Song <selsong@amazon.com> Co-authored-by: Kai Huang <105710027+ahkcs@users.noreply.github.com> Co-authored-by: Louis Chu <lingzhichu.clz@gmail.com> Co-authored-by: Xinyu Hao <75524174+ishaoxy@users.noreply.github.com> Co-authored-by: ritvibhatt <53196324+ritvibhatt@users.noreply.github.com> (cherry picked from commit f16f1c0) * Fixing compatibility issues Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fixing JDK compatibility issues Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * fixing java 11 compatibility issues Signed-off-by: Aaron Alvarez <aaarone@amazon.com> --------- Signed-off-by: Aaron Alvarez <aaarone@amazon.com> Signed-off-by: Lantao Jin <ltjin@amazon.com> Signed-off-by: Vamsi Manohar <reddyvam@amazon.com> Signed-off-by: Songkan Tang <songkant@amazon.com> Signed-off-by: Simeon Widdis <sawiddis@amazon.com> Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Peng Huo <penghuo@gmail.com> Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Anas Alkouz <aalkouz@amazon.com> Signed-off-by: opensearch-ci <opensearch-infra@amazon.com> Signed-off-by: opensearch-ci-bot <opensearch-infra@amazon.com> Signed-off-by: Selina Song <selsong@amazon.com> Signed-off-by: Kai Huang <ahkcs@amazon.com> Signed-off-by: Kai Huang <105710027+ahkcs@users.noreply.github.com> Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> Signed-off-by: Aaron Alvarez <900908alvarezaaron@gmail.com> Co-authored-by: Aaron Alvarez <aaarone@amazon.com> Co-authored-by: Lantao Jin <ltjin@amazon.com> Co-authored-by: Vamsi Manohar <reddyvam@amazon.com> Co-authored-by: Songkan Tang <songkant@amazon.com> Co-authored-by: Simeon Widdis <sawiddis@gmail.com> Co-authored-by: Yuanchun Shen <yuanchu@amazon.com> Co-authored-by: Peng Huo <penghuo@gmail.com> Co-authored-by: qianheng <qianheng@amazon.com> Co-authored-by: Anas Alkouz <aalkouz@amazon.com> Co-authored-by: Zelin Hao <zelinhao@amazon.com> Co-authored-by: Sayali Gaikawad <gaiksaya@amazon.com> Co-authored-by: opensearch-ci <83309141+opensearch-ci-bot@users.noreply.github.com> Co-authored-by: opensearch-trigger-bot[bot] <98922864+opensearch-trigger-bot[bot]@users.noreply.github.com> Co-authored-by: opensearch-ci-bot <opensearch-infra@amazon.com> Co-authored-by: Selina Song <selinasong6@gmail.com> Co-authored-by: Selina Song <selsong@amazon.com> Co-authored-by: Kai Huang <105710027+ahkcs@users.noreply.github.com> Co-authored-by: Louis Chu <lingzhichu.clz@gmail.com> Co-authored-by: Xinyu Hao <75524174+ishaoxy@users.noreply.github.com> Co-authored-by: ritvibhatt <53196324+ritvibhatt@users.noreply.github.com>
…ch-project#4055) (opensearch-project#4098) * Filter with isnull or other expression should be pushded down * revert unexpected deletion * Fix conflicts --------- (cherry picked from commit fce8383) Signed-off-by: Lantao Jin <ltjin@amazon.com>
Signed-off-by: Kai Huang <ahkcs@amazon.com> Signed-off-by: Kai Huang <105710027+ahkcs@users.noreply.github.com> (cherry picked from commit 71076f7)
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
20 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Cherry-picked from #3878