Merge main into 1.x #473

vamsi-amazon · 2022-03-08T23:51:02Z

Description

[Describe what this change achieves]

Issues Resolved

[List any issues this PR will resolve]

Check List

New functionality includes testing.
- All tests pass, including unit test, integration test and doctest
New functionality has been documented.
- New functionality has javadoc added
- New functionality has user manual doc added
Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Signed-off-by: Peter Zhu <zhujiaxi@amazon.com>

Signed-off-by: Chen Dai <daichen@amazon.com>

…pensearch-project#146) * Bump OpenSearch version from rc1 to 1.0.0 Signed-off-by: Chen Dai <daichen@amazon.com> * Rename JDBC artifact by removing -rc1 Signed-off-by: Chen Dai <daichen@amazon.com> * Remove rc1 qualifier in build workflow Signed-off-by: Chen Dai <daichen@amazon.com> * Remove rc1 from build tools version Signed-off-by: Chen Dai <daichen@amazon.com> * Fix IT failure Signed-off-by: Chen Dai <daichen@amazon.com> * Rollback build tools to rc1 due to known issue Signed-off-by: Chen Dai <daichen@amazon.com> * Bump CLI version Signed-off-by: Chen Dai <daichen@amazon.com> * Bump query workbench version Signed-off-by: Chen Dai <daichen@amazon.com> * Build against 1.0.0 Signed-off-by: Chen Dai <daichen@amazon.com> * Update release notes drafter Signed-off-by: Chen Dai <daichen@amazon.com> * Update nodejs to 10.24.1 Signed-off-by: Chen Dai <daichen@amazon.com>

Signed-off-by: Chen Dai <daichen@amazon.com>

* Add release notes Signed-off-by: Chen Dai <daichen@amazon.com> * Change release date Signed-off-by: Chen Dai <daichen@amazon.com> * Add bug fixes section Signed-off-by: Chen Dai <daichen@amazon.com>

…rch-integtest-demo

Signed-off-by: Peter Zhu <zhujiaxi@amazon.com>

…nsearch-integtest-demo Add Integtest.sh for OpenSearch integtest setups (workbench)

…roject#166) * Support implicit type conversion for bool and string Signed-off-by: Chen Dai <daichen@amazon.com> * Fix lucene query pushdown issue Signed-off-by: Chen Dai <daichen@amazon.com> * Refactor lucene query methods Signed-off-by: Chen Dai <daichen@amazon.com> * Refactor builtin repo methods Signed-off-by: Chen Dai <daichen@amazon.com> * Add comparison test Signed-off-by: Chen Dai <daichen@amazon.com> * Fix comparison test Signed-off-by: Chen Dai <daichen@amazon.com> * Add doc test for user manual Signed-off-by: Chen Dai <daichen@amazon.com> * Fix doc test Signed-off-by: Chen Dai <daichen@amazon.com> * Fix design doc link Signed-off-by: Chen Dai <daichen@amazon.com> * Fix RST render issue Signed-off-by: Chen Dai <daichen@amazon.com> * Fix cast function pushdown issue Signed-off-by: Chen Dai <daichen@amazon.com> * Improve javadoc for PR Signed-off-by: Chen Dai <daichen@amazon.com> * Upload design doc Signed-off-by: Chen Dai <daichen@amazon.com> * Add more user manual Signed-off-by: Chen Dai <daichen@amazon.com> * Add more user manual Signed-off-by: Chen Dai <daichen@amazon.com> * Fix doctest Signed-off-by: Chen Dai <daichen@amazon.com>

* Support construct AggregationResponseParser during Aggregator build stage (opensearch-project#108) * Support construct AggregationResponseParser during Aggregator build stage * modify the doc Signed-off-by: penghuo <penghuo@gmail.com> * support distinct count aggregation Signed-off-by: chloe-zh <chloezh1102@gmail.com> * fixed tests Signed-off-by: chloe-zh <chloezh1102@gmail.com> * Merge remote-tracking branch 'upstream/develop' into issue/opensearch-project#100 Signed-off-by: chloe-zh <chloezh1102@gmail.com> # Conflicts: # opensearch/src/main/java/org/opensearch/sql/opensearch/storage/script/aggregation/dsl/MetricAggregationBuilder.java * update Signed-off-by: chloe-zh <chloezh1102@gmail.com> * updated user doc Signed-off-by: chloe-zh <chloezh1102@gmail.com> * Update: support only count for distinct aggregations Signed-off-by: chloe-zh <chloezh1102@gmail.com> * Update doc; removed distinct start Signed-off-by: chloe-zh <chloezh1102@gmail.com> * Removed unnecessary methods Signed-off-by: chloe-zh <chloezh1102@gmail.com> * update Signed-off-by: chloe-zh <chloezh1102@gmail.com> * Impl stddev and variance function in SQL and PPL (opensearch-project#115) * impl variance frontend and backend * Support construct AggregationResponseParser during Aggregator build stage * add var and varp for PPL Signed-off-by: penghuo <penghuo@gmail.com> * add UT Signed-off-by: penghuo <penghuo@gmail.com> * fix UT Signed-off-by: penghuo <penghuo@gmail.com> * fix doc format Signed-off-by: penghuo <penghuo@gmail.com> * fix doc format Signed-off-by: penghuo <penghuo@gmail.com> * fix the doc Signed-off-by: penghuo <penghuo@gmail.com> * add stddev_samp and stddev_pop Signed-off-by: penghuo <penghuo@gmail.com> * fix UT coverage * address comments Signed-off-by: penghuo <penghuo@gmail.com> * Fix the aggregation filter missing in named aggregators (opensearch-project#123) * Take the condition expression as property to the named aggregator when wrapping the delegated aggregator Signed-off-by: chloe-zh <chloezh1102@gmail.com> * update Signed-off-by: chloe-zh <chloezh1102@gmail.com> * Added test case where filtered agg is not pushed down Signed-off-by: chloe-zh <chloezh1102@gmail.com> * update Signed-off-by: chloe-zh <chloezh1102@gmail.com> * update Signed-off-by: chloe-zh <chloezh1102@gmail.com> * update Signed-off-by: chloe-zh <chloezh1102@gmail.com> * modified comparison test Signed-off-by: chloe-zh <chloezh1102@gmail.com> * removed a comparison test and added it to aggregationIT Signed-off-by: chloe-zh <chloezh1102@gmail.com> * added ppl IT test cases; added window function test cases Signed-off-by: chloe-zh <chloezh1102@gmail.com> * moved distinct window function test cases to WindowsIT Signed-off-by: chloe-zh <chloezh1102@gmail.com> * added ut Signed-off-by: chloe-zh <chloezh1102@gmail.com> * update Signed-off-by: chloe-zh <chloezh1102@gmail.com> * update Signed-off-by: chloe-zh <chloezh1102@gmail.com> * addressed comments Signed-off-by: chloe-zh <chloezh1102@gmail.com> * added test cases to meet the coverage requirement Signed-off-by: chloe-zh <chloezh1102@gmail.com> * added test cases for distinct count map and array types Signed-off-by: chloe-zh <chloezh1102@gmail.com> Co-authored-by: Peng Huo <penghuo@gmail.com>

…project#171) * Support implicit cast from string to temporal types Signed-off-by: Chen Dai <daichen@amazon.com> * Add comparison test Signed-off-by: Chen Dai <daichen@amazon.com> * Add doctest Signed-off-by: Chen Dai <daichen@amazon.com> * Fix doctest Signed-off-by: Chen Dai <daichen@amazon.com> * Add more user manual Signed-off-by: Chen Dai <daichen@amazon.com> * Add more user manual Signed-off-by: Chen Dai <daichen@amazon.com> * Fix doctest Signed-off-by: Chen Dai <daichen@amazon.com> * Update user manual Signed-off-by: Chen Dai <daichen@amazon.com>

Signed-off-by: dblock <dblock@amazon.com>

Use externally-defined OpenSearch version when specified.

…#181) Signed-off-by: dblock <dblock@amazon.com>

…#178) Bumps [path-parse](https://github.com/jbgutierrez/path-parse) from 1.0.6 to 1.0.7. - [Release notes](https://github.com/jbgutierrez/path-parse/releases) - [Commits](https://github.com/jbgutierrez/path-parse/commits/v1.0.7) --- updated-dependencies: - dependency-name: path-parse dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

…project#199)

Signed-off-by: Joshua Li <joshuali925@gmail.com>

…t#202) * bump workbench versions to 1.1 Signed-off-by: David Cui <davidcui@amazon.com> * bump version to 1.1.0.0 for 1.1 release Signed-off-by: David Cui <davidcui@amazon.com> * add release notes for 1.1 Signed-off-by: David Cui <davidcui@amazon.com> * bump odbc files to 1.1.0.0 Signed-off-by: David Cui <davidcui@amazon.com>

* Downscope request to local method Signed-off-by: Chen Dai <daichen@amazon.com> * Fix checkstyle Signed-off-by: Chen Dai <daichen@amazon.com>

Signed-off-by: dblock <dblock@dblock.org>

* supported match in sql and ppl where Signed-off-by: chloe-zh <chloezh1102@gmail.com> * added legacy syntax in new parser Signed-off-by: chloe-zh <chloezh1102@gmail.com> * added integration test Signed-off-by: chloe-zh <chloezh1102@gmail.com> * updated user manual Signed-off-by: chloe-zh <chloezh1102@gmail.com> * updated user manual Signed-off-by: chloe-zh <chloezh1102@gmail.com> * update Signed-off-by: chloe-zh <chloezh1102@gmail.com> * added ppl integ test, updated ppl manual Signed-off-by: chloe-zh <chloezh1102@gmail.com> * update Signed-off-by: chloe-zh <chloezh1102@gmail.com> * update Signed-off-by: chloe-zh <chloezh1102@gmail.com> * update Signed-off-by: chloe-zh <chloezh1102@gmail.com>

…est (opensearch-project#230) Signed-off-by: chloe-zh <chloezh1102@gmail.com>

…arch-project#228)

* Update build to use public Maven repo Signed-off-by: Abbas Hussain <abbas_10690@yahoo.com> * Update runner to use macos-11 for odbc builds Signed-off-by: Abbas Hussain <abbas_10690@yahoo.com>

Signed-off-by: chloe-zh <chloezh1102@gmail.com>

* compatible with opensearch and all versions in odfe with sql plugin Signed-off-by: chloe-zh <chloezh1102@gmail.com> * address comments Signed-off-by: chloe-zh <chloezh1102@gmail.com>

…hsql` (opensearch-project#392) * rename sql-cli to opensearchsql Signed-off-by: Zhongnan Su <szhongna@amazon.com> * update workflow Signed-off-by: Zhongnan Su <szhongna@amazon.com>

…project#406) Signed-off-by: penghuo <penghuo@gmail.com>

…#394) Signed-off-by: Joshua Li <joshuali925@gmail.com>

* bumping jackson-databind to 2.12.6 Signed-off-by: Shenoy Pratik <sgguruda@amazon.com> * bumping jackson-core & dataformat-cbor Signed-off-by: Shenoy Pratik <sgguruda@amazon.com> * updated missed gradle files Signed-off-by: Shenoy Pratik <sgguruda@amazon.com> * updated junit & added resolutions Signed-off-by: Shenoy Pratik <sgguruda@amazon.com>

…ject#412) Signed-off-by: penghuo <penghuo@gmail.com>

Signed-off-by: penghuo <penghuo@gmail.com>

Signed-off-by: Joshua Li <joshuali925@gmail.com>

…rch-project#417) * Support combination of group field and span in stats command Signed-off-by: penghuo <penghuo@gmail.com>

Signed-off-by: penghuo <penghuo@gmail.com>

Signed-off-by: Joshua Li <joshuali925@gmail.com>

Signed-off-by: Eugene Lee <eugenesk@amazon.com>

…h-project#437) Signed-off-by: penghuo <penghuo@gmail.com>

Add auto backport workflow and documentation. Signed-off-by: David Cui <davidcui@amazon.com>

* update backport workflow and add auto delete Signed-off-by: David Cui <davidcui@amazon.com>

Signed-off-by: penghuo <penghuo@gmail.com>

Add default version as JDK 11 and add CI matrix for JDK 11 and 14 Signed-off-by: David Cui <davidcui@amazon.com>

Signed-off-by: Guian Gumpac <guiang@bitquilltech.com> Signed-off-by: Yury Fridlyand <yuryf@bitquilltech.com> Co-authored-by: Yury Fridlyand <yuryf@bitquilltech.com> Co-authored-by: Yury-Fridlyand <88679692+Yury-Fridlyand@users.noreply.github.com>

…oject#459) Fixed opensearch-project#423.

Signed-off-by: Joshua Li <joshuali925@gmail.com>

Signed-off-by: penghuo <penghuo@gmail.com>

Optimize sort rewrite logic by using first and last parameter in Composite aggregation. Signed-off-by: penghuo <penghuo@gmail.com>

Support ISO 8601 (%y-%m-%dT%TZ) in Date Formats. In order to do that we have started supporting plain characters in date format without a % in front.

Signed-off-by: vamsi-amazon <reddyvam@amazon.com>

codecov-commenter · 2022-03-09T00:45:22Z

Codecov Report

Merging #473 (9242b2b) into 1.x (3762023) will decrease coverage by 3.33%.
The diff coverage is 100.00%.

@@             Coverage Diff              @@
##                1.x     #473      +/-   ##
============================================
- Coverage     99.89%   96.56%   -3.34%     
- Complexity     2512     2722     +210     
============================================
  Files           245      269      +24     
  Lines          5853     7242    +1389     
  Branches        342      527     +185     
============================================
+ Hits           5847     6993    +1146     
- Misses            5      196     +191     
- Partials          1       53      +52

Flag	Coverage Δ
query-workbench	`62.91% <ø> (?)`
sql-engine	`99.92% <100.00%> (?)`

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
...rch/sql/analysis/ExpressionReferenceOptimizer.java	`100.00% <ø> (ø)`
...ensearch/sql/analysis/NamedExpressionAnalyzer.java	`100.00% <ø> (ø)`
...org/opensearch/sql/analysis/QualifierAnalyzer.java	`100.00% <ø> (ø)`
...nsearch/sql/analysis/SelectExpressionAnalyzer.java	`100.00% <ø> (ø)`
...a/org/opensearch/sql/analysis/TypeEnvironment.java	`100.00% <ø> (ø)`
...nsearch/sql/analysis/WindowExpressionAnalyzer.java	`100.00% <ø> (ø)`
.../org/opensearch/sql/analysis/symbol/Namespace.java	`100.00% <ø> (ø)`
...rg/opensearch/sql/analysis/symbol/SymbolTable.java	`100.00% <ø> (ø)`
...search/sql/data/model/AbstractExprNumberValue.java	`100.00% <ø> (ø)`
...g/opensearch/sql/data/model/AbstractExprValue.java	`100.00% <ø> (ø)`
... and 252 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 3762023...9242b2b. Read the comment docs.

peterzhuamazon and others added 30 commits June 16, 2021 14:51

Add sql integTest script for opensearch

8dd0904

Signed-off-by: Peter Zhu <zhujiaxi@amazon.com>

Refactor readme (opensearch-project#148)

5a6a3e8

Signed-off-by: Chen Dai <daichen@amazon.com>

Change grammar and add UT (opensearch-project#150)

399881d

Signed-off-by: Chen Dai <daichen@amazon.com>

Add release notes for OpenSearch GA (opensearch-project#151)

c37c423

* Add release notes Signed-off-by: Chen Dai <daichen@amazon.com> * Change release date Signed-off-by: Chen Dai <daichen@amazon.com> * Add bug fixes section Signed-off-by: Chen Dai <daichen@amazon.com>

Merge branch 'main' of github.com:opensearch-project/sql into opensea…

3be6351

…rch-integtest-demo

Add sql dashboards tests for workbench

4ca0eab

Signed-off-by: Peter Zhu <zhujiaxi@amazon.com>

Merge pull request opensearch-project#157 from opensearch-project/ope…

27e8b3c

…nsearch-integtest-demo Add Integtest.sh for OpenSearch integtest setups (workbench)

[1] Fixed aws init and shutdown behaviour (opensearch-project#163)

e2714e7

Use externally-defined OpenSearch version when specified.

fa58ce8

Signed-off-by: dblock <dblock@amazon.com>

Merge pull request opensearch-project#179 from dblock/fix-snapshot

90eb1a9

Use externally-defined OpenSearch version when specified.

Use OpenSearch 1.1 and build snapshot by default. (opensearch-project…

abd97e7

…#181) Signed-off-by: dblock <dblock@amazon.com>

Workbench: auto dump cypress test data, support security (opensearch-…

b6fdd0d

…project#199)

Workbench: remove curl commands in integtest.sh (opensearch-project#200)

d68547d

Signed-off-by: Joshua Li <joshuali925@gmail.com>

Fix import path for cypress constant (opensearch-project#201)

a274c4b

Bump opensearch ref to 1.1 in CI (opensearch-project#205)

74d9fa0

Fix PPL request concurrency handling issue (opensearch-project#207)

7cda06b

* Downscope request to local method Signed-off-by: Chen Dai <daichen@amazon.com> * Fix checkstyle Signed-off-by: Chen Dai <daichen@amazon.com>

Removed integtest.sh. (opensearch-project#208)

012cc03

Signed-off-by: dblock <dblock@dblock.org>

Changed the ODBC mac installer build platform to MacOS 10.15 from lat…

efd2f4a

…est (opensearch-project#230) Signed-off-by: chloe-zh <chloezh1102@gmail.com>

Renamed plugin helper config file to consistent name with OSD (opense…

1d1c0d8

…arch-project#228)

Update build to use public Maven repo (opensearch-project#225)

c0dabae

* Update build to use public Maven repo Signed-off-by: Abbas Hussain <abbas_10690@yahoo.com> * Update runner to use macos-11 for odbc builds Signed-off-by: Abbas Hussain <abbas_10690@yahoo.com>

Address security vulnerability CVE-2021-3795 (opensearch-project#231)

374f9eb

Signed-off-by: chloe-zh <chloezh1102@gmail.com>

Addressed security vulnerability CVE-2021-3807 (opensearch-project#233)

0bfae4e

Support ODBC compatibility with ODFE SQL (opensearch-project#238)

12eb45a

* compatible with opensearch and all versions in odfe with sql plugin Signed-off-by: chloe-zh <chloezh1102@gmail.com> * address comments Signed-off-by: chloe-zh <chloezh1102@gmail.com>

zhongnansu and others added 24 commits January 25, 2022 10:48

rename package name of sql-cli from opensearch-sqlcli to `opensearc…

91e86b8

…hsql` (opensearch-project#392) * rename sql-cli to opensearchsql Signed-off-by: Zhongnan Su <szhongna@amazon.com> * update workflow Signed-off-by: Zhongnan Su <szhongna@amazon.com>

Parse none type field as null instead of throw exception (opensearch-…

b0ef5e0

…project#406) Signed-off-by: penghuo <penghuo@gmail.com>

Add how to setup aws credentials for ODBC Tableau (opensearch-project…

1a55970

…#394) Signed-off-by: Joshua Li <joshuali925@gmail.com>

Bug Fix, disable html escape when formatting response (opensearch-pro…

f0ff78e

…ject#412) Signed-off-by: penghuo <penghuo@gmail.com>

Support multiple indices in PPL and SQL (opensearch-project#408)

fe20fb1

Signed-off-by: penghuo <penghuo@gmail.com>

Revert to windows 2019 for odbc CI (opensearch-project#413)

2714c2f

Signed-off-by: Joshua Li <joshuali925@gmail.com>

Support combination of group field and span in stats command (opensea…

1a2bfe2

…rch-project#417) * Support combination of group field and span in stats command Signed-off-by: penghuo <penghuo@gmail.com>

Support In clause in SQL and PPL (opensearch-project#420)

0fd7f2f

Signed-off-by: penghuo <penghuo@gmail.com>

Add cast function to PPL (opensearch-project#433)

f186d04

Signed-off-by: Joshua Li <joshuali925@gmail.com>

Version bump to 1.3 (opensearch-project#419)

f7bbcc0

Signed-off-by: Eugene Lee <eugenesk@amazon.com>

Span expression should always be first in by list if exist (opensearc…

8ad144e

…h-project#437) Signed-off-by: penghuo <penghuo@gmail.com>

Add auto backport functionality for SQL (opensearch-project#445)

b9a642c

Add auto backport workflow and documentation. Signed-off-by: David Cui <davidcui@amazon.com>

Update backport and add auto-delete workflows (opensearch-project#446)

d726c57

* update backport workflow and add auto delete Signed-off-by: David Cui <davidcui@amazon.com>

Version Bump: springframework and jackson (opensearch-project#443)

aceedb8

Signed-off-by: penghuo <penghuo@gmail.com>

Version Bump: H2 1.x -> 2.x (opensearch-project#444)

5e1f657

Signed-off-by: penghuo <penghuo@gmail.com>

Add CI Matrix for JDK 11 and 14 (opensearch-project#451)

ab6a43f

Add default version as JDK 11 and add CI matrix for JDK 11 and 14 Signed-off-by: David Cui <davidcui@amazon.com>

Treating ExpressionEvaluationException as client Error (opensearch-pr…

0d9da48

…oject#459) Fixed opensearch-project#423.

Add parse command to PPL (opensearch-project#411)

bb7d3bc

Signed-off-by: Joshua Li <joshuali925@gmail.com>

Add limitation section in PPL docs (opensearch-project#456)

1823cea

Signed-off-by: penghuo <penghuo@gmail.com>

[Enhancement] optimize sort rewrite logic (opensearch-project#434)

d6ccc8e

Optimize sort rewrite logic by using first and last parameter in Composite aggregation. Signed-off-by: penghuo <penghuo@gmail.com>

Support ISO 8601 Format in Date Format. (opensearch-project#460)

10a9846

Support ISO 8601 (%y-%m-%dT%TZ) in Date Formats. In order to do that we have started supporting plain characters in date format without a % in front.

Merge branch 'main' into 1.x

9242b2b

Signed-off-by: vamsi-amazon <reddyvam@amazon.com>

vamsi-amazon requested review from penghuo and joshuali925 March 8, 2022 23:54

joshuali925 approved these changes Mar 9, 2022

View reviewed changes

penghuo approved these changes Mar 9, 2022

View reviewed changes

vamsi-amazon merged commit 5c09444 into opensearch-project:1.x Mar 9, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Merge main into 1.x #473

Merge main into 1.x #473

vamsi-amazon commented Mar 8, 2022

codecov-commenter commented Mar 9, 2022 •

edited

Loading

Merge main into 1.x #473

Merge main into 1.x #473

Conversation

vamsi-amazon commented Mar 8, 2022

Description

Issues Resolved

Check List

codecov-commenter commented Mar 9, 2022 • edited Loading

Codecov Report

codecov-commenter commented Mar 9, 2022 •

edited

Loading