Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Merge main to 2.x branch #771

Merged
merged 21 commits into from
Aug 17, 2022
Merged

Merge main to 2.x branch #771

merged 21 commits into from
Aug 17, 2022

Conversation

joshuali925
Copy link
Member

Description

[Describe what this change achieves]

Issues Resolved

[List any issues this PR will resolve]

Check List

  • New functionality includes testing.
    • All tests pass, including unit test, integration test and doctest
  • New functionality has been documented.
    • New functionality has javadoc added
    • New functionality has user manual doc added
  • Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

MitchellGale and others added 21 commits August 2, 2022 14:23
@MitchellGale
Two single or double quote escapes single or double quote when string…
68663d1 
… is surrounded by same type of quote (opensearch-project#696)

Signed-off-by: mitchellg <mitchellg@bitquilltech.com>

Co-authored-by: Andrew Carbonetto <andrewc@AndrewsMBP2022.local>
@Yury-Fridlyand
Remove BWC tests (opensearch-project#721)
d26cde3 
Signed-off-by: Yury Fridlyand <yuryf@bitquilltech.com>
@MitchellGale
Github Actions fix for reference to OpenSearch-Dashboard not existing…
6a99b56 
… outside of main OS sql project (opensearch-project#704)

Signed-off-by: mitchellg <mitchellg@bitquilltech.com>
@forestmvey
Add Highlight In SQL (opensearch-project#717)
8f1b31d 
* Add support for highlight to parser and AstExpressionBuilder

Signed-off-by: MaxKsyunz <maxk@bitquilltech.com>

* Add unit test for highlight in AstExpressionBuilder

Signed-off-by: MaxKsyunz <maxk@bitquilltech.com>

* Add unit test for highlight in AstBuilderTest

Signed-off-by: MaxKsyunz <maxk@bitquilltech.com>

* Support highlight as an Unresolved expression.

Signed-off-by: MaxKsyunz <maxk@bitquilltech.com>

* Represent highlight as UnresolvedExpression.

Signed-off-by: MaxKsyunz <maxk@bitquilltech.com>

* Support highlight in Analyzer.

Signed-off-by: MaxKsyunz <maxk@bitquilltech.com>

* Treat highlight as a proper function in AST

In particular, highlightField is an expression now.

Signed-off-by: MaxKsyunz <maxk@bitquilltech.com>

* Add support for highlight in Analyzer

HighlightFunction is converted to LogicalHighlight logical plan.

Signed-off-by: MaxKsyunz <maxk@bitquilltech.com>

* Add a simple IT test for highlight.

Signed-off-by: MaxKsyunz <maxk@bitquilltech.com>

* Register highlight function in the BuiltInFunctionRepository

Signed-off-by: MaxKsyunz <maxk@bitquilltech.com>

* Partial support for highlight in physical plan.

Signed-off-by: MaxKsyunz <maxk@bitquilltech.com>

* Add HighlightOperator.

Signed-off-by: MaxKsyunz <maxk@bitquilltech.com>

* Highlight alpha complete.

Signed-off-by: MaxKsyunz <maxk@bitquilltech.com>

* Initial implementation to upporting highlight('*')

Signed-off-by: forestmvey <forestv@bitquilltech.com>

* Add support for multiple highlight calls in select statement.

Signed-off-by: forestmvey <forestv@bitquilltech.com>

* Removed OpenSearchLogicalIndexScan highlightFields and dependencies. Improved test coverage and fixing checkstyle errors.

Signed-off-by: forestmvey <forestv@bitquilltech.com>

* Added HighlightExpressionTest

Signed-off-by: forestmvey <forestv@bitquilltech.com>

* Added javadocs, minor PR revisions, and fixed jacoco errors by improving coverage for OpenSearchIndexScan

Signed-off-by: forestmvey <forestv@bitquilltech.com>

* Code cleanup, adding parsing failure tests, and adding tests for highlight acceptance as a string literal as well as qualified name.

Signed-off-by: forestmvey <forestv@bitquilltech.com>

* Removing HighlightOperator functionality and unnecessary visitHighlight call in PhysicalPlanNodeVisitor..

Signed-off-by: forestmvey <forestv@bitquilltech.com>

* Adding highlight function to functions.rst and removing unecessary function call in OpenSearchIndexScan.

Signed-off-by: forestmvey <forestv@bitquilltech.com>

* Change highlight fields returned format to array list. Changed highlight all and wildcard to unsupported to open up output formatting changes for multiple returned highlight fields. Change tests for updated coverage and disable highlight all and wildcard tests.

Signed-off-by: forestmvey <forestv@bitquilltech.com>

* Fix bug where invalid schema name was being used for returned highlight fields

Signed-off-by: forestmvey <forestv@bitquilltech.com>

* Fix failing integration tests due to schema changes for highlight expression type.

Signed-off-by: forestmvey <forestv@bitquilltech.com>

Co-authored-by: MaxKsyunz <maxk@bitquilltech.com>
@penghuo
Bump version to 2.2.0 (opensearch-project#729)
df32384 
Signed-off-by: penghuo <penghuo@gmail.com>
@joshuali925
Add release notes for 2.2.0.0 (opensearch-project#730)
1df89e3 
Signed-off-by: Joshua Li <joshuali925@gmail.com>
@penghuo
Bump workbench version to 2.2.0.0 (opensearch-project#736)
792d3ff 
Signed-off-by: penghuo <penghuo@gmail.com>
@MitchellGale @Yury-Fridlyand
Adding documentation about double quote implementation (opensearch-pr…
921a28c 
…oject#723)

Signed-off-by: mitchellg <mitchellg@bitquilltech.com>
Co-authored-by: Yury Fridlyand <yuryf@bitquilltech.com>
@joshuali925
Update snapshots for jest tests on workbench (opensearch-project#740)
e6a5ac9 
Signed-off-by: Joshua Li <joshuali925@gmail.com>
@dai-chen
Bump JDBC driver's dependency version (opensearch-project#735)
29f1c52 
* Bump wiremock version

Signed-off-by: Chen Dai <daichen@amazon.com>

* Bump guava and json-smart version

Signed-off-by: Chen Dai <daichen@amazon.com>

* Bump httpclient version

Signed-off-by: Chen Dai <daichen@amazon.com>
@dai-chen
Bump Jackson dependency version (opensearch-project#745)
6fb3fac 
* Bump jackson version and add version config in root gradle script

Signed-off-by: Chen Dai <daichen@amazon.com>

* Bump jackson version in JDBC driver

Signed-off-by: Chen Dai <daichen@amazon.com>

Signed-off-by: Chen Dai <daichen@amazon.com>
@seankao-az
Extend query size limit using scroll (opensearch-project#716)
f3c9c29 
* add maxResultWindow to LogicalRelation

Signed-off-by: Sean Kao <seankao@amazon.com>

* add maxResultWindow to OpenSearchLogicalIndexScan

Signed-off-by: Sean Kao <seankao@amazon.com>

* OpenSearchRequestBuilder init

Signed-off-by: Sean Kao <seankao@amazon.com>

* request builder: push down and build

Signed-off-by: Sean Kao <seankao@amazon.com>

* plan.build() for building request

Signed-off-by: Sean Kao <seankao@amazon.com>

* maxResultWindow for test utils

Signed-off-by: Sean Kao <seankao@amazon.com>

* fix style

Signed-off-by: Sean Kao <seankao@amazon.com>

* remove plan.build()

Signed-off-by: Sean Kao <seankao@amazon.com>

* fetch result in batches

Signed-off-by: Sean Kao <seankao@amazon.com>

* get index.max_result_window settings

Signed-off-by: Sean Kao <seankao@amazon.com>

* use index.max_result_window to decide scroll

Signed-off-by: Sean Kao <seankao@amazon.com>

* maxResultWindow for aggregation

Signed-off-by: Sean Kao <seankao@amazon.com>

* fix fetch size & for aggregation query

Signed-off-by: Sean Kao <seankao@amazon.com>

* fix rest client get max result window

Signed-off-by: Sean Kao <seankao@amazon.com>

* remove maxResultWindow from logical plan

Signed-off-by: Sean Kao <seankao@amazon.com>

* get max result window when building physical plan

Signed-off-by: Sean Kao <seankao@amazon.com>

* move source builder init to request builder

Signed-off-by: Sean Kao <seankao@amazon.com>

* fix max result window for test & rest client

Signed-off-by: Sean Kao <seankao@amazon.com>

* include request builder in equal comparison

Signed-off-by: Sean Kao <seankao@amazon.com>

* rename getIndexMaxResultWindows

Signed-off-by: Sean Kao <seankao@amazon.com>

* open search rest client test

Signed-off-by: Sean Kao <seankao@amazon.com>

* test: request builder, scroll index scan

Signed-off-by: Sean Kao <seankao@amazon.com>

* fix style

Signed-off-by: Sean Kao <seankao@amazon.com>

* remove getMaxResultWindow from base Table

Signed-off-by: Sean Kao <seankao@amazon.com>

* remove unused import from OpenSearchIndexScan

Signed-off-by: Sean Kao <seankao@amazon.com>

* test index scan

Signed-off-by: Sean Kao <seankao@amazon.com>

* integ test for head command

Signed-off-by: Sean Kao <seankao@amazon.com>

* keep request query size for aggregation

Signed-off-by: Sean Kao <seankao@amazon.com>

* fix rest client test coverage

Signed-off-by: Sean Kao <seankao@amazon.com>

* test for node client

Signed-off-by: Sean Kao <seankao@amazon.com>

* test node client default settings

Signed-off-by: Sean Kao <seankao@amazon.com>

* change Elasticsearch to OpenSearch in comment

Signed-off-by: Sean Kao <seankao@amazon.com>

* fix comments

Signed-off-by: Sean Kao <seankao@amazon.com>

* more test for Head IT

Signed-off-by: Sean Kao <seankao@amazon.com>

* ignore some head IT

Signed-off-by: Sean Kao <seankao@amazon.com>

Signed-off-by: Sean Kao <seankao@amazon.com>
@Yury-Fridlyand
Replace 2x LogUtils by QueryContext (opensearch-project#747)
deececb 
Signed-off-by: Yury Fridlyand <yuryf@bitquilltech.com>
@MitchellGale
Add any case of arguments in relevancy based functions to be allowed (o…
ac9b63a 
…pensearch-project#744)

* Added new functionality around argument names not case specific for relevancy based functions. Added unit tests to cover updates.

Signed-off-by: MitchellGale-BitQuill <mitchellg@bitquilltech.com>

* Fixed test cases that used uppercase values in arguments.

Signed-off-by: MitchellGale-BitQuill <mitchellg@bitquilltech.com>

Signed-off-by: MitchellGale-BitQuill <mitchellg@bitquilltech.com>
Callout distinction between original and current SQL engines. (opense…
45d84d0 
…arch-project#728)

Signed-off-by: MaxKsyunz <maxk@bitquilltech.com>

Signed-off-by: MaxKsyunz <maxk@bitquilltech.com>
@joshuali925
Update PPL docs link for workbench (opensearch-project#758)
95a24b2 
Signed-off-by: Joshua Li <joshuali925@gmail.com>

Signed-off-by: Joshua Li <joshuali925@gmail.com>
@Yury-Fridlyand
Add maketime and makedate (opensearch-project#102) (opensearch-pr…
9f602c3 
…oject#755)

Signed-off-by: Yury Fridlyand <yuryf@bitquilltech.com>

Signed-off-by: Yury Fridlyand <yuryf@bitquilltech.com>
@joshuali925
Upgrade yarn dependencies for workbench (opensearch-project#765)
23a4a88 
Signed-off-by: Joshua Li <joshuali925@gmail.com>
@penghuo
Schedule request in worker thread (opensearch-project#748)
ce15448 
Signed-off-by: penghuo <penghuo@gmail.com>
@joshuali925
Merge branch 'main' into 2.x
8729c32
@joshuali925
Resolve conflicts
8185127 
Signed-off-by: Joshua Li <joshuali925@gmail.com>
@joshuali925 joshuali925 requested a review from a team as a code owner August 17, 2022 18:02
@codecov-commenter
Copy link

Codecov Report

Merging #771 (8185127) into 2.x (dabe1d7) will decrease coverage by 34.98%.
The diff coverage is 0.00%.

@@              Coverage Diff              @@
##                2.x     #771       +/-   ##
=============================================
- Coverage     97.74%   62.76%   -34.99%     
=============================================
  Files           273       10      -263     
  Lines          7021      658     -6363     
  Branches        442      118      -324     
=============================================
- Hits           6863      413     -6450     
- Misses          157      192       +35     
- Partials          1       53       +52
Flag Coverage Δ
query-workbench 62.76% <0.00%> (?)
sql-engine ?

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
workbench/public/components/Main/main.tsx 53.00% <0.00%> (ø)
...ain/java/org/opensearch/sql/analysis/Analyzer.java
...rg/opensearch/sql/analysis/ExpressionAnalyzer.java
...ensearch/sql/expression/ExpressionNodeVisitor.java
...arch/sql/expression/datetime/DateTimeFunction.java
...h/sql/expression/function/BuiltinFunctionName.java
...h/sql/expression/function/OpenSearchFunctions.java
...opensearch/sql/planner/logical/LogicalPlanDSL.java
...ch/sql/planner/logical/LogicalPlanNodeVisitor.java
...ensearch/sql/planner/physical/PhysicalPlanDSL.java
... and 274 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

@dai-chen
Copy link
Collaborator

@joshuali925 Thanks for syncing main and 2.x. I found some PRs are already merged to 2.x earlier or being reviewed in auto backport PR. Probably in future we need to make sure all PRs merged to main have backport PR and merged to ?.x automatically. For now, should we remove those and cherry-pick the remaining?

@penghuo
Copy link
Collaborator

penghuo commented Aug 17, 2022

@joshuali925 Thanks for syncing main and 2.x. I found some PRs are already merged to 2.x earlier or being reviewed in auto backport PR. Probably in future we need to make sure all PRs merged to main have backport PR and merged to ?.x automatically. For now, should we remove those and cherry-pick the remaining?

Yes, we should close all the auto-porting PR

@penghuo penghuo mentioned this pull request Aug 17, 2022
Closed
@joshuali925 joshuali925 merged commit 9819635 into opensearch-project:2.x Aug 17, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@dai-chen dai-chen dai-chen approved these changes

@penghuo penghuo penghuo approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
8 participants
@joshuali925 @codecov-commenter @dai-chen @penghuo @MitchellGale @Yury-Fridlyand @forestmvey @seankao-az