Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Backport 2.9] Add spark connector #1842

Merged
merged 1 commit into from
Jul 11, 2023
Merged

Conversation

opensearch-trigger-bot[bot]
Copy link
Contributor

Backport a816a58 from #1780

* Create Spark Connector

Signed-off-by: Vamsi Manohar <reddyvam@amazon.com>

* Add spark client and engine

Signed-off-by: Rupal Mahajan <maharup@amazon.com>

* Remove vars

Signed-off-by: Rupal Mahajan <maharup@amazon.com>

* Spark connector draft

Signed-off-by: Rupal Mahajan <maharup@amazon.com>

* nit

Signed-off-by: Rupal Mahajan <maharup@amazon.com>

* Fix checkstyle errors

Signed-off-by: Rupal Mahajan <maharup@amazon.com>

* nit

Signed-off-by: Rupal Mahajan <maharup@amazon.com>

* Fix license header

Signed-off-by: Rupal Mahajan <maharup@amazon.com>

* Add spark storage test

Signed-off-by: Rupal Mahajan <maharup@amazon.com>

* Update comments

Signed-off-by: Rupal Mahajan <maharup@amazon.com>

* Fix checkstyle in comments

Signed-off-by: Rupal Mahajan <maharup@amazon.com>

* Update tests

Signed-off-by: Rupal Mahajan <maharup@amazon.com>

* Address PR comments

Signed-off-by: Rupal Mahajan <maharup@amazon.com>

* Refactor class name

Signed-off-by: Rupal Mahajan <maharup@amazon.com>

* Address PR comment

Signed-off-by: Rupal Mahajan <maharup@amazon.com>

---------

Signed-off-by: Vamsi Manohar <reddyvam@amazon.com>
Signed-off-by: Rupal Mahajan <maharup@amazon.com>
Co-authored-by: Vamsi Manohar <reddyvam@amazon.com>
(cherry picked from commit a816a58)
@codecov
Copy link

codecov bot commented Jul 11, 2023

Codecov Report

Merging #1842 (cb567b3) into 2.9 (2144ddd) will increase coverage by 0.01%.
The diff coverage is 100.00%.

@@             Coverage Diff              @@
##                2.9    #1842      +/-   ##
============================================
+ Coverage     97.35%   97.37%   +0.01%     
- Complexity     4443     4475      +32     
============================================
  Files           388      394       +6     
  Lines         10998    11066      +68     
  Branches        784      789       +5     
============================================
+ Hits          10707    10775      +68     
  Misses          284      284              
  Partials          7        7              
Flag Coverage Δ
sql-engine 97.37% <100.00%> (+0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
...implementation/SparkSqlFunctionImplementation.java 100.00% <100.00%> (ø)
...ctions/resolver/SparkSqlTableFunctionResolver.java 100.00% <100.00%> (ø)
...nctions/scan/SparkSqlFunctionTableScanBuilder.java 100.00% <100.00%> (ø)
...ensearch/sql/spark/storage/SparkStorageEngine.java 100.00% <100.00%> (ø)
...nsearch/sql/spark/storage/SparkStorageFactory.java 100.00% <100.00%> (ø)
...a/org/opensearch/sql/spark/storage/SparkTable.java 100.00% <100.00%> (ø)

@penghuo penghuo merged commit 748fc65 into 2.9 Jul 11, 2023
@github-actions github-actions bot deleted the backport/backport-1780-to-2.9 branch July 11, 2023 17:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants