[KYUUBI #6253] Support running JDBC engine on YARN AM #6275
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
lsm1
changed the title
github-actions
bot
added
kind:documentation
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #6275 +/- ##
============================================
+ Coverage 58.48% 58.84% +0.36%
Complexity 24 24
============================================
Files 651 654 +3
Lines 39513 40875 +1362
Branches 5441 5768 +327
============================================
+ Hits 23109 24053 +944
- Misses 13937 14313 +376
- Partials 2467 2509 +42 ☔ View full report in Codecov by Sentry. |
yikf
reviewed
Apr 9, 2024
kyuubi-common/src/main/scala/org/apache/kyuubi/engine/deploy/yarn/EngineYarnModeSubmitter.scala
Outdated
Show resolved
Hide resolved
...-jdbc-engine/src/main/scala/org/apache/kyuubi/engine/jdbc/deploy/JdbcYarnModeSubmitter.scala
Outdated
Show resolved
Hide resolved
...-jdbc-engine/src/main/scala/org/apache/kyuubi/engine/jdbc/deploy/JdbcYarnModeSubmitter.scala
Outdated
Show resolved
Hide resolved
...-engine/src/test/scala/org/apache/kyuubi/engine/jdbc/deploy/JdbcYarnModeSubmitterSuite.scala
Outdated
Show resolved
Hide resolved
...c/test/scala/org/apache/kyuubi/it/jdbc/doris/WithKyuubiServerAndDorisContainerYarnMode.scala
Outdated
Show resolved
Hide resolved
kyuubi-server/src/main/scala/org/apache/kyuubi/engine/jdbc/JdbcProcessBuilder.scala
Outdated
Show resolved
Hide resolved
kyuubi-server/src/main/scala/org/apache/kyuubi/engine/jdbc/JdbcYarnModeProcessBuilder.scala
Outdated
Show resolved
Hide resolved
pan3793
reviewed
Apr 10, 2024
kyuubi-common/src/main/scala/org/apache/kyuubi/engine/deploy/yarn/EngineYarnModeSubmitter.scala
Outdated
Show resolved
Hide resolved
pan3793
approved these changes
Apr 11, 2024
pan3793
reviewed
Apr 11, 2024
externals/kyuubi-jdbc-engine/src/main/scala/org/apache/kyuubi/engine/jdbc/JdbcSQLEngine.scala
Outdated
Show resolved
Hide resolved
pan3793
reviewed
Apr 11, 2024
| s"${KyuubiConf.ENGINE_PRINCIPAL.key} and " + | ||
| s"${KyuubiConf.ENGINE_KEYTAB.key} must be set when submit " + | ||
| s"${JdbcSQLEngine.getClass.getSimpleName.stripSuffix("$")} to YARN") | ||
| } |
There was a problem hiding this comment.
actually, this condition could be relaxed. but this can be done later, independently. cc @cxzl25
for hive, we require keytab to run MR jobs, but for JDBC, most DB does not require kerberos authentication.
pan3793
reviewed
Apr 11, 2024
externals/kyuubi-jdbc-engine/src/main/scala/org/apache/kyuubi/engine/jdbc/JdbcSQLEngine.scala
Outdated
Show resolved
Hide resolved
pan3793
reviewed
Apr 11, 2024
...e-sql-engine/src/main/scala/org/apache/kyuubi/engine/hive/deploy/HiveYarnModeSubmitter.scala
Show resolved
Hide resolved
|
Just minor comments, LGTM |
pan3793
reviewed
Apr 11, 2024
externals/kyuubi-jdbc-engine/src/main/scala/org/apache/kyuubi/engine/jdbc/JdbcSQLEngine.scala
Show resolved
Hide resolved
pan3793
reviewed
Apr 11, 2024
kyuubi-common/src/main/scala/org/apache/kyuubi/engine/deploy/yarn/EngineYarnModeSubmitter.scala
Outdated
Show resolved
Hide resolved
cxzl25
reviewed
Apr 11, 2024
kyuubi-server/src/main/scala/org/apache/kyuubi/engine/jdbc/JdbcProcessBuilder.scala
Outdated
Show resolved
Hide resolved
...c/test/scala/org/apache/kyuubi/it/jdbc/mysql/WithKyuubiServerAndMySQLContainerYarnMode.scala
Outdated
Show resolved
Hide resolved
yikf
approved these changes
Apr 12, 2024
There was a problem hiding this comment.
Overall, LGTM, but I would also like @zhouyifan279 to take a look at the Kerberos authentication-related code if you find a time.
cxzl25
approved these changes
Apr 15, 2024
|
Thanks, merged to master. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
🔍 Description
Issue References 🔗
This pull request fixes #6253
Describe Your Solution 🔧
Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. List any dependencies that are required for this change.
Types of changes 🔖
Test Plan 🧪
Behavior Without This Pull Request ⚰️
Behavior With This Pull Request 🎉
Related Unit Tests
Checklist 📝
Be nice. Be informative.