[SPARK-17819][SQL] Support default database in connection URIs for Spark Thrift Server #15399

dongjoon-hyun · 2016-10-08T01:14:22Z

What changes were proposed in this pull request?

Currently, Spark Thrift Server ignores the default database in URI. This PR supports that like the following.

$ bin/beeline -u jdbc:hive2://localhost:10000 -e "create database testdb"
$ bin/beeline -u jdbc:hive2://localhost:10000/testdb -e "create table t(a int)"
$ bin/beeline -u jdbc:hive2://localhost:10000/testdb -e "show tables"
...
+------------+--------------+--+
| tableName  | isTemporary  |
+------------+--------------+--+
| t          | false        |
+------------+--------------+--+
1 row selected (0.347 seconds)
$ bin/beeline -u jdbc:hive2://localhost:10000 -e "show tables"
...
+------------+--------------+--+
| tableName  | isTemporary  |
+------------+--------------+--+
+------------+--------------+--+
No rows selected (0.098 seconds)

How was this patch tested?

Manual.

Note: I tried to add a test case for this, but I cannot found a suitable testsuite for this. I'll add the testcase if some advice is given.

dongjoon-hyun · 2016-10-08T01:22:12Z

Hi, @rxin .
Sorry, but could you give me some advice about a proper testsuite for this kind of PR?

SparkQA · 2016-10-08T01:24:08Z

Test build #66551 has finished for PR 15399 at commit f9a294e.

This patch fails to build.
This patch merges cleanly.
This patch adds no public classes.

dongjoon-hyun · 2016-10-08T01:28:37Z

Oops. Sorry. I'll fix soon.

SparkQA · 2016-10-08T02:13:31Z

Test build #66555 has finished for PR 15399 at commit 26f4b25.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

dongjoon-hyun · 2016-10-08T02:29:44Z

Retest this please.

SparkQA · 2016-10-08T03:01:07Z

Test build #66557 has finished for PR 15399 at commit 26f4b25.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

SparkQA · 2016-10-08T03:30:02Z

Test build #66560 has finished for PR 15399 at commit a832760.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

dongjoon-hyun · 2016-10-09T02:56:40Z

Hi, @gatorsmile .
Could you review this PR when you have sometime?

SparkQA · 2016-10-10T02:53:45Z

Test build #66617 has finished for PR 15399 at commit d027421.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

gatorsmile · 2016-10-10T04:01:19Z

First, I am not familiar with the code in this component. Thus, I am not the right person to review it.

Second, when I going over the pending JIRA list, I found many bugs that are reported in Thrift Server.

Third, I do not know what is the current strategy for supporting beeline and spark-sql. If this is the focus, I expect at least 50+ bug fixes in this area

dongjoon-hyun · 2016-10-10T16:55:20Z

Thank you for commenting, @gatorsmile . So far, I asked @rxin and you. I'll find another committer more closer this part.

dongjoon-hyun · 2016-10-10T18:51:36Z

Hi, @liancheng .
Could you review this PR about Thrift Server?

rxin · 2016-10-10T21:55:03Z

Can we add a test case using the jdbc client?

dongjoon-hyun · 2016-10-10T22:16:50Z

Thank you for your advice, @rxin !!
Which suite do you mean? The suites inside HiveThriftServer2Suites.scala look not proper to me for this kind of cases.

dongjoon-hyun · 2016-10-10T22:19:03Z

Or, should I create a new testsuite for this kind of testcases?

SparkQA · 2016-10-12T21:38:19Z

Test build #66841 has finished for PR 15399 at commit 96633d1.

This patch passes all tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
- class JdbcConnectionUriSuite extends HiveThriftServer2Test

dongjoon-hyun · 2016-10-12T21:49:45Z

Hi, @rxin .
Since this kind of tests need to change their URI and also assume that the non-default database exists prior, I made a new testsuite for this.
Could you review this again?

dongjoon-hyun · 2016-10-14T04:25:14Z

Hi, @rxin .
Could you review this again?

dongjoon-hyun · 2016-10-14T20:56:01Z

Hi, @rxin or @liancheng .
Could you review this PR when you have some time?

…necting to thriftserver

SparkQA · 2016-10-17T00:54:23Z

Test build #67041 has finished for PR 15399 at commit 7ef307a.

This patch passes all tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
- class JdbcConnectionUriSuite extends HiveThriftServer2Test

rxin · 2016-10-17T03:14:09Z

Thanks - merging in master.

rxin · 2016-10-17T03:15:59Z

Can you create a backport for branch-2.0?

dongjoon-hyun · 2016-10-17T03:17:07Z

Thank you, @rxin !
Sure. I'll create a backport PR.

…ark Thrift Server ## What changes were proposed in this pull request? Currently, Spark Thrift Server ignores the default database in URI. This PR supports that like the following. ```sql $ bin/beeline -u jdbc:hive2://localhost:10000 -e "create database testdb" $ bin/beeline -u jdbc:hive2://localhost:10000/testdb -e "create table t(a int)" $ bin/beeline -u jdbc:hive2://localhost:10000/testdb -e "show tables" ... +------------+--------------+--+ | tableName | isTemporary | +------------+--------------+--+ | t | false | +------------+--------------+--+ 1 row selected (0.347 seconds) $ bin/beeline -u jdbc:hive2://localhost:10000 -e "show tables" ... +------------+--------------+--+ | tableName | isTemporary | +------------+--------------+--+ +------------+--------------+--+ No rows selected (0.098 seconds) ``` ## How was this patch tested? Manual. Note: I tried to add a test case for this, but I cannot found a suitable testsuite for this. I'll add the testcase if some advice is given. Author: Dongjoon Hyun <dongjoon@apache.org> Closes apache#15399 from dongjoon-hyun/SPARK-17819.

dongjoon-hyun changed the title ~~[SPARK-17819][SQL] Specified database in JDBC URL is ignored when con…~~ [SPARK-17819][SQL] Support default database in connection URIs for Spark Thrift Server Oct 8, 2016

dongjoon-hyun added 2 commits October 16, 2016 17:05

[SPARK-17819][SQL] Specified database in JDBC URL is ignored when con…

411752f

…necting to thriftserver

Add JdbcConnectionUriSuite

7ef307a

asfgit closed this in 59e3eb5 Oct 17, 2016

dongjoon-hyun deleted the SPARK-17819 branch November 7, 2016 00:49

[SPARK-17819][SQL] Support default database in connection URIs for Spark Thrift Server #15399

[SPARK-17819][SQL] Support default database in connection URIs for Spark Thrift Server #15399

Uh oh!

Conversation

dongjoon-hyun commented Oct 8, 2016 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What changes were proposed in this pull request?

How was this patch tested?

Uh oh!

dongjoon-hyun commented Oct 8, 2016 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

SparkQA commented Oct 8, 2016

Uh oh!

dongjoon-hyun commented Oct 8, 2016

Uh oh!

SparkQA commented Oct 8, 2016

Uh oh!

dongjoon-hyun commented Oct 8, 2016

Uh oh!

SparkQA commented Oct 8, 2016

Uh oh!

SparkQA commented Oct 8, 2016

Uh oh!

dongjoon-hyun commented Oct 9, 2016

Uh oh!

SparkQA commented Oct 10, 2016

Uh oh!

gatorsmile commented Oct 10, 2016

Uh oh!

dongjoon-hyun commented Oct 10, 2016

Uh oh!

dongjoon-hyun commented Oct 10, 2016

Uh oh!

rxin commented Oct 10, 2016

Uh oh!

dongjoon-hyun commented Oct 10, 2016

Uh oh!

dongjoon-hyun commented Oct 10, 2016

Uh oh!

SparkQA commented Oct 12, 2016

Uh oh!

dongjoon-hyun commented Oct 12, 2016

Uh oh!

dongjoon-hyun commented Oct 14, 2016

Uh oh!

dongjoon-hyun commented Oct 14, 2016

Uh oh!

SparkQA commented Oct 17, 2016

Uh oh!

rxin commented Oct 17, 2016

Uh oh!

rxin commented Oct 17, 2016

Uh oh!

dongjoon-hyun commented Oct 17, 2016

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

dongjoon-hyun commented Oct 8, 2016 •

edited

Loading

dongjoon-hyun commented Oct 8, 2016 •

edited

Loading