Skip to content

[SPARK-10036][SQL]Load JDBC driver in DataFrameReader.jdbc and DataFrameWriter.jdbc #8232

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
zsxwing wants to merge 2 commits into from
Closed

[SPARK-10036][SQL]Load JDBC driver in DataFrameReader.jdbc and DataFrameWriter.jdbc #8232

zsxwing wants to merge 2 commits into from

Conversation

@zsxwing
Copy link
Member

@zsxwing zsxwing commented Aug 16, 2015

This PR uses JDBCRDD.getConnector to load JDBC driver before creating connection in DataFrameReader.jdbc and DataFrameWriter.jdbc.

@zsxwing
Copy link
Member Author

zsxwing commented Aug 16, 2015

DataFrameReader.jdbc and DataFrameWriter.jdbc ignore extraOptions currently. Is it intentional, or a bug?

@SparkQA
Copy link

SparkQA commented Aug 16, 2015

Test build #40997 has finished for PR 8232 at commit 57f59d4.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

rxin
rxin reviewed Aug 17, 2015
View reviewed changes
sql/core/src/main/scala/org/apache/spark/sql/DataFrameReader.scala Outdated
Copy link
Contributor

@rxin rxin Aug 17, 2015

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i think that's a bug

Copy link
Member Author

@zsxwing zsxwing Aug 17, 2015

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okey. Fixed it.

@SparkQA
Copy link

SparkQA commented Aug 17, 2015

Test build #41012 has finished for PR 8232 at commit adf75de.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@marmbrus
Copy link
Contributor

marmbrus commented Aug 17, 2015

Thanks! Merged to master and 1.5

asfgit pushed a commit that referenced this pull request Aug 17, 2015
@zsxwing @marmbrus
[SPARK-10036] [SQL] Load JDBC driver in DataFrameReader.jdbc and Data…
4daf79f 
…FrameWriter.jdbc

This PR uses `JDBCRDD.getConnector` to load JDBC driver before creating connection in `DataFrameReader.jdbc` and `DataFrameWriter.jdbc`.

Author: zsxwing <zsxwing@gmail.com>

Closes #8232 from zsxwing/SPARK-10036 and squashes the following commits:

adf75de [zsxwing] Add extraOptions to the connection properties
57f59d4 [zsxwing] Load JDBC driver in DataFrameReader.jdbc and DataFrameWriter.jdbc

(cherry picked from commit f10660f)
Signed-off-by: Michael Armbrust <michael@databricks.com>
@asfgit asfgit closed this in f10660f Aug 17, 2015
@zsxwing zsxwing deleted the SPARK-10036 branch August 18, 2015 00:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@zsxwing @SparkQA @marmbrus @rxin