Skip to content

[SPARK-35433][DOCS] Move CSV data source options from Python and Scala into a single page #32658

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
itholic wants to merge 5 commits into from
Closed

[SPARK-35433][DOCS] Move CSV data source options from Python and Scala into a single page #32658

itholic wants to merge 5 commits into from

Conversation

@itholic
Copy link
Contributor

@itholic itholic commented May 25, 2021 •
edited
Loading

What changes were proposed in this pull request?

This PR proposes move CSV data source options from Python, Scala and Java into a single page.

Why are the changes needed?

So far, the documentation for CSV data source options is separated into different pages for each language API documents. However, this makes managing many options inconvenient, so it is efficient to manage all options in a single page and provide a link to that page in the API of each language.

Does this PR introduce any user-facing change?

Yes, the documents will be shown below after this change:

  • "CSV Files" page

Screen Shot 2021-05-27 at 12 35 36 PM

  • Python

Screen Shot 2021-05-25 at 4 12 10 PM

  • Scala

Screen Shot 2021-05-25 at 4 12 39 PM

  • Java

Screen Shot 2021-05-25 at 4 13 09 PM

How was this patch tested?

Manually build docs and confirm the page.

@itholic itholic changed the title [SPARK-35433][DOCS] Move CSV data source options from Python and Scal… [SPARK-35433][DOCS] Move CSV data source options from Python and Scala into a single page May 25, 2021
@SparkQA
Copy link

SparkQA commented May 25, 2021

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/43434/

@SparkQA
Copy link

SparkQA commented May 25, 2021

Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/43434/

HyukjinKwon
HyukjinKwon reviewed May 25, 2021
View reviewed changes
HyukjinKwon
HyukjinKwon reviewed May 25, 2021
View reviewed changes
HyukjinKwon
HyukjinKwon reviewed May 25, 2021
View reviewed changes
HyukjinKwon
HyukjinKwon reviewed May 25, 2021
View reviewed changes
@SparkQA
Copy link

SparkQA commented May 25, 2021

Test build #138912 has finished for PR 32658 at commit 5ba96ec.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA
Copy link

SparkQA commented May 26, 2021

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/43489/

HyukjinKwon
HyukjinKwon reviewed May 26, 2021
View reviewed changes
HyukjinKwon
HyukjinKwon reviewed May 26, 2021
View reviewed changes
HyukjinKwon
HyukjinKwon reviewed May 26, 2021
View reviewed changes
@SparkQA
Copy link

SparkQA commented May 26, 2021

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/43489/

@SparkQA
Copy link

SparkQA commented May 26, 2021

Test build #138974 has finished for PR 32658 at commit 0e75d8d.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@itholic itholic mentioned this pull request May 27, 2021
Closed
itholic
itholic commented May 27, 2021
View reviewed changes
HyukjinKwon
HyukjinKwon reviewed May 27, 2021
View reviewed changes
@SparkQA
Copy link

SparkQA commented May 27, 2021

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/43521/

@SparkQA
Copy link

SparkQA commented May 27, 2021

Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/43521/

@SparkQA
Copy link

SparkQA commented May 27, 2021

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/43523/

@SparkQA
Copy link

SparkQA commented May 27, 2021

Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/43523/

@SparkQA
Copy link

SparkQA commented May 27, 2021

Test build #139003 has finished for PR 32658 at commit 673e26b.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA
Copy link

SparkQA commented May 27, 2021

Test build #139005 has finished for PR 32658 at commit 9df317f.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

HyukjinKwon
HyukjinKwon reviewed May 27, 2021
View reviewed changes
HyukjinKwon
HyukjinKwon reviewed May 27, 2021
View reviewed changes
@SparkQA
Copy link

SparkQA commented May 28, 2021

Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/43561/

@SparkQA
Copy link

SparkQA commented May 28, 2021

Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/43562/

@SparkQA
Copy link

SparkQA commented May 28, 2021

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/43564/

@SparkQA
Copy link

SparkQA commented May 28, 2021

Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/43564/

@SparkQA
Copy link

SparkQA commented May 28, 2021

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/43560/

@SparkQA
Copy link

SparkQA commented May 28, 2021

Test build #139042 has finished for PR 32658 at commit 1991031.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA
Copy link

SparkQA commented May 28, 2021

Test build #139044 has finished for PR 32658 at commit a4a2bb2.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@itholic
Copy link
Contributor Author

itholic commented May 31, 2021

@itholic the generated doc looks a bit weird:

Screen Shot 2021-05-28 at 1 09 38 PM

It includes # noqa

Can you double check and fix? Seems like we should fix other places for JSON, etc.

Thanks, @HyukjinKwon . Just fixed them in every place.

@SparkQA
Copy link

SparkQA commented May 31, 2021

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/43614/

@SparkQA
Copy link

SparkQA commented May 31, 2021

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/43614/

HyukjinKwon
HyukjinKwon reviewed May 31, 2021
View reviewed changes
@SparkQA
Copy link

SparkQA commented May 31, 2021

Test build #139093 has finished for PR 32658 at commit 3bda8db.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

HyukjinKwon
HyukjinKwon reviewed May 31, 2021
View reviewed changes
HyukjinKwon
HyukjinKwon reviewed May 31, 2021
View reviewed changes
HyukjinKwon
HyukjinKwon reviewed May 31, 2021
View reviewed changes
HyukjinKwon
HyukjinKwon reviewed May 31, 2021
View reviewed changes
HyukjinKwon
HyukjinKwon reviewed Jun 1, 2021
View reviewed changes
HyukjinKwon
HyukjinKwon reviewed Jun 1, 2021
View reviewed changes
HyukjinKwon
HyukjinKwon approved these changes Jun 1, 2021
View reviewed changes
Copy link
Member

@HyukjinKwon HyukjinKwon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM otherwise.

@HyukjinKwon HyukjinKwon mentioned this pull request Jun 1, 2021
Closed
@HyukjinKwon
Copy link
Member

HyukjinKwon commented Jun 1, 2021

Merged to master.

@SparkQA
Copy link

SparkQA commented Jun 1, 2021

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/43646/

@SparkQA
Copy link

SparkQA commented Jun 1, 2021

Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/43646/

@SparkQA
Copy link

SparkQA commented Jun 1, 2021

Test build #139126 has finished for PR 32658 at commit f55a2fa.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@HyukjinKwon HyukjinKwon HyukjinKwon approved these changes

Assignees

No one assigned

Labels

CORE DOCS PYTHON SQL STRUCTURED STREAMING

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@itholic @SparkQA @HyukjinKwon