[SPARK-16772] Correct API doc references to PySpark classes + formatting fixes #14393
Conversation
nchammas
changed the title
|
Jenkins, test this please. |
|
Test build #62972 has finished for PR 14393 at commit
|
nchammas
changed the title
nchammas
changed the title
|
Test build #62976 has finished for PR 14393 at commit
|
nchammas
changed the title
|
Test build #62983 has finished for PR 14393 at commit
|
|
Apologies for making a fairly "noisy" PR, with changes in several scattered places. However, as a PySpark user it's important to me that the API docs be properly formatted and that docstring class references work. Feel free to ping me on Python docstring changes in the future. I would be happy to review them. |
|
Test build #62984 has finished for PR 14393 at commit
|
|
@nchammas did you build the docs to verify? If yes I'm going to merge it. |
|
Yes, I built the docs and reviewed several (but not all) of the changes locally in my browser and confirmed that the corrections I wanted took place as expected. (Apologies about not using the PR template when I first opened the PR. GitHub Desktop seems not to support that yet. I've updated the PR description to include this info now.) |
|
Thanks - merging in master and 2.0. |
…ing fixes ## What's Been Changed The PR corrects several broken or missing class references in the Python API docs. It also correct formatting problems. For example, you can see [here](http://spark.apache.org/docs/2.0.0/api/python/pyspark.sql.html#pyspark.sql.SQLContext.registerFunction) how Sphinx is not picking up the reference to `DataType`. That's because the reference is relative to the current module, whereas `DataType` is in a different module. You can also see [here](http://spark.apache.org/docs/2.0.0/api/python/pyspark.sql.html#pyspark.sql.SQLContext.createDataFrame) how the formatting for byte, tinyint, and so on is italic instead of monospace. That's because in ReST single backticks just make things italic, unlike in Markdown. ## Testing I tested this PR by [building the Python docs](https://github.com/apache/spark/tree/master/docs#generating-the-documentation-html) and reviewing the results locally in my browser. I confirmed that the broken or missing class references were resolved, and that the formatting was corrected. Author: Nicholas Chammas <nicholas.chammas@gmail.com> Closes #14393 from nchammas/python-docstring-fixes. (cherry picked from commit 274f3b9) Signed-off-by: Reynold Xin <rxin@databricks.com>
| from ``data``, which should be an RDD of :class:`Row`, | ||
| or :class:`namedtuple`, or :class:`dict`. | ||
| When ``schema`` is :class:`DataType` or datatype string, it must match the real data, or |
There was a problem hiding this comment.
I made a mistake here, thinking "datatype string" was actually meant to be StringType(). I understand now that a datatype string is actually a thing.
Correction incoming...
3 participants
What's Been Changed
The PR corrects several broken or missing class references in the Python API docs. It also correct formatting problems.
For example, you can see here how Sphinx is not picking up the reference to
DataType. That's because the reference is relative to the current module, whereasDataTypeis in a different module.You can also see here how the formatting for byte, tinyint, and so on is italic instead of monospace. That's because in ReST single backticks just make things italic, unlike in Markdown.
Testing
I tested this PR by building the Python docs and reviewing the results locally in my browser. I confirmed that the broken or missing class references were resolved, and that the formatting was corrected.