ARROW-13729: [Website] setup datafusion python binding docs #10982
Conversation
|
Thanks for opening a pull request! If this is not a minor PR. Could you open an issue for this pull request on JIRA? https://issues.apache.org/jira/browse/ARROW Opening JIRAs ahead of time contributes to the Openness of the Apache Arrow project. Then could you also rename pull request title in the following format? or See also: |
jimexist
changed the title
|
Looks amazing, thanks @jimexist ! @alamb @jorgecarleitao and @andygrove, do we need to wait for a voted datafusion 5.1.0 release before we can merge this PR? |
|
@houqp I do not think any official vote is required to release documentation as I wouldn't personally consider it part of the release. |
jimexist
force-pushed
the
setup-datafusion-docs
branch
from
7e9d777 to
16cf420
Compare
houqp
requested review from
lidavidm,
pitrou,
alamb,
andygrove,
Dandandan and
jorgecarleitao
|
@jimexist do we need to update https://github.com/apache/arrow/blob/master/docs/source/developers/documentation.rst as well to mention installation of the datafusion package? I am not familiar with the automated doc build and publish pipeline, @kszucs @pitrou @lidavidm @wesm @fsaintjacques do we need to update automation to build and install the datafusion python package? |
| '..', '../..') | ||
|
|
||
| ]) | ||
| sys.path.extend([os.path.join(os.path.dirname(__file__), "..", "../..")]) |
There was a problem hiding this comment.
Most of this file is auto-generated. By making style changes we will make it more difficult to diff the current file against a pristine file.
|
I don't think Datafusion is a "supported environment" at the same level as C++, Python, etc. Since this documents the Python Datafusion library, it could go into the existing "Python" section. However, since Datafusion is a distinct library from PyArrow (and has its own release schedule), it seems to me it would be better if its documentation lived in a different place altogether. That could be on https://arrow.apache.org, but it could also be hosted on https://readthedocs.org/, for example. @amol- What do you think? |
I think I would vote for having datafusion into its own documentation (on readthedocs or anywhere else as you suggested), surely it's not one of the supported environments, but in the end is not even part of libarrow/pyarrow itself so I'd avoid mixing its documentation into the one of pyarrow. |
|
Relevant mailing list discussion: https://lists.apache.org/thread.html/r0ed76cc60cdf651e8cf5c82a21cc64114c1f6d174dc5487434bd32ef%40%3Cdev.arrow.apache.org%3E |
|
based on the last reply from @kszucs (https://lists.apache.org/thread.html/r5c9341fe5360ae249532724da2bf92bd2ed661d1c58f599212e82107%40%3Cdev.arrow.apache.org%3E), how about we reuse the current sphinx setup/theme for datafusion, but have a new self-contained website created under https://github.com/apache/arrow-site/tree/asf-site's |
|
Following up on this, @pitrou @amol- based on @wesm's suggestion in https://lists.apache.org/thread.html/r9500355019f7e438ed2417bce577e1a76dbcba742c2c7e1008dcffd5%40%3Cdev.arrow.apache.org%3E, are you all cool with us hosting datafusion related docs in its own website under https://arrow.apache.org/datafusion? |
|
@houqp thanks for migrating this code. maybe now I can close this pull request? |
|
I think so, all of your change in this PR should have been merged into datafusion repo now :) |
closes apache/datafusion#837
cc @houqp @alamb @jorgecarleitao and @andygrove