-
Notifications
You must be signed in to change notification settings - Fork 29k
[SPARK-5469] restructure pyspark.sql into multiple files #4479
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
|
Test build #27121 has started for PR 4479 at commit
|
Conflicts: python/pyspark/sql.py
|
Test build #27122 has started for PR 4479 at commit
|
|
Test build #27121 has finished for PR 4479 at commit
|
|
Test FAILed. |
|
Test build #27122 has finished for PR 4479 at commit
|
|
Test FAILed. |
|
@rxin I think this PR is ready, the failed tests are flaky, not related to this one. |
|
Jenkins, retest this please. |
|
Test build #27128 has started for PR 4479 at commit
|
|
Test build #594 has started for PR 4479 at commit
|
|
Test build #27128 has finished for PR 4479 at commit
|
|
Test FAILed. |
|
Test build #594 has finished for PR 4479 at commit
|
|
Jenkins, retest this please. |
|
Test build #27151 has started for PR 4479 at commit
|
|
Test build #27151 has finished for PR 4479 at commit
|
|
Test PASSed. |
|
Thanks. I'm merging this in master and 1.3. |
All the DataTypes moved into pyspark.sql.types
The changes can be tracked by `--find-copies-harder -M25`
```
davieslocalhost:~/work/spark/python$ git diff --find-copies-harder -M25 --numstat master..
2 5 python/docs/pyspark.ml.rst
0 3 python/docs/pyspark.mllib.rst
10 2 python/docs/pyspark.sql.rst
1 1 python/pyspark/mllib/linalg.py
21 14 python/pyspark/{mllib => sql}/__init__.py
14 2108 python/pyspark/{sql.py => sql/context.py}
10 1772 python/pyspark/{sql.py => sql/dataframe.py}
7 6 python/pyspark/{sql_tests.py => sql/tests.py}
8 1465 python/pyspark/{sql.py => sql/types.py}
4 2 python/run-tests
1 1 sql/core/src/main/scala/org/apache/spark/sql/test/ExamplePointUDT.scala
```
Also `git blame -C -C python/pyspark/sql/context.py` to track the history.
Author: Davies Liu <davies@databricks.com>
Closes #4479 from davies/sql and squashes the following commits:
1b5f0a5 [Davies Liu] Merge branch 'master' of github.com:apache/spark into sql
2b2b983 [Davies Liu] restructure pyspark.sql
(cherry picked from commit 08488c1)
Signed-off-by: Reynold Xin <rxin@databricks.com>
All the DataTypes moved into pyspark.sql.types
The changes can be tracked by
--find-copies-harder -M25Also
git blame -C -C python/pyspark/sql/context.pyto track the history.