Skip to content

[SQL][DataFrame] Fix column computability bug. #4519

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
rxin wants to merge 2 commits into from
Closed

[SQL][DataFrame] Fix column computability bug. #4519

rxin wants to merge 2 commits into from

Conversation

@rxin
Copy link
Contributor

@rxin rxin commented Feb 11, 2015

Do not recursively strip out projects. Only strip the first level project.

df("colA") + df("colB").as("colC")

Previously, the above would construct an invalid plan.

@rxin
[SQL][DataFrame] Fix column computability.
015c4fc 
Do not recursively strip out projects. Only strip the first level project.
@SparkQA
Copy link

SparkQA commented Feb 11, 2015

Test build #27261 has started for PR 4519 at commit 015c4fc.

  • This patch merges cleanly.

marmbrus
marmbrus reviewed Feb 11, 2015
View reviewed changes
sql/core/src/main/scala/org/apache/spark/sql/Column.scala Outdated
Copy link
Contributor

@marmbrus marmbrus Feb 11, 2015

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: I would consider splitting this up since matching in the constructor is not super easy to read.

@SparkQA
Copy link

SparkQA commented Feb 11, 2015

Test build #27263 has started for PR 4519 at commit 87ff763.

  • This patch merges cleanly.

@SparkQA
Copy link

SparkQA commented Feb 11, 2015

Test build #27261 has finished for PR 4519 at commit 015c4fc.

  • This patch fails Spark unit tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@AmplabJenkins
Copy link

AmplabJenkins commented Feb 11, 2015

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/27261/
Test FAILed.

@marmbrus
Copy link
Contributor

marmbrus commented Feb 11, 2015

This passed catalyst/sql/hive tests. The core tests are know flakey. Merging after talking with @pwendell so this can be included in the snapshot.

@marmbrus
Copy link
Contributor

marmbrus commented Feb 11, 2015

test this please

asfgit pushed a commit that referenced this pull request Feb 11, 2015
@rxin @marmbrus
[SQL][DataFrame] Fix column computability bug.
e477e91 
Do not recursively strip out projects. Only strip the first level project.

```scala
df("colA") + df("colB").as("colC")
```

Previously, the above would construct an invalid plan.

Author: Reynold Xin <rxin@databricks.com>

Closes #4519 from rxin/computability and squashes the following commits:

87ff763 [Reynold Xin] Code review feedback.
015c4fc [Reynold Xin] [SQL][DataFrame] Fix column computability.

(cherry picked from commit 7e24249)
Signed-off-by: Michael Armbrust <michael@databricks.com>
@asfgit asfgit closed this in 7e24249 Feb 11, 2015
@SparkQA
Copy link

SparkQA commented Feb 11, 2015

Test build #27263 has finished for PR 4519 at commit 87ff763.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@AmplabJenkins
Copy link

AmplabJenkins commented Feb 11, 2015

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/27263/
Test PASSed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@rxin @SparkQA @AmplabJenkins @marmbrus