Skip to content

Conversation

@hvanhovell
Copy link
Contributor

What changes were proposed in this pull request?

This PR adds the ColumnName for the Spark Connect Scala Client. This is a stepping stone to implement the SQLImplicits.

Why are the changes needed?

API parity with the current API.

Does this PR introduce any user-facing change?

Yes. It adds new API.

How was this patch tested?

Added existing tests tot ColumnTestSuite.

ProblemFilters.exclude[Problem]("org.apache.spark.sql.functions.typedLit"),

// RelationalGroupedDataset
ProblemFilters.exclude[Problem]("org.apache.spark.sql.RelationalGroupedDataset.apply"),
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@zhenlineo why didn't this fail during CI?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you so much for fixing this, @hvanhovell ! I also spent some time to dig this.

@hvanhovell
Copy link
Contributor Author

Merging.

hvanhovell added a commit that referenced this pull request Feb 27, 2023
### What changes were proposed in this pull request?
This PR adds the ColumnName for the Spark Connect Scala Client. This is a stepping stone to implement the SQLImplicits.

### Why are the changes needed?
API parity with the current API.

### Does this PR introduce _any_ user-facing change?
Yes. It adds new API.

### How was this patch tested?
Added existing tests tot `ColumnTestSuite`.

Closes #40179 from hvanhovell/SPARK-42560.

Authored-by: Herman van Hovell <herman@databricks.com>
Signed-off-by: Herman van Hovell <herman@databricks.com>
(cherry picked from commit 24f0c45)
Signed-off-by: Herman van Hovell <herman@databricks.com>
Copy link
Member

@dongjoon-hyun dongjoon-hyun left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1, late LGTM.

snmvaughan pushed a commit to snmvaughan/spark that referenced this pull request Jun 20, 2023
### What changes were proposed in this pull request?
This PR adds the ColumnName for the Spark Connect Scala Client. This is a stepping stone to implement the SQLImplicits.

### Why are the changes needed?
API parity with the current API.

### Does this PR introduce _any_ user-facing change?
Yes. It adds new API.

### How was this patch tested?
Added existing tests tot `ColumnTestSuite`.

Closes apache#40179 from hvanhovell/SPARK-42560.

Authored-by: Herman van Hovell <herman@databricks.com>
Signed-off-by: Herman van Hovell <herman@databricks.com>
(cherry picked from commit 24f0c45)
Signed-off-by: Herman van Hovell <herman@databricks.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants