Skip to content

Conversation

@zhengruifeng
Copy link
Contributor

What changes were proposed in this pull request?

Make json_tuple throw PySparkValueError for empty fields

Why are the changes needed?

Python side should have the same check as the Scala side:

@scala.annotation.varargs
def json_tuple(json: Column, fields: String*): Column = {
require(fields.nonEmpty, "at least 1 field name should be given.")
Column.fn("json_tuple", json +: fields.map(lit): _*)
}

Does this PR introduce any user-facing change?

yes

How was this patch tested?

added ut

Was this patch authored or co-authored using generative AI tooling?

no

@zhengruifeng
Copy link
Contributor Author

cc @HyukjinKwon and @itholic

fix

init
@zhengruifeng
Copy link
Contributor Author

thanks, merged to master

@zhengruifeng zhengruifeng deleted the py_check_functions branch January 2, 2024 03:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants