Skip to content

Commit c84bc40

Browse files
liyuanjianxuanyuanking
authored andcommitted
[SPARK-25072][PYSPARK] Forbid extra value for custom Row
## What changes were proposed in this pull request? Add value length check in `_create_row`, forbid extra value for custom Row in PySpark. ## How was this patch tested? New UT in pyspark-sql Closes #22140 from xuanyuanking/SPARK-25072. Lead-authored-by: liyuanjian <liyuanjian@baidu.com> Co-authored-by: Yuanjian Li <xyliyuanjian@gmail.com> Signed-off-by: Bryan Cutler <cutlerb@gmail.com>
1 parent 3b6591b commit c84bc40

File tree

2 files changed

+7
-0
lines changed

2 files changed

+7
-0
lines changed

python/pyspark/sql/tests.py

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -277,6 +277,10 @@ def test_struct_field_type_name(self):
277277
struct_field = StructField("a", IntegerType())
278278
self.assertRaises(TypeError, struct_field.typeName)
279279

280+
def test_invalid_create_row(self):
281+
row_class = Row("c1", "c2")
282+
self.assertRaises(ValueError, lambda: row_class(1, 2, 3))
283+
280284

281285
class SQLTests(ReusedSQLTestCase):
282286

python/pyspark/sql/types.py

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1500,6 +1500,9 @@ def __contains__(self, item):
15001500
# let object acts like class
15011501
def __call__(self, *args):
15021502
"""create new Row object"""
1503+
if len(args) > len(self):
1504+
raise ValueError("Can not create Row with fields %s, expected %d values "
1505+
"but got %s" % (self, len(self), args))
15031506
return _create_row(self, args)
15041507

15051508
def __getitem__(self, item):

0 commit comments

Comments
 (0)