Skip to content

Commit f7aeba3

Browse files
marmbrusAndreSchumacher
authored andcommitted
[SPARK-1982] Support for ByteType and ShortType.
1 parent 3104886 commit f7aeba3

File tree

1 file changed

+3
-0
lines changed

1 file changed

+3
-0
lines changed

sql/core/src/main/scala/org/apache/spark/sql/parquet/ParquetTypes.scala

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -170,6 +170,9 @@ private[parquet] object ParquetTypesConverter {
170170
Some(ParquetPrimitiveTypeName.FIXED_LEN_BYTE_ARRAY)
171171
case FloatType => Some(ParquetPrimitiveTypeName.FLOAT)
172172
case IntegerType => Some(ParquetPrimitiveTypeName.INT32)
173+
// There is no type for Byte or Short so we promote them to INT32.
174+
case ShortType => Some(ParquetPrimitiveTypeName.INT32)
175+
case ByteType => Some(ParquetPrimitiveTypeName.INT32)
173176
case LongType => Some(ParquetPrimitiveTypeName.INT64)
174177
case _ => None
175178
}

0 commit comments

Comments
 (0)