[NSE-943] improve rowtocolumn operator (#990)

* [NSE-943] Optimize String/Binary Type for Row2Columnar * Fix TPCDS queries * Add __AVX512BW__ Check * [WIP][NSE-943] Utilize CPU Cache by first-row-second-column and fixed-width type Optimization * Extract vector from functin * Add optimizations * Add remaining optimization * Remove ListType in Native R2C * Fix Spark UT * Clean code * Fix clang format
oap-project · Jul 13, 2022 · fadf473 · fadf473
1 parent a6be543
commit fadf473
Show file tree

Hide file tree

Showing 3 changed files with 259 additions and 1,274 deletions.
diff --git a/native-sql-engine/core/src/main/scala/com/intel/oap/execution/ArrowRowToColumnarExec.scala b/native-sql-engine/core/src/main/scala/com/intel/oap/execution/ArrowRowToColumnarExec.scala
@@ -64,7 +64,6 @@ case class ArrowRowToColumnarExec(child: SparkPlan) extends UnaryExecNode {
         case d: DecimalType =>
         case d: TimestampType =>
         case d: BinaryType =>
-        case d: ArrayType => ConverterUtils.checkIfTypeSupported(d.elementType)
         case _ =>
           throw new UnsupportedOperationException(s"${field.dataType} " +
             s"is not supported in ArrowRowToColumnarExec.")

diff --git a/...-sql-engine/core/src/test/scala/com/intel/oap/execution/ArrowRowToColumnarExecSuite.scala b/...-sql-engine/core/src/test/scala/com/intel/oap/execution/ArrowRowToColumnarExecSuite.scala