Skip to content
This repository has been archived by the owner on Sep 18, 2023. It is now read-only.

Spark UT of "randomSplit on reordered partitions" encountered "Invalid: Map array child array should have no nulls" issue #788

Closed
zhixingheyi-tian opened this issue Mar 23, 2022 · 0 comments
Labels
bug Something isn't working

Comments

@zhixingheyi-tian
Copy link
Collaborator

Spark UT of "randomSplit on reordered partitions" encountered "Invalid: Map array child array should have no nulls" issue

org.apache.spark.sql.execution.ProjectExec(List(class org.apache.spark.sql.execution.SerializeFromObjectExec))
Enter doExecuteColumnar()
/tmp/spark_columnar_plugin_865028915242406534/libarrow.so.400.0.0(+0x27daf0)[0x7f35d259caf0]
/tmp/spark_columnar_plugin_865028915242406534/libarrow.so.400.0.0(_ZN5arrow4util8ArrowLogD2Ev+0xdd)[0x7f35d2824add]
/tmp/spark_columnar_plugin_865028915242406534/libarrow.so.400.0.0(_ZN5arrow8MapArray7SetDataERKSt10shared_ptrINS_9ArrayDataEE+0x1b8)[0x7f35d25f4208]
/tmp/spark_columnar_plugin_865028915242406534/libarrow.so.400.0.0(_ZN5arrow8MapArrayC2ERKSt10shared_ptrINS_9ArrayDataEE+0x74)[0x7f35d25f57e4]
/tmp/spark_columnar_plugin_865028915242406534/libarrow.so.400.0.0(_ZN5arrow9MakeArrayERKSt10shared_ptrINS_9ArrayDataEE+0x268)[0x7f35d265af68]
/tmp/spark_columnar_plugin_865028915242406534/libarrow.so.400.0.0(_ZNK5arrow11RecordBatch7columnsEv+0x288)[0x7f35d26c50c8]
/tmp/spark_columnar_plugin_865028915242406534/libspark_columnar_jni.so(_ZN19sparkcolumnarplugin7codegen12arrowcompute30SortArraysToIndicesVisitorImpl4EvalEv+0x55)[0x7f35c8701c65]
/tmp/spark_columnar_plugin_865028915242406534/libspark_columnar_jni.so(_ZN19sparkcolumnarplugin7codegen12arrowcompute11ExprVisitor4EvalEv+0x8f)[0x7f35c86f4e2f]
/tmp/spark_columnar_plugin_865028915242406534/libspark_columnar_jni.so(_ZN19sparkcolumnarplugin7codegen12arrowcompute11ExprVisitor4EvalERSt10shared_ptrIN5arrow11RecordBatchEE+0x51)[0x7f35c86f4f31]
/tmp/spark_columnar_plugin_865028915242406534/libspark_columnar_jni.so(_ZN19sparkcolumnarplugin7codegen12arrowcompute25ArrowComputeCodeGenerator8evaluateERSt10shared_ptrIN5arrow11RecordBatchEEPSt6vectorIS6_SaIS6_EE+0x103)[0x7f35c86abde3]
/tmp/spark_columnar_plugin_865028915242406534/libspark_columnar_jni.so(Java_com_intel_oap_vectorized_ExpressionEvaluatorJniWrapper_nativeEvaluate2+0x15b)[0x7f35c8694dcb]
[0x7f3675017b94]
/tmp/spark_columnar_plugin_865028915242406534/libarrow.so.400.0.0(+0x27daf0)[0x7f35d259caf0]
/tmp/spark_columnar_plugin_865028915242406534/libarrow.so.400.0.0(_ZN5arrow4util8ArrowLogD2Ev+0xdd)[0x7f35d2824add]
/home/shen/zhixingheyi-tian/arrow/cpp/src/arrow/array/array_nested.cc:375:  Check failed: _s.ok() Operation failed: ValidateChildData(data->child_data)
Bad status: Invalid: Map array child array should have no nulls
/home/shen/zhixingheyi-tian/arrow/cpp/src/arrow/array/array_nested.cc:375:  Check failed: _s.ok() Operation failed: ValidateChildData(data->child_data)
Bad status: Invalid: Map array child array should have no nulls
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants