Skip to content
This repository has been archived by the owner on Sep 18, 2023. It is now read-only.

Create table failed with TPCH partiton dataset #727

Closed
zhouyuan opened this issue Feb 7, 2022 · 2 comments
Closed

Create table failed with TPCH partiton dataset #727

zhouyuan opened this issue Feb 7, 2022 · 2 comments
Labels
bug Something isn't working

Comments

@zhouyuan
Copy link
Collaborator

zhouyuan commented Feb 7, 2022

Describe the bug
The issue seems due to some wrong escape on special characters

java.lang.RuntimeException: Opening HDFS file '/user/sparkuser/t_tpch_parquet_100/part/p_brand=Brand%252311/part-00000-21a924cf-440b-416a-a2f6-5a7ac39d960e.c000.snappy.parquet' failed
  at org.apache.arrow.dataset.jni.JniWrapper.inspectSchema(Native Method)
  at org.apache.arrow.dataset.jni.NativeDatasetFactory.inspect(NativeDatasetFactory.java:63)
  at com.intel.oap.spark.sql.execution.datasources.v2.arrow.ArrowUtils$.readSchema(ArrowUtils.scala:47)
  at com.intel.oap.spark.sql.execution.datasources.v2.arrow.ArrowUtils$.readSchema(ArrowUtils.scala:59)
  at com.intel.oap.spark.sql.execution.datasources.arrow.ArrowFileFormat.convert(ArrowFileFormat.scala:59)
  at com.intel.oap.spark.sql.execution.datasources.arrow.ArrowFileFormat.inferSchema(ArrowFileFormat.scala:65)

To Reproduce
create TPCH partitioned tables in hive metastore

Expected behavior
create table successful

Additional context
N/A

@zhouyuan zhouyuan added the bug Something isn't working label Feb 7, 2022
@zhouyuan zhouyuan changed the title create database failed with TPCH partiton dataset create table failed with TPCH partiton dataset Feb 7, 2022
@zhztheplayer
Copy link
Collaborator

Will try to reproduce. Thanks yuan.

@zhztheplayer zhztheplayer changed the title create table failed with TPCH partiton dataset Create table failed with TPCH partiton dataset Feb 9, 2022
@zhouyuan
Copy link
Collaborator Author

close #734

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants