bug: load parquet data to databend failed if had timestamp columns

[FANNG1]

Search before asking

• I had searched in the issues and found no similar issues.

Version

80e09b3

What's Wrong?

could't load parquet data to databend

How to Reproduce?

1. create table with sql "create table mytime(id string, t_bool boolean, t_float float, t_double double, t_timestamp timestamp, t_data date, t_array array(int));"
2. load parquet data, curl -XPUT 'http://root:@127.0.0.1:8081/v1/streaming_load' -H 'insert_sql: insert into mytime format Parquet' -F 'upload=@"./mytime.parquet"'

parquet data: https://github.com/sandflee/databend/blob/parquet/mytime.parquet

Are you willing to submit PR?

• Yes I am willing to submit a PR!

[FANNG1]

databend transfrom timestamp to int64 arrow type, while the phiscal type of "t_timestamp" in parquet is int96, this may cause the error, any advise?

[sundy-li]

Oh, databend's timestamp is always int64. But parquet may have int96 timestamp.
There is a method to handle this, maybe we should map timestamp to arrow2's timestamp datatype.
Let me find a way to fix this.

[FANNG1]

using arrow::timestamp seems reasonable, could compatible parquet int64 and int96