Logical types

[sundy-li]

I hereby agree to the terms of the CLA available at: https://datafuse.rs/policies/cla/

Summary

tasks:

• Custom logic types on arrow2 using ARROW::extension::name in field's metadata, extension types
• DataArray/DataValue will be related only with physical types, so DateTime32 and UInt32 will share same physical array inside.
• Refactor DataArray<T>
• Support Date16, Date32, DateTime32 types
• Add now(), today() functions

Summary about this PR

Changelog

• Improvement

Related Issues

Fixes #1627

Test Plan

Unit Tests

Stateless Tests

[databend-bot]

Thanks for the contribution!
I have applied any labels matching special text in your PR Changelog.

Please review the labels and make any necessary changes.

[sundy-li]

Works now:

mysql> select toDateTime(number ) as c  from numbers(3) order by c;
+---------------------+
| c                   |
+---------------------+
| 1970-01-01 00:00:00 |
| 1970-01-01 00:00:01 |
| 1970-01-01 00:00:02 |
+---------------------+
3 rows in set (0.02 sec)

mysql> select toDate(number ) as c  from numbers(3) order by c;
+------------+
| c          |
+------------+
| 1970-01-01 |
| 1970-01-02 |
| 1970-01-03 |
+------------+
3 rows in set (0.01 sec)

[codecov-commenter]

Codecov Report

Merging #1648 (31fdb41) into master (51cdf29) will decrease coverage by 0%.
The diff coverage is 51%.

@@          Coverage Diff           @@
##           master   #1648   +/-   ##
======================================
- Coverage      73%     73%   -1%     
======================================
  Files         544     564   +20     
  Lines       33132   33085   -47     
======================================
- Hits        24437   24362   -75     
- Misses       8695    8723   +28     

Impacted Files	Coverage Δ
common/datavalues/src/arrays/binary/iterator.rs	0% <0%> (ø)
common/datavalues/src/arrays/ops/contain.rs	100% <ø> (ø)
...ommon/datavalues/src/arrays/ops/group_hash_test.rs	100% <ø> (ø)
...mmon/datavalues/src/arrays/ops/take_random_test.rs	100% <ø> (ø)
common/datavalues/src/arrays/ops/take_single.rs	29% <0%> (ø)
common/datavalues/src/arrays/ops/take_test.rs	97% <ø> (ø)
common/datavalues/src/arrays/ops/vec_hash.rs	54% <0%> (ø)
common/datavalues/src/arrays/struct/mod.rs	0% <0%> (ø)
common/datavalues/src/data_group_value.rs	25% <ø> (+6%)	⬆️
common/datavalues/src/series/de.rs	88% <ø> (+8%)	⬆️
... and 145 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 51cdf29...31fdb41. Read the comment docs.

[sundy-li]

The cached building is broken

[BohuTANG]

The cached building is broken

I have updated the cache key.

[BohuTANG]

The cached building is broken

I have updated the cache key.

Still not working, @PsiACE may help :)

[PsiACE]

Still not working, @PsiACE may help :)

set CARGO_INCREMENTAL: '0' for workflows

fyi, https://github.com/datafuselabs/datafuse/blob/238074515b158f00e58444771baf63f1160a2031/.github/workflows/unit-tests.yml#L48

[BohuTANG]

Still not working, @PsiACE may help :)

set CARGO_INCREMENTAL: '0' for workflows

fyi,

https://github.com/datafuselabs/datafuse/blob/238074515b158f00e58444771baf63f1160a2031/.github/workflows/unit-tests.yml#L48

In this barnch, it's already CARGO_INCREMENTAL: '0', please see:
https://github.com/datafuselabs/datafuse/blob/238074515b158f00e58444771baf63f1160a2031/.github/workflows/unit-tests.yml#L48

[BohuTANG]

@PsiACE
Found a build log:

with:
    path: ~/.cargo/bin/
  ~/.cargo/registry/index/
  ~/.cargo/registry/cache/
  ~/.cargo/git/db/
  target/
  
    key: Linux-cargo-clippycache--939e19488e3de5d8898a4c80eb4c56293237222f3dbe16428133ed00aa480d15
    restore-keys: Linux-cargo-clippycache--
  Linux-cargo-clippycache-
  Linux-cargo-
  
  env:
    CARGO_TERM_COLOR: always
Received 218103808 of 2157467133 (10.1%), 207.8 MBs/sec
Received 469762048 of 2157467133 (21.8%), 223.6 MBs/sec
Received 729808896 of 2157467133 (33.8%), 231.5 MBs/sec
Received 981467136 of 2157467133 (45.5%), 233.6 MBs/sec
Received 1237319680 of 2157467133 (57.4%), 235.6 MBs/sec
Received 1484783616 of 2157467133 (68.8%), 235.7 MBs/sec
Received 1732247552 of 2157467133 (80.3%), 235.8 MBs/sec
Received 1983905792 of 2157467133 (92.0%), 236.2 MBs/sec
Received 2147483647 of 2157467133 (99.5%), 112.7 MBs/sec
Received 2157467133 of 2157467133 (100.0%), 112.3 MBs/sec
Cache Size: ~2058 MB (2157467133 B)
/usr/bin/tar --use-compress-program zstd -d -xf /home/runner/work/_temp/15d7f035-0c68-456c-835d-baf447aa1c06/cache.tzst -P -C /home/runner/work/datafuse/datafuse
Cache restored successfully
Cache restored from key: Linux-cargo-clippycache--939e19488e3de5d8898a4c80eb4c56293237222f3dbe16428133ed00aa480d15

Not sure how to clean the cache.

[zhang2014]

LGTM

[databend-bot]

CI Passed
Reviewer Approved
Let's Merge

[databend-bot]

CI Passed
Reviewer Approved
Let's Merge