-
Notifications
You must be signed in to change notification settings - Fork 437
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[CH-186] Support RangePartitioning
#524
Conversation
Thanks for opening a pull request! Could you open an issue for this pull request on Github Issues? https://github.com/oap-project/gluten/issues Then could you also rename commit message and pull request title in the following format?
See also: |
3aafdc7
to
b2f54f2
Compare
919bcc0
to
2d6bec5
Compare
414853a
to
0a9a918
Compare
retest this please |
gluten-core/src/main/scala/io/glutenproject/extension/ColumnarOverrides.scala
Outdated
Show resolved
Hide resolved
There is something wrong under aqe |
7bb984b
to
e71ff7c
Compare
Build a projection node and pass it to the native splitter, the native splitter will compute the expressions now. |
0fd3af1
to
8617faf
Compare
gluten-core/src/main/scala/io/glutenproject/extension/ColumnarOverrides.scala
Outdated
Show resolved
Hide resolved
gluten-core/src/main/scala/io/glutenproject/extension/ColumnarOverrides.scala
Outdated
Show resolved
Hide resolved
shims/common/src/main/scala/io/glutenproject/GlutenConfig.scala
Outdated
Show resolved
Hide resolved
...nds-clickhouse/src/main/scala/io/glutenproject/backendsapi/clickhouse/CHTransformerApi.scala
Outdated
Show resolved
Hide resolved
backends-clickhouse/src/main/scala/org/apache/spark/sql/execution/utils/CHExecUtil.scala
Outdated
Show resolved
Hide resolved
...ickhouse/src/main/scala/org/apache/spark/shuffle/utils/RangePartitionerBoundsGenerator.scala
Outdated
Show resolved
Hide resolved
backends-clickhouse/src/test/scala/io/glutenproject/execution/GlutenClickHouseTPCHSuite.scala
Show resolved
Hide resolved
68105d8
to
a095aef
Compare
RangePartitionoing
RangePartitioning
RangePartitioning
RangePartitioning
What changes were proposed in this pull request?
Implement a new
NativePartitioning
to supportRangePartitioning
.what cases are supported
don't support order by a complex data type.
in case the backend has functions incompleted, supply option
spark.gluten.sql.columnar.rangepartitioning
to disable this columnar range partitioning.How was this patch tested?
unit tests
(If this patch involves UI changes, please attach a screenshot; otherwise, remove this)
test by CH[[189]]