Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Convert first, last aggregate function to UDAF #10648

Merged
merged 86 commits into from
May 27, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
86 commits
Select commit Hold shift + click to select a range
e506662
move out the ordering ruel
jayzhan211 Apr 10, 2024
2812886
introduce rule
jayzhan211 Apr 10, 2024
d20dde3
revert test result
jayzhan211 Apr 10, 2024
c671033
pass mulit order test
jayzhan211 Apr 10, 2024
3f43dc1
cleanup
jayzhan211 Apr 10, 2024
c8b46a4
with new childes
jayzhan211 Apr 10, 2024
fcb297b
revert slt
jayzhan211 Apr 10, 2024
ab10168
revert back
jayzhan211 Apr 10, 2024
fffb54a
rm rewrite in new child
jayzhan211 Apr 10, 2024
fec3eaf
backup
jayzhan211 Apr 10, 2024
5b68b48
only move conversion to optimizer
jayzhan211 Apr 10, 2024
2ab74c9
find test that do reverse
jayzhan211 Apr 11, 2024
03abf19
add test for first and last
jayzhan211 Apr 11, 2024
83c721a
pass all test
jayzhan211 Apr 11, 2024
34b8f01
upd test
jayzhan211 Apr 11, 2024
b17e8f7
upd test
jayzhan211 Apr 12, 2024
a313af0
cleanup
jayzhan211 Apr 12, 2024
a37fb3e
cleanup
jayzhan211 Apr 12, 2024
8de501c
cleanup
jayzhan211 Apr 12, 2024
107f4c8
add aggregate test
jayzhan211 Apr 12, 2024
345d4c6
cleanup
jayzhan211 Apr 12, 2024
030d7b9
final draft
jayzhan211 Apr 12, 2024
cf3f33e
cleanup again
jayzhan211 Apr 12, 2024
6d9ee9f
pull out finer ordering code and reuse
jayzhan211 Apr 13, 2024
abd40c6
clippy
jayzhan211 Apr 13, 2024
3e2c186
remove finer in optimize rule
jayzhan211 Apr 13, 2024
d7c2590
add comments and clenaup
jayzhan211 Apr 13, 2024
fed356a
rename fun
jayzhan211 Apr 13, 2024
0d3d461
rename fun
jayzhan211 Apr 13, 2024
d3daa4e
fmt
jayzhan211 Apr 13, 2024
dcb3e90
avoid unnecessary recursion and rename
jayzhan211 Apr 13, 2024
4983fee
Minor changes
mustafasrepo Apr 15, 2024
4f0105d
Add new API for aggregate optimization
mustafasrepo Apr 15, 2024
4b1e24f
Minor changes
mustafasrepo Apr 15, 2024
d95d6de
Minor changes
mustafasrepo Apr 15, 2024
99f0fa0
Merge branch 'apache_main' into feature/10061_subsequent
mustafasrepo Apr 15, 2024
25119bd
Remove old code
mustafasrepo Apr 15, 2024
61f5b43
Minor changes
mustafasrepo Apr 15, 2024
857eedb
Minor changes
mustafasrepo Apr 15, 2024
98ce018
Minor changes
mustafasrepo Apr 15, 2024
a514db7
Minor changes
mustafasrepo Apr 15, 2024
99c369b
Minor changes
mustafasrepo Apr 15, 2024
c6fb7a2
Minor changes
mustafasrepo Apr 15, 2024
36951b9
Update comments
mustafasrepo Apr 15, 2024
d00d529
Minor changes
berkaysynnada Apr 17, 2024
782d144
Minor changes
mustafasrepo Apr 17, 2024
a4c5ca0
Merge branch 'apache_main' into feature/10061_subsequent
mustafasrepo Apr 24, 2024
56c7629
Review Part 1
ozankabak Apr 24, 2024
e1883d9
TMP
mustafasrepo Apr 25, 2024
fa9cdb1
Merge branch 'feature/10061_subsequent' of https://github.com/synnada…
mustafasrepo Apr 25, 2024
ca56d38
Update display of aggregate fun exprs
mustafasrepo Apr 25, 2024
5012fbd
TMP
mustafasrepo Apr 25, 2024
fef594e
TMP
mustafasrepo Apr 25, 2024
25ba8e8
Update tests
mustafasrepo Apr 25, 2024
a530dc4
TMP buggy
mustafasrepo Apr 25, 2024
98320b0
modify name in place
mustafasrepo Apr 25, 2024
257abbb
Minor changes
mustafasrepo Apr 25, 2024
0f9a903
Tmp
mustafasrepo Apr 25, 2024
e8dac2e
Merge branch 'apache_main' into feature/10061_subsequent
mustafasrepo Apr 26, 2024
b316885
Tmp
mustafasrepo Apr 26, 2024
df5d68c
Merge branch 'apache_main' into feature/10061_subsequent_pr
mustafasrepo Apr 29, 2024
782fa2c
Tmp
mustafasrepo Apr 29, 2024
f39e1fb
TMP
mustafasrepo Apr 29, 2024
274ea1c
Merge branch 'apache_main' into feature/10061_subsequent_pr
mustafasrepo May 9, 2024
23a48f3
Simplifications
mustafasrepo May 9, 2024
0fd29aa
Tmp
mustafasrepo May 9, 2024
78f798e
Tmp
mustafasrepo May 10, 2024
bd2eeed
Merge branch 'apache_main' into feature/10061_subsequent_pr
mustafasrepo May 23, 2024
9364129
Compiles
mustafasrepo May 23, 2024
2816c5b
Resolve linter errors
mustafasrepo May 23, 2024
910f0ec
Resolve linter errors
mustafasrepo May 23, 2024
ba403dc
Minor changes
mustafasrepo May 23, 2024
1da66f5
Simplifications
mustafasrepo May 23, 2024
5fa6b5a
Minor chagnes
mustafasrepo May 23, 2024
2aa4244
Move cast to common
mustafasrepo May 23, 2024
7393158
Minor changes
mustafasrepo May 23, 2024
92fec67
Fix test
mustafasrepo May 23, 2024
809dc7f
Minor changes
mustafasrepo May 23, 2024
e091afc
Simplifications
mustafasrepo May 23, 2024
89a601b
Review
ozankabak May 24, 2024
9e50da6
Address reviews
mustafasrepo May 24, 2024
c9bcc89
Address reviews
mustafasrepo May 27, 2024
13eed8b
Merge branch 'apache_main' into feature/10061_subsequent_pr
mustafasrepo May 27, 2024
93cccdb
Update documentation, rename method
mustafasrepo May 27, 2024
4c430d3
Minor changes
mustafasrepo May 27, 2024
cbd450c
Merge branch 'apache_main' into feature/10061_subsequent_pr
mustafasrepo May 27, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions datafusion-cli/Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions datafusion/core/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -105,6 +105,7 @@ datafusion-functions-aggregate = { workspace = true }
datafusion-functions-array = { workspace = true, optional = true }
datafusion-optimizer = { workspace = true }
datafusion-physical-expr = { workspace = true }
datafusion-physical-expr-common = { workspace = true }
datafusion-physical-plan = { workspace = true }
datafusion-sql = { workspace = true }
flate2 = { version = "1.0.24", optional = true }
Expand Down
260 changes: 0 additions & 260 deletions datafusion/core/src/physical_optimizer/convert_first_last.rs

This file was deleted.

2 changes: 1 addition & 1 deletion datafusion/core/src/physical_optimizer/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,6 @@
pub mod aggregate_statistics;
pub mod coalesce_batches;
pub mod combine_partial_final_agg;
mod convert_first_last;
pub mod enforce_distribution;
pub mod enforce_sorting;
pub mod join_selection;
Expand All @@ -37,6 +36,7 @@ pub mod pruning;
pub mod replace_with_order_preserving_variants;
mod sort_pushdown;
pub mod topk_aggregation;
mod update_aggr_exprs;
mod utils;

#[cfg(test)]
Expand Down
2 changes: 1 addition & 1 deletion datafusion/core/src/physical_optimizer/optimizer.rs
Original file line number Diff line number Diff line change
Expand Up @@ -19,8 +19,8 @@

use std::sync::Arc;

use super::convert_first_last::OptimizeAggregateOrder;
use super::projection_pushdown::ProjectionPushdown;
use super::update_aggr_exprs::OptimizeAggregateOrder;
use crate::config::ConfigOptions;
use crate::physical_optimizer::aggregate_statistics::AggregateStatistics;
use crate::physical_optimizer::coalesce_batches::CoalesceBatches;
Expand Down
Loading
Loading