-
Notifications
You must be signed in to change notification settings - Fork 1.7k
Implement tree explain for PartialSortExec #15066
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks @irenjj -- this looks great. I think we could make it simpler though . Let me know what you think
// TODO: collect info | ||
write!(f, "") | ||
let common_prefix_length = self.common_prefix_length; | ||
match self.fetch { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I recommend we make this simpler -- specifically the common_prefix_length is too detailed for the TreeRender per
datafusion/datafusion/physical-plan/src/display.rs
Lines 48 to 75 in 2458693
/// TreeRender, displayed in the `tree` explain type. | |
/// | |
/// This format is inspired by DuckDB's explain plans. The information | |
/// presented should be "user friendly", and contain only the most relevant | |
/// information for understanding a plan. It should NOT contain the same level | |
/// of detail information as the [`Self::Default`] format. | |
/// | |
/// In this mode, each line contains a key=value pair. | |
/// Everything before the first `=` is treated as the key, and everything after the | |
/// first `=` is treated as the value. | |
/// | |
/// For example, if the output of `TreeRender` is this: | |
/// ```text | |
/// partition_sizes=[1] | |
/// partitions=1 | |
/// ``` | |
/// | |
/// It is rendered in the center of a box in the following way: | |
/// | |
/// ```text | |
/// ┌───────────────────────────┐ | |
/// │ DataSourceExec │ | |
/// │ -------------------- │ | |
/// │ partition_sizes: [1] │ | |
/// │ partitions: 1 │ | |
/// └───────────────────────────┘ | |
/// ``` | |
TreeRender, |
I also think the TopK / Fetch thing is confusing
What I would recommend is:
01)┌───────────────────────────┐
02)│ PartialSortExec │
03)│ -------------------- │
07)│ sort: │
08)│[a@1 ASC NULLS LAST, b@2...│
09)└─────────────┬─────────────┘
10)┌─────────────┴─────────────┐
11)│ StreamingTableExec │
12)└───────────────────────────┘
And
01)┌───────────────────────────┐
02)│ PartialSortExec │
03)│ -------------------- │
04)│ fetch=50 │
09)│ sort: │
10)│[a@1 ASC NULLS LAST, b@2...│
11)└─────────────┬─────────────┘
Here is what duckdb does for refernece:
D create table foo (x int);
D insert into foo values (1);
D explain select * from foo order by x;
┌─────────────────────────────┐
│┌───────────────────────────┐│
││ Physical Plan ││
│└───────────────────────────┘│
└─────────────────────────────┘
┌───────────────────────────┐
│ PROJECTION │
│ ──────────────────── │
│__internal_decompress_integ│
│ ral_integer(#0, 1) │
│ │
│ ~1 Rows │
└─────────────┬─────────────┘
┌─────────────┴─────────────┐
│ ORDER_BY │
│ ──────────────────── │
│ memory.main.foo.x ASC │
└─────────────┬─────────────┘
┌─────────────┴─────────────┐
│ PROJECTION │
│ ──────────────────── │
│__internal_compress_integra│
│ l_utinyint(#0, 1) │
│ │
│ ~1 Rows │
└─────────────┬─────────────┘
┌─────────────┴─────────────┐
│ SEQ_SCAN │
│ ──────────────────── │
│ Table: foo │
│ Type: Sequential Scan │
│ Projections: x │
│ │
│ ~1 Rows │
└───────────────────────────┘
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks @irenjj
DisplayFormatType::TreeRender => match self.fetch { | ||
Some(fetch) => { | ||
writeln!(f, "limit={fetch}")?; | ||
writeln!(f, "sort keys=[{}]", self.expr) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I recommend avoiding the [
and ]
if possible
5816313
to
2006ebb
Compare
Thanks again @irenjj |
Which issue does this PR close?
PartialSortExec
#15064Rationale for this change
What changes are included in this PR?
Are these changes tested?
Are there any user-facing changes?