-
Notifications
You must be signed in to change notification settings - Fork 3.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
opt: inline all filters with virtual columns #60553
opt: inline all filters with virtual columns #60553
Conversation
7be450f
to
8cccb58
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Reviewable status: complete! 1 of 0 LGTMs obtained (waiting on @rytaft)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Reviewed 4 of 4 files at r1, 3 of 3 files at r2.
Reviewable status: complete! 2 of 0 LGTMs obtained (waiting on @mgartner and @RaduBerinde)
pkg/sql/opt/norm/inline_funcs.go, line 244 at r2 (raw file):
} // Initialize inlined lazily.
inlined -> inlinableFilters
pkg/sql/opt/norm/rules/inline.opt, line 189 at r2 (raw file):
# a INT, # b INT, # v INT AS (abs(a)),
should this have the tag VIRTUAL
?
pkg/sql/opt/xform/testdata/rules/select, line 1735 at r1 (raw file):
└── b:3 = 10 [outer=(3), constraints=(/3: [/10 - /10]; tight), fd=()-->(3)] # Test that we can constrain an partial index with a virtual column in the
[nit] an partial -> a partial
Previously, `InlineSelectVirtualColumns` inlined virtual column expressions in filters and pushed those filters below a Project only if the virtual columns were indexed. Now, all virtual column expressions are inlined. This allows exploration rules to generate expressions that utilize partial indexes with predicates that reference virtual columns. We considered extending `InlineSelectVirtualColumns` to also inline a virtual columns if they are referenced in partial index predicates. This would require addition metadata to keep track of the set of such virtual columns. Virtual columns have already been inlined in the predicates in the table metadata, so simply fetching the outer columns of those predicates would not suffice. Because this rule already is highly selective (only operating on virtual columns), we opted to simplify the rule to inline all virtual columns instead. There is no release note because virtual columns are gated behind an experimental session setting. Release note: None
Previously, running the `optsteps` test command panicked with "could not find path to expr" for queries when `InlineSelectVirtualColumns` was matched. This commit fixes the panic by eliminating the anti-pattern of calling a constructor during the pattern-match phase of a normalization rule. The `optsteps` test command displays step `n` of optimization by running the optimizer twice and halting after `n` and `n - 1` rules have been applied. The difference in the two resulting expressions represents the change in step `n`. `InlineSelectVirtualColumns` previously called `ConstructFiltersItem` during the pattern-match phase of the rule, but only added the constructed filters item to the memo in the replace phase of the rule. When some normalization rule applied during the call to `ConstructFiltersItem` was the last rule applied in a step, the constructed item was not added to the memo because the limit on the number of rules prevented the replace phase of `InlineSelectVirtualColumns`. As a result, `optsteps` panicked when it could not find the normalized `FiltersItem` in the memo. Release note: None
8cccb58
to
2ac4d8c
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Reviewable status: complete! 2 of 0 LGTMs obtained (waiting on @rytaft)
pkg/sql/opt/norm/inline_funcs.go, line 244 at r2 (raw file):
Previously, rytaft (Rebecca Taft) wrote…
inlined -> inlinableFilters
Done.
pkg/sql/opt/norm/rules/inline.opt, line 189 at r2 (raw file):
Previously, rytaft (Rebecca Taft) wrote…
should this have the tag
VIRTUAL
?
Done.
pkg/sql/opt/xform/testdata/rules/select, line 1735 at r1 (raw file):
Previously, rytaft (Rebecca Taft) wrote…
[nit] an partial -> a partial
Done.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Reviewed 4 of 4 files at r3, 3 of 3 files at r4.
Reviewable status: complete! 0 of 0 LGTMs obtained (and 2 stale)
bors r+ |
Build succeeded: |
opt: inline all filters with virtual columns
Previously,
InlineSelectVirtualColumns
inlined virtual columnexpressions in filters and pushed those filters below a Project only if
the virtual columns were indexed. Now, all virtual column expressions
are inlined. This allows exploration rules to generate expressions that
utilize partial indexes with predicates that reference virtual columns.
We considered extending
InlineSelectVirtualColumns
to also inline avirtual columns if they are referenced in partial index predicates.
This would require addition metadata to keep track of the set of such
virtual columns. Virtual columns have already been inlined in the
predicates in the table metadata, so simply fetching the outer columns
of those predicates would not suffice. Because this rule already is
highly selective (only operating on virtual columns), we opted to
simplify the rule to inline all virtual columns instead.
There is no release note because virtual columns are gated behind an
experimental session setting.
Release note: None
opt: fix optsteps panic from InlineSelectVirtualColumns
Previously, running the
optsteps
test command panicked with "could notfind path to expr" for queries when
InlineSelectVirtualColumns
wasmatched. This commit fixes the panic by eliminating the anti-pattern of
calling a constructor during the pattern-match phase of a normalization
rule.
The
optsteps
test command displays stepn
of optimization by runningthe optimizer twice and halting after
n
andn - 1
rules have beenapplied. The difference in the two resulting expressions represents the
change in step
n
.InlineSelectVirtualColumns
previously calledConstructFiltersItem
during the pattern-match phase of the rule, butonly added the constructed filters item to the memo in the replace
phase of the rule. When some normalization rule applied during the call
to
ConstructFiltersItem
was the last rule applied in a step, theconstructed item was not added to the memo because the limit on the
number of rules prevented the replace phase of
InlineSelectVirtualColumns
. As a result,optsteps
panicked when itcould not find the normalized
FiltersItem
in the memo.Release note: None